From 04caa3398dd64b792b76aaab4aa16790ac3bded1 Mon Sep 17 00:00:00 2001 From: yuanqi Date: Wed, 14 May 2025 09:08:41 +0800 Subject: [PATCH 1/2] remove not support jit in hook programming --- .../source_en/custom_program/hook_program.md | 8 ++--- .../custom_program/hook_program.ipynb | 32 +++++++++---------- 2 files changed, 20 insertions(+), 20 deletions(-) diff --git a/tutorials/source_en/custom_program/hook_program.md b/tutorials/source_en/custom_program/hook_program.md index 98d0d6a1bc..7bb04eb38e 100644 --- a/tutorials/source_en/custom_program/hook_program.md +++ b/tutorials/source_en/custom_program/hook_program.md @@ -45,7 +45,7 @@ For more descriptions of the HookBackward operator, refer to the [API documentat ## register_forward_pre_hook Function in Cell Object -The user can use the `register_forward_pre_hook` function on the Cell object to register a custom Hook function to capture data that is passed to that Cell object. This function does not work in static graph mode and inside functions modified with `@jit`. The `register_forward_pre_hook` function takes the Hook function as an input and returns a `handle` object that corresponds to the Hook function. The user can remove the corresponding Hook function by calling the `remove()` function of the `handle` object. Each call to the `register_forward_pre_hook` function returns a different `handle` object. Hook functions should be defined in the following way. +The user can use the `register_forward_pre_hook` function on the Cell object to register a custom Hook function to capture data that is passed to that Cell object. The `register_forward_pre_hook` function takes the Hook function as an input and returns a `handle` object that corresponds to the Hook function. The user can remove the corresponding Hook function by calling the `remove()` function of the `handle` object. Each call to the `register_forward_pre_hook` function returns a different `handle` object. Hook functions should be defined in the following way. ```python def forward_pre_hook_fn(cell, inputs): @@ -149,7 +149,7 @@ For more information about the `register_forward_pre_hook` function of the Cell ## register_forward_hook Function of Cell Object -The user can use the `register_forward_hook` function on the Cell object to register a custom Hook function that captures the data passed forward to the Cell object and the output data of the Cell object. This function does not work in static graph mode and inside functions modified with `@jit`. The `register_forward_hook` function takes the Hook function as an input and returns a `handle` object that corresponds to the Hook function. The user can remove the corresponding Hook function by calling the `remove()` function of the `handle` object. Each call to the `register_forward_hook` function returns a different `handle` object. Hook functions should be defined in the following way. +The user can use the `register_forward_hook` function on the Cell object to register a custom Hook function that captures the data passed forward to the Cell object and the output data of the Cell object. The `register_forward_hook` function takes the Hook function as an input and returns a `handle` object that corresponds to the Hook function. The user can remove the corresponding Hook function by calling the `remove()` function of the `handle` object. Each call to the `register_forward_hook` function returns a different `handle` object. Hook functions should be defined in the following way. The sample code is as follows: @@ -214,7 +214,7 @@ For more information about the `register_forward_hook` function of the Cell obje ## register_backward_pre_hook Function of Cell Object -The user can use the `register_backward_pre_hook` function on the Cell object to register a custom Hook function that captures the gradient associated with the Cell object when the network is back propagated. This function does not work in graph mode or inside functions modified with `@jit`. The `register_backward_pre_hook` function takes the Hook function as an input and returns a `handle` object that corresponds to the Hook function. The user can remove the corresponding Hook function by calling the `remove()` function of the `handle` object. Each call to the `register_backward_pre_hook` function will return a different `handle` object. +The user can use the `register_backward_pre_hook` function on the Cell object to register a custom Hook function that captures the gradient associated with the Cell object when the network is back propagated. The `register_backward_pre_hook` function takes the Hook function as an input and returns a `handle` object that corresponds to the Hook function. The user can remove the corresponding Hook function by calling the `remove()` function of the `handle` object. Each call to the `register_backward_pre_hook` function will return a different `handle` object. Unlike the custom Hook function used by the HookBackward operator, the inputs of the Hook function used by `register_backward_pre_hook` contains `cell`, which represents the information of the Cell object, the gradient passed to the Cell object in reverse of the Cell object. @@ -279,7 +279,7 @@ For more information about the `register_backward_pre_hook` function of the Cell ## register_backward_hook Function of Cell Object -The user can use the `register_backward_hook` function on the Cell object to register a custom Hook function that captures the gradient associated with the Cell object when the network is back propagated. This function does not work in graph mode or inside functions modified with `@jit`. The `register_backward_hook` function takes the Hook function as an input and returns a `handle` object that corresponds to the Hook function. The user can remove the corresponding Hook function by calling the `remove()` function of the `handle` object. Each call to the `register_backward_hook` function will return a different `handle` object. +The user can use the `register_backward_hook` function on the Cell object to register a custom Hook function that captures the gradient associated with the Cell object when the network is back propagated. The `register_backward_hook` function takes the Hook function as an input and returns a `handle` object that corresponds to the Hook function. The user can remove the corresponding Hook function by calling the `remove()` function of the `handle` object. Each call to the `register_backward_hook` function will return a different `handle` object. Unlike the custom Hook function used by the HookBackward operator, the inputs of the Hook function used by `register_backward_hook` contains `cell`, which represents the information of the Cell object, the gradient passed to the Cell object in reverse, and the gradient of the reverse output of the Cell object. diff --git a/tutorials/source_zh_cn/custom_program/hook_program.ipynb b/tutorials/source_zh_cn/custom_program/hook_program.ipynb index 0cc13c2eda..0eddc34389 100644 --- a/tutorials/source_zh_cn/custom_program/hook_program.ipynb +++ b/tutorials/source_zh_cn/custom_program/hook_program.ipynb @@ -26,12 +26,23 @@ }, { "cell_type": "code", + "execution_count": 15, "metadata": { "ExecuteTime": { "end_time": "2024-08-15T03:32:04.585336Z", "start_time": "2024-08-15T03:32:04.578481Z" } }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "hook_fn print grad_out: (Tensor(shape=[], dtype=Float32, value= 2),)\n", + "output: (Tensor(shape=[], dtype=Float32, value= 4), Tensor(shape=[], dtype=Float32, value= 4))\n" + ] + } + ], "source": [ "import mindspore as ms\n", "from mindspore import ops\n", @@ -54,18 +65,7 @@ "\n", "output = net(ms.Tensor(1, ms.float32), ms.Tensor(2, ms.float32))\n", "print(\"output:\", output)" - ], - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "hook_fn print grad_out: (Tensor(shape=[], dtype=Float32, value= 2),)\n", - "output: (Tensor(shape=[], dtype=Float32, value= 4), Tensor(shape=[], dtype=Float32, value= 4))\n" - ] - } - ], - "execution_count": 15 + ] }, { "cell_type": "markdown", @@ -75,7 +75,7 @@ "\n", "## Cell对象的register_forward_pre_hook功能\n", "\n", - "用户可以对Cell对象使用`register_forward_pre_hook`函数来注册一个自定义的Hook函数，用来捕获正向传入该Cell对象的数据。该功能在静态图模式下和在使用`@jit`修饰的函数内不起作用。`register_forward_pre_hook`函数接收Hook函数作为入参，并返回一个与Hook函数一一对应的`handle`对象。用户可以通过调用`handle`对象的`remove()`函数来删除与之对应的Hook函数。每一次调用`register_forward_pre_hook`函数，都会返回一个不同的`handle`对象。Hook函数应该按照以下的方式进行定义。" + "用户可以对Cell对象使用`register_forward_pre_hook`函数来注册一个自定义的Hook函数，用来捕获正向传入该Cell对象的数据。`register_forward_pre_hook`函数接收Hook函数作为入参，并返回一个与Hook函数一一对应的`handle`对象。用户可以通过调用`handle`对象的`remove()`函数来删除与之对应的Hook函数。每一次调用`register_forward_pre_hook`函数，都会返回一个不同的`handle`对象。Hook函数应该按照以下的方式进行定义。" ] }, { @@ -227,7 +227,7 @@ "\n", "## Cell对象的register_forward_hook功能\n", "\n", - "用户可以在Cell对象上使用`register_forward_hook`函数来注册一个自定义的Hook函数，用来捕获正向传入Cell对象的数据和Cell对象的输出数据。该功能在静态图模式下和在使用`@jit`修饰的函数内不起作用。`register_forward_hook`函数接收Hook函数作为入参，并返回一个与Hook函数一一对应的`handle`对象。用户可以通过调用`handle`对象的`remove()`函数来删除与之对应的Hook函数。每一次调用`register_forward_hook`函数，都会返回一个不同的`handle`对象。Hook函数应该按照以下的方式进行定义。\n", + "用户可以在Cell对象上使用`register_forward_hook`函数来注册一个自定义的Hook函数，用来捕获正向传入Cell对象的数据和Cell对象的输出数据。`register_forward_hook`函数接收Hook函数作为入参，并返回一个与Hook函数一一对应的`handle`对象。用户可以通过调用`handle`对象的`remove()`函数来删除与之对应的Hook函数。每一次调用`register_forward_hook`函数，都会返回一个不同的`handle`对象。Hook函数应该按照以下的方式进行定义。\n", "\n", "示例代码如下：" ] @@ -321,7 +321,7 @@ "\n", "## Cell对象的register_backward_pre_hook功能\n", "\n", - "用户可以在Cell对象上使用`register_backward_pre_hook`函数来注册一个自定义的Hook函数，用来捕获网络反向传播时与Cell对象相关联的梯度。该功能在图模式下或者在使用`@jit`修饰的函数内不起作用。`register_backward_pre_hook`函数接收Hook函数作为入参，并返回一个与Hook函数一一对应的`handle`对象。用户可以通过调用`handle`对象的`remove()`函数来删除与之对应的Hook函数。每一次调用`register_backward_pre_hook`函数，都会返回一个不同的`handle`对象。\n", + "用户可以在Cell对象上使用`register_backward_pre_hook`函数来注册一个自定义的Hook函数，用来捕获网络反向传播时与Cell对象相关联的梯度。`register_backward_pre_hook`函数接收Hook函数作为入参，并返回一个与Hook函数一一对应的`handle`对象。用户可以通过调用`handle`对象的`remove()`函数来删除与之对应的Hook函数。每一次调用`register_backward_pre_hook`函数，都会返回一个不同的`handle`对象。\n", "\n", "与HookBackward算子所使用的自定义Hook函数有所不同，`register_backward_pre_hook`使用的Hook函数的入参中，包含了表示Cell对象信息`cell`以及反向传入到Cell对象的梯度。\n", "示例代码如下：" @@ -414,7 +414,7 @@ "\n", "## Cell对象的register_backward_hook功能\n", "\n", - "用户可以在Cell对象上使用`register_backward_hook`函数来注册一个自定义的Hook函数，用来捕获网络反向传播时与Cell对象相关联的梯度。该功能在图模式下或者在使用`@jit`修饰的函数内不起作用。`register_backward_hook`函数接收Hook函数作为入参，并返回一个与Hook函数一一对应的`handle`对象。用户可以通过调用`handle`对象的`remove()`函数来删除与之对应的Hook函数。每一次调用`register_backward_hook`函数，都会返回一个不同的`handle`对象。\n", + "用户可以在Cell对象上使用`register_backward_hook`函数来注册一个自定义的Hook函数，用来捕获网络反向传播时与Cell对象相关联的梯度。`register_backward_hook`函数接收Hook函数作为入参，并返回一个与Hook函数一一对应的`handle`对象。用户可以通过调用`handle`对象的`remove()`函数来删除与之对应的Hook函数。每一次调用`register_backward_hook`函数，都会返回一个不同的`handle`对象。\n", "\n", "与HookBackward算子所使用的自定义Hook函数有所不同，`register_backward_hook`使用的Hook函数的入参中，包含了表示Cell对象信息`cell`、反向传入到Cell对象的梯度、以及Cell对象的反向输出的梯度。\n", "\n", -- Gitee From 94b343daff8ad7c6c32c2554925bf9a16ac705cc Mon Sep 17 00:00:00 2001 From: chaijinwei Date: Thu, 22 May 2025 15:16:32 +0800 Subject: [PATCH 2/2] add raise docs --- tutorials/source_en/compile/statements.md | 2 +- tutorials/source_zh_cn/compile/statements.ipynb | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tutorials/source_en/compile/statements.md b/tutorials/source_en/compile/statements.md index bcf75e2472..9957840710 100644 --- a/tutorials/source_en/compile/statements.md +++ b/tutorials/source_en/compile/statements.md @@ -8,7 +8,7 @@ Support the use of `raise` to trigger an exception. `raise` syntax format: `raise[Exception [, args]]`. The `Exception` in the statement is the type of the exception, and the `args` is the user-supplied argument to the exception, usually a string or other object. The following types of errors are supported: NoExceptionType, UnknownError, ArgumentError, NotSupportError, NotExistsError, DeviceProcessError, AbortedError, IndexError, ValueError, TypeError, KeyError, AttributeError, NameError, AssertionError, BaseException, KeyboardInterrupt, Exception, StopIteration, OverflowError, ZeroDivisionError, EnvironmentError, IOError, OSError, ImportError, MemoryError, UnboundLocalError, RuntimeError, NotImplementedError, IndentationError, RuntimeWarning. -It is worth noting that the ability of the raise statement in the variable scenario in static graph mode is supported through extended syntax, that is, it needs to be supported only when the JIT syntax support level option 'jit_syntax_level' is 'LAX'. +The raise syntax in graph mode does not support variables of type `Dict`. For example: diff --git a/tutorials/source_zh_cn/compile/statements.ipynb b/tutorials/source_zh_cn/compile/statements.ipynb index ba517c5f0a..952f4f8e6a 100644 --- a/tutorials/source_zh_cn/compile/statements.ipynb +++ b/tutorials/source_zh_cn/compile/statements.ipynb @@ -15,7 +15,7 @@ "\n", "支持使用`raise`触发异常，`raise`语法格式：`raise[Exception [, args]]`。语句中的`Exception`是异常的类型，`args`是用户提供的异常参数，通常可以是字符串或者其他对象。目前支持的异常类型有：NoExceptionType、UnknownError、ArgumentError、NotSupportError、NotExistsError、DeviceProcessError、AbortedError、IndexError、ValueError、TypeError、KeyError、AttributeError、NameError、AssertionError、BaseException、KeyboardInterrupt、Exception、StopIteration、OverflowError、ZeroDivisionError、EnvironmentError、IOError、OSError、ImportError、MemoryError、UnboundLocalError、RuntimeError、NotImplementedError、IndentationError、RuntimeWarning。\n", "\n", - "值得注意的是，raise语句在静态图模式下的变量场景的能力是通过扩展语法支持，即需要在JIT语法支持级别选项`jit_syntax_level`为`LAX`时才支持。\n", + "图模式下的raise语法不支持`Dict`类型的变量。\n", "\n", "例如：" ] @@ -1068,4 +1068,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} \ No newline at end of file +} -- Gitee