diff --git a/docs/mindspore/programming_guide/source_zh_cn/debug_in_pynative_mode.ipynb b/docs/mindspore/programming_guide/source_zh_cn/debug_in_pynative_mode.ipynb index 3316b3d75d9518dc9aaca0a1d632026987305bb4..2dc4605b2589ac4f33c00571bc82a3be2e3eb940 100644 --- a/docs/mindspore/programming_guide/source_zh_cn/debug_in_pynative_mode.ipynb +++ b/docs/mindspore/programming_guide/source_zh_cn/debug_in_pynative_mode.ipynb @@ -2,7 +2,6 @@ "cells": [ { "cell_type": "markdown", - "id": "d8f28388", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:06:23.745016Z", @@ -37,7 +36,6 @@ { "cell_type": "code", "execution_count": 1, - "id": "1f073178", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:15:13.240496Z", @@ -53,7 +51,6 @@ }, { "cell_type": "markdown", - "id": "7c2d61c8", "metadata": {}, "source": [ "## 执行单算子" @@ -62,7 +59,6 @@ { "cell_type": "code", "execution_count": 2, - "id": "02da4fc9", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:08:29.337515Z", @@ -109,7 +105,6 @@ }, { "cell_type": "markdown", - "id": "cb199d5a", "metadata": {}, "source": [ "## 执行函数" @@ -118,7 +113,6 @@ { "cell_type": "code", "execution_count": 3, - "id": "ae94d0c8", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:08:53.065585Z", @@ -156,7 +150,6 @@ }, { "cell_type": "markdown", - "id": "4750f0de", "metadata": {}, "source": [ "## 执行网络\n", @@ -167,7 +160,6 @@ { "cell_type": "code", "execution_count": 4, - "id": "b27a3b82", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:09:16.498705Z", @@ -208,7 +200,6 @@ }, { "cell_type": "markdown", - "id": "e8fd0a3f", "metadata": {}, "source": [ "## 构建网络\n", @@ -219,7 +210,6 @@ { "cell_type": "code", "execution_count": 5, - "id": "587ec18c", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:09:54.055532Z", @@ -264,7 +254,6 @@ }, { "cell_type": "markdown", - "id": "63cd2da8", "metadata": {}, "source": [ "## 设置Loss函数及优化器\n", @@ -279,7 +268,6 @@ }, { "cell_type": "markdown", - "id": "12463c2b", "metadata": {}, "source": [ "## 保存模型参数\n", @@ -297,7 +285,6 @@ }, { "cell_type": "markdown", - "id": "efe5d247", "metadata": {}, "source": [ "## 训练网络\n", @@ -326,7 +313,6 @@ { "cell_type": "code", "execution_count": 6, - "id": "642d8765", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:11:12.072192Z", @@ -376,7 +362,6 @@ }, { "cell_type": "markdown", - "id": "d3acddf1", "metadata": {}, "source": [ "上述示例代码中,在`TensorAddNet`类的`construct`之前加装了`ms_function`装饰器,该装饰器会将`construct`方法编译成计算图,在给定输入之后,以图的形式下发执行,而上一示例代码中的`add`会直接以普通的PyNative的方式执行。\n", @@ -389,7 +374,6 @@ { "cell_type": "code", "execution_count": 7, - "id": "0d5f8edc", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:11:37.208346Z", @@ -432,7 +416,6 @@ }, { "cell_type": "markdown", - "id": "51aba8b2", "metadata": {}, "source": [ "如果被装饰的函数中包含了需要进行参数训练的算子(如`Convolution`、`BatchNorm`等算子),则这些算子必须在被装饰的函数之外完成实例化操作,如下例所示。\n", @@ -443,7 +426,6 @@ { "cell_type": "code", "execution_count": 8, - "id": "b768cd62", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:12:01.721275Z", @@ -513,7 +495,6 @@ }, { "cell_type": "markdown", - "id": "2c22412b", "metadata": {}, "source": [ "更多ms_function的功能可以参考[ms_function文档](https://mindspore.cn/docs/programming_guide/zh-CN/master/ms_function.html)。\n", @@ -528,7 +509,6 @@ { "cell_type": "code", "execution_count": 9, - "id": "733a67bc", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:12:28.228442Z", @@ -542,7 +522,6 @@ }, { "cell_type": "markdown", - "id": "198071c8", "metadata": {}, "source": [ "示例代码:\n", @@ -577,7 +556,6 @@ }, { "cell_type": "markdown", - "id": "bc31de88", "metadata": {}, "source": [ "输出:此时算子为同步执行,当算子执行错误时,可以看到完整的调用栈,找到出错的代码行。\n", @@ -604,7 +582,6 @@ }, { "cell_type": "markdown", - "id": "76c1c4d4", "metadata": {}, "source": [ "## Hook功能\n", @@ -621,7 +598,6 @@ { "cell_type": "code", "execution_count": 10, - "id": "7c478a5b", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:13:38.660885Z", @@ -667,7 +643,6 @@ }, { "cell_type": "markdown", - "id": "7c591014", "metadata": {}, "source": [ "更多HookBackward算子的说明可以参考[API文档](https://mindspore.cn/docs/api/zh-CN/master/api_python/ops/mindspore.ops.HookBackward.html)。\n", @@ -691,7 +666,6 @@ { "cell_type": "code", "execution_count": 11, - "id": "be24bd1e", "metadata": {}, "outputs": [ { @@ -745,7 +719,6 @@ }, { "cell_type": "markdown", - "id": "7c591014", "metadata": {}, "source": [ "用户如果在Hook函数中直接返回新创建的数据,而不是返回由原始输入数据经过计算后得到的数据,那么梯度的反向传播将会在该Cell对象上截止。\n", @@ -756,7 +729,6 @@ { "cell_type": "code", "execution_count": 12, - "id": "ef703b78", "metadata": {}, "outputs": [ { @@ -801,7 +773,6 @@ }, { "cell_type": "markdown", - "id": "7527515d", "metadata": {}, "source": [ "为了避免脚本在切换到图模式时运行失败,不建议在Cell对象的 `construct` 函数中调用 `register_forward_pre_hook` 函数和 `handle` 对象的 `remove()` 函数。在PyNative模式下,如果在Cell对象的 `construct` 函数中调用 `register_forward_pre_hook` 函数,那么Cell对象每次运行都将新注册一个Hook函数。\n", @@ -828,7 +799,6 @@ { "cell_type": "code", "execution_count": 13, - "id": "bf9a9441", "metadata": {}, "outputs": [ { @@ -880,7 +850,6 @@ }, { "cell_type": "markdown", - "id": "6c43f47d", "metadata": {}, "source": [ "用户如果在Hook函数中直接返回新创建的数据,而不是将原始的输出数据经过计算后,将得到的新输出数据返回,那么梯度的反向传播将会在该Cell对象上截止。该现象可以参考 `register_forward_pre_hook` 函数的用例说明。\n", @@ -891,7 +860,6 @@ }, { "cell_type": "markdown", - "id": "785b3be4", "metadata": {}, "source": [ "### Cell对象的register_backward_hook功能\n", @@ -906,7 +874,6 @@ { "cell_type": "code", "execution_count": 14, - "id": "9826935c", "metadata": {}, "outputs": [], "source": [ @@ -917,7 +884,6 @@ }, { "cell_type": "markdown", - "id": "acb32235", "metadata": {}, "source": [ "这里的 `cell_id` 是Cell对象的名称以及ID信息, `grad_input` 是网络反向传播时,传入到Cell对象的梯度,它对应于正向过程中下一个算子的反向输出梯度; `grad_output` 是Cell对象反向输出的梯度。因此,用户可以使用 `register_backward_hook` 函数来捕获网络中某一个Cell对象的反向传入和反向输出梯度。用户可以在Hook函数中自定义对梯度的操作,比如查看、打印梯度,或者返回新的输出梯度。使用方式上与 `register_forward_pre_hook` 和 `register_forward_hook` 函数类似。\n", @@ -928,7 +894,6 @@ { "cell_type": "code", "execution_count": 15, - "id": "6aed31a5", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:14:26.523389Z", @@ -994,7 +959,6 @@ }, { "cell_type": "markdown", - "id": "acb32235", "metadata": {}, "source": [ "当 `register_backward_hook` 函数和 `register_forward_pre_hook` 函数、 `register_forward_hook` 函数同时作用于同一Cell对象时,如果 `register_forward_pre_hook` 和 `register_forward_hook` 函数中有添加其他算子进行数据处理,这些新增算子会在Cell对象执行前或者执行后参与数据的正向计算,但是这些新增算子的反向梯度不在 `register_backward_hook` 函数的捕获范围内。 `register_backward_hook` 中注册的Hook函数仅捕获原始Cell对象的输入、输出梯度。\n", @@ -1005,7 +969,6 @@ { "cell_type": "code", "execution_count": 16, - "id": "bf9a9441", "metadata": {}, "outputs": [ { @@ -1068,7 +1031,6 @@ }, { "cell_type": "markdown", - "id": "5a0eb08c", "metadata": {}, "source": [ "这里的 `grad_input` 是梯度反向传播时传入`net.mul`的梯度,而不是传入 `forward_hook_fn` 函数中,新增的 `Add` 算子的梯度。这里的 `grad_output` 是梯度反向传播时 `net.mul` 反向输出的梯度,而不是 `forward_pre_hook_fn` 函数中新增 `Add` 算子的反向输出梯度。 `register_forward_pre_hook` 函数和 `register_forward_hook` 函数是在Cell对象执行前后起作用,不会影响Cell对象上反向Hook函数的梯度捕获范围。\n", @@ -1086,7 +1048,6 @@ { "cell_type": "code", "execution_count": 17, - "id": "342c346e", "metadata": { "ExecuteTime": { "end_time": "2022-03-02T09:14:55.896896Z",