diff --git a/docs/mindspore/source_en/model_train/program_form/static_graph.rst b/docs/mindspore/source_en/model_train/program_form/static_graph.rst index 318c06327ae5e7829cccb7b8bdca4af5a9800b61..3eb5bfe5576ea4a77c14ad2acc44ed6f41af40c2 100644 --- a/docs/mindspore/source_en/model_train/program_form/static_graph.rst +++ b/docs/mindspore/source_en/model_train/program_form/static_graph.rst @@ -1360,6 +1360,90 @@ The results are as follows: ret:(Tensor(shape=[1], dtype=Int64, value= [1]), Tensor(shape=[1], dtype=Int64, value= [1])) + +Inplace and View Functions +In MindSpore's graph mode, memory management and computational graph optimization are core design objectives. Inplace and View are two important memory optimization techniques. Below, we introduce their behaviors and practices in graph mode. + +Inplace Operations +Inplace (in-place operations) is a technique that optimizes memory usage by directly modifying the memory data of the input tensor instead of creating a new tensor. MindSpore supports Inplace operations through certain operators, which can effectively reduce memory overhead and improve computational efficiency in both training and inference scenarios. This feature is suitable for memory-sensitive large-scale models or environments with limited hardware resources. It supports operations on both ``Parameter`` and ``Tensor``. + +Usage +1. Explicit Inplace API + +.. code:: python + + import mindspore as ms + from mindspore import nn, ops, set_context + + set_context(mode=ms.GRAPH_MODE) + + class Net(nn.Cell): + def __init__(self): + super(Net, self).__init__() + self.assign = ops.Assign() + + def construct(self, x, y): + self.assign(x, y) + return x + + net = Net() + input_x = ms.Tensor([1.0], ms.float64) + input_y = ms.Tensor([2.0], ms.float64) + net(input_x, input_y) + +2. Tensor Slice Assignment + +.. code:: python + + import mindspore as ms + from mindspore import nn, ops, set_context + + set_context(mode=ms.GRAPH_MODE) + + class Net(nn.Cell): + def construct(self, x): + x[0:2] = 0 + return x + + net = Net() + input_x = ms.Tensor([1.0, 2.0, 3.0], ms.float64) + net(input_x) + +View Operations +View operations (such as Transpose) generate a new Tensor that shares memory with the original data, without copying the data. The lifecycle of the view Tensor is controlled by the original Tensor. If the original data is released, the view will become invalid. + +Usage +.. code:: python + + import mindspore as ms + from mindspore import nn, ops, set_context + + set_context(mode=ms.GRAPH_MODE) + + class ViewOut(nn.Cell): + def __init__(self): + super(ViewOut, self).__init__() + self.transpose = ops.Transpose() + self.assign = ops.Assign() + + def construct(self, x): + self.transpose(x, (0, 1, 2)) + self.assign(x, x * 2) + return x * 3 + + x1 = ms.Tensor(np.array([[[1, 0, 0, 0], [0, 0, 0, 0], [-1, -1, 0, -1]], + [[0, -1, 0, 0], [0, 0, 0, 0], [0, 1, 0, 0]]), ms.int32) + net = ViewOut() + + out_graph = net(x1) + +Notes +1. In dynamic graph mode, using Inplace operations may lead to errors in backpropagation, but they are supported in static graph mode. + +2. Currently, augmented assignment statements in static graph mode still use copy implementation and do not have Inplace capabilities. Future versions will support this. + +3. Both Inplace and View operations can reduce memory usage and operate on Tensors. + Syntax Constraints of Basic Syntaxes ------------------------------------ diff --git a/docs/mindspore/source_zh_cn/model_train/program_form/static_graph.rst b/docs/mindspore/source_zh_cn/model_train/program_form/static_graph.rst index 6a6f6d3b70628b19a6f15e6e81e20e243b14ee0d..4f80a6d7b79489a7d60b0744c8378ade58cbdbe4 100644 --- a/docs/mindspore/source_zh_cn/model_train/program_form/static_graph.rst +++ b/docs/mindspore/source_zh_cn/model_train/program_form/static_graph.rst @@ -1154,6 +1154,98 @@ Python内置函数 ret:(Tensor(shape=[1], dtype=Int64, value= [1]), Tensor(shape=[1], dtype=Int64, value= [1])) +Inplace和View功能 + +在MindSpore的图模式下，内存管理和计算图优化是核心设计目标。Inplace和View是两种重要的内存优化技术，下面介绍二者在图模式下的行为和实践。 + +Inplace操作 + +1、Inplace（原地操作）是一种通过直接修改输入张量的内存数据而非创建新张量来优化内存使用的技术。MindSpore通过部分算子支持Inplace操作，在训练和推理场景中可有效减少内存开销，提升计算效率。该功能适用于对内存敏感的大规模模型或硬件资源受限的环境。支持对Parameter和Tensor进行操作。 + +使用方法 + +1、显示Inplace API + +.. code:: python + + import mindspore as ms + from mindspore import nn, ops, set_context + + set_context(mode=ms.GRAPH_MODE) + + class Net(nn.Cell): + def __init__(self): + super(Net, self).__init__() + self.assign = ops.Assign() + + def construct(self, x, y): + self.assign(x, y) + return x + + net = Net() + input_x = ms.Tensor([1.0], ms.float64) + input_y = ms.Tensor([2.0], ms.float64) + net(input_x, input_y) + +2、张量切片赋值 + +.. code:: python + + import mindspore as ms + from mindspore import nn, ops, set_context + + set_context(mode=ms.GRAPH_MODE) + + class Net(nn.Cell): + def construct(self, x): + x[0:2] = 0 + return x + + net = Net() + input_x = ms.Tensor([1.0, 2.0, 3.0], ms.float64) + net(input_x) + + +View操作 + +View操作（如Transpose）生成的新Tensor与原始数据共享内存，无数据复制。视图Tensor的生命周期受原始Tensor控制，若原始数据被释放，视图将失效。 + +使用方法 + +.. code:: python + + import mindspore as ms + from mindspore import nn, ops, set_context + + set_context(mode=ms.GRAPH_MODE) + + class ViewOut(nn.Cell): + def __init__(self): + super(ViewOut, self).__init__() + self.transpose = P.TransposeView() + self.assign = P.Assign() + + def construct(self, x): + self.transpose(x, (0, 1, 2)) + self.assign(x, x * 2) + return x * 3 + + x1 = ms.Tensor(np.array([[[1, 0, 0, 0], [0, 0, 0, 0], [-1, -1, 0, -1]], + [[0, -1, 0, 0], [0, 0, 0, 0], [0, 1, 0, 0]]]), ms.int32) + net = ViewOut() + + out_graph = net(x1) + +注意事项 + +1、在动态图模式下，使用Inplace操作可能导致反向传播错误，在静态图模式下可以支持。 + +2、当前静态图模式下的增强赋值语句仍然使用拷贝实现，不具备Inplace能力，后续的版本将支持。 + +3、Inplace和View操作均可减少内存，对Tensor进行操作。 + + + 基础语法的语法约束 ------------------ diff --git a/docs/mindspore/source_zh_cn/model_train/program_form/static_graph_syntax/static_graph_syntax_support.ipynb b/docs/mindspore/source_zh_cn/model_train/program_form/static_graph_syntax/static_graph_syntax_support.ipynb index 863c988a11182aa04e34aeafdd82905c935cb2e0..4ed0fef0a3dd2ffd1ed95a099e3c227bd2747b5e 100644 --- a/docs/mindspore/source_zh_cn/model_train/program_form/static_graph_syntax/static_graph_syntax_support.ipynb +++ b/docs/mindspore/source_zh_cn/model_train/program_form/static_graph_syntax/static_graph_syntax_support.ipynb @@ -1480,6 +1480,100 @@ "print('ret:{}'.format(ret))" ] }, + { + "cell_type": "markdown", + "id": "8eefb7b3", + "metadata": {}, + "source": [ + "### Inplace和View功能\n", + "\n", + "在MindSpore的图模式下，内存管理和计算图优化是核心设计目标。Inplace和View是两种重要的内存优化技术，下面介绍二者在图模式下的行为和实践。\n", + "\n", + "### Inplace操作\n", + "\n", + "1、Inplace（原地操作）是一种通过直接修改输入张量的内存数据而非创建新张量来优化内存使用的技术。MindSpore通过部分算子支持Inplace操作，在训练和推理场景中可有效减少内存开销，提升计算效率。该功能适用于对内存敏感的大规模模型或硬件资源受限的环境。支持对Parameter和Tensor进行操作。\n", + "\n", + "#### 使用方法\n", + "\n", + "1、显示Inplace API\n", + "\n", + "```python\n", + " import mindspore as ms\n", + " from mindspore import nn, ops, set_context\n", + "\n", + " set_context(mode=ms.GRAPH_MODE)\n", + "\n", + " class Net(nn.Cell):\n", + " def __init__(self):\n", + " super(Net, self).__init__()\n", + " self.assign = ops.Assign()\n", + "\n", + " def construct(self, x, y):\n", + " self.assign(x, y)\n", + " return x\n", + "\n", + " net = Net()\n", + " input_x = ms.Tensor([1.0], ms.float64)\n", + " input_y = ms.Tensor([2.0], ms.float64)\n", + " net(input_x, input_y)\n", + " ```\n", + "\n", + "2、张量切片赋值\n", + "\n", + "```python\n", + " import mindspore as ms\n", + " from mindspore import nn, ops, set_context\n", + "\n", + " set_context(mode=ms.GRAPH_MODE)\n", + "\n", + " class Net(nn.Cell):\n", + " def construct(self, x):\n", + " x[0:2] = 0\n", + " return x\n", + "\n", + " net = Net()\n", + " input_x = ms.Tensor([1.0, 2.0, 3.0], ms.float64)\n", + " net(input_x)\n", + " ```\n", + "\n", + "### View操作\n", + "\n", + "View操作（如Transpose）生成的新Tensor与原始数据共享内存，无数据复制。视图Tensor的生命周期受原始Tensor控制，若原始数据被释放，视图将失效。\n", + "\n", + "#### 使用方法\n", + "\n", + "```python\n", + " import mindspore as ms\n", + " from mindspore import nn, ops, set_context\n", + "\n", + " set_context(mode=ms.GRAPH_MODE)\n", + "\n", + " class ViewOut(nn.Cell):\n", + " def __init__(self):\n", + " super(ViewOut, self).__init__()\n", + " self.transpose = P.TransposeView()\n", + " self.assign = P.Assign()\n", + "\n", + " def construct(self, x):\n", + " self.transpose(x, (0, 1, 2))\n", + " self.assign(x, x * 2)\n", + " return x * 3\n", + "\n", + " x1 = ms.Tensor(np.array([[[1, 0, 0, 0], [0, 0, 0, 0], [-1, -1, 0, -1]],\n", + " [[0, -1, 0, 0], [0, 0, 0, 0], [0, 1, 0, 0]]]), ms.int32)\n", + " net = ViewOut()\n", + " out_graph = net(x1)\n", + " ```\n", + "\n", + "### 注意事项\n", + "\n", + "1、在动态图模式下，使用Inplace操作可能导致反向传播错误，在静态图模式下可以支持。\n", + "\n", + "2、当前静态图模式下的增强赋值语句仍然使用拷贝实现，不具备Inplace能力，后续的版本将支持。\n", + "\n", + "3、Inplace和View操作均可减少内存，对Tensor进行操作。" + ] + }, { "cell_type": "markdown", "id": "9b823a13",