diff --git a/tutorials/source_en/custom_program/op_custom.rst b/tutorials/source_en/custom_program/op_custom.rst index 7607bdb5a1e91e7c7ff9352cf4681c77efaac138..5421c51a5c10a32c19bb94c11155b1b442a22cc6 100644 --- a/tutorials/source_en/custom_program/op_custom.rst +++ b/tutorials/source_en/custom_program/op_custom.rst @@ -12,7 +12,6 @@ Custom Operators operation/op_custom_prim operation/op_custom_ascendc operation/op_custom_aot - operation/op_custom_julia operation/op_custom_adv operation/op_customopbuilder operation/cpp_api_for_custom_ops diff --git a/tutorials/source_en/custom_program/operation/op_custom_julia.md b/tutorials/source_en/custom_program/operation/op_custom_julia.md deleted file mode 100644 index 652e05fac8f04db51e32a10e6a9b1f972268eb4d..0000000000000000000000000000000000000000 --- a/tutorials/source_en/custom_program/operation/op_custom_julia.md +++ /dev/null @@ -1,151 +0,0 @@ -# Custom Operator with Third Party Frontend - -[![View Source On Gitee](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.7.0rc1/resource/_static/logo_source_en.svg)](https://gitee.com/mindspore/docs/blob/r2.7.0rc1/tutorials/source_en/custom_program/operation/op_custom_julia.md) - -As one of the future development goals of MindSpore, the fusion of AI and scientific computing draws more and more attention from the industry. Based on the flexibility of the representation, MindSpore custom operator also makes exploration on the scientific computing, and introduces the programming frontend for HPC to MindSpore via custom operator. - -## Defining Custom Operator of julia Type - -Julia is a high level general programming language which has high performance and is easy to use. Julia is firstly designed for scientific computing, and also gain the favor of general users due to its high effience. -The custom operator of julia type uses Julia to describe the internal calculation logic of the operator. The framework will automatically call this function during the network runtime. - -Operator output shape and data type inference can be realized by defining Python functions to describe the inference logic of the operator output shape and the data type. - -If the custom operator only supports specific input and output data types, you need to define the operator information. For the creation of operator information, please refer to [Registering the Operator Information](https://www.mindspore.cn/tutorials/en/r2.7.0rc1/custom_program/operation/op_custom_adv.html#registering-the-operator-information). - -## Custom Operator Use Cases of julia Type - -Takes the function of adding two input tensors as an example to introduce how to define a custom operator of julia type. - -Firstly, users need to implement Julia functions via separate files, such as (add.jl): - -```julia -# add.jl -module Add -# inputs: x, y, output: z, output should use .= to inplace assign -function add(x, y, z) - z .= x + y -end -end -``` - -Secondly, refer to the Julia function written above in a custom operator in the network script, taking test_custom_julia.py as an example: - -```python -import numpy as np -import mindspore as ms -import mindspore.ops as ops - -ms.set_device(device_target="CPU") - -if __name__ == "__main__": - # Define custom operators of type julia - op = ops.Custom("./add.jl:Add:add", out_shape=lambda x, _: x, out_dtype=lambda x, _: x, func_type="julia") - x0 = np.array([[0.0, 0.0], [1.0, 1.0]]).astype(np.float32) - x1 = np.array([[2.0, 2.0], [3.0, 3.0]]).astype(np.float32) - output = op(ms.Tensor(x0), ms.Tensor(x1)) - print(output) -``` - -The following points need to be explained in this example: - -- Use Python lambda functions to infer the output shape and data type, and pass them to the `out_shape` and `out_dtype` parameters of the `Custom` primitive. In this example, the lambda function indicates that the output shape and data type are the same as the information of the first input tensor. -- The operator information is not registered, so the operator information of the custom operator will be inferred from the inputs. - -Execute case: - -```bash -python test_custom_julia.py -``` - -The execution result is as follows: - -```text -[[2. 2.] - [4. 4.]] -``` - -Matters need attention: - -1. User should make sure to download the correct version of Julia, that is, version >= 1.6.0. -2. User is required to set `julia/lib` to `LD_LIBRARY_PATH`, because the Julia C API called at runtime is obtained from `libjulia.so`, taking julia-1.6.5 as an example: - - ```bash - # download julia-1.6.5 - wget https://julialang-s3.julialang.org/bin/linux/x64/1.6/julia-1.6.5-linux-x86_64.tar.gz - # for arm server - # wget https://julialang-s3.julialang.org/bin/linux/aarch64/1.6/julia-1.6.5-linux-aarch64.tar.gz - # extract file - tar xvf julia-1.6.5-linux-x86_64.tar.gz - # if $JULIA_DIR not exist - export LD_LIBRARY_PATH=$PWD/julia-1.6.5/lib:$LD_LIBRARY_PATH - # else - export LD_LIBRARY_PATH=$JULIA_DIR/lib:$LD_LIBRARY_PATH - ``` - -3. `Custom` operator's first arg `func` should keep format like `file_name:module_name:func_name`, `file_name` should include path, suggest using absolute path. -4. Julia file should include `module`, `module` include `function`, both ends with `end`. -5. The input and output order of the Julia function needs to be consistent with the input and output order of the operator. -6. The final output of the Julia function, i.e. assignment of kernel output, needs to use `.=`, otherwise the result cannot be written to memory. -7. Julia code supports [Julia](https://docs.julialang.org/en/v1/)'s common syntax, users need to ensure that the syntax is correct and the function can be executed correctly. -8. Users who want to use Julia's third-party software packages in Julia files need to download the corresponding software to ensure that they can call it correctly, which can be called through `import pkg; pkg.add ("somepkg")` to install. -9. `julia array` is `column major` arranged in memory, while `numpy array` is `row major`. If Julia and numpy are compared, non-elemwise calculations need to consider memory arrangement. In the Julia function, the conversion of `numpy array` and `julia array` can be performed by following the following code example:An example of MatMul: - - ```julia - function change_input_to_row_major(x) - return permutedims(reshape(x, reverse(size(x))), length(size(x)):-1:1) - end - - function change_output_to_row_major(x) - return reshape(permutedims(x, length(size(x)):-1:1), size(x)) - end - ``` - - Taking matrix multiplication as an example: - - ```julia - # julia array is column-major, numpy array is row-major - # user should change julia or numpy's layout to keep same behavior - #= EXAMPLE - A[2,3] B[3,4] C[2,4] - NUMPY: - [[1, 2, 3] [[1, 2, 3, 4] [[38, 44, 50, 56] - [4, 5, 6]] [5, 6, 7, 8] [83, 98, 113,128]] - [9,10,11,12]] - JULIA: - change_input_to_row_major: - 1.inputs read numpy data from memory: - [[1, 3, 5] [[1, 4, 7,10] - [2, 4, 6]] [2, 5, 8,11] - [3, 6, 9,12]] - 2.inputs after reshape(reverse(shape)): - [[1, 4] [[1, 5, 9] - [2, 5] [2, 6,10] - [3, 6]] [3, 7,11] - [4, 8,12]] - 3.inputs after transpose/permutedims: - [[1, 2, 3] [[1, 2, 3, 4] [[38, 44, 50, 56] - [4, 5, 6]] [5, 6, 7, 8] [83, 98, 113,128]] - [9,10,11,12]] - change_output_to_row_major: - 1.output after transpose/permutedims: - [[38, 83] - [44, 98] - [50,113] - [56,128] - 2.output after reshape: - [[38, 50, 83, 113] - [44, 56, 98, 128]] - 3.output read numpy data from memory: - [[38, 44, 50, 56] - [83, 98,113, 128]] - =# - function foo!(x, y, z) - x = change_input_to_row_major(x) - y = change_input_to_row_major(y) - z .= gemm(x, y, z) - z .= change_output_to_row_major(z) - end - ``` - -For more complete examples of julia-type custom operators, see the [use cases](https://gitee.com/mindspore/mindspore/blob/v2.7.0-rc1/tests/st/graph_kernel/custom/test_custom_julia.py) in the MindSpore source code. diff --git a/tutorials/source_en/custom_program/operation/op_custom_prim.rst b/tutorials/source_en/custom_program/operation/op_custom_prim.rst index ba7997374d2fc15f95ad5ac198e061226959a4ca..54ae4da4bc7960eba0bff162dfdf45841b44c35c 100644 --- a/tutorials/source_en/custom_program/operation/op_custom_prim.rst +++ b/tutorials/source_en/custom_program/operation/op_custom_prim.rst @@ -23,7 +23,7 @@ Compared with traditional custom operator creating methods, creating custom oper Custom operator classification and adaptation scenarios ----------------------------------------------------------- -The operator development methods supported by custom operator based on the `Custom `_ primitive include: aot, pyfunc, and julia. +The operator development methods supported by custom operator based on the `Custom `_ primitive include: aot, pyfunc. The difference between these operator development methods are as follows: @@ -43,10 +43,6 @@ The difference between these operator development methods are as follows: - Ascend C/CUDA/C++ - `Ascend` `GPU` `CPU` - high-performance scenarios - * - `julia `_ - - Julia - - `CPU` - - Science compute scenarios / use Julia scenarios Different custom operator defining methods use different development languages to implement the operator, but the development process is the same, including operator implementation, shape inference, data type inference, and operator information registration (optional). Developers can choose which one to use based on needs. The defining methods of these custom operators will be introduced here, and examples are provided for each method. When developers are developing custom operators, they can refer to the following methods to select the corresponding type: @@ -55,12 +51,10 @@ Different custom operator defining methods use different development languages t - Quick verification scenario: If users want to do quick verification and development based on MindSpore, with low performance requirements, or want to interact based on Python, then choose custom operators with Pyfunc type; - High-performance scenario: If users want to do high-performance computing based on MindSpore or need to interface with third-party operator libraries, then choose custom operators of AOT type; - - Scientific computing scenario: If users need to use Julia for scientific computing tasks, then choose custom operators of Julia type. To help you better use custom operators, we have used [the pyfunc-type custom operator](#an-example-of-custom-operators) as an example of a custom operator. In addition, we provide tutorials for other custom operators including: - AOT-type custom op on `Ascend backend `_ and `GPU/CPU backend `_ ; -- `Julia-type custom op `_ ; - `Advanced usage of custom operators `_ : registering the operator information and defining the backward functions for operators. .. note:: diff --git a/tutorials/source_zh_cn/custom_program/op_custom.rst b/tutorials/source_zh_cn/custom_program/op_custom.rst index c3166021a669388fd5a183236a75f94705dabf91..11c245387f5712396db966defef2df6082750ce9 100644 --- a/tutorials/source_zh_cn/custom_program/op_custom.rst +++ b/tutorials/source_zh_cn/custom_program/op_custom.rst @@ -12,7 +12,6 @@ operation/op_custom_prim operation/op_custom_ascendc operation/op_custom_aot - operation/op_custom_julia operation/op_custom_adv operation/op_customopbuilder operation/cpp_api_for_custom_ops diff --git a/tutorials/source_zh_cn/custom_program/operation/op_custom_julia.md b/tutorials/source_zh_cn/custom_program/operation/op_custom_julia.md deleted file mode 100644 index f9d18ee3c0a558b3efa527b483023cefb97cdd3a..0000000000000000000000000000000000000000 --- a/tutorials/source_zh_cn/custom_program/operation/op_custom_julia.md +++ /dev/null @@ -1,151 +0,0 @@ -# Custom原语自定义算子接入第三方前端 - -[![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.7.0rc1/resource/_static/logo_source.svg)](https://gitee.com/mindspore/docs/blob/r2.7.0rc1/tutorials/source_zh_cn/custom_program/operation/op_custom_julia.md) - -作为MindSpore未来的发展方向之一，AI和科学计算的融合越来越受到业界的重视。MindSpore自定义算子基于自身表达的灵活性，也在科学计算方面做出了探索：把面向HPC的编程前端以自定义算子的方式接入MindSpore。 - -## julia类型的自定义算子开发概述 - -Julia是一种速度快且使用简单的高级通用编程语言，最初设计用于科学计算领域，而由于其高效而实用的特性，近些年来越来越受到用户的青睐，逐步迈向主流编程语言。 -julia类型的自定义算子使用Julia语法定义算子实现函数，描述算子内部计算逻辑的实现。网络运行时框架会自动调用执行相应的Julia函数。 - -算子输出shape和数据类型推导可以通过定义Python函数实现，描述算子输出shape和数据类型的推导逻辑。 - -若自定义算子只支持特定的输入输出数据类型，则需要定义算子信息，算子信息生成方式请参考[算子信息注册](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/custom_program/operation/op_custom_adv.html#算子信息注册)。 - -## julia类型的自定义算子开发用例 - -下面以两个输入张量相加为例，介绍julia类型的自定义算子开发流程: - -首先，用户需要通过单独文件实现Julia函数，如(add.jl)： - -```julia -# add.jl -module Add -# inputs: x, y, output: z, output should use .= to inplace assign -function add(x, y, z) - z .= x + y -end -end -``` - -其次，在网络脚本中通过自定义算子方式引用上面所写的Julia函数，以test_custom_julia.py为例： - -```python -import numpy as np -import mindspore as ms -import mindspore.ops as ops - -ms.set_device(device_target="CPU") - -if __name__ == "__main__": - # 定义julia类型的自定义算子 - op = ops.Custom("./add.jl:Add:add", out_shape=lambda x, _: x, out_dtype=lambda x, _: x, func_type="julia") - x0 = np.array([[0.0, 0.0], [1.0, 1.0]]).astype(np.float32) - x1 = np.array([[2.0, 2.0], [3.0, 3.0]]).astype(np.float32) - output = op(ms.Tensor(x0), ms.Tensor(x1)) - print(output) -``` - -本例中，有如下几点需要说明： - -- 用Python lambda函数定义输出shape和数据类型推理函数，并分别传给`Custom`原语的`out_shape`和`out_dtype`参数。本例中lambda函数表明输出shape和数据类型与第一个输入张量的信息相同。 -- 未注册算子信息，所以自定义算子的算子信息将会从算子输入中推理。 - -执行用例： - -```shell -python test_custom_julia.py -``` - -执行结果： - -```text -[[2. 2.] - [4. 4.]] -``` - -注意事项： - -1. 用户需确保下载正确版本的Julia，即version>=1.6.0。 -2. 由于运行时调用的Julia C API是从`libjulia.so`中获取的，因此需要用户设置`julia/lib`到`LD_LIBRARY_PATH`，以julia-1.6.5为例: - - ```bash - # download julia-1.6.5 - wget https://julialang-s3.julialang.org/bin/linux/x64/1.6/julia-1.6.5-linux-x86_64.tar.gz - # for arm server - # wget https://julialang-s3.julialang.org/bin/linux/aarch64/1.6/julia-1.6.5-linux-aarch64.tar.gz - # extract file - tar xvf julia-1.6.5-linux-x86_64.tar.gz - # if $JULIA_DIR not exist - export LD_LIBRARY_PATH=$PWD/julia-1.6.5/lib:$LD_LIBRARY_PATH - # else - export LD_LIBRARY_PATH=$JULIA_DIR/lib:$LD_LIBRARY_PATH - ``` - -3. `Custom` 第一个入参指定用户书写的Julia函数需按照`file_name:module_name:func_name`格式指定，`file_name`需包含文件路径，建议使用绝对路径。 -4. Julia代码文件需包含`module`，`module`内包含`function`，且`module`或`function`都以`end`结束。 -5. Julia函数的输入输出顺序需与算子的输入输出顺序一致。 -6. Julia函数的最终输出，即kernel output的赋值需要使用`.=`，否则结果无法写入内存。 -7. Julia代码支持[Julia](https://docs.julialang.org/en/v1/)的常用语法，用户需自行保证语法正确，函数可正确执行。 -8. 用户想在Julia文件内使用Julia的第三方软件包，需自行下载对应软件以确保能正常调用，可以通过 `import pkg; pkg.add("somepkg")`进行安装。 -9. `julia array`在内存上是按照`column major`排列的，而`numpy array`是按照`row major`排列的。如果Julia和numpy做比较，非elemwise计算需考虑内存排布。在Julia函数中，可以通过如下代码示例进行`numpy array`和`julia array`的相互转换： - - ```julia - function change_input_to_row_major(x) - return permutedims(reshape(x, reverse(size(x))), length(size(x)):-1:1) - end - - function change_output_to_row_major(x) - return reshape(permutedims(x, length(size(x)):-1:1), size(x)) - end - ``` - - 以矩阵乘为例： - - ```julia - # julia array is column-major, numpy array is row-major - # user should change julia or numpy's layout to keep same behavior - #= EXAMPLE - A[2,3] B[3,4] C[2,4] - NUMPY: - [[1, 2, 3] [[1, 2, 3, 4] [[38, 44, 50, 56] - [4, 5, 6]] [5, 6, 7, 8] [83, 98, 113,128]] - [9,10,11,12]] - JULIA: - change_input_to_row_major: - 1.inputs read numpy data from memory: - [[1, 3, 5] [[1, 4, 7,10] - [2, 4, 6]] [2, 5, 8,11] - [3, 6, 9,12]] - 2.inputs after reshape(reverse(shape)): - [[1, 4] [[1, 5, 9] - [2, 5] [2, 6,10] - [3, 6]] [3, 7,11] - [4, 8,12]] - 3.inputs after transpose/permutedims: - [[1, 2, 3] [[1, 2, 3, 4] [[38, 44, 50, 56] - [4, 5, 6]] [5, 6, 7, 8] [83, 98, 113,128]] - [9,10,11,12]] - change_output_to_row_major: - 1.output after transpose/permutedims: - [[38, 83] - [44, 98] - [50,113] - [56,128] - 2.output after reshape: - [[38, 50, 83, 113] - [44, 56, 98, 128]] - 3.output read numpy data from memory: - [[38, 44, 50, 56] - [83, 98,113, 128]] - =# - function foo!(x, y, z) - x = change_input_to_row_major(x) - y = change_input_to_row_major(y) - z .= gemm(x, y, z) - z .= change_output_to_row_major(z) - end - ``` - -更多完整的jullia类型自定义算子的例子可以参见MindSpore源码中的[用例](https://gitee.com/mindspore/mindspore/blob/v2.7.0-rc1/tests/st/graph_kernel/custom/test_custom_julia.py)。 diff --git a/tutorials/source_zh_cn/custom_program/operation/op_custom_prim.ipynb b/tutorials/source_zh_cn/custom_program/operation/op_custom_prim.ipynb index 81b3d2a434d9b9567600a7dc735120d3ec1c185c..a7e1f0f1353f8c5b92970a96afabcba09dc73c7b 100644 --- a/tutorials/source_zh_cn/custom_program/operation/op_custom_prim.ipynb +++ b/tutorials/source_zh_cn/custom_program/operation/op_custom_prim.ipynb @@ -26,13 +26,12 @@ "\n", "## 自定义算子分类及适应场景\n", "\n", - "基于[Custom](https://www.mindspore.cn/docs/zh-CN/r2.7.0rc1/api_python/ops/mindspore.ops.Custom.html#mindspore-ops-custom)原语的自定义算子支持的算子开发方式包括：pyfunc、aot和julia。不同的算子开发方式适应的场景如下：\n", + "基于[Custom](https://www.mindspore.cn/docs/zh-CN/r2.7.0rc1/api_python/ops/mindspore.ops.Custom.html#mindspore-ops-custom)原语的自定义算子支持的算子开发方式包括：pyfunc、aot。不同的算子开发方式适应的场景如下：\n", "\n", "| 算子开发方式 | 开发语言 | 支持平台 | 推荐场景 |\n", "|:-------|:------------------ |:------ |:------------------------|\n", "| [pyfunc](#自定义算子用例) | Python | `CPU` | 快速算法验证的场景 |\n", "| [aot](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/custom_program/operation/op_custom_aot.html) | Ascend C/CUDA/C++ | `Ascend` `GPU` `CPU` | 需要高性能算子的场景 |\n", - "| [julia](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/custom_program/operation/op_custom_julia.html) | Julia | `CPU` | 科学计算场景 |\n", "\n", "不同的开发方式使用不同的开发语言实现算子计算逻辑，但是自定义算子的开发流程是一致的，包括算子实现、shape推导、数据类型推理和算子信息注册（可选）。网络开发者可以根据需要选用不同的自定义算子开发方式。在开发者进行自定义算子开发的时候，可以参考如下方式选择对应类型：\n", "\n", @@ -40,12 +39,10 @@ "2. 判断场景：在使用CPU后端的时候，不同的场景对应不同类型的自定义算子推荐：\n", " - 快速验证场景：如果用户希望基于MindSpore做快速验证和开发，对于性能要求不高，或者希望基于Python进行交互，那么选取pyfunc类型的自定义算子；\n", " - 高性能场景：如果用户希望基于MindSpore做高性能计算，或者需要对接第三方算子库，那么选取aot类型自定义算子；\n", - " - 科学计算场景：如果用户在做科学计算任务时需要使用Julia，那么选取julia类型自定义算子。\n", "\n", "为了帮助大家更好地使用自定义算子，我们以[pyfunc类型自定义算子](#自定义算子用例)中作为自定义算子的范例展示。此外，我们提供了其他自定义算子的教程包括：\n", "\n", "- aot类型自定义算子：[Ascend平台](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/custom_program/operation/op_custom_ascendc.html)和[GPU/CPU平台](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/custom_program/operation/op_custom_aot.html)；\n", - "- [julia类型自定义算子](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/custom_program/operation/op_custom_julia.html)；\n", "- [自定义算子进阶用法](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/custom_program/operation/op_custom_adv.html)：算子注册和反向算子。\n", "\n", "> 更多示例可参考MindSpore源码中[tests/st/graph_kernel/custom](https://gitee.com/mindspore/mindspore/tree/v2.7.0-rc1/tests/st/graph_kernel/custom)下的用例。\n",