From 871bf16d8b75289ebbe997d737202b966f000f37 Mon Sep 17 00:00:00 2001 From: CaoJian Date: Fri, 16 Oct 2020 16:06:16 +0800 Subject: [PATCH 1/5] update LSTM support info --- docs/note/source_en/operator_list_ms.md | 3 ++- docs/note/source_zh_cn/operator_list_ms.md | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/note/source_en/operator_list_ms.md b/docs/note/source_en/operator_list_ms.md index 40e7bcb8e4..fee523856d 100644 --- a/docs/note/source_en/operator_list_ms.md +++ b/docs/note/source_en/operator_list_ms.md @@ -51,7 +51,8 @@ | [mindspore.nn.SSIM](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.SSIM) | Supported | Supported | Doing |layer/image | [mindspore.nn.PSNR](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.PSNR) | Supported |Supported | Doing |layer/image | [mindspore.nn.CentralCrop](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.CentralCrop) | Supported |Supported | Doing |layer/image -| [mindspore.nn.LSTM](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.LSTM) | Doing | Supported | Supported |layer/lstm +| [mindspore.nn.LSTM](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.LSTM) | Doing | Supported | Doing |layer/lstm +| [mindspore.nn.LSTMCell](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.LSTMCell) | Doing | Supported | Supported |layer/lstm | [mindspore.nn.GlobalBatchNorm](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.GlobalBatchNorm) | Supported |Doing | Doing |layer/normalization | [mindspore.nn.BatchNorm1d](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm1d) | Supported |Doing | Doing |layer/normalization | [mindspore.nn.BatchNorm2d](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d) | Supported | Supported | Doing |layer/normalization diff --git a/docs/note/source_zh_cn/operator_list_ms.md b/docs/note/source_zh_cn/operator_list_ms.md index f18271b91f..3fae2a6691 100644 --- a/docs/note/source_zh_cn/operator_list_ms.md +++ b/docs/note/source_zh_cn/operator_list_ms.md @@ -46,7 +46,8 @@ | [mindspore.nn.SSIM](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.SSIM) | Supported | Supported | Doing |layer/image | [mindspore.nn.PSNR](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.PSNR) | Supported |Supported | Doing |layer/image | [mindspore.nn.CentralCrop](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.CentralCrop) | Supported |Supported | Doing |layer/image -| [mindspore.nn.LSTM](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.LSTM) | Doing | Supported | Supported |layer/lstm +| [mindspore.nn.LSTM](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.LSTM) | Doing | Supported | Doing |layer/lstm +| [mindspore.nn.LSTMCell](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.LSTMCell) | Doing | Supported | Supported |layer/lstm | [mindspore.nn.GlobalBatchNorm](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.GlobalBatchNorm) | Supported |Doing | Doing |layer/normalization | [mindspore.nn.BatchNorm1d](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm1d) | Supported |Doing | Doing |layer/normalization | [mindspore.nn.BatchNorm2d](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d) | Supported | Supported | Doing |layer/normalization -- Gitee From d237a6422c7364d658fb0e66dfeb2034308e81a1 Mon Sep 17 00:00:00 2001 From: yingchen Date: Fri, 16 Oct 2020 17:16:30 +0800 Subject: [PATCH 2/5] update docs error part2 --- .../source_en/errorcode_and_metatype.md | 2 +- docs/api_cpp/source_en/tensor.md | 30 ++++---- docs/api_cpp/source_zh_cn/dataset.md | 68 +++++++++-------- docs/api_cpp/source_zh_cn/lite.md | 74 ++++++++++--------- docs/api_cpp/source_zh_cn/session.md | 59 +++++++++------ docs/api_cpp/source_zh_cn/tensor.md | 35 +++++---- docs/api_java/source_zh_cn/lite_session.md | 10 +-- docs/api_java/source_zh_cn/model.md | 7 +- docs/api_java/source_zh_cn/msconfig.md | 14 ++-- docs/api_java/source_zh_cn/mstensor.md | 5 +- 10 files changed, 163 insertions(+), 141 deletions(-) diff --git a/docs/api_cpp/source_en/errorcode_and_metatype.md b/docs/api_cpp/source_en/errorcode_and_metatype.md index 2cebba704d..6d725cb19d 100644 --- a/docs/api_cpp/source_en/errorcode_and_metatype.md +++ b/docs/api_cpp/source_en/errorcode_and_metatype.md @@ -28,6 +28,7 @@ Description of error code and meta type supported in MindSpore Lite. | RET_INPUT_PARAM_INVALID | -600 | Invalid input param by user. | ## MetaType + An **enum** type. | Type Definition | Value | Description | @@ -49,4 +50,3 @@ An **enum** type. |kNumberTypeFloat32| 43 | Indicating a data type of float32. | |kNumberTypeFloat64| 44 | Indicating a data type of float64.| |kNumberTypeEnd| 45 | The end of number type. | - diff --git a/docs/api_cpp/source_en/tensor.md b/docs/api_cpp/source_en/tensor.md index 14171fd1d8..4d10116efd 100644 --- a/docs/api_cpp/source_en/tensor.md +++ b/docs/api_cpp/source_en/tensor.md @@ -1,33 +1,35 @@ # mindspore::tensor -#include <[ms_tensor.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/ms_tensor.h)> - +\#include <[ms_tensor.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/ms_tensor.h)> ## MSTensor MSTensor defined tensor in MindSpore Lite. -**Constructors & Destructors** -``` +### Constructors & Destructors + +```cpp MSTensor() ``` + Constructor of MindSpore Lite MSTensor. - Returns Instance of MindSpore Lite MSTensor. - -``` + +```cpp virtual ~MSTensor() ``` Destructor of MindSpore Lite Model. -**Public Member Functions** +### Public Member Functions -``` +```cpp virtual TypeId data_type() const ``` + Get data type of the MindSpore Lite MSTensor. > TypeId is defined in [mindspore/mindspore/core/ir/dtype/type_id.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/core/ir/dtype/type_id.h). Only number types in TypeId enum are suitable for MSTensor. @@ -36,7 +38,7 @@ Get data type of the MindSpore Lite MSTensor. MindSpore Lite TypeId of the MindSpore Lite MSTensor. -``` +```cpp virtual std::vector shape() const ``` @@ -46,7 +48,7 @@ Get shape of the MindSpore Lite MSTensor. A vector of int as the shape of the MindSpore Lite MSTensor. -``` +```cpp virtual int DimensionSize(size_t index) const ``` @@ -60,7 +62,7 @@ Get size of the dimension of the MindSpore Lite MSTensor index by the parameter Size of dimension of the MindSpore Lite MSTensor. -``` +```cpp virtual int ElementsNum() const ``` @@ -70,7 +72,7 @@ Get number of element in MSTensor. Number of element in MSTensor. -``` +```cpp virtual size_t Size() const ``` @@ -79,15 +81,13 @@ Get byte size of data in MSTensor. - Returns Byte size of data in MSTensor. - -``` +```cpp virtual void *MutableData() const ``` Get the pointer of data in MSTensor. - > The data pointer can be used to both write and read data in MSTensor. - Returns diff --git a/docs/api_cpp/source_zh_cn/dataset.md b/docs/api_cpp/source_zh_cn/dataset.md index 5170585149..1811c7d1ca 100644 --- a/docs/api_cpp/source_zh_cn/dataset.md +++ b/docs/api_cpp/source_zh_cn/dataset.md @@ -1,14 +1,13 @@ # mindspore::dataset -#include <[lite_mat.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/minddata/dataset/kernels/image/lite_cv/lite_mat.h)> -#include <[image_process.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/minddata/dataset/kernels/image/lite_cv/image_process.h)> - +\#include <[lite_mat.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/minddata/dataset/kernels/image/lite_cv/lite_mat.h)> +\#include <[image_process.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/minddata/dataset/kernels/image/lite_cv/image_process.h)> ## image_process.h文件的函数 ### ResizeBilinear -``` +```cpp bool ResizeBilinear(LiteMat &src, LiteMat &dst, int dst_w, int dst_h) ``` @@ -26,7 +25,7 @@ bool ResizeBilinear(LiteMat &src, LiteMat &dst, int dst_w, int dst_h) ### InitFromPixel -``` +```cpp bool InitFromPixel(const unsigned char *data, LPixelType pixel_type, LDataType data_type, int w, int h, LiteMat &m) ``` @@ -46,7 +45,7 @@ bool InitFromPixel(const unsigned char *data, LPixelType pixel_type, LDataType d ### ConvertTo -``` +```cpp bool ConvertTo(LiteMat &src, LiteMat &dst, double scale = 1.0) ``` @@ -64,7 +63,7 @@ bool ConvertTo(LiteMat &src, LiteMat &dst, double scale = 1.0) ### Crop -``` +```cpp bool Crop(LiteMat &src, LiteMat &dst, int x, int y, int w, int h) ``` @@ -84,7 +83,7 @@ bool Crop(LiteMat &src, LiteMat &dst, int x, int y, int w, int h) ### SubStractMeanNormalize -``` +```cpp bool SubStractMeanNormalize(const LiteMat &src, LiteMat &dst, const std::vector &mean, const std::vector &std) ``` @@ -102,7 +101,7 @@ bool SubStractMeanNormalize(const LiteMat &src, LiteMat &dst, const std::vector< ### Pad -``` +```cpp bool Pad(const LiteMat &src, LiteMat &dst, int top, int bottom, int left, int right, PaddBorderType pad_type, uint8_t fill_b_or_gray, uint8_t fill_g, uint8_t fill_r) ``` @@ -126,7 +125,7 @@ bool Pad(const LiteMat &src, LiteMat &dst, int top, int bottom, int left, int ri ### Affine -``` +```cpp void Affine(LiteMat &src, LiteMat &out_img, double M[6], std::vector dsize, UINT8_C1 borderValue) ``` @@ -140,7 +139,7 @@ void Affine(LiteMat &src, LiteMat &out_img, double M[6], std::vector dsi - `dsize`: 输出图像的大小。 - `borderValue`: 采图之后用于填充的像素值。 -``` +```cpp void Affine(LiteMat &src, LiteMat &out_img, double M[6], std::vector dsize, UINT8_C3 borderValue) ``` @@ -156,7 +155,7 @@ void Affine(LiteMat &src, LiteMat &out_img, double M[6], std::vector dsi ### GetDefaultBoxes -``` +```cpp std::vector> GetDefaultBoxes(BoxesConfig config) ``` @@ -172,7 +171,7 @@ std::vector> GetDefaultBoxes(BoxesConfig config) ### ConvertBoxes -``` +```cpp void ConvertBoxes(std::vector> &boxes, std::vector> &default_boxes, BoxesConfig config) ``` @@ -186,7 +185,7 @@ void ConvertBoxes(std::vector> &boxes, std::vector ApplyNms(std::vector> &all_boxes, std::vector &all_scores, float thres, int max_boxes) ``` @@ -208,11 +207,11 @@ std::vector ApplyNms(std::vector> &all_boxes, std::vecto LiteMat是一个处理图像的类。 -**构造函数和析构函数** +### 构造函数和析构函数 -### LiteMat +#### LiteMat -``` +```cpp LiteMat() LiteMat(int width, LDataType data_type = LDataType::UINT8) @@ -224,17 +223,17 @@ LiteMat(int width, int height, int channel, LDataType data_type = LDataType::UIN MindSpore中dataset模块下LiteMat的构造方法，使用参数的默认值。 -``` +```cpp ~LiteMat(); ``` MindSpore dataset LiteMat的析构函数。 -**公有成员函数** +### 公有成员函数 -### Init +#### Init -``` +```cpp void Init(int width, LDataType data_type = LDataType::UINT8) void Init(int width, int height, LDataType data_type = LDataType::UINT8) @@ -244,9 +243,9 @@ void Init(int width, int height, int channel, LDataType data_type = LDataType::U 该函数用于初始化图像的通道，宽度和高度，参数不同。 -### IsEmpty +#### IsEmpty -``` +```cpp bool IsEmpty() const ``` @@ -256,19 +255,19 @@ bool IsEmpty() const 返回True或者False。 -### Release +#### Release -``` +```cpp void Release() ``` 释放内存的函数。 -**私有成员函数** +### 私有成员函数 -### AlignMalloc +#### AlignMalloc -``` +```cpp void *AlignMalloc(unsigned int size) ``` @@ -282,18 +281,17 @@ void *AlignMalloc(unsigned int size) 返回指针的大小。 -### AlignFree +#### AlignFree -``` +```cpp void AlignFree(void *ptr) ``` 释放指针内存大小的方法。 +#### InitElemSize -### InitElemSize - -``` +```cpp void InitElemSize(LDataType data_type) ``` @@ -303,7 +301,7 @@ void InitElemSize(LDataType data_type) - `data_type`: 数据的类型。 -``` +```cpp int addRef(int *p, int value) ``` @@ -312,4 +310,4 @@ void InitElemSize(LDataType data_type) - 参数 - `p`: 指向引用的对象。 - - `value`: 引用时所加的值。 \ No newline at end of file + - `value`: 引用时所加的值。 diff --git a/docs/api_cpp/source_zh_cn/lite.md b/docs/api_cpp/source_zh_cn/lite.md index b3aaa5909c..034c3bd22e 100644 --- a/docs/api_cpp/source_zh_cn/lite.md +++ b/docs/api_cpp/source_zh_cn/lite.md @@ -1,11 +1,10 @@ # mindspore::lite -#include <[context.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/context.h)> +\#include <[context.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/context.h)> -#include <[model.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/model.h)> - -#include <[version.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/version.h)> +\#include <[model.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/model.h)> +\#include <[version.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/version.h)> ## Allocator @@ -15,23 +14,23 @@ Allocator类定义了一个内存池，用于动态地分配和释放内存。 Context类用于保存执行中的环境变量。 -**构造函数和析构函数** +### 构造函数和析构函数 -``` +```cpp Context() ``` 用默认参数构造MindSpore Lite Context 对象。 -``` +```cpp ~Context() ``` MindSpore Lite Context 的析构函数。 -**公有属性** +### 公有属性 -``` +```cpp float16_priority ``` @@ -39,29 +38,29 @@ float16_priority > 使能float16推理可能会导致模型推理精度下降，因为在模型推理的中间过程中，有些变量可能会超出float16的数值范围。 -``` +```cpp device_type ``` [**DeviceType**](https://www.mindspore.cn/doc/api_cpp/zh-CN/master/lite.html#devicetype)枚举类型。默认为**DT_CPU**，用于设置设备信息。 -``` +```cpp thread_num_ ``` **int** 值，默认为**2**，设置线程数。 -``` +```cpp allocator ``` 指针类型，指向内存分配器[**Allocator**](https://www.mindspore.cn/doc/api_cpp/zh-CN/master/lite.html#allocator)的指针。 -``` -cpu_bind_mode_ +```cpp +cpu_bind_mode_ ``` -[**CpuBindMode**](https://www.mindspore.cn/doc/api_cpp/zh-CN/master/lite.html#cpubindmode)枚举类型，默认为**MID_CPU**。 +[**CpuBindMode**](https://www.mindspore.cn/doc/api_cpp/zh-CN/master/lite.html#cpubindmode)枚举类型，默认为**MID_CPU**。 ## PrimitiveC @@ -71,87 +70,89 @@ PrimitiveC定义为算子的原型。 Model定义了MindSpore Lite中的模型，便于计算图管理。 -**析构函数** +### 析构函数 -``` +```cpp ~Model() ``` MindSpore Lite Model的析构函数。 -**公有成员函数** +### 公有成员函数 -``` +```cpp void Destroy() ``` 释放Model内的所有过程中动态分配的内存。 -``` +```cpp void Free() ``` 释放MindSpore Lite Model中的MetaGraph，用于减小运行时的内存。 -**静态公有成员函数** +### 静态公有成员函数 -``` +```cpp static Model *Import(const char *model_buf, size_t size) ``` 创建Model指针的静态方法。 -- 参数 +- 参数 - - `model_buf`: 定义了读取模型文件的缓存区。 + - `model_buf`: 定义了读取模型文件的缓存区。 - - `size`: 定义了模型缓存区的字节数。 + - `size`: 定义了模型缓存区的字节数。 - 返回值 指向MindSpore Lite的Model的指针。 - + ## CpuBindMode + 枚举类型，设置cpu绑定策略。 -**属性** +### 属性 -``` +```cpp MID_CPU = -1 ``` 优先中等CPU绑定策略。 -``` +```cpp HIGHER_CPU = 1 ``` 优先高级CPU绑定策略。 -``` +```cpp NO_BIND = 0 ``` 不绑定。 ## DeviceType + 枚举类型，设置设备类型。 -**属性** +### 属性 -``` +```cpp DT_CPU = -1 ``` 设备为CPU。 -``` +```cpp DT_GPU = 1 ``` 设备为GPU。 -``` +```cpp DT_NPU = 0 ``` @@ -159,11 +160,12 @@ DT_NPU = 0 ## Version -``` +```cpp std::string Version() ``` + 全局方法，用于获取版本的字符串。 - 返回值 - MindSpore Lite版本的字符串。 \ No newline at end of file + MindSpore Lite版本的字符串。 diff --git a/docs/api_cpp/source_zh_cn/session.md b/docs/api_cpp/source_zh_cn/session.md index adfc159b68..58f3a86018 100644 --- a/docs/api_cpp/source_zh_cn/session.md +++ b/docs/api_cpp/source_zh_cn/session.md @@ -1,36 +1,41 @@ # mindspore::session -#include <[lite_session.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/lite_session.h)> - +\#include <[lite_session.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/lite_session.h)> ## LiteSession LiteSession定义了MindSpore Lite中的会话，用于进行Model的编译和前向推理。 -**构造函数和析构函数** +### 构造函数和析构函数 -``` +```cpp LiteSession() ``` + MindSpore Lite LiteSession的构造函数，使用默认参数。 -``` + +```cpp ~LiteSession() ``` + MindSpore Lite LiteSession的析构函数。 -**公有成员函数** -``` +### 公有成员函数 + +```cpp virtual void BindThread(bool if_bind) ``` + 尝试将线程池中的线程绑定到指定的cpu内核，或从指定的cpu内核进行解绑。 - 参数 - `if_bind`: 定义了对线程进行绑定或解绑。 -``` +```cpp virtual int CompileGraph(lite::Model *model) ``` + 编译MindSpore Lite模型。 > CompileGraph必须在RunGraph方法之前调用。 @@ -43,18 +48,20 @@ virtual int CompileGraph(lite::Model *model) STATUS ，即编译图的错误码。STATUS在[errorcode.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/errorcode.h)中定义。 -``` +```cpp virtual std::vector GetInputs() const ``` + 获取MindSpore Lite模型的MSTensors输入。 - 返回值 MindSpore Lite MSTensor向量。 -``` +```cpp mindspore::tensor::MSTensor *GetInputsByName(const std::string &name) const ``` + 通过tensor名获取MindSpore Lite模型的MSTensors输入。 - 参数 @@ -65,9 +72,10 @@ mindspore::tensor::MSTensor *GetInputsByName(const std::string &name) const MindSpore Lite MSTensor。 -``` +```cpp virtual int RunGraph(const KernelCallBack &before = nullptr, const KernelCallBack &after = nullptr) ``` + 运行带有回调函数的会话。 > RunGraph必须在CompileGraph方法之后调用。 @@ -81,9 +89,10 @@ virtual int RunGraph(const KernelCallBack &before = nullptr, const KernelCallBac STATUS ，即编译图的错误码。STATUS在[errorcode.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/errorcode.h)中定义。 -``` +```cpp virtual std::vector GetOutputsByNodeName(const std::string &node_name) const ``` + 通过节点名获取MindSpore Lite模型的MSTensors输出。 - 参数 @@ -94,27 +103,30 @@ virtual std::vector GetOutputsByNodeName(const std::string MindSpore Lite MSTensor向量。 -``` +```cpp virtual std::unordered_map GetOutputs() const ``` + 获取与张量名相关联的MindSpore Lite模型的MSTensors输出。 - 返回值 包含输出张量名和MindSpore Lite MSTensor的容器类型变量。 -``` +```cpp virtual std::vector GetOutputTensorNames() const ``` + 获取由当前会话所编译的模型的输出张量名。 - 返回值 字符串向量，其中包含了按顺序排列的输出张量名。 -``` +```cpp virtual mindspore::tensor::MSTensor *GetOutputByTensorName(const std::string &tensor_name) const ``` + 通过张量名获取MindSpore Lite模型的MSTensors输出。 - 参数 @@ -125,9 +137,10 @@ virtual mindspore::tensor::MSTensor *GetOutputByTensorName(const std::string &te 指向MindSpore Lite MSTensor的指针。 -``` +```cpp virtual int Resize(const std::vector &inputs, const std::vector> &dims) ``` + 调整输入的形状。 - 参数 @@ -139,11 +152,12 @@ virtual int Resize(const std::vector &inputs, const std::ve STATUS ，即编译图的错误码。STATUS在[errorcode.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/errorcode.h)中定义。 -**静态公有成员函数** +### 静态公有成员函数 -``` +```cpp static LiteSession *CreateSession(lite::Context *context) ``` + 用于创建一个LiteSession指针的静态方法。 - 参数 @@ -153,9 +167,10 @@ static LiteSession *CreateSession(lite::Context *context) - 返回值 指向MindSpore Lite LiteSession的指针。 + ## KernelCallBack -``` +```cpp using KernelCallBack = std::function inputs, std::vector outputs, const CallBackParam &opInfo)> ``` @@ -166,12 +181,14 @@ using KernelCallBack = std::function inputs 一个结构体。CallBackParam定义了回调函数的输入参数。 **属性** -``` +```cpp name_callback_param ``` + **string** 类型变量。节点名参数。 -``` +```cpp type_callback_param ``` + **string** 类型变量。节点类型参数。 diff --git a/docs/api_cpp/source_zh_cn/tensor.md b/docs/api_cpp/source_zh_cn/tensor.md index 50e7c294b1..0d2e7e219a 100644 --- a/docs/api_cpp/source_zh_cn/tensor.md +++ b/docs/api_cpp/source_zh_cn/tensor.md @@ -1,32 +1,35 @@ # mindspore::tensor -#include <[ms_tensor.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/ms_tensor.h)> - +\#include <[ms_tensor.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/ms_tensor.h)> ## MSTensor MSTensor定义了MindSpore Lite中的张量。 -**构造函数和析构函数** -``` +### 构造函数和析构函数 + +```cpp MSTensor() ``` + MindSpore Lite MSTensor的构造函数。 - 返回值 MindSpore Lite MSTensor的实例。 - -``` + +```cpp virtual ~MSTensor() ``` + MindSpore Lite Model的析构函数。 -**公有成员函数** +### 公有成员函数 -``` +```cpp virtual TypeId data_type() const ``` + 获取MindSpore Lite MSTensor的数据类型。 > TypeId在[mindspore/mindspore/core/ir/dtype/type_id\.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/core/ir/dtype/type_id.h)中定义。只有TypeId枚举中的数字类型可用于MSTensor。 @@ -35,18 +38,20 @@ virtual TypeId data_type() const MindSpore Lite MSTensor类的MindSpore Lite TypeId。 -``` +```cpp virtual std::vector shape() const ``` + 获取MindSpore Lite MSTensor的形状。 - 返回值 一个包含MindSpore Lite MSTensor形状数值的整型向量。 -``` +```cpp virtual int DimensionSize(size_t index) const ``` + 通过参数索引获取MindSpore Lite MSTensor的维度的大小。 - 参数 @@ -57,28 +62,30 @@ virtual int DimensionSize(size_t index) const MindSpore Lite MSTensor的维度的大小。 -``` +```cpp virtual int ElementsNum() const ``` + 获取MSTensor中的元素个数。 - 返回值 MSTensor中的元素个数 -``` +```cpp virtual size_t Size() const ``` + 获取MSTensor中的数据的字节数大小。 - 返回值 MSTensor中的数据的字节数大小。 - -``` +```cpp virtual void *MutableData() const ``` + 获取MSTensor中的数据的指针。 > 该数据指针可用于对MSTensor中的数据进行读取和写入。 diff --git a/docs/api_java/source_zh_cn/lite_session.md b/docs/api_java/source_zh_cn/lite_session.md index c008d0eab9..9ed5bc8e9a 100644 --- a/docs/api_java/source_zh_cn/lite_session.md +++ b/docs/api_java/source_zh_cn/lite_session.md @@ -33,7 +33,7 @@ public boolean init(MSConfig config) - 参数 - - `MSConfig`: MSConfig类。 + - `MSConfig`: MSConfig类。 - 返回值 @@ -48,7 +48,7 @@ public void bindThread(boolean if_bind) 尝试将线程池中的线程绑定到指定的CPU内核，或从指定的CPU内核进行解绑。 - 参数 - - `if_bind`: 是否对线程进行绑定或解绑。 + - `if_bind`: 是否对线程进行绑定或解绑。 ## compileGraph @@ -156,7 +156,7 @@ public MSTensor getOutputByTensorName(String tensorName) - 参数 - - `tensorName`: 张量名。 + - `tensorName`: 张量名。 - 返回值 @@ -172,8 +172,8 @@ public boolean resize(List inputs, int[][] dims) - 参数 - - `inputs`: 模型对应的所有输入。 - - `dims`: 输入对应的新的shape，顺序注意要与inputs一致。 + - `inputs`: 模型对应的所有输入。 + - `dims`: 输入对应的新的shape，顺序注意要与inputs一致。 - 返回值 diff --git a/docs/api_java/source_zh_cn/model.md b/docs/api_java/source_zh_cn/model.md index 054bcf7068..c6081a8db8 100644 --- a/docs/api_java/source_zh_cn/model.md +++ b/docs/api_java/source_zh_cn/model.md @@ -25,8 +25,8 @@ public boolean loadModel(Context context, String modelName) - 参数 - - `context`: Android中的Context上下文 - - `modelName`: 模型文件名称 + - `context`: Android中的Context上下文 + - `modelName`: 模型文件名称 - 返回值 @@ -40,7 +40,7 @@ public boolean loadModel(String modelPath) - 参数 - - `modelPath`: 模型文件路径 + - `modelPath`: 模型文件路径 - 返回值 @@ -61,4 +61,3 @@ public void free() ``` 释放Model运行过程中动态分配的内存。 - diff --git a/docs/api_java/source_zh_cn/msconfig.md b/docs/api_java/source_zh_cn/msconfig.md index 7e868a60f1..c60019b8c6 100644 --- a/docs/api_java/source_zh_cn/msconfig.md +++ b/docs/api_java/source_zh_cn/msconfig.md @@ -26,9 +26,9 @@ public boolean init(int deviceType, int threadNum, int cpuBindMode) - 参数 - - `deviceType`: 设备类型，`deviceType`在[com.mindspore.lite.config.DeviceType](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/java/java/app/src/main/java/com/mindspore/lite/config/DeviceType.java)中定义。 - - `threadNum`: 线程数 - - `cpuBindMode`: CPU绑定模式，`cpuBindMode`在[com.mindspore.lite.config.CpuBindMode](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/java/java/app/src/main/java/com/mindspore/lite/config/CpuBindMode.java)中定义。 + - `deviceType`: 设备类型，`deviceType`在[com.mindspore.lite.config.DeviceType](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/java/java/app/src/main/java/com/mindspore/lite/config/DeviceType.java)中定义。 + - `threadNum`: 线程数 + - `cpuBindMode`: CPU绑定模式，`cpuBindMode`在[com.mindspore.lite.config.CpuBindMode](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/java/java/app/src/main/java/com/mindspore/lite/config/CpuBindMode.java)中定义。 - 返回值 @@ -42,8 +42,8 @@ public boolean init(int deviceType, int threadNum) - 参数 - - `deviceType`: 设备类型，`deviceType`在[com.mindspore.lite.config.DeviceType](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/java/java/app/src/main/java/com/mindspore/lite/config/DeviceType.java)中定义。 - - `threadNum`: 线程数。 + - `deviceType`: 设备类型，`deviceType`在[com.mindspore.lite.config.DeviceType](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/java/java/app/src/main/java/com/mindspore/lite/config/DeviceType.java)中定义。 + - `threadNum`: 线程数。 - 返回值 @@ -57,7 +57,7 @@ public boolean init(int deviceType) - 参数 - - `deviceType`: 设备类型，`deviceType`在[com.mindspore.lite.config.DeviceType](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/java/java/app/src/main/java/com/mindspore/lite/config/DeviceType.java)中定义。 + - `deviceType`: 设备类型，`deviceType`在[com.mindspore.lite.config.DeviceType](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/java/java/app/src/main/java/com/mindspore/lite/config/DeviceType.java)中定义。 - 返回值 @@ -79,4 +79,4 @@ public boolean init() public void free() ``` -释放MSConfig运行过程中动态分配的内存。LiteSession init之后即可释放。 \ No newline at end of file +释放MSConfig运行过程中动态分配的内存。LiteSession init之后即可释放。 diff --git a/docs/api_java/source_zh_cn/mstensor.md b/docs/api_java/source_zh_cn/mstensor.md index b6e582f38f..ce7db9822c 100644 --- a/docs/api_java/source_zh_cn/mstensor.md +++ b/docs/api_java/source_zh_cn/mstensor.md @@ -103,7 +103,7 @@ public void setData(byte[] data) 设定MSTensor的输入数据。 - 参数 - - `data`: byte[]类型的输入数据。 + - `data`: byte[]类型的输入数据。 ```java public void setData(ByteBuffer data) @@ -112,7 +112,7 @@ public void setData(ByteBuffer data) 设定MSTensor的输入数据。 - 参数 - - `data`: ByteBuffer类型的输入数据。 + - `data`: ByteBuffer类型的输入数据。 ## size @@ -145,4 +145,3 @@ public void free() ``` 释放MSTensor运行过程中动态分配的内存。 - -- Gitee From 01c1067ec548cca4d910b3316702b596d0b1b2ba Mon Sep 17 00:00:00 2001 From: lihongkang <[lihongkang1@huawei.com]> Date: Sat, 17 Oct 2020 14:43:35 +0800 Subject: [PATCH 3/5] update op list --- docs/note/source_en/operator_list_ms.md | 7 ++++++- docs/note/source_zh_cn/operator_list_ms.md | 7 ++++++- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/note/source_en/operator_list_ms.md b/docs/note/source_en/operator_list_ms.md index fee523856d..87bc091f8f 100644 --- a/docs/note/source_en/operator_list_ms.md +++ b/docs/note/source_en/operator_list_ms.md @@ -110,6 +110,8 @@ | [mindspore.nn.LGamma](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.LGamma) |Supported | Doing | Doing |layer/math | [mindspore.nn.ReduceLogSumExp](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.ReduceLogSumExp) |Supported | Supported | Doing |layer/math | [mindspore.nn.MSSSIM](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.MSSSIM) | Supported |Doing | Doing |layer/image +| [mindspore.nn.AvgPool1d](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.AvgPool1d) | Supported | Doing | Doing |layer/pooling +| [mindspore.nn.Unfold](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.Unfold) |Supported | Doing | Doing |layer/basic ## mindspore.ops @@ -273,7 +275,7 @@ | [mindspore.ops.GatherV2](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.GatherV2) | Supported | Supported | Supported | array_ops | [mindspore.ops.Split](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.Split) | Supported | Supported | Doing | array_ops | [mindspore.ops.Rank](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.Rank) | Supported | Supported | Supported | array_ops -| [mindspore.ops.TruncatedNormal](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.TruncatedNormal) | Supported | Supported | Supported | array_ops +| [mindspore.ops.TruncatedNormal](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.TruncatedNormal) | Doing | Supported | Supported | array_ops | [mindspore.ops.Size](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.Size) | Supported | Supported | Supported | array_ops | [mindspore.ops.Fill](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.Fill) | Supported | Supported | Supported | array_ops | [mindspore.ops.OnesLike](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.OnesLike) | Supported | Supported | Doing | array_ops @@ -371,6 +373,9 @@ | [mindspore.ops.IFMR](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.IFMR) | Supported | Doing | Doing | math_ops | [mindspore.ops.DynamicShape](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.DynamicShape) | Supported | Supported | Supported | array_ops | [mindspore.ops.Unique](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.Unique) | Doing | Doing | Doing | array_ops +| [mindspore.ops.ReduceAny](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.ReduceAny) | Supported | Doing | Doing | math_ops +| [mindspore.ops.SparseToDense](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.SparseToDense) | Doing | Doing | Doing | sparse_ops +| [mindspore.ops.CTCGreedyDecoder](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.CTCGreedyDecoder) | Doing | Doing | Doing | nn_ops ## mindspore.ops.functional diff --git a/docs/note/source_zh_cn/operator_list_ms.md b/docs/note/source_zh_cn/operator_list_ms.md index 3fae2a6691..6a299dbe98 100644 --- a/docs/note/source_zh_cn/operator_list_ms.md +++ b/docs/note/source_zh_cn/operator_list_ms.md @@ -105,6 +105,8 @@ | [mindspore.nn.LGamma](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.LGamma) |Supported | Doing | Doing |layer/math | [mindspore.nn.ReduceLogSumExp](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.ReduceLogSumExp) |Supported | Supported | Doing |layer/math | [mindspore.nn.MSSSIM](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.MSSSIM) | Supported |Doing | Doing |layer/image +| [mindspore.nn.AvgPool1d](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.AvgPool1d) | Supported | Doing | Doing |layer/pooling +| [mindspore.nn.Unfold](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.nn.html#mindspore.nn.Unfold) |Supported | Doing | Doing |layer/basic ## mindspore.ops.operations @@ -268,7 +270,7 @@ | [mindspore.ops.GatherV2](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.GatherV2) | Supported | Supported | Supported | array_ops | [mindspore.ops.Split](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.Split) | Supported | Supported | Doing | array_ops | [mindspore.ops.Rank](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.Rank) | Supported | Supported | Supported | array_ops -| [mindspore.ops.TruncatedNormal](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.TruncatedNormal) | Supported | Supported | Supported | array_ops +| [mindspore.ops.TruncatedNormal](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.TruncatedNormal) | Doing | Supported | Supported | array_ops | [mindspore.ops.Size](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.Size) | Supported | Supported | Supported | array_ops | [mindspore.ops.Fill](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.Fill) | Supported | Supported | Supported | array_ops | [mindspore.ops.OnesLike](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.OnesLike) | Supported | Supported | Doing | array_ops @@ -366,6 +368,9 @@ | [mindspore.ops.IFMR](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.IFMR) | Supported | Doing | Doing | math_ops | [mindspore.ops.DynamicShape](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.DynamicShape) | Supported | Supported | Supported | array_ops | [mindspore.ops.Unique](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.Unique) | Doing | Doing | Doing | array_ops +| [mindspore.ops.ReduceAny](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.ReduceAny) | Supported | Doing | Doing | math_ops +| [mindspore.ops.SparseToDense](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.SparseToDense) | Doing | Doing | Doing | sparse_ops +| [mindspore.ops.CTCGreedyDecoder](https://www.mindspore.cn/doc/api_python/zh-CN/master/mindspore/mindspore.ops.html#mindspore.ops.CTCGreedyDecoder) | Doing | Doing | Doing | nn_ops ## mindspore.ops.functional -- Gitee From fcdc1244f35b3de4408292588daef98427d42b81 Mon Sep 17 00:00:00 2001 From: zhaodezan Date: Sat, 17 Oct 2020 02:28:58 -0400 Subject: [PATCH 4/5] bind core api change --- docs/api_cpp/source_en/lite.md | 2 +- docs/api_cpp/source_zh_cn/lite.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/api_cpp/source_en/lite.md b/docs/api_cpp/source_en/lite.md index 26458a38eb..61ee990133 100644 --- a/docs/api_cpp/source_en/lite.md +++ b/docs/api_cpp/source_en/lite.md @@ -117,7 +117,7 @@ An **enum** type. CpuBindMode defined for holding bind cpu strategy argument. ### Attributes ```cpp -MID_CPU = -1 +MID_CPU = 2 ``` Bind middle cpu first. diff --git a/docs/api_cpp/source_zh_cn/lite.md b/docs/api_cpp/source_zh_cn/lite.md index 034c3bd22e..978178fabc 100644 --- a/docs/api_cpp/source_zh_cn/lite.md +++ b/docs/api_cpp/source_zh_cn/lite.md @@ -117,7 +117,7 @@ static Model *Import(const char *model_buf, size_t size) ### 属性 ```cpp -MID_CPU = -1 +MID_CPU = 2 ``` 优先中等CPU绑定策略。 -- Gitee From 759062ab12b843dfc27581ac239687c385572a3c Mon Sep 17 00:00:00 2001 From: yingchen Date: Mon, 19 Oct 2020 10:54:38 +0800 Subject: [PATCH 5/5] fix docs errors part3 --- docs/faq/source_en/faq.md | 5 +- docs/faq/source_zh_cn/faq.md | 4 +- docs/note/source_en/benchmark.md | 16 +- .../constraints_on_network_construction.md | 163 ++++++++++-------- .../mindarmour/differential_privacy_design.md | 23 ++- .../design/mindarmour/fuzzer_design.md | 3 +- .../design/mindinsight/graph_visual_design.md | 8 +- .../mindinsight/tensor_visual_design.md | 12 +- .../mindinsight/training_visual_design.md | 24 +-- .../design/mindspore/architecture_lite.md | 6 +- .../mindspore/distributed_training_design.md | 16 +- .../note/source_en/design/mindspore/mindir.md | 41 +++-- .../design/mindspore/profiler_design.md | 19 +- docs/note/source_en/glossary.md | 2 +- docs/note/source_en/help_seeking_path.md | 22 ++- docs/note/source_en/network_list_ms.md | 2 +- docs/note/source_en/object_detection_lite.md | 1 - docs/note/source_en/operator_list_implicit.md | 27 +-- docs/note/source_en/operator_list_lite.md | 2 +- docs/note/source_en/operator_list_ms.md | 16 +- docs/note/source_en/roadmap.md | 90 +++++----- docs/note/source_zh_cn/benchmark.md | 12 +- 22 files changed, 285 insertions(+), 229 deletions(-) diff --git a/docs/faq/source_en/faq.md b/docs/faq/source_en/faq.md index 3162c71c3a..33cea73eb8 100644 --- a/docs/faq/source_en/faq.md +++ b/docs/faq/source_en/faq.md @@ -98,6 +98,7 @@ A: You can write the frequently-used environment settings to `~/.bash_profile` o ### Verifying the Installation Q: After MindSpore is installed on a CPU of a PC, an error message `the pointer[session] is null` is displayed during code verification. The specific code is as follows. How do I verify whether MindSpore is successfully installed? + ```python import numpy as np from mindspore import Tensor @@ -194,7 +195,7 @@ A: The MindSpore CPU version can be installed on Windows 10. For details about t Q: What can I do if an error message `wrong shape of image` is displayed when I use a model trained by MindSpore to perform prediction on a `28 x 28` digital image with white text on a black background? -A: The MNIST gray scale image dataset is used for MindSpore training. Therefore, when the model is used, the data must be set to a `28 x 28 `gray scale image, that is, a single channel. +A: The MNIST gray scale image dataset is used for MindSpore training. Therefore, when the model is used, the data must be set to a `28 x 28`gray scale image, that is, a single channel.
@@ -291,4 +292,4 @@ A: For details about script or model migration, please visit the [MindSpore offi Q: Does MindSpore provide open-source e-commerce datasets? -A: No. Please stay tuned for updates on the [MindSpore official website](https://www.mindspore.cn/en). \ No newline at end of file +A: No. Please stay tuned for updates on the [MindSpore official website](https://www.mindspore.cn/en). diff --git a/docs/faq/source_zh_cn/faq.md b/docs/faq/source_zh_cn/faq.md index 67250b2d6b..4637908446 100644 --- a/docs/faq/source_zh_cn/faq.md +++ b/docs/faq/source_zh_cn/faq.md @@ -103,6 +103,7 @@ A：常用的环境变量设置写入到`~/.bash_profile` 或 `~/.bashrc`中， ### 安装验证 Q:个人电脑CPU环境安装MindSpore后验证代码时报错：`the pointer[session] is null`，具体代码如下，该如何验证是否安装成功呢？ + ```python import numpy as np from mindspore import Tensor @@ -234,6 +235,7 @@ Q：MindSpore如何实现早停功能？ A：可以自定义`callback`方法实现早停功能。 例子：当loss降到一定数值后，停止训练。 + ```python class EarlyStop(Callback): def __init__(self, control_loss=1): @@ -373,4 +375,4 @@ A：关于脚本或者模型迁移，可以查询MindSpore官网中关于[网络 Q：MindSpore是否附带开源电商类数据集？ -A：暂时还没有，可以持续关注[MindSpore官网](https://www.mindspore.cn)。 \ No newline at end of file +A：暂时还没有，可以持续关注[MindSpore官网](https://www.mindspore.cn)。 diff --git a/docs/note/source_en/benchmark.md b/docs/note/source_en/benchmark.md index 72779fa5d8..f418c9d8d3 100644 --- a/docs/note/source_en/benchmark.md +++ b/docs/note/source_en/benchmark.md @@ -15,7 +15,7 @@ -This document describes the MindSpore benchmarks. +This document describes the MindSpore benchmarks. For details about the MindSpore networks, see [Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/model_zoo). ## Training Performance @@ -28,17 +28,17 @@ For details about the MindSpore networks, see [Model Zoo](https://gitee.com/mind | | | | | Ascend: 8 * Ascend 910
CPU: 192 Cores | Mixed | 256 | 16600 images/sec | 0.98 | | | | | | Ascend: 16 * Ascend 910
CPU: 384 Cores | Mixed | 256 | 32768 images/sec | 0.96 | -1. The preceding performance is obtained based on ModelArts, the HUAWEI CLOUD AI development platform. It is the average performance obtained by the Ascend 910 AI processor during the overall training process. +1. The preceding performance is obtained based on ModelArts, the HUAWEI CLOUD AI development platform. It is the average performance obtained by the Ascend 910 AI processor during the overall training process. 2. For details about other open source frameworks, see [ResNet-50 v1.5 for TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/Classification/ConvNets/resnet50v1.5). ### BERT -| Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | +| Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | BERT-Large | Attention | zhwiki | 0.5.0-beta | Ascend: 1 * Ascend 910
CPU: 24 Cores | Mixed | 96 | 269 sentences/sec | - | | | | | | Ascend: 8 * Ascend 910
CPU: 192 Cores | Mixed | 96 | 2069 sentences/sec | 0.96 | -1. The preceding performance is obtained based on ModelArts, the HUAWEI CLOUD AI development platform. The network contains 24 hidden layers, the sequence length is 128 tokens, and the vocabulary contains 21128 tokens. +1. The preceding performance is obtained based on ModelArts, the HUAWEI CLOUD AI development platform. The network contains 24 hidden layers, the sequence length is 128 tokens, and the vocabulary contains 21128 tokens. 2. For details about other open source frameworks, see [BERT For TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT). ### Wide & Deep (data parallel) @@ -46,7 +46,7 @@ For details about the MindSpore networks, see [Model Zoo](https://gitee.com/mind | Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Wide & Deep | Recommend | Criteo | 0.6.0-beta | Ascend: 1 * Ascend 910
CPU: 24 Cores | Mixed | 16000 | 796892 samples/sec | - | -| | | | | Ascend: 8 * Ascend 910
CPU: 192 Cores | Mixed | 16000*8 | 4872849 samples/sec | 0.76 | +| | | | | Ascend: 8 *Ascend 910
CPU: 192 Cores | Mixed | 16000*8 | 4872849 samples/sec | 0.76 | 1. The preceding performance is obtained based on Atlas 800, and the model is data parallel. 2. For details about other open source frameworks, see [Wide & Deep For TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/Recommendation/WideAndDeep). @@ -56,9 +56,9 @@ For details about the MindSpore networks, see [Model Zoo](https://gitee.com/mind | Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Wide & Deep | Recommend | Criteo | 0.6.0-beta | Ascend: 1 * Ascend 910
CPU: 24 Cores | Mixed | 1000 | 68715 samples/sec | - | -| | | | | Ascend: 8 * Ascend 910
CPU: 192 Cores | Mixed | 8000*8 | 283830 samples/sec | 0.51 | -| | | | | Ascend: 16 * Ascend 910
CPU: 384 Cores | Mixed | 8000*16 | 377848 samples/sec | 0.34 | -| | | | | Ascend: 32 * Ascend 910
CPU: 768 Cores | Mixed | 8000*32 | 433423 samples/sec | 0.20 | +| | | | | Ascend: 8 *Ascend 910
CPU: 192 Cores | Mixed | 8000*8 | 283830 samples/sec | 0.51 | +| | | | | Ascend: 16 *Ascend 910
CPU: 384 Cores | Mixed | 8000*16 | 377848 samples/sec | 0.34 | +| | | | | Ascend: 32 *Ascend 910
CPU: 768 Cores | Mixed | 8000*32 | 433423 samples/sec | 0.20 | 1. The preceding performance is obtained based on Atlas 800, and the model is model parallel. 2. For details about other open source frameworks, see [Wide & Deep For TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/Recommendation/WideAndDeep). diff --git a/docs/note/source_en/constraints_on_network_construction.md b/docs/note/source_en/constraints_on_network_construction.md index 0c0a4f2cba..bedbdf3848 100644 --- a/docs/note/source_en/constraints_on_network_construction.md +++ b/docs/note/source_en/constraints_on_network_construction.md @@ -1,7 +1,7 @@ # Constraints on Network Construction Using Python `Linux` `Ascend` `GPU` `CPU` `Model Development` `Beginner` `Intermediate` `Expert` - + - [Constraints on Network Construction Using Python](#constraints-on-network-construction-using-python) @@ -28,21 +28,26 @@ ## Overview + MindSpore can compile user source code based on the Python syntax into computational graphs, and can convert common functions or instances inherited from nn.Cell into computational graphs. Currently, MindSpore does not support conversion of any Python source code into computational graphs. Therefore, there are constraints on source code compilation, including syntax constraints and network definition constraints. As MindSpore evolves, the constraints may change. ## Syntax Constraints + ### Supported Python Data Types -* Number: supports `int`, `float`, and `bool`. Complex numbers are not supported. -* String -* List: supports the append method only. Updating a list will generate a new list. -* Tuple -* Dictionary: The type of key should be String. + +- Number: supports `int`, `float`, and `bool`. Complex numbers are not supported. +- String +- List: supports the append method only. Updating a list will generate a new list. +- Tuple +- Dictionary: The type of key should be String. + ### MindSpore Extended Data Type -* Tensor: Tensor variables must be defined instances. + +- Tensor: Tensor variables must be defined instances. ### Expression Types -| Operation | Description +| Operation | Description | :----------- |:-------- | Unary operator |`+`,`-`, and`not`. The operator `+` supports only scalars. | Binary operator |`+`, `-`, `*`, `/`, `%`, `**` and `//`. @@ -81,10 +86,11 @@ | `isinstance` | The usage principle is consistent with Python, but the second input parameter can only be the type defined by mindspore. ### Function Parameters -* Default parameter value: The data types `int`, `float`, `bool`, `None`, `str`, `tuple`, `list`, and `dict` are supported, whereas `Tensor` is not supported. -* Variable parameter: Functions with variable arguments is supported for training and inference. -* Key-value pair parameter: Functions with key-value pair parameters cannot be used for backward propagation on computational graphs. -* Variable key-value pair parameter: Functions with variable key-value pairs cannot be used for backward propagation on computational graphs. + +- Default parameter value: The data types `int`, `float`, `bool`, `None`, `str`, `tuple`, `list`, and `dict` are supported, whereas `Tensor` is not supported. +- Variable parameter: Functions with variable arguments is supported for training and inference. +- Key-value pair parameter: Functions with key-value pair parameters cannot be used for backward propagation on computational graphs. +- Variable key-value pair parameter: Functions with variable key-value pairs cannot be used for backward propagation on computational graphs. ### Operators @@ -101,55 +107,56 @@ ### Index operation -The index operation includes `tuple` and` Tensor`. The following focuses on the index value assignment and assignment operation of `Tensor`. The value takes` tensor_x [index] `as an example, and the assignment takes` tensor_x [index] = u` as an example for detailed description. Among them, tensor_x is a `Tensor`, which is sliced; index means the index, u means the assigned value, which can be` scalar` or `Tensor (size = 1)`. The index types are as follows: +The index operation includes `tuple` and`Tensor`. The following focuses on the index value assignment and assignment operation of `Tensor`. The value takes` tensor_x [index] `as an example, and the assignment takes`tensor_x [index] = u` as an example for detailed description. Among them, tensor_x is a `Tensor`, which is sliced; index means the index, u means the assigned value, which can be`scalar` or `Tensor (size = 1)`. The index types are as follows: - Slice index: index is `slice` - - Value: `tensor_x[start: stop: step]`, where Slice (start: stop: step) has the same syntax as Python, and will not be repeated here. - - Assignment: `tensor_x[start: stop: step] = u`. + - Value: `tensor_x[start: stop: step]`, where Slice (start: stop: step) has the same syntax as Python, and will not be repeated here. + - Assignment: `tensor_x[start: stop: step] = u`. - Ellipsis index: index is `ellipsis` - - Value: `tensor_x [...]`. - - Assignment: `tensor_x [...] = u`. + - Value: `tensor_x [...]`. + - Assignment: `tensor_x [...] = u`. - Boolean constant index: index is `True`, index is `False` is not supported temporarily. - - Value: `tensor_x[True]`. - - Assignment: Not supported yet. + - Value: `tensor_x[True]`. + - Assignment: Not supported yet. - Tensor index: index is `Tensor` - - Value: `tensor_x [index]`, `index` must be `Tensor` of data type `int32` or `int64`, + - Value: `tensor_x [index]`, `index` must be `Tensor` of data type `int32` or `int64`, the element value range is `[0, tensor_x.shape[0])`. - - Assignment: `tensor_x [index] = U`. - - `tensor_x` data type must be one of the following: `float16`, `float32`, `int8`, `uint8`. - - `index` must be `Tensor` of data type `int32`, the element value range is `[0, tensor_x.shape [0])`. - - `U` can be `Number`, `Tensor`, `Tuple` only containing `Number`, `Tuple` only containing `Tensor`. - - Single `Number` or every `Number` in `Tuple` must be the same type as `tensor_x`, ie + - Assignment: `tensor_x [index] = U`. + - `tensor_x` data type must be one of the following: `float16`, `float32`, `int8`, `uint8`. + - `index` must be `Tensor` of data type `int32`, the element value range is `[0, tensor_x.shape [0])`. + - `U` can be `Number`, `Tensor`, `Tuple` only containing `Number`, `Tuple` only containing `Tensor`. + - Single `Number` or every `Number` in `Tuple` must be the same type as `tensor_x`, ie When the data type of `tensor_x` is `uint8` or `int8`, the `Number` type should be `int`; When the data type of `tensor_x` is `float16` or `float32`, the `Number` type should be `float`. - - Single `Tensor` or every `Tensor in Tuple` must be consistent with the data type of `tensor_x`, + - Single `Tensor` or every `Tensor in Tuple` must be consistent with the data type of `tensor_x`, when single `Tensor`, the `shape` should be equal to or broadcast as `index.shape + tensor_x.shape [1:]`. - - `Tuple` containing `Number` must meet requirement: + - `Tuple` containing `Number` must meet requirement: `len (Tuple) = (index.shape + tensor_x.shape [1:]) [-1]`. - - `Tuple` containing `Tensor` must meet requirements: + - `Tuple` containing `Tensor` must meet requirements: the `shape` of each `Tensor` should be the same, `(len (Tuple),) + Tensor.shape` should be equal to or broadcast as `index.shape + tensor_x.shape [1:]`. - None constant index: index is `None` - - Value: `tensor_x[None]`, results are consistent with numpy. - - Assignment: Not supported yet. + - Value: `tensor_x[None]`, results are consistent with numpy. + - Assignment: Not supported yet. - tuple index: index is `tuple` - - The tuple element is a slice: - - Value: for example `tensor_x[::,: 4, 3: 0: -1]`. - - Assignment: for example `tensor_x[::,: 4, 3: 0: -1] = u`. - - The tuple element is Number: - - Value: for example `tensor_x[2,1]`. - - Assignment: for example `tensor_x[1,4] = u`. - - The tuple element is a mixture of slice and ellipsis: - - Value: for example `tensor_x[..., ::, 1:]`. - - Assignment: for example `tensor_x[..., ::, 1:] = u`. - - Not supported in other situations + - The tuple element is a slice: + - Value: for example `tensor_x[::,: 4, 3: 0: -1]`. + - Assignment: for example `tensor_x[::,: 4, 3: 0: -1] = u`. + - The tuple element is Number: + - Value: for example `tensor_x[2,1]`. + - Assignment: for example `tensor_x[1,4] = u`. + - The tuple element is a mixture of slice and ellipsis: + - Value: for example `tensor_x[..., ::, 1:]`. + - Assignment: for example `tensor_x[..., ::, 1:] = u`. + - Not supported in other situations The index value operation of tuple and list type, we need to focus on the index value operation of tuple or list whose element type is `nn.Cell`. This operation is currently only supported by the GPU backend in Graph mode, and its syntax format is like `layers[index](*inputs)`, the example code is as follows: + ```python class Net(nn.Cell): def __init__(self): @@ -162,13 +169,16 @@ The index value operation of tuple and list type, we need to focus on the index x = self.layers[index](x) return x ``` + The grammar has the following constraints: -* Only the index value operation of tuple or list whose element type is `nn.Cell` is supported. -* The index is a scalar `Tensor` of type `int32`, with a value range of `[-n, n)`, where `n` is the size of the tuple, and the maximum supported tuple size is 1000. -* The number, type and shape of the input data of the `Construct` function of each Cell element in the tuple are the same, and the number of data output after the `Construct` function runs, the type and shape are also the same. -* Each element in the tuple needs to be defined before the tuple is defined. -* This syntax does not support running branches as if, while, for and other control flow, except if the control condition of the control flow is constant. for example: - - Supported example: + +- Only the index value operation of tuple or list whose element type is `nn.Cell` is supported. +- The index is a scalar `Tensor` of type `int32`, with a value range of `[-n, n)`, where `n` is the size of the tuple, and the maximum supported tuple size is 1000. +- The number, type and shape of the input data of the `Construct` function of each Cell element in the tuple are the same, and the number of data output after the `Construct` function runs, the type and shape are also the same. +- Each element in the tuple needs to be defined before the tuple is defined. +- This syntax does not support running branches as if, while, for and other control flow, except if the control condition of the control flow is constant. for example: + - Supported example: + ```python class Net(nn.Cell): def __init__(self, flag=True): @@ -183,7 +193,9 @@ The grammar has the following constraints: x = self.layers[index](x) return x ``` - - Unsupported example: + + - Unsupported example: + ```python class Net(nn.Cell): def __init__(self): @@ -202,20 +214,23 @@ Tuple also support slice value operations, but do not support slice type as Tens ### Unsupported Syntax -Currently, the following syntax is not supported in network constructors: +Currently, the following syntax is not supported in network constructors: `raise`, `yield`, `async for`, `with`, `async with`, `assert`, `import`, and `await`. ## Network Definition Constraints ### Instance Types on the Entire Network -* Common Python function with the [@ms_function](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.html#mindspore.ms_function) decorator. -* Cell subclass inherited from [nn.Cell](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.Cell). + +- Common Python function with the [@ms_function](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.html#mindspore.ms_function) decorator. +- Cell subclass inherited from [nn.Cell](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.nn.html#mindspore.nn.Cell). ### Network Input Type -* The training data input parameters of the entire network must be of the Tensor type. -* The generated ANF diagram cannot contain the following constant nodes: string constants, constants with nested tuples, and constants with nested lists. + +- The training data input parameters of the entire network must be of the Tensor type. +- The generated ANF diagram cannot contain the following constant nodes: string constants, constants with nested tuples, and constants with nested lists. ### Network Graph Optimization + During graph optimization at the ME frontend, the dataclass, dictionary, list, and key-value pair types are converted to tuple types, and the corresponding operations are converted to tuple operations. ### Network Construction Components @@ -230,33 +245,36 @@ Currently, the following syntax is not supported in network constructors: | Composite operator |[mindspore/ops/composite/*](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html). | Operator generated by constexpr |Uses the value generated by [@constexpr](https://www.mindspore.cn/doc/api_python/en/master/mindspore/mindspore.ops.html#mindspore.ops.constexpr) to calculate operators. - ### Other Constraints + 1. Input parameters of the `construct` function on the entire network and parameters of functions modified by the `ms_function` decorator are generalized during the graph compilation and cannot be passed to operators as constant input. Therefore, in graph mode, the parameter passed to the entry network can only be `Tensor`. As shown in the following example: - - * The following is an example of incorrect input: + + - The following is an example of incorrect input: + ```python class ExpandDimsTest(Cell): def __init__(self): super(ExpandDimsTest, self).__init__() self.expandDims = P.ExpandDims() - + def construct(self, input_x, input_axis): return self.expandDims(input_x, input_axis) expand_dim = ExpandDimsTest() input_x = Tensor(np.random.randn(2,2,2,2).astype(np.float32)) expand_dim(input_x, 0) ``` + In the example, `ExpandDimsTest` is a single-operator network with two inputs: `input_x` and `input_axis`. The second input of the `ExpandDims` operator must be a constant. This is because `input_axis` is required when the output dimension of the `ExpandDims` operator is deduced during graph compilation. As the network parameter input, the value of `input_axis` is generalized into a variable and cannot be determined. As a result, the output dimension of the operator cannot be deduced, causing the graph compilation failure. Therefore, the input required by deduction in the graph compilation phase must be a constant. In the API, the parameters of this type of operator that require constant input will be explained, marked `const input is needed`. - - * Directly enter the needed value or a member variable in a class for the constant input of the operator in the construct function. The following is an example of correct input: + + - Directly enter the needed value or a member variable in a class for the constant input of the operator in the construct function. The following is an example of correct input: + ```python class ExpandDimsTest(Cell): def __init__(self, axis): super(ExpandDimsTest, self).__init__() self.expandDims = P.ExpandDims() self.axis = axis - + def construct(self, input_x): return self.expandDims(input_x, self.axis) axis = 0 @@ -264,52 +282,56 @@ Currently, the following syntax is not supported in network constructors: input_x = Tensor(np.random.randn(2,2,2,2).astype(np.float32)) expand_dim(input_x) ``` - + 2. It is not allowed to modify `non-Parameter` type data members of the network. Examples are as follows: - ``` + ```python class Net(Cell): def __init__(self): super(Net, self).__init__() self.num = 2 self.par = Parameter(Tensor(np.ones((2, 3, 4))), name="par") - + def construct(self, x, y): return x + y ``` + In the network defined above, `self.num` is not a `Parameter` and cannot be modified, but `self.par` is a `Parameter` and can be modified. 3. When an undefined class member is used in the `construct` function, it will be treated as `None` instead of throwing `AttributeError` like the Python interpreter. Examples are as follows: - - ``` + + ```python class Net(Cell): def __init__(self): super(Net, self).__init__() - + def construct(self, x): return x + self.y ``` + In the network defined above, the undefined class member `self.y` is used in `construct`, and `self.y` will be treated as `None`. 4. When using the control flow of `if-else` in the `construct` function, the data types returned by `if` and `else` or the data types of the same variable after being updated must be the same. Examples are as follows: - ``` + + ```python class NetReturn(Cell): def __init__(self): super(NetReturn, self).__init__() - + def construct(self, x, y, m, n): if x > y: return m else: return n ``` + In the network `NetReturn` defined above, the `if-else` control flow is used in `construct`, then the data type of `m` returned by the `if` branch and the data type of `n` returned by the `else` branch must be consistent. - - ``` + + ```python class NetAssign(Cell): def __init__(self): super(NetAssign, self).__init__() - + def construct(self, x, y, m, n): if x > y: out = m @@ -317,4 +339,5 @@ Currently, the following syntax is not supported in network constructors: out = n return out ``` + In the network `NetAssign` defined above, the `if-else` control flow is used in the `construct`, then the data types of the `out` after the update of the `if` branch and the update of the `else` branch must be consistent. diff --git a/docs/note/source_en/design/mindarmour/differential_privacy_design.md b/docs/note/source_en/design/mindarmour/differential_privacy_design.md index c57af833f1..7038864e65 100644 --- a/docs/note/source_en/design/mindarmour/differential_privacy_design.md +++ b/docs/note/source_en/design/mindarmour/differential_privacy_design.md @@ -26,14 +26,13 @@ The Differential-Privacy module of MindArmour implements the differential privac Figure 1 shows an overall design of differential privacy training, and mainly including differential privacy noise mechanisms (DP mechanisms), a differential privacy optimizer (DP optimizer), and a privacy monitor. - ### DP Optimizer DP optimizer inherits capabilities of the MindSpore optimizer and uses the DP mechanisms to scramble and protect gradients. Currently, MindArmour provides three types of DP optimizers: constant Gaussian optimizer, adaptive Gaussian optimizer, and adaptive clipping optimizer. Each type of DP optimizer adds differential privacy protection capabilities to common optimizers such as SGD and Momentum from different perspectives. -* Constant Gaussian optimizer is a DP optimizer for non-adaptive Gaussian noise. The advantage is that the differential privacy budget ϵ can be strictly controlled. The disadvantage is that in the model training process, the noise amount added in each step is fixed. If the number of training steps is too large, the noise in the later phase of training makes the model convergence difficult, or even causes the performance to deteriorate greatly and the model availability to be poor. -* Adaptive Gaussian optimizer adaptively adjusts the standard deviation to adjust the Gaussian distribution noise. In the initial phase of model training, a large amount of noise is added. As the model gradually converges, the noise amount gradually decreases, and the impact of the noise on the model availability is reduced. A disadvantage of the adaptive Gaussian noise is that a differential privacy budget cannot be strictly controlled. -* Adaptive clipping optimizer is a DP optimizer that adaptively adjusts a clipping granularity. Gradient clipping is an important operation in differential privacy training. The adaptive clipping optimizer can control a ratio of gradient clipping to fluctuate within a given range and control the gradient clipping granularity during training steps. +- Constant Gaussian optimizer is a DP optimizer for non-adaptive Gaussian noise. The advantage is that the differential privacy budget ϵ can be strictly controlled. The disadvantage is that in the model training process, the noise amount added in each step is fixed. If the number of training steps is too large, the noise in the later phase of training makes the model convergence difficult, or even causes the performance to deteriorate greatly and the model availability to be poor. +- Adaptive Gaussian optimizer adaptively adjusts the standard deviation to adjust the Gaussian distribution noise. In the initial phase of model training, a large amount of noise is added. As the model gradually converges, the noise amount gradually decreases, and the impact of the noise on the model availability is reduced. A disadvantage of the adaptive Gaussian noise is that a differential privacy budget cannot be strictly controlled. +- Adaptive clipping optimizer is a DP optimizer that adaptively adjusts a clipping granularity. Gradient clipping is an important operation in differential privacy training. The adaptive clipping optimizer can control a ratio of gradient clipping to fluctuate within a given range and control the gradient clipping granularity during training steps. ### DP Mechanisms @@ -41,26 +40,24 @@ The noise mechanism is a basis for building a differential privacy training capa ### Monitor -Monitor provides callback functions such as Rényi differential privacy (RDP) and zero-concentrated differential privacy (ZCDP) to monitor the differential privacy budget of the model. +Monitor provides callback functions such as Rényi differential privacy (RDP) and zero-concentrated differential privacy (ZCDP) to monitor the differential privacy budget of the model. -* ZCDP[2] +- ZCDP[2] ZCDP is a loose differential privacy definition. It uses the Rényi divergence to measure the distribution difference of random functions on adjacent datasets. -* RDP[3] +- RDP[3] RDP is a more general differential privacy definition based on the Rényi divergence. It uses the Rényi divergence to measure the distribution difference between two adjacent datasets. - Compared with traditional differential privacy, ZCDP and RDP provide stricter privacy budget upper bound guarantee. - ## Code Implementation -* [mechanisms.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/mechanisms/mechanisms.py): implements the noise generation mechanism required by differential privacy training, including simple Gaussian noise, adaptive Gaussian noise, and adaptive clipping Gaussian noise. -* [optimizer.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/optimizer/optimizer.py): implements the fundamental logic of using the noise generation mechanism to add noise during backward propagation. -* [monitor.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/monitor/monitor.py): implements the callback function for computing the differential privacy budget. During model training, the current differential privacy budget is returned. -* [model.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/train/model.py): implements the logic of computing the loss and gradient as well as the gradient truncation logic of differential privacy training, which is the entry for users to use the differential privacy training capability. +- [mechanisms.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/mechanisms/mechanisms.py): implements the noise generation mechanism required by differential privacy training, including simple Gaussian noise, adaptive Gaussian noise, and adaptive clipping Gaussian noise. +- [optimizer.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/optimizer/optimizer.py): implements the fundamental logic of using the noise generation mechanism to add noise during backward propagation. +- [monitor.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/monitor/monitor.py): implements the callback function for computing the differential privacy budget. During model training, the current differential privacy budget is returned. +- [model.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/train/model.py): implements the logic of computing the loss and gradient as well as the gradient truncation logic of differential privacy training, which is the entry for users to use the differential privacy training capability. ## References diff --git a/docs/note/source_en/design/mindarmour/fuzzer_design.md b/docs/note/source_en/design/mindarmour/fuzzer_design.md index dbb4feeb0a..34cfc563dc 100644 --- a/docs/note/source_en/design/mindarmour/fuzzer_design.md +++ b/docs/note/source_en/design/mindarmour/fuzzer_design.md @@ -2,7 +2,6 @@ `Linux` `Ascend` `GPU` `CPU` `Data Preparation` `Model Development` `Model Training` `Model Optimization` `Enterprise` `Expert` - - [AI Model Security Testing](#ai-model-security-testing) @@ -71,4 +70,4 @@ Through multiple rounds of mutations, you can obtain a series of variant data in [1] Pei K, Cao Y, Yang J, et al. Deepxplore: Automated whitebox testing of deep learning systems[C]//Proceedings of the 26th Symposium on Operating Systems Principles. ACM, 2017: 1-18. -[2] Ma L, Juefei-Xu F, Zhang F, et al. Deepgauge: Multi-granularity testing criteria for deep learning systems[C]//Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. ACM, 2018: 120-131. \ No newline at end of file +[2] Ma L, Juefei-Xu F, Zhang F, et al. Deepgauge: Multi-granularity testing criteria for deep learning systems[C]//Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. ACM, 2018: 120-131. diff --git a/docs/note/source_en/design/mindinsight/graph_visual_design.md b/docs/note/source_en/design/mindinsight/graph_visual_design.md index a0e0c8918f..8633d64951 100644 --- a/docs/note/source_en/design/mindinsight/graph_visual_design.md +++ b/docs/note/source_en/design/mindinsight/graph_visual_design.md @@ -21,9 +21,9 @@ The computational graph visualization function is mainly used in the following scenarios: - - View a data flow direction of operators and a model structure when programming a deep learning neural network. - - View the input and output nodes of a specified node and attributes of a queried node. - - Trace data, including data dimension and type changes when debugging a network. +- View a data flow direction of operators and a model structure when programming a deep learning neural network. +- View the input and output nodes of a specified node and attributes of a queried node. +- Trace data, including data dimension and type changes when debugging a network. ## Overall Design @@ -71,4 +71,4 @@ RESTful API is used for data interaction between the MindInsight frontend and ba #### File API Design Data interaction between MindSpore and MindInsight uses the data format defined by [Protocol Buffer](https://developers.google.cn/protocol-buffers/docs/pythontutorial). -The main entry is the [summary.proto file](https://gitee.com/mindspore/mindinsight/blob/master/mindinsight/datavisual/proto_files/mindinsight_summary.proto). A message object of a computational graph is defined as `GraphProto`. For details about `GraphProto`, see the [anf_ir.proto file](https://gitee.com/mindspore/mindinsight/blob/master/mindinsight/datavisual/proto_files/mindinsight_anf_ir.proto). \ No newline at end of file +The main entry is the [summary.proto file](https://gitee.com/mindspore/mindinsight/blob/master/mindinsight/datavisual/proto_files/mindinsight_summary.proto). A message object of a computational graph is defined as `GraphProto`. For details about `GraphProto`, see the [anf_ir.proto file](https://gitee.com/mindspore/mindinsight/blob/master/mindinsight/datavisual/proto_files/mindinsight_anf_ir.proto). diff --git a/docs/note/source_en/design/mindinsight/tensor_visual_design.md b/docs/note/source_en/design/mindinsight/tensor_visual_design.md index f21b670f1f..86a364148c 100644 --- a/docs/note/source_en/design/mindinsight/tensor_visual_design.md +++ b/docs/note/source_en/design/mindinsight/tensor_visual_design.md @@ -60,7 +60,8 @@ In tensor visualization, there are file API and RESTful API. The file API is the #### File API Design The `summary.proto` file is the main entry. TensorProto data is stored in the summary value, as shown in the following: -``` + +```cpp { message Summary { message Image { @@ -69,7 +70,7 @@ The `summary.proto` file is the main entry. TensorProto data is stored in the su required int32 width = 2; ... } - + message Histogram { message bucket{ // Counting number of values fallen in [left, left + width). @@ -78,7 +79,7 @@ The `summary.proto` file is the main entry. TensorProto data is stored in the su required double width = 2; required int64 count = 3; } - + repeated bucket buckets = 1; ... } @@ -86,7 +87,7 @@ The `summary.proto` file is the main entry. TensorProto data is stored in the su message Value { // Tag name for the data. required string tag = 1; - + // Value associated with the tag. oneof value { float scalar_value = 3; @@ -100,4 +101,5 @@ The `summary.proto` file is the main entry. TensorProto data is stored in the su repeated Value value = 1; } ``` -TensorProto is defined in the [anf_ir.proto](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/utils/anf_ir.proto) file. \ No newline at end of file + +TensorProto is defined in the [anf_ir.proto](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/utils/anf_ir.proto) file. diff --git a/docs/note/source_en/design/mindinsight/training_visual_design.md b/docs/note/source_en/design/mindinsight/training_visual_design.md index 86451518c8..05cadfe220 100644 --- a/docs/note/source_en/design/mindinsight/training_visual_design.md +++ b/docs/note/source_en/design/mindinsight/training_visual_design.md @@ -5,16 +5,16 @@ - [Overall Design of Training Visualization](#overall-design-of-training-visualization) - - [Logical Architecture of Training Visualization](#logical-architecture-of-training-visualization) - - [Architecture of Training Information Collection](#architecture-of-training-information-collection) - - [Architecture of Training Information Analysis and Display](#architecture-of-training-information-analysis-and-display) - - [Code Organization](#code-organization) - - [Training Visualization Data Model](#training-visualization-data-model) - - [Training Information Data Flow](#training-information-data-flow) - - [Data Model](#data-model) - - [Training Job](#training-job) - - [Lineage Data](#lineage-data) - - [Training Process Data](#training-process-data) + - [Logical Architecture of Training Visualization](#logical-architecture-of-training-visualization) + - [Architecture of Training Information Collection](#architecture-of-training-information-collection) + - [Architecture of Training Information Analysis and Display](#architecture-of-training-information-analysis-and-display) + - [Code Organization](#code-organization) + - [Training Visualization Data Model](#training-visualization-data-model) + - [Training Information Data Flow](#training-information-data-flow) + - [Data Model](#data-model) + - [Training Job](#training-job) + - [Lineage Data](#lineage-data) + - [Training Process Data](#training-process-data) @@ -105,7 +105,7 @@ MindInsight uses directories to distinguish different training jobs. To distingu In MindInsight code, a training job is called a TrainJob. A TrainJob ID is the name of the directory where the training log is located, for example, ./train_my_lenet_1. -During a training process, a lineage data file (whose name ends with _lineage) and a training process data file (whose name ends with _MS) are generated. The lineage data mainly describes an invariant attribute of the training from a global perspective, for example, a dataset path used for training, an optimizer used for training, and user-defined lineage information. The most prominent feature of the lineage data file is that it does not change during the training process. The training process data mainly describes a change status of the training, for example, a loss value, parameter distribution, and image data sent to the model in a step. The most prominent feature of the training process data file is that each step changes. +During a training process, a lineage data file (whose name ends with _lineage) and a training process data file (whose name ends with_MS) are generated. The lineage data mainly describes an invariant attribute of the training from a global perspective, for example, a dataset path used for training, an optimizer used for training, and user-defined lineage information. The most prominent feature of the lineage data file is that it does not change during the training process. The training process data mainly describes a change status of the training, for example, a loss value, parameter distribution, and image data sent to the model in a step. The most prominent feature of the training process data file is that each step changes. It should be noted that the classification about whether the training information changes is not absolute. For example, the training process data file contains computational graph data, which is determined when the training starts. @@ -129,4 +129,4 @@ The lineage data describes the invariant attribute of a training from a global p - Data Query and Display - When displaying data, you might want to see how the data under a tag changes with the training process. Therefore, when querying data, you do not need to specify the step number. Instead, you can specify the training job, plugin name, and tag to query data of all steps under the tag. \ No newline at end of file + When displaying data, you might want to see how the data under a tag changes with the training process. Therefore, when querying data, you do not need to specify the step number. Instead, you can specify the training job, plugin name, and tag to query data of all steps under the tag. diff --git a/docs/note/source_en/design/mindspore/architecture_lite.md b/docs/note/source_en/design/mindspore/architecture_lite.md index d78fc89147..05c69cede3 100644 --- a/docs/note/source_en/design/mindspore/architecture_lite.md +++ b/docs/note/source_en/design/mindspore/architecture_lite.md @@ -1,7 +1,7 @@ # Overall Architecture (Lite) `Linux` `Windows` `On Device` `Inference Application` `Intermediate` `Expert` `Contributor` - + The overall architecture of MindSpore Lite is as follows: @@ -14,8 +14,8 @@ The overall architecture of MindSpore Lite is as follows: - **Backend:** optimizes graphs based on IR, including graph high level optimization (GHLO), graph low level optimization (GLLO), and quantization. GHLO is responsible for hardware-independent optimization, such as operator fusion and constant folding. GLLO is responsible for hardware-related optimization. Quantizer supports quantization methods after training, such as weight quantization and activation value quantization. -- **Runtime:** inference runtime of intelligent devices. Sessions are responsible for session management and provide external APIs. The thread pool and parallel primitives are responsible for managing the thread pool used for graph execution. Memory allocation is responsible for memory overcommitment of each operator during graph execution. The operator library provides the CPU and GPU operators. +- **Runtime:** inference runtime of intelligent devices. Sessions are responsible for session management and provide external APIs. The thread pool and parallel primitives are responsible for managing the thread pool used for graph execution. Memory allocation is responsible for memory overcommitment of each operator during graph execution. The operator library provides the CPU and GPU operators. - **Micro:** runtime of IoT devices, including the model generation .c file, thread pool, memory overcommitment, and operator library. -Runtime and Micro share the underlying infrastructure layers, such as the operator library, memory allocation, thread pool, and parallel primitives. +Runtime and Micro share the underlying infrastructure layers, such as the operator library, memory allocation, thread pool, and parallel primitives. diff --git a/docs/note/source_en/design/mindspore/distributed_training_design.md b/docs/note/source_en/design/mindspore/distributed_training_design.md index 79de486296..cf963d8a9f 100644 --- a/docs/note/source_en/design/mindspore/distributed_training_design.md +++ b/docs/note/source_en/design/mindspore/distributed_training_design.md @@ -24,7 +24,6 @@ With the rapid development of deep learning, the number of datasets and parameters are growing exponentially to improve the accuracy and generalization capability of neural networks. Parallel distributed training has become a development trend to resolve the performance bottleneck of ultra-large scale networks. MindSpore supports the mainstream distributed training paradigm and develops an automatic hybrid parallel solution. The following describes the design principles of several parallel training modes and provides guidance for users to perform custom development. - ## Concepts ### Collective Communication @@ -74,7 +73,6 @@ This section describes how the data parallel mode `ParallelMode.DATA_PARALLEL` w - [grad_reducer.py](https://gitee.com/mindspore/mindspore/blob/master/mindspore/nn/wrap/grad_reducer.py): This file implements the gradient aggregation process. After the input parameter `grads` is expanded by using `HyperMap`, the `AllReduce` operator is inserted. The global communication group is used. You can also perform custom development by referring to this section based on your network requirements. In MindSpore, standalone and distributed execution shares a set of network encapsulation APIs. In the `Cell`, `ParallelMode` is used to determine whether to perform gradient aggregation. For details about the network encapsulation APIs, see the `TrainOneStepCell` code implementation. - ## Automatic Parallelism As a key feature of MindSpore, automatic parallelism is used to implement hybrid parallel training that combines automatic data parallelism and model parallelism. It aims to help users express the parallel algorithm logic using standalone scripts, reduce the difficulty of distributed training, improve the algorithm R&D efficiency, and maintain the high performance of training. This section describes how the automatic parallel mode `ParallelMode.AUTO_PARALLEL` and semi-automatic parallel mode `ParallelMode.SEMI_AUTO_PARALLEL` work in MindSpore. @@ -86,19 +84,19 @@ As a key feature of MindSpore, automatic parallelism is used to implement hybrid 1. Distributed operator and tensor layout As shown in the preceding figure, the automatic parallel process traverses the standalone forward ANF graphs and performs shard modeling on tensors in the unit of distributed operator, indicating how the input and output tensors of an operator are distributed to each device of the cluster, that is, the tensor layout. Users do not need to know which device runs which slice of a model. The framework automatically schedules and allocates model slices. - + To obtain the tensor layout model, each operator has a shard strategy, which indicates the shard status of each input of the operator in the corresponding dimension. Generally, tensors can be sharded in any dimension as long as the value is a multiple of 2, and the even distribution principle is met. The following figure shows an example of the three-dimensional `BatchMatmul` operation. The parallel strategy consists of two tuples, indicating the sharding of `input` and `weight`, respectively. Elements in a tuple correspond to tensor dimensions one by one. `2^N` indicates the shard unit, and `1` indicates that the tuple is not sharded. If you want to express a parallel data shard strategy, that is, only data in the `batch` dimension of `input` is sharded, and data in other dimensions are not sharded, you can use `strategy=((2^N, 1, 1),(1, 1, 1))`. If you want to express a parallel model shard strategy, that is, only model in the non-`batch` dimension of `weight` is sharded, for example, only the `channel` dimension is sharded, you can use `strategy=((1, 1, 1),(1, 1, 2^N))`. If you want to express a hybrid parallel shard strategy, one of which is `strategy=((2^N, 1, 1),(1, 1, 2^N))`. ![Operator Sharding Definition](./images/operator_split.png) - - Based on the shard strategy of an operator, the framework automatically derives the distribution model of input tensors and output tensors of the operator. This distribution model consists of `device_matrix`, `tensor_shape`, and `tensor map`, which indicate the device matrix shape, tensor shape, and mapping between devices and tensor dimensions, respectively. Based on the tensor layout model, distributed operator determines whether to insert extra computation and communication operations in the graph to ensure that the operator computing logic is correct. + + Based on the shard strategy of an operator, the framework automatically derives the distribution model of input tensors and output tensors of the operator. This distribution model consists of `device_matrix`, `tensor_shape`, and `tensor map`, which indicate the device matrix shape, tensor shape, and mapping between devices and tensor dimensions, respectively. Based on the tensor layout model, distributed operator determines whether to insert extra computation and communication operations in the graph to ensure that the operator computing logic is correct. 2. Tensor Redistribution When the output tensor model of an operator is inconsistent with the input tensor model of the next operator, computation and communication operations need to be introduced to implement the change between tensor layouts. The automatic parallel process introduces the tensor redistribution algorithm, which can be used to derive the communication conversion operations between random tensor layouts. The following three examples represent a parallel computing process of the formula `Z=(X×W)×V`, that is, a `MatMul` operation of two two-dimensional matrices, and show how to perform conversion between different parallel modes. - + In example 1, the output of the first data parallel matrix multiplication is sharded in the row rection, and the input of the second model parallel matrix multiplication requires full tensors. The framework automatically inserts the `AllGather` operator to implement redistribution. - + ![Tensor Redistribution](./images/tensor_redistribution1.png) In example 2, the output of parallel matrix multiplication of the first model is sharded in the column direction, and the input of parallel matrix multiplication of the second model is sharded in the row direction. The framework automatically inserts a communication operator equivalent to the `AlltoAll` operation in collective communication to implement redistribution. @@ -114,9 +112,8 @@ As a key feature of MindSpore, automatic parallelism is used to implement hybrid 3. Efficient parallel strategy search algorithm The `SEMI_AUTO_PARALLEL` semi-automatic parallel mode indicates that you manually configure the parallel strategy for operators when you are familiar with the operator sharding representation. This mode is helpful for manual optimization, with certain commissioning difficulty. You need to master the parallel principle and obtain a high-performance parallel solution based on the network structure and cluster topology. To further help users accelerate the parallel network training process, the automatic parallel mode `AUTO_PARALLEL` introduces the automatic search feature of the parallel strategy on the basis of the semi-automatic parallel mode. Automatic parallelism builds cost models based on the hardware platform, and calculates the computation cost, memory cost, and communication cost of a certain amount of data and specific operators based on different parallel strategies Then, by using the dynamic programming algorithm or recursive programming algorithm and taking the memory upper limit of a single device as a constraint condition, a parallel strategy with optimal performance is efficiently searched out. - - Strategy search replaces manual model sharding and provides a high-performance sharding solution within a short period of time, greatly reducing the threshold for parallel training. + Strategy search replaces manual model sharding and provides a high-performance sharding solution within a short period of time, greatly reducing the threshold for parallel training. 4. Convenient distributed automatic differentiation @@ -139,6 +136,5 @@ As a key feature of MindSpore, automatic parallelism is used to implement hybrid 5. Entire graph sharding - [step_auto_parallel.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/frontend/parallel/step_auto_parallel.h), and [step_parallel.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/frontend/parallel/step_parallel.h): The two files contain the core implementation of the automatic parallel process. `step_auto_parallel.h` calls the strategy search process and generates the `OperatorInfo` of the distributed operator. Then in `step_parallel.h`, processes such as operator sharding and tensor redistribution are processed to reconstruct the standalone computing graph in distributed mode. - 6. Backward propagation of communication operators - [grad_comm_ops.py](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ops/_grad/grad_comm_ops.py): This file defines the backward propagation of communication operators, such as `AllReduce` and `AllGather`. diff --git a/docs/note/source_en/design/mindspore/mindir.md b/docs/note/source_en/design/mindspore/mindir.md index 7d3ba3385c..59f55e3195 100644 --- a/docs/note/source_en/design/mindspore/mindir.md +++ b/docs/note/source_en/design/mindspore/mindir.md @@ -21,13 +21,16 @@ ## Overview + An intermediate representation (IR) is a representation of a program between the source and target languages, which facilitates program analysis and optimization for the compiler. Therefore, the IR design needs to consider the difficulty in converting the source language to the target language, as well as the ease-of-use and performance of program analysis and optimization. MindSpore IR (MindIR) is a function-style IR based on graph representation. Its core purpose is to serve automatic differential transformation. Automatic differentiation uses the transformation method based on the function-style programming framework. Therefore, IR uses the semantics close to that of the ANF function. In addition, a manner of representation based on an explicit dependency graph is used by referring to excellent designs of Sea of Nodes[1] and Thorin[2]. ## Syntax + ANF is a simple IR commonly used during functional programming. The ANF syntax is defined as follows: -``` + +```python ::= NUMBER | STRING | VAR | BOOLEAN | PRIMOP | (lambda (VAR …) ) ::= ( …) @@ -35,17 +38,20 @@ ANF is a simple IR commonly used during functional programming. The ANF syntax i ::= (let ([VAR ]) ) | | ``` + Expressions in the ANF are classified into atomic expressions (aexp) and compound expressions (cexp). An atomic expression indicates a constant value, a variable, or an anonymous function. A compound expression consists of multiple atomic expressions, indicating that an anonymous function or primitive function call. The first input expression of a compound expression is the called function, and the other input expressions are the called parameters. The syntax of MindIR is inherited from the ANF and is defined as follows: -``` + +```python ::= | ::= Parameter ::= Scalar | Named | Tensor | Type | Shape - | Primitive | MetaFuncGraph | FuncGraph + | Primitive | MetaFuncGraph | FuncGraph ::= ( …) ::= | ``` + ANode in a MindIR corresponds to the atomic expression of ANF. ANode has two subclasses: ValueNode and ParameterNode. ValueNode refers to a constant node, which can carry a constant value (such as a scalar, symbol, tensor, type, and dimension), a primitive function (Primitive), a metafunction (MetaFuncGraph), or a common function (FuncGraph). In functional programming, the function definition itself is a value. ParameterNode refers to a parameter node, which indicates the formal parameter of a function. CNode in a MindIR corresponds to the compound expression of ANF, indicating a function call. @@ -53,7 +59,9 @@ CNode in a MindIR corresponds to the compound expression of ANF, indicating a fu During automatic differentiation of MindSpore, the gradient contribution of ParameterNode and CNode are calculated, and the final gradient of ParameterNode is returned. The gradient of ValueNode is not calculated. ## Example + The following uses a program code segment as an example to help you understand MindIR. + ```python def func(x, y): return x / y @@ -65,8 +73,10 @@ def test_f(x, y): c = b * func(a, b) return c ``` + The ANF corresponding to the Python code is as follows: -``` + +```python lambda (x, y) let a = x - 1 in let b = a + y in @@ -77,26 +87,31 @@ lambda (x, y) let c = b * %1 in c end ``` + The corresponding MindIR is [ir.dot](https://gitee.com/mindspore/docs/blob/master/docs/note/source_en/design/mindspore/images/ir/ir.dot). -![](./images/ir/ir.png) +![image](./images/ir/ir.png) In a MindIR, a function graph (FuncGraph) indicates the definition of a common function. A directed acyclic graph (DAG) usually consists of ParameterNode, ValueNode, and CNode, which clearly shows the calculation process from parameters to return values. As shown in the preceding figure, the `test_f` and `func` functions in the Python code are converted into two function graphs. The `x` and `y` parameters are converted into ParameterNode in the function graphs, and each expression is converted into a CNode. The first input of CNode links to the called functions, for example, `add`, `func`, and `return` in the figure. It should be noted that these nodes are all `ValueNode` because they are considered as constant function values. Other input of CNode links to the called parameters. The parameter values can be obtained from the ParameterNode, ValueNode, and other CNode. In the ANF, each expression is bound as a variable by using the let expression, and the dependency on the expression output is represented by referencing the variable. In the MindIR, each expression is bound as a node, and the dependency is represented by using the directed edges between nodes. ## Saving IR + `context.set_context(save_graphs=True)` is used to save the intermediate code in each compilation phase. The intermediate code can be saved in two formats. One is the text format with the suffix `.ir`, and the other is the graphical format with the suffix `.dot`. When the network scale is small, you are advised to use the graphical format that is more intuitive. When the network scale is large, you are advised to use the text format that is more efficient. You can run the graphviz command to convert a .dot file to the picture format. For example, you can run the `dot -Tpng *.dot -o *.png` command to convert a .dot file to a .png file. ## Function-style Semantics + Compared with traditional computational graphs, MindIR can not only express data dependency between operators, but also express rich function-style semantics. + ### Higher-Order Functions + In a MindIR, a function is defined by a subgraph. However, the function itself can be transferred as the input or output of other higher-order functions. In the following simple example, the `f` function is transferred as a parameter into the `g` function. Therefore, the `g` function is a higher-order function that receives function input, and the actual call site of the `f` function is inside the `g` function. -``` +```python @ms_function def hof(x): def f(x): @@ -108,14 +123,16 @@ def hof(x): ``` The corresponding MindIR is [hof.dot](https://gitee.com/mindspore/docs/blob/master/docs/note/source_en/design/mindspore/images/ir/hof.dot). -![](./images/ir/hof.png) +![image](./images/ir/hof.png) In the actual network training scripts, the automatic derivation generic function `GradOperation` and `Partial` and `HyperMap` that are commonly used in the optimizer are typical high-order functions. Higher-order semantics greatly improve the flexibility and simplicity of MindSpore representations. ### Control Flows + In a MindIR, control flows are expressed in the form of high-order function selection and calling. This form transforms a control flow into a data flow of higher-order functions, making the automatic differential algorithm more powerful. It not only supports automatic differentiation of data flows, but also supports automatic differentiation of control flows such as conditional jumps, loops, and recursion. The following uses a simple Fibonacci instance as an example. + ```python @ms_function def fibonacci(n): @@ -128,15 +145,16 @@ def fibonacci(n): ``` The corresponding MindIR is [cf.dot](https://gitee.com/mindspore/docs/blob/master/docs/note/source_en/design/mindspore/images/ir/cf.dot). -![](./images/ir/cf.png) +![image](./images/ir/cf.png) `fibonacci` is a top-level function graph. Two function graphs at the top level are selected and called by `switch`. `✓fibonacci` is the True branch of the first `if`, and `✗fibonacci` is the False branch of the first `if`. `✓✗fibonacci` called in `✗fibonacci` is the True branch of `elif`, and `✗✗fibonacci` is the False branch of `elif`. The key is, in a MindIR, conditional jumps and recursion are represented in the form of higher-order control flows. For example, `✓✗fibonacci` and `✗fibonacci` are transferred in as parameters of the `switch` operator. `switch` selects a function as the return value based on the condition parameter. In this way, `switch` performs a binary selection operation on the input functions as common values and does not call the functions. The real function call is completed on CNode following `switch`. - ### Free Variables and Closures + Closure is a programming language feature that refers to the combination of code blocks and scope environment. A free variable refers to a variable in the scope environment referenced in a code block instead of a local variable. In a MindIR, a code block is represented as a function graph. The scope environment can be considered as the context where the function is called. The capture method of free variables is value copy instead of reference. A typical closure instance is as follows: + ```python @ms_function def func_outer(a, b): @@ -153,14 +171,15 @@ def ms_closure(): ``` The corresponding MindIR is [closure.dot](https://gitee.com/mindspore/docs/blob/master/docs/note/source_en/design/mindspore/images/ir/closure.dot). -![](./images/ir/closure.png) +![image](./images/ir/closure.png) In the example, `a` and `b` are free variables because the variables `a` and `b` in `func_inner` are parameters defined in the referenced parent graph `func_outer`. The variable `closure` is a closure, which is the combination of the function `func_inner` and its context `func_outer(1, 2)`. Therefore, the result of `out1` is 4, which is equivalent to `1+2+1`, and the result of `out2` is 5, which is equivalent to `1+2+2`. ## References + [1] C. Click and M. Paleczny. A simple graph-based intermediate representation. SIGPLAN Not., 30:35–49, March 1995. [2] Roland Leißa, Marcel Köster, and Sebastian Hack. A graph-based higher-order intermediate representation. In Proceedings of the 13th Annual IEEE/ACM International Symposium on -Code Generation and Optimization, pages 202–212. IEEE Computer Society, 2015. \ No newline at end of file +Code Generation and Optimization, pages 202–212. IEEE Computer Society, 2015. diff --git a/docs/note/source_en/design/mindspore/profiler_design.md b/docs/note/source_en/design/mindspore/profiler_design.md index 9acb50afb7..e8dd67c9ff 100644 --- a/docs/note/source_en/design/mindspore/profiler_design.md +++ b/docs/note/source_en/design/mindspore/profiler_design.md @@ -33,6 +33,7 @@ To support model development and performance debugging in MindSpore, an easy-to-use profile tool is required to intuitively display the performance information of each dimension of a network model, provide users with easy-to-use and abundant profiling functions, and help users quickly locate network performance faults. ## Profiler Architecture Design + The Profiler architecture design is introduced from the following three aspects: the overall context interaction relationship of Profiler; the internal structure of Profiler, including the module structure and module layers; the interactive calling relationship between modules. ### Context @@ -50,6 +51,7 @@ As shown in the preceding figure, the interaction between the Profiler and other 2. MindSpore Profiler parses the original data in the user script and generates the intermediate data results in the specified folder. 3. MindInsight Profiler connects to the intermediate data and provides the visualized Profiler function for users. + ### Module Structure Modules are classified into the following layers: @@ -58,8 +60,8 @@ Modules are classified into the following layers: Figure 2 Relationships between modules at different layers - Module functions are as follows: + 1. ProfilerAPI is a calling entry provided by code, including the performance collection startup API and analysis API. 2. Controller is a module at a layer lower than that of ProfilerAPI. It is called by the startup API of ProfilerAPI to start or stop the performance collection function. The original data is written to a fixed position by ada. 3. Parser is a module for parsing original performance data which is collected on the device and cannot be directly understood by users. Parser parses, combines, and converts the data to generate intermediate results that can be understood by users and analyzed by upper layers. @@ -67,6 +69,7 @@ Module functions are as follows: 5. RESTful is used to call the common API provided by the backend Analyser to obtain objective data and use RESTful to connect to the frontend. ### Internal Module Interaction + Users can use API or RESTful to complete internal module interaction process. The following uses the API as an example: ![time_order_profiler.png](./images/time_order_profiler.png) @@ -82,20 +85,22 @@ The interaction process of each module is as follows: 3. Profiler API analysis API uses the Parser module to parse performance data, generates intermediate results, calls the Aalayser module to analyze the results, and returns various information to users. ## Sub-Module Design + ### ProfilerAPI and Controller #### Description + ProfilerAPI provides an entry API in the training script for users to start performance collection and analyze performance data. ProfilerAPI delivers commands through Controller to control the startup of ada. #### Design + ProfilerAPI belongs to the API layer of upper-layer application and is integrated by the training script. The function is divided into two parts: - Before training, call the bottom-layer Controller API to deliver a command to start a profiling task. - After training, call the bottom-layer Controller API to deliver commands to stop the profiling task, call the Analyser and Parser APIs to parse data files and generate result data such as operator performance statistics and training trace statistics. - Controller provides an API for the upper layer, calls API of the lower-layer performance collection module, and delivers commands for starting and stopping performance collection. The generated original performance data includes: @@ -106,9 +111,13 @@ The generated original performance data includes: - `training_trace.46.dev.profiler_default_tag` file: stores the start and end time of each step and time of step interval, forward and backward propagation, and step tail. ### Parser + #### Description + Parser is a module for parsing original performance data which is collected on the device and cannot be directly understood by users. Parser parses, combines, and converts the data to generate intermediate results that can be understood by users and analyzed by upper layers. + #### Design + ![parser_module_profiler.png](./images/parser_module_profiler.png) Figure 4 Parser module @@ -123,6 +132,7 @@ As shown in the preceding figure, there are HWTS Parser, AI CPU Parser, Framewor ### Analyser #### Description + Analyzer is used to filter, sort, query, and page the intermediate results generated at the parsing stage. #### Design @@ -142,9 +152,10 @@ Currently, there are two types of analyzers for operator information: To hide the internal implementation of Analyser and facilitate calling, the simple factory mode is used to obtain the specified Analyser through AnalyserFactory. - ### Proposer + #### Description + Proposer is a Profiler performance optimization suggestion module. Proposer calls the Analyser module to obtain performance data, analyzes the performance data based on optimization rules, and displays optimization suggestions for users through the UI and API. #### Design @@ -172,4 +183,4 @@ Figure 7 Proposer class As shown in the preceding figure: - Proposers of various types inherit the abstract class Proposer and implement the analyze methods. -- API and CLI call the ProposerFactory to obtain the Proposer and call the Proposer.analyze function to obtain the optimization suggestions of each type of Proposer. \ No newline at end of file +- API and CLI call the ProposerFactory to obtain the Proposer and call the Proposer.analyze function to obtain the optimization suggestions of each type of Proposer. diff --git a/docs/note/source_en/glossary.md b/docs/note/source_en/glossary.md index 852afab1c8..6ec11c85d3 100644 --- a/docs/note/source_en/glossary.md +++ b/docs/note/source_en/glossary.md @@ -4,7 +4,7 @@ -| Acronym and Abbreviation | Description | +| Acronym and Abbreviation | Description | | ----- | ----- | | ACL | Ascend Computer Language, for users to develop deep neural network applications, which provides the C++ API library including device management, context management, stream management, memory management, model loading and execution, operator loading and execution, media data processing, etc. | | Ascend | Name of Huawei Ascend series chips. | diff --git a/docs/note/source_en/help_seeking_path.md b/docs/note/source_en/help_seeking_path.md index ecaa964ae6..9ac8c6bb6d 100644 --- a/docs/note/source_en/help_seeking_path.md +++ b/docs/note/source_en/help_seeking_path.md @@ -10,22 +10,20 @@ This document describes how to seek help and support when you encounter problems - Website search - - Go to the [official search page](https://www.mindspore.cn/search/en). - - When encountering a problem, search on the official website first, which is simple and efficient. - - Enter a keyword in the search box and click the search icon. The related content is displayed. - - Resolve the problem based on the search result. - + - Go to the [official search page](https://www.mindspore.cn/search/en). + - When encountering a problem, search on the official website first, which is simple and efficient. + - Enter a keyword in the search box and click the search icon. The related content is displayed. + - Resolve the problem based on the search result. - User group consultation - - If you cannot solve the problem using the website search method and want a quick consultation. Get support by joining the [Slack group](https://mindspore.slack.com/join/shared_invite/zt-dgk65rli-3ex4xvS4wHX7UDmsQmfu8w#/ ) and start a conversation with our members. - - Resolve the problem by asking experts or communicating with other users. - + - If you cannot solve the problem using the website search method and want a quick consultation. Get support by joining the [Slack group](https://mindspore.slack.com/join/shared_invite/zt-dgk65rli-3ex4xvS4wHX7UDmsQmfu8w#/ ) and start a conversation with our members. + - Resolve the problem by asking experts or communicating with other users. - Forum Help-Seeking - - If you want a detailed solution, start a help post on the [Ascend forum](https://forum.huawei.com/enterprise/en/forum-100504.html). - - After the post is sent, a forum moderator collects the question and contacts technical experts to answer the question. The question will be resolved within three working days. - - Resolve the problem by referring to solutions provided by technical experts. + - If you want a detailed solution, start a help post on the [Ascend forum](https://forum.huawei.com/enterprise/en/forum-100504.html). + - After the post is sent, a forum moderator collects the question and contacts technical experts to answer the question. The question will be resolved within three working days. + - Resolve the problem by referring to solutions provided by technical experts. - If the expert test result shows that the MindSpore function needs to be improved, you are advised to submit an issue in the [MindSpore repository](https://gitee.com/mindspore). Issues will be resolved in later versions. \ No newline at end of file + If the expert test result shows that the MindSpore function needs to be improved, you are advised to submit an issue in the [MindSpore repository](https://gitee.com/mindspore). Issues will be resolved in later versions. diff --git a/docs/note/source_en/network_list_ms.md b/docs/note/source_en/network_list_ms.md index 8cbf3ba980..169d2137cc 100644 --- a/docs/note/source_en/network_list_ms.md +++ b/docs/note/source_en/network_list_ms.md @@ -13,7 +13,7 @@ ## Model Zoo -| Domain | Sub Domain | Network | Ascend(Graph) | Ascend(PyNative) | GPU(Graph) | GPU(PyNative)| CPU(Graph) +| Domain | Sub Domain | Network | Ascend(Graph) | Ascend(PyNative) | GPU(Graph) | GPU(PyNative)| CPU(Graph) |:------ |:------| :----------- |:------ |:------ |:------ |:------ |:----- |Computer Vision (CV) | Image Classification | [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/alexnet/src/alexnet.py) | Supported | Supported | Supported | Supported | Doing | Computer Vision (CV) | Image Classification | [GoogleNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/googlenet/src/googlenet.py) | Supported | Supported | Supported | Supported | Doing diff --git a/docs/note/source_en/object_detection_lite.md b/docs/note/source_en/object_detection_lite.md index 6e02865426..10089f2a80 100644 --- a/docs/note/source_en/object_detection_lite.md +++ b/docs/note/source_en/object_detection_lite.md @@ -23,4 +23,3 @@ The following table shows the data of some object detection models using MindSpo | Model name | Size | mAP(IoU=0.50:0.95) | CPU 4 thread delay (ms) | |-----------------------| :----------: | :----------: | :-----------: | | [MobileNetv2-SSD](https://download.mindspore.cn/model_zoo/official/lite/ssd_mobilenetv2_lite/ssd.ms) | 16.7 | 0.22 | 25.4 | - diff --git a/docs/note/source_en/operator_list_implicit.md b/docs/note/source_en/operator_list_implicit.md index 955ea0a2fe..5eb631df80 100644 --- a/docs/note/source_en/operator_list_implicit.md +++ b/docs/note/source_en/operator_list_implicit.md @@ -17,24 +17,26 @@ ## Implicit Type Conversion ### conversion rules -* Scalar and Tensor operations: during operation, the scalar is automatically converted to Tensor, and the data type is consistent with the Tensor data type involved in the operation; + +- Scalar and Tensor operations: during operation, the scalar is automatically converted to Tensor, and the data type is consistent with the Tensor data type involved in the operation; when Tensor is bool data type and the scalar is int or float, both the scalar and Tensor are converted to the Tensor with the data type of int32 or float32; when Tensor is int or uint data type and the scalar is float, both the scalar and Tensor are converted to the Tensor with the data type of float32. -* Tensor operation of different data types: the priority of data type is bool < uint8 < int8 < int16 < int32 < int64 < float16 < float32 - + 本文介绍MindSpore的基准性能。MindSpore网络定义可参考[Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/model_zoo)。 @@ -32,7 +32,7 @@ ### BERT -| Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | +| Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | BERT-Large | Attention | zhwiki | 0.5.0-beta | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 96 | 269 sentences/sec | - | | | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 96 | 2069 sentences/sec | 0.96 | @@ -45,7 +45,7 @@ | Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Wide & Deep | Recommend | Criteo | 0.6.0-beta | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 16000 | 796892 samples/sec | - | -| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 16000*8 | 4872849 samples/sec | 0.76 | +| | | | | Ascend: 8 *Ascend 910
CPU：192 Cores | Mixed | 16000*8 | 4872849 samples/sec | 0.76 | 1. 以上数据基于Atlas 800测试获得，且网络模型为数据并行。 2. 业界其他开源框架数据可参考：[Wide & Deep For TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/Recommendation/WideAndDeep)。 @@ -55,9 +55,9 @@ | Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Wide & Deep | Recommend | Criteo | 0.6.0-beta | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 8000 | 68715 samples/sec | - | -| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 8000*8 | 283830 samples/sec | 0.51 | -| | | | | Ascend: 16 * Ascend 910
CPU：384 Cores | Mixed | 8000*16 | 377848 samples/sec | 0.34 | -| | | | | Ascend: 32 * Ascend 910
CPU：768 Cores | Mixed | 8000*32 | 433423 samples/sec | 0.20 | +| | | | | Ascend: 8 *Ascend 910
CPU：192 Cores | Mixed | 8000*8 | 283830 samples/sec | 0.51 | +| | | | | Ascend: 16 *Ascend 910
CPU：384 Cores | Mixed | 8000*16 | 377848 samples/sec | 0.34 | +| | | | | Ascend: 32 *Ascend 910
CPU：768 Cores | Mixed | 8000*32 | 433423 samples/sec | 0.20 | 1. 以上数据基于Atlas 800测试获得，且网络模型为模型并行。 2. 业界其他开源框架数据可参考：[Wide & Deep For TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/Recommendation/WideAndDeep)。 -- Gitee