diff --git a/api/Makefile b/docs/api_cpp/Makefile similarity index 100% rename from api/Makefile rename to docs/api_cpp/Makefile diff --git a/api/requirements.txt b/docs/api_cpp/requirements.txt similarity index 100% rename from api/requirements.txt rename to docs/api_cpp/requirements.txt diff --git a/lite/docs/source_en/apicc/apicc.rst b/docs/api_cpp/source_en/apicc.rst similarity index 100% rename from lite/docs/source_en/apicc/apicc.rst rename to docs/api_cpp/source_en/apicc.rst diff --git a/lite/docs/source_en/apicc/class_list.md b/docs/api_cpp/source_en/class_list.md similarity index 100% rename from lite/docs/source_en/apicc/class_list.md rename to docs/api_cpp/source_en/class_list.md diff --git a/lite/docs/source_en/apicc/dataset.md b/docs/api_cpp/source_en/dataset.md similarity index 97% rename from lite/docs/source_en/apicc/dataset.md rename to docs/api_cpp/source_en/dataset.md index 984ffc15eaa2fc44ed5e17c87f89b561083a5eae..4f58dd043666484bfaeebfdea4c11931ae3f099c 100644 --- a/lite/docs/source_en/apicc/dataset.md +++ b/docs/api_cpp/source_en/dataset.md @@ -91,7 +91,7 @@ Normalize image, currently the supports data type is float. Return True or False. ``` -bool Padd(LiteMat &src, LiteMat &dst, const int top, const int bottom, const int left, const int right, const PaddBorderType pad_type, uint8_t fill_r, uint8_t fill_g, uint8_t fill_b) +bool Pad(LiteMat &src, LiteMat &dst, const int top, const int bottom, const int left, const int right, const PaddBorderType pad_type, uint8_t fill_r, uint8_t fill_g, uint8_t fill_b) ``` Padd image, the channel supports is 3 and 1. diff --git a/lite/docs/source_en/apicc/errorcode_and_metatype.md b/docs/api_cpp/source_en/errorcode_and_metatype.md similarity index 100% rename from lite/docs/source_en/apicc/errorcode_and_metatype.md rename to docs/api_cpp/source_en/errorcode_and_metatype.md diff --git a/lite/docs/source_en/apicc/lite.md b/docs/api_cpp/source_en/lite.md similarity index 100% rename from lite/docs/source_en/apicc/lite.md rename to docs/api_cpp/source_en/lite.md diff --git a/lite/docs/source_en/apicc/session.md b/docs/api_cpp/source_en/session.md similarity index 100% rename from lite/docs/source_en/apicc/session.md rename to docs/api_cpp/source_en/session.md diff --git a/lite/docs/source_en/apicc/tensor.md b/docs/api_cpp/source_en/tensor.md similarity index 100% rename from lite/docs/source_en/apicc/tensor.md rename to docs/api_cpp/source_en/tensor.md diff --git a/docs/api_cpp/source_zh_cn/apicc.rst b/docs/api_cpp/source_zh_cn/apicc.rst new file mode 100644 index 0000000000000000000000000000000000000000..82bbf145ca283f9d79be97c93986fcf03d9e2aed --- /dev/null +++ b/docs/api_cpp/source_zh_cn/apicc.rst @@ -0,0 +1,12 @@ +C++ API +======= + +.. toctree:: + :maxdepth: 1 + + class_list + lite + session + tensor + dataset + errorcode_and_metatype \ No newline at end of file diff --git a/docs/api_cpp/source_zh_cn/class_list.md b/docs/api_cpp/source_zh_cn/class_list.md new file mode 100644 index 0000000000000000000000000000000000000000..3eddc864670fdbfda9390e020164edddb676392c --- /dev/null +++ b/docs/api_cpp/source_zh_cn/class_list.md @@ -0,0 +1,15 @@ +# 类列表 + +MindSpore Lite中的类定义及其所属命名空间和描述： + +| 命名空间 | 类 | 描述 | +| --- | --- | --- | +| mindspore::lite | [Allocator](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#allocator) | Allocator定义了一个内存池，用于动态地分配和释放内存。 | +| mindspore::lite | [Context](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#context) | Context用于保存执行期间的环境变量。 | +| mindspore::lite | [ModelImpl](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#modelimpl) | ModelImpl定义了MindSpore Lite中的Model的实现类。 | +| mindspore::lite | [PrimitiveC](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#primitivec) | PrimitiveC定义为算子的原型。 | +| mindspore::lite | [Model](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#model) | Model定义了MindSpore Lite中的模型，便于计算图管理。 | +| mindspore::lite | [ModelBuilder](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#modelbuilder) | ModelBuilder定义了MindSpore Lite中的模型构建器。 | +| mindspore::session | [LiteSession](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/session.html#litesession) | LiteSession定义了MindSpore Lite中的会话，用于进行Model的编译和前向推理。 | +| mindspore::tensor | [MSTensor](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/tensor.html#mstensor) | MSTensor定义了MindSpore Lite中的张量。 | +| mindspore::dataset | [LiteMat](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/dataset.html#litemat) |LiteMat是一个处理图像的类。 | diff --git a/docs/api_cpp/source_zh_cn/dataset.md b/docs/api_cpp/source_zh_cn/dataset.md new file mode 100644 index 0000000000000000000000000000000000000000..379d3e11632327b3075c0f8a56d53c852cdeae80 --- /dev/null +++ b/docs/api_cpp/source_zh_cn/dataset.md @@ -0,0 +1,282 @@ +# mindspore::dataset + +#include <[lite_mat.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/minddata/dataset/kernels/image/lite_cv/lite_mat.h)> +#include <[image_process.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/ccsrc/minddata/dataset/kernels/image/lite_cv/image_process.h)> + + +## image_process.h文件的函数 + +``` +bool ResizeBilinear(LiteMat &src, LiteMat &dst, int dst_w, int dst_h) +``` + +通过双线性算法调整图像大小，当前仅支持的数据类型为uint8，当前支持的通道为3和1。 + +- 参数 + + - `src`: 输入的图片数据。 + - `dst`: 输出的图片数据。 + - `dst_w`: 输出图片数据的宽度。 + - `dst_h`: 输出图片数据的高度。 +- 返回值 + + 返回True或者False。 + +``` +bool InitFromPixel(const unsigned char *data, LPixelType pixel_type, LDataType data_type, int w, int h, LiteMat &m) +``` + +从像素初始化LiteMat，当前支持的转换是rbgaTorgb和rgbaTobgr。 + +- 参数 + + - `data`: 输入的数据。 + - `pixel_type`: 像素点的类型。 + - `data_type`: 数据的类型。 + - `w`: 输出数据的宽度。 + - `h`: 输出数据的高度。 + - `mat`: 用于存储图像数据。 +- 返回值 + + 返回True或者False。 + +``` +bool ConvertTo(LiteMat &src, LiteMat &dst, double scale = 1.0) +``` + +转换数据类型，当前支持的转换是将uint8转换为float。 + +- 参数 + + - `src`: 输入的图片数据。 + - `dst`: 输出图像数据。 + - `scale`: 对像素做尺度(默认值为1.0)。 + +- 返回值 + + 返回True或者False。 + +``` +bool Crop(LiteMat &src, LiteMat &dst, int x, int y, int w, int h) +``` + +裁剪图像，通道支持为3和1。 + +- 参数 + + - `src`: 输入的图片数据。 + - `dst`: 输出图像数据。 + - `x`: 屏幕截图起点的x坐标值。 + - `y`: 屏幕截图起点的y坐标值。 + - `w`: 截图的宽度。 + - `h`: 截图的高度。 +- 返回值 + + 返回True或者False。 + +``` +bool SubStractMeanNormalize(LiteMat &src, LiteMat &dst, const float *mean, float *norm) +``` + +规一化图像，当前支持的数据类型为float。 + +- 参数 + + - `src`: 输入的图片数据。 + - `dst`: 输出图像数据。 + - `mean`: 数据集的均值。 + - `norm`: 数据集的方差。 +- 返回值 + + 返回True或者False。 + +``` +bool Padd(LiteMat &src, LiteMat &dst, const int top, const int bottom, const int left, const int right, const PaddBorderType pad_type, uint8_t fill_r, uint8_t fill_g, uint8_t fill_b) +``` + +填充图像，通道支持为3和1。 + +- 参数 + + - `src`: 输入的图片数据。 + - `dst`: 输出图像数据。 + - `top`: 图片顶部长度。 + - `bottom`: 图片底部长度。 + - `left`: 图片左边长度。 + - `right`: 图片右边长度。 + - `pad_type`: padding的类型。 + - `fill_r`: R. + - `fill_g`: G. + - `fill_b`: B. +- 返回值 + + 返回True或者False。 + +``` +void Affine(LiteMat &src, LiteMat &out_img, double M[6], std::vector dsize, UINT8_C1 borderValue) +``` + +对1通道图像应用仿射变换。 + +- 参数 + + - `src`: 输入图片数据。 + - `out_img`: 输出图片数据。 + - `M[6]`: 仿射变换矩阵。 + - `dsize`: 输出图像的大小。 + - `borderValue`: 采图之后用于填充的像素值。 + +``` +void Affine(LiteMat &src, LiteMat &out_img, double M[6], std::vector dsize, UINT8_C3 borderValue) +``` + +对3通道图像应用仿射变换。 + +- 参数 + + - `src`: 输入图片数据。 + - `out_img`: 输出图片数据。 + - `M[6]`: 仿射变换矩阵。 + - `dsize`: 输出图像的大小。 + - `borderValue`: 采图之后用于填充的像素值。 + +``` +std::vector> GetDefaultBoxes(BoxesConfig config) +``` + +获取Faster R-CNN，SSD，YOLO等的默认框。 + +- 参数 + + - `config`: BoxesConfig结构体对象。 + +- 返回值 + + 返回默认框。 + +``` +void ConvertBoxes(std::vector> &boxes, std::vector> &default_boxes, BoxesConfig config) +``` + +将预测框转换为（y，x，h，w）的实际框。 + +- 参数 + + - `boxes`: 实际框的大小。 + - `default_boxes`: 默认框。 + - `config`: BoxesConfig结构体对象。 + +``` +std::vector ApplyNms(std::vector> &all_boxes, std::vector &all_scores, float thres, int max_boxes) +``` + +对实际框的非极大值抑制。 + +- 参数 + + - `all_boxes`: 所有输入的框。 + - `all_scores`: 通过网络执行后所有框的得分。 + - `thres`: IOU的预值。 + - `max_boxes`: 输出框的最大值。 +- 返回值 + + 返回框的id。 + +  + +## LiteMat + +LiteMat是一个处理图像的类。 + +**构造函数和析构函数** + + +``` +LiteMat() + +LiteMat(int width, LDataType data_type = LDataType::UINT8) + +LiteMat(int width, int height, LDataType data_type = LDataType::UINT8) + +LiteMat(int width, int height, int channel, LDataType data_type = LDataType::UINT8) +``` + +MindSpore中dataset模块下LiteMat的构造方法，使用参数的默认值。 + +``` +~LiteMat(); +``` + +MindSpore dataset LiteMat的析构函数。 + +**公有成员函数** + + +``` +void Init(int width, LDataType data_type = LDataType::UINT8) + +void Init(int width, int height, LDataType data_type = LDataType::UINT8) + +void Init(int width, int height, int channel, LDataType data_type = LDataType::UINT8) +``` + +该函数用于初始化图像的通道，宽度和高度，参数不同。 + +``` +bool IsEmpty() const +``` + +确定对象是否为空的函数。 + +- 返回值 + + 返回True或者False。 + +``` +void Release() +``` + +释放内存的函数。 + +**私有成员函数** + +``` +void *AlignMalloc(unsigned int size) +``` + +申请内存对齐的函数。 + +- 参数 + + - `size`: 内存大小。 + +- 返回值 + + 返回指针的大小。 + +``` +void AlignFree(void *ptr) +``` + +释放指针内存大小的方法。 + +``` +void InitElemSize(LDataType data_type) +``` + +通过data_type初始化元素字节数的值。 + +- 参数 + + - `data_type`: 数据的类型。 + +``` + int addRef(int *p, int value) +``` + +用于计算引用该函数次数的函数。 + +- 参数 + + - `p`: 指向引用的对象。 + - `value`: 引用时所加的值。 \ No newline at end of file diff --git a/docs/api_cpp/source_zh_cn/errorcode_and_metatype.md b/docs/api_cpp/source_zh_cn/errorcode_and_metatype.md new file mode 100644 index 0000000000000000000000000000000000000000..4195eaedcfa2cda8e0470d3db06950e35e2050d8 --- /dev/null +++ b/docs/api_cpp/source_zh_cn/errorcode_and_metatype.md @@ -0,0 +1,51 @@ +# 错误码及元类型 + +以下表格描述了MindSpore Lite中支持的错误码和元类型。 + +## ErrorCode + +| 定义 | 值 | 描述 | +| --- | --- | --- | +| RET_OK | 0 | 执行成功。 | +| RET_ERROR | -1 | 通用错误码。 | +| RET_NULL_PTR | -2 | 返回空指针。 | +| RET_PARAM_INVALID | -3 | 无效参数。 | +| RET_NO_CHANGE | -4 | 无改变。 | +| RET_SUCCESS_EXIT | -5 | 无错误退出。 | +| RET_MEMORY_FAILED | -6 | 创建内存失败。 | +| RET_OUT_OF_TENSOR_RANGE | -101 | 输出检查越界。 | +| RET_INPUT_TENSOR_ERROR | -102 | 输入检查越界。 | +| RET_REENTRANT_ERROR | -103 | 存在运行中的执行器。 | +| RET_GRAPH_FILE_ERR | -201 | 图文件识别失败。 | +| RET_NOT_FIND_OP | -301 | 无法找到算子。 | +| RET_INVALID_OP_NAME | -302 | 无效算子名。 | +| RET_INVALID_OP_ATTR | -303 | 无效算子属性。 | +| RET_OP_EXECUTE_FAILURE | -304 | 算子执行失败。 | +| RET_FORMAT_ERR | -401 | 张量格式检查失败。 | +| RET_INFER_ERR | -501 | 维度推理失败。 | +| RET_INFER_INVALID | -502 | 无效的维度推理。 | + +## MetaType + + **enum**类型变量。 + +| 类型定义 | 值 | 描述 | +| --- | --- | --- | +|kNumberTypeBegin| 29 | 表示Number类型的起始。 | +|kNumberTypeBool| 30 | 表示Bool数据类型。 | +|kNumberTypeInt| 31 | 表示Int数据类型。 | +|kNumberTypeInt8| 32 | 表示Int8数据类型。 | +|kNumberTypeInt16| 33 | 表示Int16数据类型。 | +|kNumberTypeInt32| 34 | 表示Int32数据类型。 | +|kNumberTypeInt64| 35 | 表示Int64数据类型。 | +|kNumberTypeUInt| 36 | 表示UInt数据类型。 | +|kNumberTypeUInt8| 37 | 表示UInt8数据类型。 | +|kNumberTypeUInt16| 38 | 表示UInt16数据类型。 | +|kNumberTypeUInt32| 39 | 表示UInt32数据类型。 | +|kNumberTypeUInt64| 40 | 表示UInt64数据类型。 | +|kNumberTypeFloat| 41 | 表示Float数据类型。 | +|kNumberTypeFloat16| 42 | 表示Float16数据类型。 | +|kNumberTypeFloat32| 43 | 表示Float32数据类型。 | +|kNumberTypeFloat64| 44 | 表示Float64数据类型。| +|kNumberTypeEnd| 45 | 表示Number类型的结尾。 | + diff --git a/docs/api_cpp/source_zh_cn/lite.md b/docs/api_cpp/source_zh_cn/lite.md new file mode 100644 index 0000000000000000000000000000000000000000..8c321676b42d8905532063343d4dde9f03f12857 --- /dev/null +++ b/docs/api_cpp/source_zh_cn/lite.md @@ -0,0 +1,197 @@ +# mindspore::lite + +#include <[context.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/context.h)> + +#include <[model.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/model.h)> + +#include <[version.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/version.h)> + + +## Allocator + +Allocator类定义了一个内存池，用于动态地分配和释放内存。 + +## Context + +Context类用于保存执行中的环境变量。 + +**构造函数和析构函数** + +``` +Context() +``` + +用默认参数构造MindSpore Lite Context 对象。 + +``` +Context(int thread_num, std::shared_ptr allocator, DeviceContext device_ctx) +``` + +根据输入参数构造MindSpore Lite Context 对象。 + +- 参数 + + - `thread_num`: 定义了执行线程数。 + + - `allocator`: 定义了内存分配器。 + + - `device_ctx`: 定义了设备信息。 + +- 返回值 + + MindSpore Lite Context 指针。 + +``` +~Context() +``` + +MindSpore Lite Context 的析构函数。 + +**公有属性** + +``` +float16_priority +``` + +**bool** 值，默认为**false**，用于使能float16 推理。 + +``` +device_ctx_{DT_CPU} +``` + +[**DeviceContext**](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#devicecontext)结构体。用于设置设备信息。 + +``` +thread_num_ +``` + +**int** 值，默认为**2**，设置线程数。 + +``` +allocator +``` + +指针类型，指向内存分配器[**Allocator**](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#allocator)的指针。 + +``` +cpu_bind_mode_ +``` + +[**CpuBindMode**](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#cpubindmode)枚举类型，默认为**MID_CPU**。 + +## PrimitiveC + +PrimitiveC定义为算子的原型。 + +## Model + +Model定义了MindSpore Lite中的模型，便于计算图管理。 + +**析构函数** + +``` +~Model() +``` + +MindSpore Lite Model的析构函数。 + +**公有成员函数** + +``` +void Destroy() +``` + +释放Model内的所有过程中动态分配的内存。 + +``` +void Free() +``` + +释放MindSpore Lite Model中的MetaGraph。 + +**静态公有成员函数** + +``` +static Model *Import(const char *model_buf, size_t size) +``` + +创建Model指针的静态方法。 + +- 参数 + + - `model_buf`: 定义了读取模型文件的缓存区。 + + - `size`: 定义了模型缓存区的字节数。 + +- 返回值 + + 指向MindSpore Lite的Model的指针。 + +## CpuBindMode +枚举类型，设置cpu绑定策略。 + +**属性** + +``` +MID_CPU = -1 +``` + +优先中等CPU绑定策略。 + +``` +HIGHER_CPU = 1 +``` + +优先高级CPU绑定策略。 + +``` +NO_BIND = 0 +``` + +不绑定。 + +## DeviceType +枚举类型，设置设备类型。 + +**属性** + +``` +DT_CPU = -1 +``` + +设备为CPU。 + +``` +DT_GPU = 1 +``` + +设备为GPU。 + +``` +DT_NPU = 0 +``` + +设备为NPU，暂不支持。 + +## DeviceContext + +定义设备类型的结构体。 + +**属性** + +``` +type +``` + +[**DeviceType**](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/lite.html#devicetype) 变量。设备类型。 + +## Version + +``` +std::string Version() +``` +全局方法，用于获取版本的字符串。 + +- 返回值 + + MindSpore Lite版本的字符串。 \ No newline at end of file diff --git a/docs/api_cpp/source_zh_cn/session.md b/docs/api_cpp/source_zh_cn/session.md new file mode 100644 index 0000000000000000000000000000000000000000..37a7bcb8d732dbedcc1ec69e0423c25d91f54704 --- /dev/null +++ b/docs/api_cpp/source_zh_cn/session.md @@ -0,0 +1,177 @@ +# mindspore::session + +#include <[lite_session.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/lite_session.h)> + + +## LiteSession + +LiteSession定义了MindSpore Lite中的会话，用于进行Model的编译和前向推理。 + +**构造函数和析构函数** + +``` +LiteSession() +``` +MindSpore Lite LiteSession的构造函数，使用默认参数。 +``` +~LiteSession() +``` +MindSpore Lite LiteSession的析构函数。 + +**公有成员函数** +``` +virtual void BindThread(bool if_bind) +``` +尝试将线程池中的线程绑定到指定的cpu内核，或从指定的cpu内核进行解绑。 + +- 参数 + + - `if_bind`: 定义了对线程进行绑定或解绑。 + +``` +virtual int CompileGraph(lite::Model *model) +``` +编译MindSpore Lite模型。 + +> 注意： CompileGraph必须在RunGraph方法之后调用。 + +- 参数 + + - `model`: 定义了需要被编译的模型。 + +- 返回值 + + STATUS ，即编译图的错误码。STATUS在[errorcode.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/errorcode.h)中定义。 + +``` +virtual std::vector GetInputs() const +``` +获取MindSpore Lite模型的MSTensors输入。 + +- 返回值 + + MindSpore Lite MSTensor向量。 + +``` +std::vector GetInputsByName(const std::string &node_name) const +``` +通过节点名获取MindSpore Lite模型的MSTensors输入。 + +- 参数 + + - `node_name`: 定义了节点名。 + +- 返回值 + + MindSpore Lite MSTensor向量。 + +``` +virtual int RunGraph(const KernelCallBack &before = nullptr, const KernelCallBack &after = nullptr) +``` +运行带有回调函数的会话。 +> 注意： RunGraph必须在CompileGraph方法之后调用。 + +- 参数 + + - `before`: 一个[**KernelCallBack**](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/session.html#kernelcallback) 结构体。定义了运行每个节点之前调用的回调函数。 + + - `after`: 一个[**KernelCallBack**](https://www.mindspore.cn/lite/docs/zh-CN/master/apicc/session.html#kernelcallback) 结构体。定义了运行每个节点之后调用的回调函数。 + +- 返回值 + + STATUS ，即编译图的错误码。STATUS在[errorcode.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/errorcode.h)中定义。 + +``` +virtual std::vector GetOutputsByNodeName(const std::string &node_name) const +``` +通过节点名获取MindSpore Lite模型的MSTensors输出。 + +- 参数 + + - `node_name`: 定义了节点名。 + +- 返回值 + + MindSpore Lite MSTensor向量。 + +``` +virtual std::unordered_map GetOutputs() const +``` +获取与张量名相关联的MindSpore Lite模型的MSTensors输出。 + +- 返回值 + + 包含输出张量名和MindSpore Lite MSTensor的容器类型变量。 + +``` +virtual std::vector GetOutputTensorNames() const +``` +获取由当前会话所编译的模型的输出张量名。 + +- 返回值 + + 字符串向量，其中包含了按顺序排列的输出张量名。 + +``` +virtual mindspore::tensor::MSTensor *GetOutputByTensorName(const std::string &tensor_name) const +``` +通过张量名获取MindSpore Lite模型的MSTensors输出。 + +- 参数 + + - `tensor_name`: 定义了张量名。 + +- 返回值 + + 指向MindSpore Lite MSTensor的指针。 + +``` +virtual int Resize(const std::vector &inputs, const std::vector> &dims) +``` +调整输入的形状。 + +- 参数 + + - `inputs`: 模型对应的所有输入。 + - `dims`: 输入对应的新的shape，顺序注意要与inputs一致。 + +- 返回值 + + STATUS ，即编译图的错误码。STATUS在[errorcode.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/errorcode.h)中定义。 + +**静态公有成员函数** + +``` +static LiteSession *CreateSession(lite::Context *context) +``` +用于创建一个LiteSession指针的静态方法。 + +- 参数 + + - `context`: 定义了所要创建的session的上下文。 + +- 返回值 + + 指向MindSpore Lite LiteSession的指针。 +## KernelCallBack + +``` +using KernelCallBack = std::function inputs, std::vector outputs, const CallBackParam &opInfo)> +``` + +一个函数包装器。KernelCallBack 定义了指向回调函数的指针。 + +## CallBackParam + +一个结构体。CallBackParam定义了回调函数的输入参数。 +**属性** + +``` +name_callback_param +``` +**string** 类型变量。节点名参数。 + +``` +type_callback_param +``` +**string** 类型变量。节点类型参数。 \ No newline at end of file diff --git a/docs/api_cpp/source_zh_cn/tensor.md b/docs/api_cpp/source_zh_cn/tensor.md new file mode 100644 index 0000000000000000000000000000000000000000..e9eae1f0fd9a62aa59e7b578b09a455bab843f1d --- /dev/null +++ b/docs/api_cpp/source_zh_cn/tensor.md @@ -0,0 +1,142 @@ +# mindspore::tensor + +#include <[ms_tensor.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/include/ms_tensor.h)> + + +## MSTensor + +MSTensor定义了MindSpore Lite中的张量。 + +**构造函数和析构函数** +``` +MSTensor() +``` +MindSpore Lite MSTensor的构造函数。 + +- 返回值 + + MindSpore Lite MSTensor 的实例。 + +``` +virtual ~MSTensor() +``` +MindSpore Lite Model的析构函数。 + +**公有成员函数** + +``` +virtual TypeId data_type() const +``` +获取MindSpore Lite MSTensor的数据类型。 + +> 注意：TypeId在[mindspore/mindspore/core/ir/dtype/type_id\.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/core/ir/dtype/type_id.h)中定义。只有TypeId枚举中的数字类型可用于MSTensor。 + +- 返回值 + + MindSpore Lite MSTensor类的MindSpore Lite TypeId。 + +``` +virtual TypeId set_data_type(TypeId data_type) +``` +设置MindSpore Lite MSTensor的数据类型。 + +- 参数 + + - `data_type`: 定义了MindSpore Lite MSTensor所需设置的MindSpore Lite TypeId。 + +- 返回值 + + 设置后的MindSpore Lite MSTensor的MindSpore Lite TypeI。 + +``` +virtual std::vector shape() const +``` +获取MindSpore Lite MSTensor的形状。 + +- 返回值 + + 一个包含MindSpore Lite MSTensor形状数值的整型向量。 + +``` +virtual size_t set_shape(const std::vector &shape) +``` +设置MindSpore Lite MSTensor的形状. + +- 参数 + + - `shape`: 定义了一个整型向量，包含了所需设置的MindSpore Lite MSTensor形状数值。 + +- 返回值 + + 设置形状后的MindSpore Lite MSTensor的大小。 + +``` +virtual int DimensionSize(size_t index) const +``` +Get size of the dimension of the MindSpore Lite MSTensor index by the parameter index. + +- 参数 + + - `index`: 定义了返回的维度的索引。 + +- 返回值 + + MindSpore Lite MSTensor的维度的大小。 + +``` +virtual int ElementsNum() const +``` +获取MSTensor中的元素个数。 + +- 返回值 + + MSTensor中的元素个数 + +``` +virtual std::size_t hash() const +``` +获取MindSpore Lite MSTensor的哈希码。 + +- 返回值 + + MindSpore Lite MSTensor的哈希码。 + +``` +virtual size_t Size() const +``` +获取MSTensor中的数据的字节数大小。 + +- 返回值 + + MSTensor中的数据的字节数大小。 + + +``` +virtual void *MutableData() const +``` +获取MSTensor中的数据的指针。 + +> 注意：该数据指针可用于对MSTensor中的数据进行读取和写入。 + +- 返回值 + + 指向MSTensor中的数据的指针。 + +**静态公有成员函数** + +``` +static MSTensor *CreateTensor(TypeId data_type, const std::vector &shape) +``` +创建MSTensor指针的静态方法。 + +> 注意：TypeId在[mindspore/mindspore/core/ir/dtype/type_id\.h](https://gitee.com/mindspore/mindspore/blob/master/mindspore/core/ir/dtype/type_id.h)中定义。只有TypeId枚举中的数字类型可用于MSTensor。 + +- 参数 + + - `data_type`: 定义了所要创建的张量的数据类型。 + + - `shape`: 定义了所要创建的张量的形状。 + +- 返回值 + + 指向MSTensor的指针。 \ No newline at end of file diff --git a/docs/Makefile b/docs/api_java/Makefile similarity index 100% rename from docs/Makefile rename to docs/api_java/Makefile diff --git a/docs/requirements.txt b/docs/api_java/requirements.txt similarity index 100% rename from docs/requirements.txt rename to docs/api_java/requirements.txt diff --git a/lite/tutorials/Makefile b/docs/api_python/Makefile similarity index 100% rename from lite/tutorials/Makefile rename to docs/api_python/Makefile diff --git a/api/numpy_objects.inv b/docs/api_python/numpy_objects.inv similarity index 100% rename from api/numpy_objects.inv rename to docs/api_python/numpy_objects.inv diff --git a/api/python_objects.inv b/docs/api_python/python_objects.inv similarity index 100% rename from api/python_objects.inv rename to docs/api_python/python_objects.inv diff --git a/docs/api_python/requirements.txt b/docs/api_python/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..162b50040286bb9a0177801c580a31013082a360 --- /dev/null +++ b/docs/api_python/requirements.txt @@ -0,0 +1,6 @@ +sphinx >= 2.2.1, <= 2.4.4 +recommonmark +sphinx-markdown-tables +sphinx_rtd_theme +numpy +jieba diff --git a/api/run.sh b/docs/api_python/run.sh similarity index 100% rename from api/run.sh rename to docs/api_python/run.sh diff --git a/api/source_en/_static/logo_source.png b/docs/api_python/source_en/_static/logo_source.png similarity index 100% rename from api/source_en/_static/logo_source.png rename to docs/api_python/source_en/_static/logo_source.png diff --git a/api/source_en/api/python/mindarmour/mindarmour.adv_robustness.attacks.rst b/docs/api_python/source_en/api/python/mindarmour/mindarmour.adv_robustness.attacks.rst similarity index 100% rename from api/source_en/api/python/mindarmour/mindarmour.adv_robustness.attacks.rst rename to docs/api_python/source_en/api/python/mindarmour/mindarmour.adv_robustness.attacks.rst diff --git a/api/source_en/api/python/mindarmour/mindarmour.adv_robustness.defenses.rst b/docs/api_python/source_en/api/python/mindarmour/mindarmour.adv_robustness.defenses.rst similarity index 100% rename from api/source_en/api/python/mindarmour/mindarmour.adv_robustness.defenses.rst rename to docs/api_python/source_en/api/python/mindarmour/mindarmour.adv_robustness.defenses.rst diff --git a/api/source_en/api/python/mindarmour/mindarmour.adv_robustness.detectors.rst b/docs/api_python/source_en/api/python/mindarmour/mindarmour.adv_robustness.detectors.rst similarity index 100% rename from api/source_en/api/python/mindarmour/mindarmour.adv_robustness.detectors.rst rename to docs/api_python/source_en/api/python/mindarmour/mindarmour.adv_robustness.detectors.rst diff --git a/api/source_en/api/python/mindarmour/mindarmour.adv_robustness.evaluations.rst b/docs/api_python/source_en/api/python/mindarmour/mindarmour.adv_robustness.evaluations.rst similarity index 100% rename from api/source_en/api/python/mindarmour/mindarmour.adv_robustness.evaluations.rst rename to docs/api_python/source_en/api/python/mindarmour/mindarmour.adv_robustness.evaluations.rst diff --git a/api/source_en/api/python/mindarmour/mindarmour.fuzz_testing.rst b/docs/api_python/source_en/api/python/mindarmour/mindarmour.fuzz_testing.rst similarity index 100% rename from api/source_en/api/python/mindarmour/mindarmour.fuzz_testing.rst rename to docs/api_python/source_en/api/python/mindarmour/mindarmour.fuzz_testing.rst diff --git a/api/source_en/api/python/mindarmour/mindarmour.privacy.diff_privacy.rst b/docs/api_python/source_en/api/python/mindarmour/mindarmour.privacy.diff_privacy.rst similarity index 100% rename from api/source_en/api/python/mindarmour/mindarmour.privacy.diff_privacy.rst rename to docs/api_python/source_en/api/python/mindarmour/mindarmour.privacy.diff_privacy.rst diff --git a/api/source_en/api/python/mindarmour/mindarmour.privacy.evaluation.rst b/docs/api_python/source_en/api/python/mindarmour/mindarmour.privacy.evaluation.rst similarity index 100% rename from api/source_en/api/python/mindarmour/mindarmour.privacy.evaluation.rst rename to docs/api_python/source_en/api/python/mindarmour/mindarmour.privacy.evaluation.rst diff --git a/api/source_en/api/python/mindarmour/mindarmour.rst b/docs/api_python/source_en/api/python/mindarmour/mindarmour.rst similarity index 100% rename from api/source_en/api/python/mindarmour/mindarmour.rst rename to docs/api_python/source_en/api/python/mindarmour/mindarmour.rst diff --git a/api/source_en/api/python/mindarmour/mindarmour.utils.rst b/docs/api_python/source_en/api/python/mindarmour/mindarmour.utils.rst similarity index 100% rename from api/source_en/api/python/mindarmour/mindarmour.utils.rst rename to docs/api_python/source_en/api/python/mindarmour/mindarmour.utils.rst diff --git a/api/source_en/api/python/mindspore/mindspore.common.initializer.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.common.initializer.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.common.initializer.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.common.initializer.rst diff --git a/api/source_en/api/python/mindspore/mindspore.communication.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.communication.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.communication.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.communication.rst diff --git a/api/source_en/api/python/mindspore/mindspore.context.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.context.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.context.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.context.rst diff --git a/api/source_en/api/python/mindspore/mindspore.dataset.config.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.dataset.config.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.dataset.config.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.dataset.config.rst diff --git a/api/source_en/api/python/mindspore/mindspore.dataset.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.dataset.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.dataset.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.dataset.rst diff --git a/api/source_en/api/python/mindspore/mindspore.dataset.text.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.dataset.text.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.dataset.text.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.dataset.text.rst diff --git a/api/source_en/api/python/mindspore/mindspore.dataset.transforms.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.dataset.transforms.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.dataset.transforms.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.dataset.transforms.rst diff --git a/api/source_en/api/python/mindspore/mindspore.dataset.vision.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.dataset.vision.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.dataset.vision.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.dataset.vision.rst diff --git a/api/source_en/api/python/mindspore/mindspore.dtype.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.dtype.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.dtype.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.dtype.rst diff --git a/api/source_en/api/python/mindspore/mindspore.hub.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.hub.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.hub.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.hub.rst diff --git a/api/source_en/api/python/mindspore/mindspore.mindrecord.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.mindrecord.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.mindrecord.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.mindrecord.rst diff --git a/api/source_en/api/python/mindspore/mindspore.nn.dynamic_lr.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.nn.dynamic_lr.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.nn.dynamic_lr.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.nn.dynamic_lr.rst diff --git a/api/source_en/api/python/mindspore/mindspore.nn.learning_rate_schedule.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.nn.learning_rate_schedule.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.nn.learning_rate_schedule.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.nn.learning_rate_schedule.rst diff --git a/api/source_en/api/python/mindspore/mindspore.nn.probability.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.nn.probability.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.nn.probability.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.nn.probability.rst diff --git a/api/source_en/api/python/mindspore/mindspore.nn.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.nn.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.nn.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.nn.rst diff --git a/api/source_en/api/python/mindspore/mindspore.ops.composite.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.ops.composite.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.ops.composite.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.ops.composite.rst diff --git a/api/source_en/api/python/mindspore/mindspore.ops.operations.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.ops.operations.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.ops.operations.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.ops.operations.rst diff --git a/api/source_en/api/python/mindspore/mindspore.ops.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.ops.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.ops.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.ops.rst diff --git a/api/source_en/api/python/mindspore/mindspore.profiler.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.profiler.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.profiler.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.profiler.rst diff --git a/api/source_en/api/python/mindspore/mindspore.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.rst diff --git a/api/source_en/api/python/mindspore/mindspore.train.rst b/docs/api_python/source_en/api/python/mindspore/mindspore.train.rst similarity index 100% rename from api/source_en/api/python/mindspore/mindspore.train.rst rename to docs/api_python/source_en/api/python/mindspore/mindspore.train.rst diff --git a/api/source_en/api/python/mindspore_hub/mindspore_hub.rst b/docs/api_python/source_en/api/python/mindspore_hub/mindspore_hub.rst similarity index 100% rename from api/source_en/api/python/mindspore_hub/mindspore_hub.rst rename to docs/api_python/source_en/api/python/mindspore_hub/mindspore_hub.rst diff --git a/api/source_en/conf.py b/docs/api_python/source_en/conf.py similarity index 100% rename from api/source_en/conf.py rename to docs/api_python/source_en/conf.py diff --git a/api/source_en/index.rst b/docs/api_python/source_en/index.rst similarity index 100% rename from api/source_en/index.rst rename to docs/api_python/source_en/index.rst diff --git a/api/source_zh_cn/_static/logo_source.png b/docs/api_python/source_zh_cn/_static/logo_source.png similarity index 100% rename from api/source_zh_cn/_static/logo_source.png rename to docs/api_python/source_zh_cn/_static/logo_source.png diff --git a/api/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.attacks.rst b/docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.attacks.rst similarity index 100% rename from api/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.attacks.rst rename to docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.attacks.rst diff --git a/api/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.defenses.rst b/docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.defenses.rst similarity index 100% rename from api/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.defenses.rst rename to docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.defenses.rst diff --git a/api/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.detectors.rst b/docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.detectors.rst similarity index 100% rename from api/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.detectors.rst rename to docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.detectors.rst diff --git a/api/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.evaluations.rst b/docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.evaluations.rst similarity index 100% rename from api/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.evaluations.rst rename to docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.adv_robustness.evaluations.rst diff --git a/api/source_zh_cn/api/python/mindarmour/mindarmour.fuzz_testing.rst b/docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.fuzz_testing.rst similarity index 100% rename from api/source_zh_cn/api/python/mindarmour/mindarmour.fuzz_testing.rst rename to docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.fuzz_testing.rst diff --git a/api/source_zh_cn/api/python/mindarmour/mindarmour.privacy.diff_privacy.rst b/docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.privacy.diff_privacy.rst similarity index 100% rename from api/source_zh_cn/api/python/mindarmour/mindarmour.privacy.diff_privacy.rst rename to docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.privacy.diff_privacy.rst diff --git a/api/source_zh_cn/api/python/mindarmour/mindarmour.privacy.evaluation.rst b/docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.privacy.evaluation.rst similarity index 100% rename from api/source_zh_cn/api/python/mindarmour/mindarmour.privacy.evaluation.rst rename to docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.privacy.evaluation.rst diff --git a/api/source_zh_cn/api/python/mindarmour/mindarmour.rst b/docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.rst similarity index 100% rename from api/source_zh_cn/api/python/mindarmour/mindarmour.rst rename to docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.rst diff --git a/api/source_zh_cn/api/python/mindarmour/mindarmour.utils.rst b/docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.utils.rst similarity index 100% rename from api/source_zh_cn/api/python/mindarmour/mindarmour.utils.rst rename to docs/api_python/source_zh_cn/api/python/mindarmour/mindarmour.utils.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.common.initializer.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.common.initializer.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.common.initializer.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.common.initializer.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.communication.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.communication.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.communication.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.communication.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.context.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.context.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.context.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.context.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.dataset.config.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.config.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.dataset.config.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.config.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.dataset.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.dataset.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.dataset.text.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.text.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.dataset.text.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.text.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.dataset.transforms.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.transforms.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.dataset.transforms.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.transforms.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.dataset.vision.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.vision.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.dataset.vision.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dataset.vision.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.dtype.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dtype.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.dtype.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.dtype.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.hub.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.hub.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.hub.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.hub.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.mindrecord.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.mindrecord.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.mindrecord.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.mindrecord.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.nn.dynamic_lr.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.nn.dynamic_lr.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.nn.dynamic_lr.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.nn.dynamic_lr.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.nn.learning_rate_schedule.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.nn.learning_rate_schedule.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.nn.learning_rate_schedule.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.nn.learning_rate_schedule.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.nn.probability.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.nn.probability.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.nn.probability.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.nn.probability.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.nn.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.nn.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.nn.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.nn.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.ops.composite.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.ops.composite.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.ops.composite.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.ops.composite.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.ops.operations.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.ops.operations.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.ops.operations.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.ops.operations.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.ops.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.ops.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.ops.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.ops.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.profiler.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.profiler.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.profiler.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.profiler.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.rst diff --git a/api/source_zh_cn/api/python/mindspore/mindspore.train.rst b/docs/api_python/source_zh_cn/api/python/mindspore/mindspore.train.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore/mindspore.train.rst rename to docs/api_python/source_zh_cn/api/python/mindspore/mindspore.train.rst diff --git a/api/source_zh_cn/api/python/mindspore_hub/mindspore_hub.rst b/docs/api_python/source_zh_cn/api/python/mindspore_hub/mindspore_hub.rst similarity index 100% rename from api/source_zh_cn/api/python/mindspore_hub/mindspore_hub.rst rename to docs/api_python/source_zh_cn/api/python/mindspore_hub/mindspore_hub.rst diff --git a/api/source_zh_cn/conf.py b/docs/api_python/source_zh_cn/conf.py similarity index 100% rename from api/source_zh_cn/conf.py rename to docs/api_python/source_zh_cn/conf.py diff --git a/api/source_zh_cn/index.rst b/docs/api_python/source_zh_cn/index.rst similarity index 100% rename from api/source_zh_cn/index.rst rename to docs/api_python/source_zh_cn/index.rst diff --git a/tutorials/Makefile b/docs/faq/Makefile similarity index 100% rename from tutorials/Makefile rename to docs/faq/Makefile diff --git a/docs/faq/requirements.txt b/docs/faq/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..162b50040286bb9a0177801c580a31013082a360 --- /dev/null +++ b/docs/faq/requirements.txt @@ -0,0 +1,6 @@ +sphinx >= 2.2.1, <= 2.4.4 +recommonmark +sphinx-markdown-tables +sphinx_rtd_theme +numpy +jieba diff --git a/docs/source_en/FAQ.md b/docs/faq/source_en/FAQ.md similarity index 100% rename from docs/source_en/FAQ.md rename to docs/faq/source_en/FAQ.md diff --git a/docs/source_zh_cn/FAQ.md b/docs/faq/source_zh_cn/FAQ.md similarity index 100% rename from docs/source_zh_cn/FAQ.md rename to docs/faq/source_zh_cn/FAQ.md diff --git a/docs/note/Makefile b/docs/note/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..1eff8952707bdfa503c8d60c1e9a903053170ba2 --- /dev/null +++ b/docs/note/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source_zh_cn +BUILDDIR = build_zh_cn + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/note/requirements.txt b/docs/note/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..162b50040286bb9a0177801c580a31013082a360 --- /dev/null +++ b/docs/note/requirements.txt @@ -0,0 +1,6 @@ +sphinx >= 2.2.1, <= 2.4.4 +recommonmark +sphinx-markdown-tables +sphinx_rtd_theme +numpy +jieba diff --git a/docs/source_en/_static/logo_source.png b/docs/note/source_en/_static/logo_source.png similarity index 100% rename from docs/source_en/_static/logo_source.png rename to docs/note/source_en/_static/logo_source.png diff --git a/docs/source_en/architecture.md b/docs/note/source_en/architecture.md similarity index 100% rename from docs/source_en/architecture.md rename to docs/note/source_en/architecture.md diff --git a/lite/docs/source_en/architecture.md b/docs/note/source_en/architecture_lite.md similarity index 100% rename from lite/docs/source_en/architecture.md rename to docs/note/source_en/architecture_lite.md diff --git a/docs/source_en/benchmark.md b/docs/note/source_en/benchmark.md similarity index 100% rename from docs/source_en/benchmark.md rename to docs/note/source_en/benchmark.md diff --git a/docs/source_en/community.rst b/docs/note/source_en/community.rst similarity index 100% rename from docs/source_en/community.rst rename to docs/note/source_en/community.rst diff --git a/docs/source_en/conf.py b/docs/note/source_en/conf.py similarity index 100% rename from docs/source_en/conf.py rename to docs/note/source_en/conf.py diff --git a/docs/source_en/constraints_on_network_construction.md b/docs/note/source_en/constraints_on_network_construction.md similarity index 100% rename from docs/source_en/constraints_on_network_construction.md rename to docs/note/source_en/constraints_on_network_construction.md diff --git a/docs/source_en/design.rst b/docs/note/source_en/design.rst similarity index 100% rename from docs/source_en/design.rst rename to docs/note/source_en/design.rst diff --git a/docs/source_en/design/mindinsight/graph_visual_design.md b/docs/note/source_en/design/mindinsight/graph_visual_design.md similarity index 100% rename from docs/source_en/design/mindinsight/graph_visual_design.md rename to docs/note/source_en/design/mindinsight/graph_visual_design.md diff --git a/docs/source_en/design/mindinsight/images/graph_visual_class_design.png b/docs/note/source_en/design/mindinsight/images/graph_visual_class_design.png similarity index 100% rename from docs/source_en/design/mindinsight/images/graph_visual_class_design.png rename to docs/note/source_en/design/mindinsight/images/graph_visual_class_design.png diff --git a/docs/source_en/design/mindinsight/images/graph_visual_main.png b/docs/note/source_en/design/mindinsight/images/graph_visual_main.png similarity index 100% rename from docs/source_en/design/mindinsight/images/graph_visual_main.png rename to docs/note/source_en/design/mindinsight/images/graph_visual_main.png diff --git a/docs/source_en/design/mindinsight/images/graph_visual_right_side.png b/docs/note/source_en/design/mindinsight/images/graph_visual_right_side.png similarity index 100% rename from docs/source_en/design/mindinsight/images/graph_visual_right_side.png rename to docs/note/source_en/design/mindinsight/images/graph_visual_right_side.png diff --git a/docs/source_en/design/mindinsight/images/tensor_histogram.png b/docs/note/source_en/design/mindinsight/images/tensor_histogram.png similarity index 100% rename from docs/source_en/design/mindinsight/images/tensor_histogram.png rename to docs/note/source_en/design/mindinsight/images/tensor_histogram.png diff --git a/docs/source_en/design/mindinsight/images/tensor_table.png b/docs/note/source_en/design/mindinsight/images/tensor_table.png similarity index 100% rename from docs/source_en/design/mindinsight/images/tensor_table.png rename to docs/note/source_en/design/mindinsight/images/tensor_table.png diff --git a/docs/source_en/design/mindinsight/images/training_visualization_architecture.png b/docs/note/source_en/design/mindinsight/images/training_visualization_architecture.png similarity index 100% rename from docs/source_en/design/mindinsight/images/training_visualization_architecture.png rename to docs/note/source_en/design/mindinsight/images/training_visualization_architecture.png diff --git a/docs/source_en/design/mindinsight/images/training_visualization_data_flow.png b/docs/note/source_en/design/mindinsight/images/training_visualization_data_flow.png similarity index 100% rename from docs/source_en/design/mindinsight/images/training_visualization_data_flow.png rename to docs/note/source_en/design/mindinsight/images/training_visualization_data_flow.png diff --git a/docs/source_en/design/mindinsight/images/training_visualization_data_model.png b/docs/note/source_en/design/mindinsight/images/training_visualization_data_model.png similarity index 100% rename from docs/source_en/design/mindinsight/images/training_visualization_data_model.png rename to docs/note/source_en/design/mindinsight/images/training_visualization_data_model.png diff --git a/docs/source_en/design/mindinsight/tensor_visual_design.md b/docs/note/source_en/design/mindinsight/tensor_visual_design.md similarity index 100% rename from docs/source_en/design/mindinsight/tensor_visual_design.md rename to docs/note/source_en/design/mindinsight/tensor_visual_design.md diff --git a/docs/source_en/design/mindinsight/training_visual_design.md b/docs/note/source_en/design/mindinsight/training_visual_design.md similarity index 100% rename from docs/source_en/design/mindinsight/training_visual_design.md rename to docs/note/source_en/design/mindinsight/training_visual_design.md diff --git a/docs/source_en/design/mindspore/images/ir/cf.dot b/docs/note/source_en/design/mindspore/images/ir/cf.dot similarity index 100% rename from docs/source_en/design/mindspore/images/ir/cf.dot rename to docs/note/source_en/design/mindspore/images/ir/cf.dot diff --git a/docs/source_en/design/mindspore/images/ir/cf.png b/docs/note/source_en/design/mindspore/images/ir/cf.png similarity index 100% rename from docs/source_en/design/mindspore/images/ir/cf.png rename to docs/note/source_en/design/mindspore/images/ir/cf.png diff --git a/docs/source_en/design/mindspore/images/ir/closure.dot b/docs/note/source_en/design/mindspore/images/ir/closure.dot similarity index 100% rename from docs/source_en/design/mindspore/images/ir/closure.dot rename to docs/note/source_en/design/mindspore/images/ir/closure.dot diff --git a/docs/source_en/design/mindspore/images/ir/closure.png b/docs/note/source_en/design/mindspore/images/ir/closure.png similarity index 100% rename from docs/source_en/design/mindspore/images/ir/closure.png rename to docs/note/source_en/design/mindspore/images/ir/closure.png diff --git a/docs/source_en/design/mindspore/images/ir/hof.dot b/docs/note/source_en/design/mindspore/images/ir/hof.dot similarity index 100% rename from docs/source_en/design/mindspore/images/ir/hof.dot rename to docs/note/source_en/design/mindspore/images/ir/hof.dot diff --git a/docs/source_en/design/mindspore/images/ir/hof.png b/docs/note/source_en/design/mindspore/images/ir/hof.png similarity index 100% rename from docs/source_en/design/mindspore/images/ir/hof.png rename to docs/note/source_en/design/mindspore/images/ir/hof.png diff --git a/docs/source_en/design/mindspore/images/ir/ir.dot b/docs/note/source_en/design/mindspore/images/ir/ir.dot similarity index 100% rename from docs/source_en/design/mindspore/images/ir/ir.dot rename to docs/note/source_en/design/mindspore/images/ir/ir.dot diff --git a/docs/source_en/design/mindspore/images/ir/ir.png b/docs/note/source_en/design/mindspore/images/ir/ir.png similarity index 100% rename from docs/source_en/design/mindspore/images/ir/ir.png rename to docs/note/source_en/design/mindspore/images/ir/ir.png diff --git a/docs/source_en/design/mindspore/ir.md b/docs/note/source_en/design/mindspore/ir.md similarity index 100% rename from docs/source_en/design/mindspore/ir.md rename to docs/note/source_en/design/mindspore/ir.md diff --git a/docs/source_en/glossary.md b/docs/note/source_en/glossary.md similarity index 100% rename from docs/source_en/glossary.md rename to docs/note/source_en/glossary.md diff --git a/lite/docs/source_en/glossary.md b/docs/note/source_en/glossary_lite.md similarity index 100% rename from lite/docs/source_en/glossary.md rename to docs/note/source_en/glossary_lite.md diff --git a/docs/source_en/help_seeking_path.md b/docs/note/source_en/help_seeking_path.md similarity index 100% rename from docs/source_en/help_seeking_path.md rename to docs/note/source_en/help_seeking_path.md diff --git a/docs/note/source_en/images/MindSpore-Lite-architecture.png b/docs/note/source_en/images/MindSpore-Lite-architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..abf28796690f5649f8bc92382dfd4c2c83187620 Binary files /dev/null and b/docs/note/source_en/images/MindSpore-Lite-architecture.png differ diff --git a/docs/source_en/images/architecture.eddx b/docs/note/source_en/images/architecture.eddx similarity index 100% rename from docs/source_en/images/architecture.eddx rename to docs/note/source_en/images/architecture.eddx diff --git a/docs/source_en/images/architecture.png b/docs/note/source_en/images/architecture.png similarity index 100% rename from docs/source_en/images/architecture.png rename to docs/note/source_en/images/architecture.png diff --git a/docs/source_en/images/help_seeking_path.png b/docs/note/source_en/images/help_seeking_path.png similarity index 100% rename from docs/source_en/images/help_seeking_path.png rename to docs/note/source_en/images/help_seeking_path.png diff --git a/docs/source_en/index.rst b/docs/note/source_en/index.rst similarity index 100% rename from docs/source_en/index.rst rename to docs/note/source_en/index.rst diff --git a/lite/docs/source_en/index.rst b/docs/note/source_en/index_lite.rst similarity index 100% rename from lite/docs/source_en/index.rst rename to docs/note/source_en/index_lite.rst diff --git a/docs/source_en/network_list.md b/docs/note/source_en/network_list.md similarity index 99% rename from docs/source_en/network_list.md rename to docs/note/source_en/network_list.md index 897111be5078687a3c4b4671c0c9f05904226128..d8cc516092656354d957cbd072e3d0fc850f409c 100644 --- a/docs/source_en/network_list.md +++ b/docs/note/source_en/network_list.md @@ -1,60 +1,60 @@ -# Network List - -`Linux` `Ascend` `GPU` `CPU` `Model Development` `Intermediate` `Expert` - - - -- [Network List](#network-list) - - [Model Zoo](#model-zoo) - - [Pre-trained Models](#pre-trained-models) - - - - - -## Model Zoo - -| Domain | Sub Domain | Network | Ascend | GPU | CPU -|:------ |:------| :----------- |:------ |:------ |:----- -|Computer Vision (CV) | Image Classification | [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/alexnet/src/alexnet.py) | Supported | Supported | Doing -| Computer Vision (CV) | Image Classification | [GoogleNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/googlenet/src/googlenet.py) | Supported | Doing | Doing -| Computer Vision (CV) | Image Classification | [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/lenet/src/lenet.py) | Supported | Supported | Supported -| Computer Vision (CV) | Image Classification | [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported | Supported | Doing -|Computer Vision (CV) | Image Classification | [ResNet-101](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported |Doing | Doing -|Computer Vision (CV) | Image Classification | [SE-ResNet50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported |Doing | Doing -|Computer Vision (CV) | Image Classification | [ResNext50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnext50/src/image_classification.py) | Supported | Supported | Doing -| Computer Vision (CV) | Image Classification | [VGG16](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/vgg16/src/vgg.py) | Supported | Doing | Doing -| Computer Vision (CV) | Image Classification | [InceptionV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/inceptionv3/src/inception_v3.py) | Supported | Doing | Doing -| Computer Vision (CV) | Mobile Image Classification
Image Classification
Semantic Tegmentation | [MobileNetV2](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/mobilenetv2/src/mobilenetV2.py) | Supported | Supported | Doing -| Computer Vision (CV) | Mobile Image Classification
Image Classification
Semantic Tegmentation | [MobileNetV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/mobilenetv3/src/mobilenetV3.py) | Doing | Supported | Doing -|Computer Vision (CV) | Targets Detection | [SSD](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/ssd/src/ssd.py) | Supported |Doing | Doing -| Computer Vision (CV) | Targets Detection | [YoloV3-ResNet18](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_resnet18/src/yolov3.py) | Supported | Doing | Doing -| Computer Vision (CV) | Targets Detection | [YoloV3-DarkNet53](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_darknet53/src/yolo.py) | Supported | Doing | Doing -| Computer Vision (CV) | Targets Detection | [FasterRCNN](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/faster_rcnn/src/FasterRcnn/faster_rcnn_r50.py) | Supported | Doing | Doing -| Computer Vision (CV) | Semantic Segmentation | [DeeplabV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/deeplabv3/src/deeplabv3.py) | Supported | Doing | Doing -| Computer Vision（CV） | Targets Detection | [WarpCTC](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/warpctc/src/warpctc.py) | Doing | Supported | Doing -| Natural Language Processing (NLP) | Natural Language Understanding | [BERT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py) | Supported | Doing | Doing -| Natural Language Processing (NLP) | Natural Language Understanding | [Transformer](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/transformer/src/transformer_model.py) | Supported | Doing | Doing -| Natural Language Processing (NLP) | Natural Language Understanding | [SentimentNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/lstm/src/lstm.py) | Doing | Supported | Supported -| Natural Language Processing (NLP) | Natural Language Understanding | [MASS](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/mass/src/transformer/transformer_for_train.py) | Supported | Doing | Doing -| Natural Language Processing (NLP) | Natural Language Understanding | [TinyBert](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/tinybert/src/tinybert_model.py) | Supported | Supported | Doing -| Recommender | Recommender System, CTR prediction | [DeepFM](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/recommend/deepfm/src/deepfm.py) | Supported | Supported | Doing -| Recommender | Recommender System, Search ranking | [Wide&Deep](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/recommend/wide_and_deep/src/wide_and_deep.py) | Supported | Supported | Doing -| Graph Neural Networks（GNN）| Text Classification | [GCN](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/gnn/gcn/src/gcn.py) | Supported | Doing | Doing -| Graph Neural Networks（GNN）| Text Classification | [GAT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/gnn/gat/src/gat.py) | Supported | Doing | Doing - -> You can also use [MindWizard Tool](https://gitee.com/mindspore/mindinsight/tree/master/mindinsight/wizard/) to quickly generate classic network scripts. - -## Pre-trained Models -*It refers to the released MindSpore version. The hardware platforms that support model training are CPU, GPU and Ascend. As shown in the table below, ✓ indicates that the pre-trained model run on the selected platform. - -| Domain | Sub Domain| Network | Dataset | CPU | GPU | Ascend | 0.5.0-beta* -|:------ |:------ | :------- |:------ |:------ |:------ |:----- |:----- -|Computer Vision (CV) | Image Classification| [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/alexnet/src/alexnet.py) | CIFAR-10| | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/cv/alexnet/alexnet_ascend_0.5.0_cifar10_official_classification_20200716.tar.gz) -|Computer Vision (CV) | Image Classification| [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/lenet/src/lenet.py)| MNIST | | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/cv/lenet/lenet_ascend_0.5.0_mnist_official_classification_20200716.tar.gz) -|Computer Vision (CV) | Image Classification| [VGG16](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/vgg16/src/vgg.py)| CIFAR-10 | | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/cv/vgg/vgg16_ascend_0.5.0_cifar10_official_classification_20200715.tar.gz) -|Computer Vision (CV) | Image Classification| [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | CIFAR-10| | | ✓ |[Download](http://download.mindspore.cn/model_zoo/official/cv/resnet/resnet50_v1.5_ascend_0.3.0_cifar10_official_classification_20200718.tar.gz) -|Computer Vision (CV) | Targets Detection| [YoloV3-DarkNet53](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_darknet53/src/yolo.py) | COCO 2014| | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/cv/yolo/yolov3_darknet53_ascend_0.5.0_coco2014_official_object_detection_20200717.tar.gz) -| Natural Language Processing (NLP) | Natural Language Understanding| [BERT_Base](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py) | zhwiki | | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/nlp/bert/bert_base_ascend_0.5.0_cn-wiki_official_nlp_20200720.tar.gz) -| Natural Language Processing (NLP) | Natural Language Understanding| [BERT_NEZHA](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py)| zhwiki| | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/nlp/bert/bert_nezha_ascend_0.5.0_cn-wiki_official_nlp_20200720.tar.gz) -| Natural Language Processing (NLP) | Natural Language Understanding| [Transformer](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/transformer/src/transformer_model.py)| WMT English-German| | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/nlp/transformer/transformer_ascend_0.5.0_wmtende_official_machine_translation_20200713.tar.gz) +# Network List + +`Linux` `Ascend` `GPU` `CPU` `Model Development` `Intermediate` `Expert` + + + +- [Network List](#network-list) + - [Model Zoo](#model-zoo) + - [Pre-trained Models](#pre-trained-models) + + + + + +## Model Zoo + +| Domain | Sub Domain | Network | Ascend | GPU | CPU +|:------ |:------| :----------- |:------ |:------ |:----- +|Computer Vision (CV) | Image Classification | [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/alexnet/src/alexnet.py) | Supported | Supported | Doing +| Computer Vision (CV) | Image Classification | [GoogleNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/googlenet/src/googlenet.py) | Supported | Doing | Doing +| Computer Vision (CV) | Image Classification | [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/lenet/src/lenet.py) | Supported | Supported | Supported +| Computer Vision (CV) | Image Classification | [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported | Supported | Doing +|Computer Vision (CV) | Image Classification | [ResNet-101](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported |Doing | Doing +|Computer Vision (CV) | Image Classification | [SE-ResNet50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported |Doing | Doing +|Computer Vision (CV) | Image Classification | [ResNext50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnext50/src/image_classification.py) | Supported | Supported | Doing +| Computer Vision (CV) | Image Classification | [VGG16](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/vgg16/src/vgg.py) | Supported | Doing | Doing +| Computer Vision (CV) | Image Classification | [InceptionV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/inceptionv3/src/inception_v3.py) | Supported | Doing | Doing +| Computer Vision (CV) | Mobile Image Classification
Image Classification
Semantic Tegmentation | [MobileNetV2](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/mobilenetv2/src/mobilenetV2.py) | Supported | Supported | Doing +| Computer Vision (CV) | Mobile Image Classification
Image Classification
Semantic Tegmentation | [MobileNetV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/mobilenetv3/src/mobilenetV3.py) | Doing | Supported | Doing +|Computer Vision (CV) | Targets Detection | [SSD](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/ssd/src/ssd.py) | Supported |Doing | Doing +| Computer Vision (CV) | Targets Detection | [YoloV3-ResNet18](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_resnet18/src/yolov3.py) | Supported | Doing | Doing +| Computer Vision (CV) | Targets Detection | [YoloV3-DarkNet53](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_darknet53/src/yolo.py) | Supported | Doing | Doing +| Computer Vision (CV) | Targets Detection | [FasterRCNN](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/faster_rcnn/src/FasterRcnn/faster_rcnn_r50.py) | Supported | Doing | Doing +| Computer Vision (CV) | Semantic Segmentation | [DeeplabV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/deeplabv3/src/deeplabv3.py) | Supported | Doing | Doing +| Computer Vision（CV） | Targets Detection | [WarpCTC](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/warpctc/src/warpctc.py) | Doing | Supported | Doing +| Natural Language Processing (NLP) | Natural Language Understanding | [BERT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py) | Supported | Doing | Doing +| Natural Language Processing (NLP) | Natural Language Understanding | [Transformer](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/transformer/src/transformer_model.py) | Supported | Doing | Doing +| Natural Language Processing (NLP) | Natural Language Understanding | [SentimentNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/lstm/src/lstm.py) | Doing | Supported | Supported +| Natural Language Processing (NLP) | Natural Language Understanding | [MASS](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/mass/src/transformer/transformer_for_train.py) | Supported | Doing | Doing +| Natural Language Processing (NLP) | Natural Language Understanding | [TinyBert](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/tinybert/src/tinybert_model.py) | Supported | Supported | Doing +| Recommender | Recommender System, CTR prediction | [DeepFM](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/recommend/deepfm/src/deepfm.py) | Supported | Supported | Doing +| Recommender | Recommender System, Search ranking | [Wide&Deep](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/recommend/wide_and_deep/src/wide_and_deep.py) | Supported | Supported | Doing +| Graph Neural Networks（GNN）| Text Classification | [GCN](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/gnn/gcn/src/gcn.py) | Supported | Doing | Doing +| Graph Neural Networks（GNN）| Text Classification | [GAT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/gnn/gat/src/gat.py) | Supported | Doing | Doing + +> You can also use [MindWizard Tool](https://gitee.com/mindspore/mindinsight/tree/master/mindinsight/wizard/) to quickly generate classic network scripts. + +## Pre-trained Models +*It refers to the released MindSpore version. The hardware platforms that support model training are CPU, GPU and Ascend. As shown in the table below, ✓ indicates that the pre-trained model run on the selected platform. + +| Domain | Sub Domain| Network | Dataset | CPU | GPU | Ascend | 0.5.0-beta* +|:------ |:------ | :------- |:------ |:------ |:------ |:----- |:----- +|Computer Vision (CV) | Image Classification| [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/alexnet/src/alexnet.py) | CIFAR-10| | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/cv/alexnet/alexnet_ascend_0.5.0_cifar10_official_classification_20200716.tar.gz) +|Computer Vision (CV) | Image Classification| [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/lenet/src/lenet.py)| MNIST | | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/cv/lenet/lenet_ascend_0.5.0_mnist_official_classification_20200716.tar.gz) +|Computer Vision (CV) | Image Classification| [VGG16](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/vgg16/src/vgg.py)| CIFAR-10 | | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/cv/vgg/vgg16_ascend_0.5.0_cifar10_official_classification_20200715.tar.gz) +|Computer Vision (CV) | Image Classification| [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | CIFAR-10| | | ✓ |[Download](http://download.mindspore.cn/model_zoo/official/cv/resnet/resnet50_v1.5_ascend_0.3.0_cifar10_official_classification_20200718.tar.gz) +|Computer Vision (CV) | Targets Detection| [YoloV3-DarkNet53](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_darknet53/src/yolo.py) | COCO 2014| | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/cv/yolo/yolov3_darknet53_ascend_0.5.0_coco2014_official_object_detection_20200717.tar.gz) +| Natural Language Processing (NLP) | Natural Language Understanding| [BERT_Base](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py) | zhwiki | | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/nlp/bert/bert_base_ascend_0.5.0_cn-wiki_official_nlp_20200720.tar.gz) +| Natural Language Processing (NLP) | Natural Language Understanding| [BERT_NEZHA](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py)| zhwiki| | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/nlp/bert/bert_nezha_ascend_0.5.0_cn-wiki_official_nlp_20200720.tar.gz) +| Natural Language Processing (NLP) | Natural Language Understanding| [Transformer](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/transformer/src/transformer_model.py)| WMT English-German| | | ✓ | [Download](http://download.mindspore.cn/model_zoo/official/nlp/transformer/transformer_ascend_0.5.0_wmtende_official_machine_translation_20200713.tar.gz) diff --git a/docs/source_en/operator_list.md b/docs/note/source_en/operator_list.md similarity index 100% rename from docs/source_en/operator_list.md rename to docs/note/source_en/operator_list.md diff --git a/lite/docs/source_en/operator_list.md b/docs/note/source_en/operator_list_lite.md similarity index 100% rename from lite/docs/source_en/operator_list.md rename to docs/note/source_en/operator_list_lite.md diff --git a/docs/source_en/roadmap.md b/docs/note/source_en/roadmap.md similarity index 100% rename from docs/source_en/roadmap.md rename to docs/note/source_en/roadmap.md diff --git a/docs/source_zh_cn/_static/logo_source.png b/docs/note/source_zh_cn/_static/logo_source.png similarity index 100% rename from docs/source_zh_cn/_static/logo_source.png rename to docs/note/source_zh_cn/_static/logo_source.png diff --git a/docs/source_zh_cn/architecture.md b/docs/note/source_zh_cn/architecture.md similarity index 100% rename from docs/source_zh_cn/architecture.md rename to docs/note/source_zh_cn/architecture.md diff --git a/lite/docs/source_zh_cn/architecture.md b/docs/note/source_zh_cn/architecture_lite.md similarity index 100% rename from lite/docs/source_zh_cn/architecture.md rename to docs/note/source_zh_cn/architecture_lite.md diff --git a/docs/source_zh_cn/benchmark.md b/docs/note/source_zh_cn/benchmark.md similarity index 100% rename from docs/source_zh_cn/benchmark.md rename to docs/note/source_zh_cn/benchmark.md diff --git a/docs/source_zh_cn/community.rst b/docs/note/source_zh_cn/community.rst similarity index 100% rename from docs/source_zh_cn/community.rst rename to docs/note/source_zh_cn/community.rst diff --git a/docs/source_zh_cn/conf.py b/docs/note/source_zh_cn/conf.py similarity index 100% rename from docs/source_zh_cn/conf.py rename to docs/note/source_zh_cn/conf.py diff --git a/docs/source_zh_cn/constraints_on_network_construction.md b/docs/note/source_zh_cn/constraints_on_network_construction.md similarity index 100% rename from docs/source_zh_cn/constraints_on_network_construction.md rename to docs/note/source_zh_cn/constraints_on_network_construction.md diff --git a/docs/source_zh_cn/design.rst b/docs/note/source_zh_cn/design.rst similarity index 100% rename from docs/source_zh_cn/design.rst rename to docs/note/source_zh_cn/design.rst diff --git a/docs/source_zh_cn/design/mindarmour/differential_privacy_design.md b/docs/note/source_zh_cn/design/mindarmour/differential_privacy_design.md similarity index 98% rename from docs/source_zh_cn/design/mindarmour/differential_privacy_design.md rename to docs/note/source_zh_cn/design/mindarmour/differential_privacy_design.md index 276219655693f4c3424f23906cde901cd465217a..b919d27912be4ef70a9361823b39dc304347da0d 100644 --- a/docs/source_zh_cn/design/mindarmour/differential_privacy_design.md +++ b/docs/note/source_zh_cn/design/mindarmour/differential_privacy_design.md @@ -1,70 +1,70 @@ -# 差分隐私 - -`Linux` `Ascend` `模型开发` `模型调优` `框架开发` `企业` `高级` `贡献者` - - - -- [差分隐私](#差分隐私) - - [总体设计](#总体设计) - - [差分隐私优化器](#差分隐私优化器) - - [差分隐私的噪声机制](#差分隐私的噪声机制) - - [Monitor](#monitor) - - [代码实现](#代码实现) - - [参考文献](#参考文献) - - - - - -## 总体设计 - -MindArmour的Differential-Privacy模块实现了差分隐私训练的能力。模型的训练主要由构建训练数据集、计算损失、计算梯度以及更新模型参数等过程组成，目前MindArmour的差分隐私训练主要着力于计算梯度的过程，通过相应的算法对梯度进行裁剪、加噪等处理，从而保护用户数据隐私。 - -![dp_arch](./images/dp_arch.png) - -
图1 差分隐私总体设计
- -图1是差分隐私训练的总体设计，主要由差分隐私噪声机制(DP Mechanisms)、差分隐私优化器(DP Optimizer)、差分隐私监控器(Privacy Monitor)组成。 - - -### 差分隐私优化器 - -差分隐私优化器继承了MindSpore优化器的能力，并使用差分隐私的噪声机制对梯度加扰保护。目前，MindArmour提供三类差分隐私优化器：固定高斯优化器、自适应高斯优化器、自适应裁剪优化器，每类差分隐私优化器从不同的角度为SGD、Momentum等常规优化器增加差分隐私保护的能力。 - -* 固定高斯优化器，是一种非自适应高斯噪声的差分隐私优化器。其优势在于可以严格控制差分隐私预算ϵ，缺点是在模型训练过程中，每个Step添加的噪声量固定，若迭代次数过大，训练后期的噪声使得模型收敛困难，甚至导致性能大幅下跌，模型可用性差。 -* 自适应高斯优化器，通过自适应调整标准差，来调整高斯分布噪声的大小，在模型训练初期，添加的噪声量较大，随着模型逐渐收敛，噪声量逐渐减小，噪声对于模型可用性的影响减小。自适应高斯噪声的缺点是不能严格控制差分隐私预算。 -* 自适应裁剪优化器，是一种自适应调整调整裁剪粒度的差分隐私优化器，梯度裁剪是差分隐私训练的一个重要操作，自适应裁剪优化器能够自适应的控制梯度裁剪的的比例在给定的范围波动，控制迭代训练过程中梯度裁剪的粒度。 - -### 差分隐私的噪声机制 - -噪声机制是构建差分隐私训练能力的基础，不同的噪声机制满足不同差分隐私优化器的需求，包括固定高斯分布噪声、自适应高斯分布噪声、自适应裁剪高斯分布噪声、拉普拉斯分布噪声等多种机制。 - -### Monitor - -Monitor提供RDP、ZCDP等回调函数，用于监测模型的差分隐私预算。 - -* ZCDP[2] - - ZCDP，zero-concentrated differential privacy，是一种宽松的差分隐私定义，利用Rényi散度来度量随机函数在相邻数据集上的分布差异。 - -* RDP[3] - - RDP，Rényi Differential Privacy，是一种更通用的基于R'enyi散度的差分隐私定义，利用Rényi散度来度量两个相邻数据集的分布差异。 - -相对于传统差分隐私，ZCDP和RDP都能能够提供更加严格的隐私预算上界保证。 - - -## 代码实现 - -* [mechanisms.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/mechanisms/mechanisms.py): 这个文件实现了差分隐私训练所需的噪声生成机制，包括简单高斯噪声、自适应高斯噪声、自适应裁剪高斯噪声等。 -* [optimizer.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/optimizer/optimizer.py): 这个文件实现了使用噪声生成机制在反向传播时添加噪声的根本逻辑。 -* [monitor.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/monitor/monitor.py): 实现了计算差分隐私预算的回调函数，模型训练过程中，会反馈当前的差分隐私预算。 -* [model.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/train/model.py): 这个文件实现了计算损失和梯度的逻辑，差分隐私训练的梯度截断逻辑在此文件中实现，且model.py是用户使用差分隐私训练能力的入口。 - -## 参考文献 - -[1] Dwork, Cynthia, and Jing Lei. "Differential privacy and robust statistics." *Proceedings of the forty-first annual ACM symposium on Theory of computing*. 2009. - -[2] Lee, Jaewoo, and Daniel Kifer. "Concentrated differentially private gradient descent with adaptive per-iteration privacy budget." *Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining*. 2018. - -[3] Mironov, Ilya. "Rényi differential privacy." *2017 IEEE 30th Computer Security Foundations Symposium (CSF)*. IEEE, 2017. +# 差分隐私 + +`Linux` `Ascend` `模型开发` `模型调优` `框架开发` `企业` `高级` `贡献者` + + + +- [差分隐私](#差分隐私) + - [总体设计](#总体设计) + - [差分隐私优化器](#差分隐私优化器) + - [差分隐私的噪声机制](#差分隐私的噪声机制) + - [Monitor](#monitor) + - [代码实现](#代码实现) + - [参考文献](#参考文献) + + + + + +## 总体设计 + +MindArmour的Differential-Privacy模块实现了差分隐私训练的能力。模型的训练主要由构建训练数据集、计算损失、计算梯度以及更新模型参数等过程组成，目前MindArmour的差分隐私训练主要着力于计算梯度的过程，通过相应的算法对梯度进行裁剪、加噪等处理，从而保护用户数据隐私。 + +![dp_arch](./images/dp_arch.png) + +
图1 差分隐私总体设计
+ +图1是差分隐私训练的总体设计，主要由差分隐私噪声机制(DP Mechanisms)、差分隐私优化器(DP Optimizer)、差分隐私监控器(Privacy Monitor)组成。 + + +### 差分隐私优化器 + +差分隐私优化器继承了MindSpore优化器的能力，并使用差分隐私的噪声机制对梯度加扰保护。目前，MindArmour提供三类差分隐私优化器：固定高斯优化器、自适应高斯优化器、自适应裁剪优化器，每类差分隐私优化器从不同的角度为SGD、Momentum等常规优化器增加差分隐私保护的能力。 + +* 固定高斯优化器，是一种非自适应高斯噪声的差分隐私优化器。其优势在于可以严格控制差分隐私预算ϵ，缺点是在模型训练过程中，每个Step添加的噪声量固定，若迭代次数过大，训练后期的噪声使得模型收敛困难，甚至导致性能大幅下跌，模型可用性差。 +* 自适应高斯优化器，通过自适应调整标准差，来调整高斯分布噪声的大小，在模型训练初期，添加的噪声量较大，随着模型逐渐收敛，噪声量逐渐减小，噪声对于模型可用性的影响减小。自适应高斯噪声的缺点是不能严格控制差分隐私预算。 +* 自适应裁剪优化器，是一种自适应调整调整裁剪粒度的差分隐私优化器，梯度裁剪是差分隐私训练的一个重要操作，自适应裁剪优化器能够自适应的控制梯度裁剪的的比例在给定的范围波动，控制迭代训练过程中梯度裁剪的粒度。 + +### 差分隐私的噪声机制 + +噪声机制是构建差分隐私训练能力的基础，不同的噪声机制满足不同差分隐私优化器的需求，包括固定高斯分布噪声、自适应高斯分布噪声、自适应裁剪高斯分布噪声、拉普拉斯分布噪声等多种机制。 + +### Monitor + +Monitor提供RDP、ZCDP等回调函数，用于监测模型的差分隐私预算。 + +* ZCDP[2] + + ZCDP，zero-concentrated differential privacy，是一种宽松的差分隐私定义，利用Rényi散度来度量随机函数在相邻数据集上的分布差异。 + +* RDP[3] + + RDP，Rényi Differential Privacy，是一种更通用的基于R'enyi散度的差分隐私定义，利用Rényi散度来度量两个相邻数据集的分布差异。 + +相对于传统差分隐私，ZCDP和RDP都能能够提供更加严格的隐私预算上界保证。 + + +## 代码实现 + +* [mechanisms.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/mechanisms/mechanisms.py): 这个文件实现了差分隐私训练所需的噪声生成机制，包括简单高斯噪声、自适应高斯噪声、自适应裁剪高斯噪声等。 +* [optimizer.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/optimizer/optimizer.py): 这个文件实现了使用噪声生成机制在反向传播时添加噪声的根本逻辑。 +* [monitor.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/monitor/monitor.py): 实现了计算差分隐私预算的回调函数，模型训练过程中，会反馈当前的差分隐私预算。 +* [model.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/privacy/diff_privacy/train/model.py): 这个文件实现了计算损失和梯度的逻辑，差分隐私训练的梯度截断逻辑在此文件中实现，且model.py是用户使用差分隐私训练能力的入口。 + +## 参考文献 + +[1] Dwork, Cynthia, and Jing Lei. "Differential privacy and robust statistics." *Proceedings of the forty-first annual ACM symposium on Theory of computing*. 2009. + +[2] Lee, Jaewoo, and Daniel Kifer. "Concentrated differentially private gradient descent with adaptive per-iteration privacy budget." *Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining*. 2018. + +[3] Mironov, Ilya. "Rényi differential privacy." *2017 IEEE 30th Computer Security Foundations Symposium (CSF)*. IEEE, 2017. diff --git a/docs/source_zh_cn/design/mindarmour/fuzzer_design.md b/docs/note/source_zh_cn/design/mindarmour/fuzzer_design.md similarity index 98% rename from docs/source_zh_cn/design/mindarmour/fuzzer_design.md rename to docs/note/source_zh_cn/design/mindarmour/fuzzer_design.md index 81a730c30a9ff3804fea82b355544b9894bfa8c1..f7554141b1532dfa8aca60fc3135328ecc2c90ed 100644 --- a/docs/source_zh_cn/design/mindarmour/fuzzer_design.md +++ b/docs/note/source_zh_cn/design/mindarmour/fuzzer_design.md @@ -1,73 +1,73 @@ -# AI模型安全测试 - -`Linux` `Ascend` `GPU` `CPU` `数据准备` `模型开发` `模型训练` `模型调优` `企业` `高级` - - - -- [AI模型安全测试](#ai模型安全测试) - - [背景](#背景) - - [Fuzzer设计图](#Fuzzer设计图) - - [Fuzzer流程](#Fuzzer流程) - - [代码实现](#代码实现) - - [参考文献](#参考文献) - - - - - -## 背景 - -不同于[传统程序的Fuzz安全测试](https://zhuanlan.zhihu.com/p/43432370)，MindArmour针对深度神经网络，提供AI模型安全测试模块Fuzzer。根据神经网络的特点，引入神经元覆盖率[1]的概念，作为Fuzz的测试指导，引导Fuzz朝神经元覆盖率增加的方向生成样本，让输入能够激活更多的神经元，神经元值的分布范围更广，以充分测试DNN，探索不同类型的模型输出结果、模型错误行为。 - -## Fuzzer设计图 - -AI模型安全测试设计图如下。 - -![fuzz_architecture](./images/fuzz_architecture.png) - -在用户接口层，需要用户提供原始数据集`DataSet`、被测试模型`Model`和配置Fuzzer参数`Fuzzer configuration`。Fuzzer模块对模型和数据进行Fuzz测试后，返回安全评估报告`Security Report`。 - -Fuzzer架构主要包括三个模块： - -1. Natural Threat/Adversarial Example Generator（数据变异模块）： - - 随机选择变异方法对种子数据变异生成多个变种。支持多种样本的变异策略， 包括： - - - 图像仿射变换方法如：平移、旋转、缩放、错切。 - - 基于图像像素值变化的方法如：改变对比度、亮度、模糊、加噪。 - - 基于对抗攻击的白盒、黑盒对抗样本生成方法，如FGSM、PGD、MDIIM。 - -2. Fuzzer moduler（变异指导模块）: - - 对变异生成的数据进行fuzz测试，观察神经元覆盖率的变化情况，如果生成的数据使得神经元覆盖率增加，则加入变异的种子队列，用于下一轮的数据变异。目前支持的神经元覆盖率指标包括KMNC、NBC、SNAC[2]。 - -3. Evaluation（评估模块）： - - 评估Fuzzer效果，生成数据的质量，变异方法的强度。支持3个类型5种指标，包括通用评价指标：accuracy，神经元覆盖率指标：kmnc， nbc，snac，对抗攻击评价指标：attack_success_rate。 - -## Fuzzer流程 - -![fuzz_process](./images/fuzz_process.png) - -具体的Fuzzer流程如下： - -1. 根据策略从种子队列中选择一个种子A。 -2. 随机选择变异策略，对种子A进行变异，生成多个变种数据A1，A2... -3. 用目标模型对变种A1，A2...进行预测，如果变种使得目标模型预测错误，则改变种进入Failed tests。 -4. 若目标模型对于变种的预测结果是正确的，用神经元覆盖率指标进行分析。 -5. 如果变种使得覆盖率增加，那么将该变种放入种子队列，用于下一轮变异。 - -通过多轮循环，我们获得一系列变异数据Fuzzed Tests，并进一步分析，从多个角度给出安全报告。可以用于深入分析神经网络模型的缺陷，从而针对这些缺陷，进行模型增强等，改善提升模型的通用性、鲁棒性。 - -## 代码实现 - -1. [fuzzing.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/fuzz_testing/fuzzing.py)：Fuzzer总体流程。 -2. [model_coverage_metrics.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/fuzz_testing/model_coverage_metrics.py)：神经元覆盖率指标，包括KMNC，NBC，SNAC。 -3. [image_transform.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/fuzz_testing/image_transform.py)：图像变异方法，包括基于像素值的变化方法和仿射变化方法。 -4. [adversarial attacks](https://gitee.com/mindspore/mindarmour/tree/master/mindarmour/adv_robustness/attacks)：对抗样本攻击方法，包含多种黑盒、白盒攻击方法。 - -## 参考文献 - -[1] Pei K, Cao Y, Yang J, et al. Deepxplore: Automated whitebox testing of deep learning systems[C]//Proceedings of the 26th Symposium on Operating Systems Principles. ACM, 2017: 1-18. - -[2]Ma L, Juefei-Xu F, Zhang F, et al. Deepgauge: Multi-granularity testing criteria for deep learning systems[C]//Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. ACM, 2018: 120-131. +# AI模型安全测试 + +`Linux` `Ascend` `GPU` `CPU` `数据准备` `模型开发` `模型训练` `模型调优` `企业` `高级` + + + +- [AI模型安全测试](#ai模型安全测试) + - [背景](#背景) + - [Fuzzer设计图](#Fuzzer设计图) + - [Fuzzer流程](#Fuzzer流程) + - [代码实现](#代码实现) + - [参考文献](#参考文献) + + + + + +## 背景 + +不同于[传统程序的Fuzz安全测试](https://zhuanlan.zhihu.com/p/43432370)，MindArmour针对深度神经网络，提供AI模型安全测试模块Fuzzer。根据神经网络的特点，引入神经元覆盖率[1]的概念，作为Fuzz的测试指导，引导Fuzz朝神经元覆盖率增加的方向生成样本，让输入能够激活更多的神经元，神经元值的分布范围更广，以充分测试DNN，探索不同类型的模型输出结果、模型错误行为。 + +## Fuzzer设计图 + +AI模型安全测试设计图如下。 + +![fuzz_architecture](./images/fuzz_architecture.png) + +在用户接口层，需要用户提供原始数据集`DataSet`、被测试模型`Model`和配置Fuzzer参数`Fuzzer configuration`。Fuzzer模块对模型和数据进行Fuzz测试后，返回安全评估报告`Security Report`。 + +Fuzzer架构主要包括三个模块： + +1. Natural Threat/Adversarial Example Generator（数据变异模块）： + + 随机选择变异方法对种子数据变异生成多个变种。支持多种样本的变异策略， 包括： + + - 图像仿射变换方法如：平移、旋转、缩放、错切。 + - 基于图像像素值变化的方法如：改变对比度、亮度、模糊、加噪。 + - 基于对抗攻击的白盒、黑盒对抗样本生成方法，如FGSM、PGD、MDIIM。 + +2. Fuzzer moduler（变异指导模块）: + + 对变异生成的数据进行fuzz测试，观察神经元覆盖率的变化情况，如果生成的数据使得神经元覆盖率增加，则加入变异的种子队列，用于下一轮的数据变异。目前支持的神经元覆盖率指标包括KMNC、NBC、SNAC[2]。 + +3. Evaluation（评估模块）： + + 评估Fuzzer效果，生成数据的质量，变异方法的强度。支持3个类型5种指标，包括通用评价指标：accuracy，神经元覆盖率指标：kmnc， nbc，snac，对抗攻击评价指标：attack_success_rate。 + +## Fuzzer流程 + +![fuzz_process](./images/fuzz_process.png) + +具体的Fuzzer流程如下： + +1. 根据策略从种子队列中选择一个种子A。 +2. 随机选择变异策略，对种子A进行变异，生成多个变种数据A1，A2... +3. 用目标模型对变种A1，A2...进行预测，如果变种使得目标模型预测错误，则改变种进入Failed tests。 +4. 若目标模型对于变种的预测结果是正确的，用神经元覆盖率指标进行分析。 +5. 如果变种使得覆盖率增加，那么将该变种放入种子队列，用于下一轮变异。 + +通过多轮循环，我们获得一系列变异数据Fuzzed Tests，并进一步分析，从多个角度给出安全报告。可以用于深入分析神经网络模型的缺陷，从而针对这些缺陷，进行模型增强等，改善提升模型的通用性、鲁棒性。 + +## 代码实现 + +1. [fuzzing.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/fuzz_testing/fuzzing.py)：Fuzzer总体流程。 +2. [model_coverage_metrics.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/fuzz_testing/model_coverage_metrics.py)：神经元覆盖率指标，包括KMNC，NBC，SNAC。 +3. [image_transform.py](https://gitee.com/mindspore/mindarmour/blob/master/mindarmour/fuzz_testing/image_transform.py)：图像变异方法，包括基于像素值的变化方法和仿射变化方法。 +4. [adversarial attacks](https://gitee.com/mindspore/mindarmour/tree/master/mindarmour/adv_robustness/attacks)：对抗样本攻击方法，包含多种黑盒、白盒攻击方法。 + +## 参考文献 + +[1] Pei K, Cao Y, Yang J, et al. Deepxplore: Automated whitebox testing of deep learning systems[C]//Proceedings of the 26th Symposium on Operating Systems Principles. ACM, 2017: 1-18. + +[2]Ma L, Juefei-Xu F, Zhang F, et al. Deepgauge: Multi-granularity testing criteria for deep learning systems[C]//Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. ACM, 2018: 120-131. diff --git a/docs/source_zh_cn/design/mindarmour/images/dp_arch.png b/docs/note/source_zh_cn/design/mindarmour/images/dp_arch.png similarity index 100% rename from docs/source_zh_cn/design/mindarmour/images/dp_arch.png rename to docs/note/source_zh_cn/design/mindarmour/images/dp_arch.png diff --git a/docs/source_zh_cn/design/mindarmour/images/fuzz_architecture.png b/docs/note/source_zh_cn/design/mindarmour/images/fuzz_architecture.png similarity index 100% rename from docs/source_zh_cn/design/mindarmour/images/fuzz_architecture.png rename to docs/note/source_zh_cn/design/mindarmour/images/fuzz_architecture.png diff --git a/docs/source_zh_cn/design/mindarmour/images/fuzz_process.png b/docs/note/source_zh_cn/design/mindarmour/images/fuzz_process.png similarity index 100% rename from docs/source_zh_cn/design/mindarmour/images/fuzz_process.png rename to docs/note/source_zh_cn/design/mindarmour/images/fuzz_process.png diff --git a/docs/source_zh_cn/design/mindinsight/graph_visual_design.md b/docs/note/source_zh_cn/design/mindinsight/graph_visual_design.md similarity index 100% rename from docs/source_zh_cn/design/mindinsight/graph_visual_design.md rename to docs/note/source_zh_cn/design/mindinsight/graph_visual_design.md diff --git a/docs/source_zh_cn/design/mindinsight/images/analyser_class_profiler.png b/docs/note/source_zh_cn/design/mindinsight/images/analyser_class_profiler.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/analyser_class_profiler.png rename to docs/note/source_zh_cn/design/mindinsight/images/analyser_class_profiler.png diff --git a/docs/source_zh_cn/design/mindinsight/images/context_profiler.png b/docs/note/source_zh_cn/design/mindinsight/images/context_profiler.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/context_profiler.png rename to docs/note/source_zh_cn/design/mindinsight/images/context_profiler.png diff --git a/docs/source_zh_cn/design/mindinsight/images/graph_visual_class_design.png b/docs/note/source_zh_cn/design/mindinsight/images/graph_visual_class_design.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/graph_visual_class_design.png rename to docs/note/source_zh_cn/design/mindinsight/images/graph_visual_class_design.png diff --git a/docs/source_zh_cn/design/mindinsight/images/graph_visual_main.png b/docs/note/source_zh_cn/design/mindinsight/images/graph_visual_main.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/graph_visual_main.png rename to docs/note/source_zh_cn/design/mindinsight/images/graph_visual_main.png diff --git a/docs/source_zh_cn/design/mindinsight/images/graph_visual_right_side.png b/docs/note/source_zh_cn/design/mindinsight/images/graph_visual_right_side.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/graph_visual_right_side.png rename to docs/note/source_zh_cn/design/mindinsight/images/graph_visual_right_side.png diff --git a/docs/source_zh_cn/design/mindinsight/images/module_profiler.png b/docs/note/source_zh_cn/design/mindinsight/images/module_profiler.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/module_profiler.png rename to docs/note/source_zh_cn/design/mindinsight/images/module_profiler.png diff --git a/docs/source_zh_cn/design/mindinsight/images/parser_module_profiler.png b/docs/note/source_zh_cn/design/mindinsight/images/parser_module_profiler.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/parser_module_profiler.png rename to docs/note/source_zh_cn/design/mindinsight/images/parser_module_profiler.png diff --git a/docs/source_zh_cn/design/mindinsight/images/proposer_class_profiler.png b/docs/note/source_zh_cn/design/mindinsight/images/proposer_class_profiler.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/proposer_class_profiler.png rename to docs/note/source_zh_cn/design/mindinsight/images/proposer_class_profiler.png diff --git a/docs/source_zh_cn/design/mindinsight/images/proposer_module_profiler.png b/docs/note/source_zh_cn/design/mindinsight/images/proposer_module_profiler.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/proposer_module_profiler.png rename to docs/note/source_zh_cn/design/mindinsight/images/proposer_module_profiler.png diff --git a/docs/source_zh_cn/design/mindinsight/images/tensor_histogram.png b/docs/note/source_zh_cn/design/mindinsight/images/tensor_histogram.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/tensor_histogram.png rename to docs/note/source_zh_cn/design/mindinsight/images/tensor_histogram.png diff --git a/docs/source_zh_cn/design/mindinsight/images/tensor_table.png b/docs/note/source_zh_cn/design/mindinsight/images/tensor_table.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/tensor_table.png rename to docs/note/source_zh_cn/design/mindinsight/images/tensor_table.png diff --git a/docs/source_zh_cn/design/mindinsight/images/time_order_profiler.png b/docs/note/source_zh_cn/design/mindinsight/images/time_order_profiler.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/time_order_profiler.png rename to docs/note/source_zh_cn/design/mindinsight/images/time_order_profiler.png diff --git a/docs/source_zh_cn/design/mindinsight/images/training_visualization_architecture.png b/docs/note/source_zh_cn/design/mindinsight/images/training_visualization_architecture.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/training_visualization_architecture.png rename to docs/note/source_zh_cn/design/mindinsight/images/training_visualization_architecture.png diff --git a/docs/source_zh_cn/design/mindinsight/images/training_visualization_data_flow.png b/docs/note/source_zh_cn/design/mindinsight/images/training_visualization_data_flow.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/training_visualization_data_flow.png rename to docs/note/source_zh_cn/design/mindinsight/images/training_visualization_data_flow.png diff --git a/docs/source_zh_cn/design/mindinsight/images/training_visualization_data_model.png b/docs/note/source_zh_cn/design/mindinsight/images/training_visualization_data_model.png similarity index 100% rename from docs/source_zh_cn/design/mindinsight/images/training_visualization_data_model.png rename to docs/note/source_zh_cn/design/mindinsight/images/training_visualization_data_model.png diff --git a/docs/source_zh_cn/design/mindinsight/profiler_design.md b/docs/note/source_zh_cn/design/mindinsight/profiler_design.md similarity index 98% rename from docs/source_zh_cn/design/mindinsight/profiler_design.md rename to docs/note/source_zh_cn/design/mindinsight/profiler_design.md index 0171bd3f9bcd4176f75796f6f54c5344be1872de..cb0832e23e6be6b079e8307b74189842e432f7ea 100644 --- a/docs/source_zh_cn/design/mindinsight/profiler_design.md +++ b/docs/note/source_zh_cn/design/mindinsight/profiler_design.md @@ -1,174 +1,174 @@ -# Profiler设计文档 - -`Linux` `Ascend` `GPU` `模型开发` `模型调优` `框架开发` `中级` `高级` `贡献者` - - - -- [Profiler设计文档](#profiler设计文档) - - [背景](#背景) - - [Profiler框架设计](#profiler架构设计) - - [上下文](#上下文) - - [模块层级结构](#模块层级结构) - - [内部模块交互](#内部模块交互) - - [子模块设计](#准备训练脚本) - - [ProfilerAPI和Controller](#profiler-api-controller) - - [ProfilerAPI和Controller模块介绍](#profiler-api-controller模块介绍) - - [Analyser](#analyser) - - [Analyser模块介绍](#analyser模块介绍) - - [Analyser模块设计](#analyser模块设计) - - [Parser](#parser) - - [Parser模块介绍](#parser模块介绍) - - [Parser模块设计](#parser模块设计) - - [Proposer](#proposer) - - [Proposer模块介绍](#proposer模块介绍) - - [Proposer模块设计](#proposer模块设计) - - - - - -## 背景 - -为了支持用户在MindSpore进行模型开发性能调试，需要提供易用的Profile工具，直观地展现网络模型各维度的性能信息，为用户提供易用、丰富的性能分析功能，帮助用户快速定位网络中性能问题。 - -## Profiler架构设计 -这一章将介绍Profiler的架构设计，第一节从整体Profiler的角度出发介绍其上下文交互关系，第二节将打开Profiler内部，介绍模块层架结构以及模块划分，第三节将介绍模块间的交互调用关系。 - -### 上下文 - -Profiler是MindSpore调试调优工具的一部分，在整个使用过程中的上下文环境如下图所示： - -![context_profiler.png](./images/context_profiler.png) - -图1：上下文关系图 - -如上图所示，Profiler与其他部分的交互包括： - -1. 在训练脚本中调用MindSpore的Profiler向MindSpore的ada通信模块发送启动收集性能数据的命令，最终由ada生成性能原始数据； - -2. MindSpore侧Profiler将在用户脚本中对原始数据进行解析，并在用户指定的文件夹下面生成中间数据结果； - -3. Mindinsight侧Profiler对接中间数据，提供可视化Profiler功能供用户使用。 -### 模块层级结构 - -模块层级划分如下： - -![module_profiler.png](./images/module_profiler.png) - -图2：层级模块关系图 - - -如上图所示，各个模块功能介绍如下： -1. ProfilerAPI是代码侧对用户提供的调用入口，为用户提供了性能收集启动接口以及分析接口； -2. Controller是ProfilerAPI下层的模块，被ProfilerAPI中的启动接口调用，负责控制下方性能收集功能的启动停止，原始数据会被ada写入固定位置； -3. Parser是性能原始数据解析模块，由于性能原始数据是在设备侧收集的信息，所以信息不能直接被用户所理解，该模块负责将信息进行解析、组合、转换，最终形成用户可理解、上层可分析的中间结果； -4. Analyser获取下层Parser解析出的中间结果，负责对中间结果的封装、筛选、排序，最终按照信息分类，返回各个类别对应的信息，提供给上层的表现层Profiler API、RESTful使用； -5. 通过RESTful调用后端Analyser提供的common API，获取目标数据，以RESTful接口对接前端。 - -### 内部模块交互 -从用户角度，有两种使用形式API、RESTful，我们以API为例，阐述一个完整的内部模块交互流程： - -![time_order_profiler.png](./images/time_order_profiler.png) - -图3：模块交互图 - -如上图所示，各个模块交互流程如下： - -1. ProfilerAPI会调用下层Controller的控制函数，控制下层收集模块进行收集，目前收集模块（ada）是以常驻进程的方式接受命令，并独立工作收集性能信息的； - -2. 用户在训练结束后会调用ProfilerAPI的分析接口； - -3. Profiler API分析接口首先使用Parser模块对性能数据进行解析，产生中间结果，再调用Aalayser进行中间结果分析，最终将各类信息返回至用户侧。 - -## 子模块设计 -### ProfilerAPI和Controller - -#### ProfilerAPI和Controller模块说明 -ProfilerAPI为用户在训练脚本侧提供入口API，用户通过ProfilerAPI启动性能收集以及对性能数据进行分析。 -ProfilerAPI通过Controller下发命令，完成对ada启动的控制。 - -#### ProfilerAPI和Controller模块设计 -ProfilerAPI模块，属于上层应用接口层，由训练脚本集成。功能分为两部分： - -- 训练前调用底层Controller接口，下发命令，启动profiling统计任务。 - -- 训练完成后，调用底层Controller接口，下发命令，停止性能统计任务，再调用Analyser、Parser模块接口解析数据文件，生成算子性能统计、training trace统计等结果数据。 - - -Controller模块提供对上层接口，并调用底层性能收集模块接口，下发启动和停止性能收集的命令。 - -最终生成的性能原始数据主要包含： - -- `hwts.log.data.45.dev.profiler_default_tag`文件：存储算子执行信息，包括task的开始/结束，stream id的信息等； -- `DATA_PREPROCESS.dev.AICPU`文件：AI CPU算子的执行各阶段的执行时间信息； -- `Framework.host.task_desc_info`文件：存储算子id与算子名称的对应关系，以及每个算子的输入输出信息； -- `training_trace.46.dev.profiler_default_tag`文件：存储每个step的开始结束时刻，迭代间隙、迭代前向反向、迭代拖尾的时刻信息。 - -### Parser -#### Parser模块介绍 -Parser是原始性能数据解析模块，由于原始性能数据是在设备侧收集的信息，所以信息不能直接被用户所理解，该模块负责将信息进行解析、组合、转换，最终形成用户可理解、上层可分析的中间结果。 -#### Parser模块设计 -![parser_module_profiler.png](./images/parser_module_profiler.png) - -图4：Parser模块图 - -如上图所示，Parser模块主要由HWTS Parser、AI CPU Parser、Framework Parser、Training Trace Parser组成，每个模块对应解析一种原始数据，通过解析原始数据得到用户能读懂的中间文件。 - -- HWTS Parser：解析`hwts.log.data.45.dev.profiler_default_tag`文件，获得Device基于task的统计信息，如每个task的开始/结束，stream id等数据，用于算子执行时间的计算。 -- AI CPU Parser：解析`DATA_PREPROCESS.dev.AICPU`文件，获得AI CPU算子的执行各阶段的执行时间信息。 -- Framework Parser：解析`Framework.host.task_desc_info`文件，用于获取AI Core算子与task的对应关系，算子关键信息等内容。 -- Training Trace Parser：解析`training_trace.46.dev.profiler_default_tag`文件，用于分析训练各阶段的时间。 - -### Analyser - -#### Analyser模块介绍 -分析器的作用是对解析阶段生成的中间结果，进行筛选、排序、查询、分页等相关操作。 - -#### Analyser模块设计 - -该模块负责解析Parser生成的中间文件，为上层数据分析提供通用接口，将分析后的数据返回给上层展示给用户，由于各种中间文件有一定的共同点，可以抽象出公共内容，所以Analyser类设计如下图所示： - -![analyser_class_profiler.png](./images/analyser_class_profiler.png) - -图5：Analyser类图 - -如上图所示，针对期望查询的不同内容，实现多个Analyser，每个Analyser可以定义筛选、排序、分页条件。每个Analyser知道自己需要哪些中间文件来进行数据的合并、筛选、排序。Analyser与Parser是通过Parser生成的中间文件关联起来的，本身不存在函数调用的情况，这样对两个模块进行了解耦。 - -针对算子信息的Analyser，目前存在两种： - -- 针对算子类型平均信息的筛选。 -- 针对每个算子详细平均信息的筛选，分别在两个Analyser中实现（AicoreTypeAnalyser、AicoreDetailAnalyser）。 - -为了隐藏Analyser内部实现，方便调用，使用简单工厂模式，通过AnalyserFactory获取指定的Analyser。 - - -### Proposer -#### Proposer模块介绍 -Proposer是Profiler性能优化建议模块，Proposer调用Analyser模块获取性能数据，通过调优规则对性能数据进行分析，输出调优建议由UI、API接口展示给用户。 - -#### Proposer模块设计 - -模块划分如下所示： - -![proposer_module_profiler.png](./images/proposer_module_profiler.png) - -图6：Proposer模块图 - -模块设计如上图所示： - -- Proposer提供接口用于API、RESTful调用以获取优化建议。 -- Proposer调用Analyser接口，获取性能数据并根据优化规则，获得优化建议。 -- Proposer调用Analyser工厂获得Analyser对象。 - -调用Analyser对象的query接口获取信息，包括：按时间排序TOP N的AICore、AICoreType、AICpu算子信息、traning trace各阶段的时间信息。 - -模块类设计如下所示： - -![proposer_class_profiler.png](./images/proposer_class_profiler.png) - -图7：Proposer类图 - -如上模块类图所示： - -- 各类型Proposer继承抽象类Proposer并实现analyze方法； +# Profiler设计文档 + +`Linux` `Ascend` `GPU` `模型开发` `模型调优` `框架开发` `中级` `高级` `贡献者` + + + +- [Profiler设计文档](#profiler设计文档) + - [背景](#背景) + - [Profiler框架设计](#profiler架构设计) + - [上下文](#上下文) + - [模块层级结构](#模块层级结构) + - [内部模块交互](#内部模块交互) + - [子模块设计](#准备训练脚本) + - [ProfilerAPI和Controller](#profiler-api-controller) + - [ProfilerAPI和Controller模块介绍](#profiler-api-controller模块介绍) + - [Analyser](#analyser) + - [Analyser模块介绍](#analyser模块介绍) + - [Analyser模块设计](#analyser模块设计) + - [Parser](#parser) + - [Parser模块介绍](#parser模块介绍) + - [Parser模块设计](#parser模块设计) + - [Proposer](#proposer) + - [Proposer模块介绍](#proposer模块介绍) + - [Proposer模块设计](#proposer模块设计) + + + + + +## 背景 + +为了支持用户在MindSpore进行模型开发性能调试，需要提供易用的Profile工具，直观地展现网络模型各维度的性能信息，为用户提供易用、丰富的性能分析功能，帮助用户快速定位网络中性能问题。 + +## Profiler架构设计 +这一章将介绍Profiler的架构设计，第一节从整体Profiler的角度出发介绍其上下文交互关系，第二节将打开Profiler内部，介绍模块层架结构以及模块划分，第三节将介绍模块间的交互调用关系。 + +### 上下文 + +Profiler是MindSpore调试调优工具的一部分，在整个使用过程中的上下文环境如下图所示： + +![context_profiler.png](./images/context_profiler.png) + +图1：上下文关系图 + +如上图所示，Profiler与其他部分的交互包括： + +1. 在训练脚本中调用MindSpore的Profiler向MindSpore的ada通信模块发送启动收集性能数据的命令，最终由ada生成性能原始数据； + +2. MindSpore侧Profiler将在用户脚本中对原始数据进行解析，并在用户指定的文件夹下面生成中间数据结果； + +3. Mindinsight侧Profiler对接中间数据，提供可视化Profiler功能供用户使用。 +### 模块层级结构 + +模块层级划分如下： + +![module_profiler.png](./images/module_profiler.png) + +图2：层级模块关系图 + + +如上图所示，各个模块功能介绍如下： +1. ProfilerAPI是代码侧对用户提供的调用入口，为用户提供了性能收集启动接口以及分析接口； +2. Controller是ProfilerAPI下层的模块，被ProfilerAPI中的启动接口调用，负责控制下方性能收集功能的启动停止，原始数据会被ada写入固定位置； +3. Parser是性能原始数据解析模块，由于性能原始数据是在设备侧收集的信息，所以信息不能直接被用户所理解，该模块负责将信息进行解析、组合、转换，最终形成用户可理解、上层可分析的中间结果； +4. Analyser获取下层Parser解析出的中间结果，负责对中间结果的封装、筛选、排序，最终按照信息分类，返回各个类别对应的信息，提供给上层的表现层Profiler API、RESTful使用； +5. 通过RESTful调用后端Analyser提供的common API，获取目标数据，以RESTful接口对接前端。 + +### 内部模块交互 +从用户角度，有两种使用形式API、RESTful，我们以API为例，阐述一个完整的内部模块交互流程： + +![time_order_profiler.png](./images/time_order_profiler.png) + +图3：模块交互图 + +如上图所示，各个模块交互流程如下： + +1. ProfilerAPI会调用下层Controller的控制函数，控制下层收集模块进行收集，目前收集模块（ada）是以常驻进程的方式接受命令，并独立工作收集性能信息的； + +2. 用户在训练结束后会调用ProfilerAPI的分析接口； + +3. Profiler API分析接口首先使用Parser模块对性能数据进行解析，产生中间结果，再调用Aalayser进行中间结果分析，最终将各类信息返回至用户侧。 + +## 子模块设计 +### ProfilerAPI和Controller + +#### ProfilerAPI和Controller模块说明 +ProfilerAPI为用户在训练脚本侧提供入口API，用户通过ProfilerAPI启动性能收集以及对性能数据进行分析。 +ProfilerAPI通过Controller下发命令，完成对ada启动的控制。 + +#### ProfilerAPI和Controller模块设计 +ProfilerAPI模块，属于上层应用接口层，由训练脚本集成。功能分为两部分： + +- 训练前调用底层Controller接口，下发命令，启动profiling统计任务。 + +- 训练完成后，调用底层Controller接口，下发命令，停止性能统计任务，再调用Analyser、Parser模块接口解析数据文件，生成算子性能统计、training trace统计等结果数据。 + + +Controller模块提供对上层接口，并调用底层性能收集模块接口，下发启动和停止性能收集的命令。 + +最终生成的性能原始数据主要包含： + +- `hwts.log.data.45.dev.profiler_default_tag`文件：存储算子执行信息，包括task的开始/结束，stream id的信息等； +- `DATA_PREPROCESS.dev.AICPU`文件：AI CPU算子的执行各阶段的执行时间信息； +- `Framework.host.task_desc_info`文件：存储算子id与算子名称的对应关系，以及每个算子的输入输出信息； +- `training_trace.46.dev.profiler_default_tag`文件：存储每个step的开始结束时刻，迭代间隙、迭代前向反向、迭代拖尾的时刻信息。 + +### Parser +#### Parser模块介绍 +Parser是原始性能数据解析模块，由于原始性能数据是在设备侧收集的信息，所以信息不能直接被用户所理解，该模块负责将信息进行解析、组合、转换，最终形成用户可理解、上层可分析的中间结果。 +#### Parser模块设计 +![parser_module_profiler.png](./images/parser_module_profiler.png) + +图4：Parser模块图 + +如上图所示，Parser模块主要由HWTS Parser、AI CPU Parser、Framework Parser、Training Trace Parser组成，每个模块对应解析一种原始数据，通过解析原始数据得到用户能读懂的中间文件。 + +- HWTS Parser：解析`hwts.log.data.45.dev.profiler_default_tag`文件，获得Device基于task的统计信息，如每个task的开始/结束，stream id等数据，用于算子执行时间的计算。 +- AI CPU Parser：解析`DATA_PREPROCESS.dev.AICPU`文件，获得AI CPU算子的执行各阶段的执行时间信息。 +- Framework Parser：解析`Framework.host.task_desc_info`文件，用于获取AI Core算子与task的对应关系，算子关键信息等内容。 +- Training Trace Parser：解析`training_trace.46.dev.profiler_default_tag`文件，用于分析训练各阶段的时间。 + +### Analyser + +#### Analyser模块介绍 +分析器的作用是对解析阶段生成的中间结果，进行筛选、排序、查询、分页等相关操作。 + +#### Analyser模块设计 + +该模块负责解析Parser生成的中间文件，为上层数据分析提供通用接口，将分析后的数据返回给上层展示给用户，由于各种中间文件有一定的共同点，可以抽象出公共内容，所以Analyser类设计如下图所示： + +![analyser_class_profiler.png](./images/analyser_class_profiler.png) + +图5：Analyser类图 + +如上图所示，针对期望查询的不同内容，实现多个Analyser，每个Analyser可以定义筛选、排序、分页条件。每个Analyser知道自己需要哪些中间文件来进行数据的合并、筛选、排序。Analyser与Parser是通过Parser生成的中间文件关联起来的，本身不存在函数调用的情况，这样对两个模块进行了解耦。 + +针对算子信息的Analyser，目前存在两种： + +- 针对算子类型平均信息的筛选。 +- 针对每个算子详细平均信息的筛选，分别在两个Analyser中实现（AicoreTypeAnalyser、AicoreDetailAnalyser）。 + +为了隐藏Analyser内部实现，方便调用，使用简单工厂模式，通过AnalyserFactory获取指定的Analyser。 + + +### Proposer +#### Proposer模块介绍 +Proposer是Profiler性能优化建议模块，Proposer调用Analyser模块获取性能数据，通过调优规则对性能数据进行分析，输出调优建议由UI、API接口展示给用户。 + +#### Proposer模块设计 + +模块划分如下所示： + +![proposer_module_profiler.png](./images/proposer_module_profiler.png) + +图6：Proposer模块图 + +模块设计如上图所示： + +- Proposer提供接口用于API、RESTful调用以获取优化建议。 +- Proposer调用Analyser接口，获取性能数据并根据优化规则，获得优化建议。 +- Proposer调用Analyser工厂获得Analyser对象。 + +调用Analyser对象的query接口获取信息，包括：按时间排序TOP N的AICore、AICoreType、AICpu算子信息、traning trace各阶段的时间信息。 + +模块类设计如下所示： + +![proposer_class_profiler.png](./images/proposer_class_profiler.png) + +图7：Proposer类图 + +如上模块类图所示： + +- 各类型Proposer继承抽象类Proposer并实现analyze方法； - API、CLI通过调用工厂ProposerFactory获取Proposer，并调用Proposer.analyze函数获取各类型的Proposer分析的优化建议。 \ No newline at end of file diff --git a/docs/source_zh_cn/design/mindinsight/tensor_visual_design.md b/docs/note/source_zh_cn/design/mindinsight/tensor_visual_design.md similarity index 100% rename from docs/source_zh_cn/design/mindinsight/tensor_visual_design.md rename to docs/note/source_zh_cn/design/mindinsight/tensor_visual_design.md diff --git a/docs/source_zh_cn/design/mindinsight/training_visual_design.md b/docs/note/source_zh_cn/design/mindinsight/training_visual_design.md similarity index 100% rename from docs/source_zh_cn/design/mindinsight/training_visual_design.md rename to docs/note/source_zh_cn/design/mindinsight/training_visual_design.md diff --git a/docs/source_zh_cn/design/mindspore/distributed_training_design.md b/docs/note/source_zh_cn/design/mindspore/distributed_training_design.md similarity index 100% rename from docs/source_zh_cn/design/mindspore/distributed_training_design.md rename to docs/note/source_zh_cn/design/mindspore/distributed_training_design.md diff --git a/docs/source_zh_cn/design/mindspore/images/auto_parallel.png b/docs/note/source_zh_cn/design/mindspore/images/auto_parallel.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/auto_parallel.png rename to docs/note/source_zh_cn/design/mindspore/images/auto_parallel.png diff --git a/docs/source_zh_cn/design/mindspore/images/data_parallel.png b/docs/note/source_zh_cn/design/mindspore/images/data_parallel.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/data_parallel.png rename to docs/note/source_zh_cn/design/mindspore/images/data_parallel.png diff --git a/docs/source_zh_cn/design/mindspore/images/ir/cf.dot b/docs/note/source_zh_cn/design/mindspore/images/ir/cf.dot similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/ir/cf.dot rename to docs/note/source_zh_cn/design/mindspore/images/ir/cf.dot diff --git a/docs/source_zh_cn/design/mindspore/images/ir/cf.png b/docs/note/source_zh_cn/design/mindspore/images/ir/cf.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/ir/cf.png rename to docs/note/source_zh_cn/design/mindspore/images/ir/cf.png diff --git a/docs/source_zh_cn/design/mindspore/images/ir/closure.dot b/docs/note/source_zh_cn/design/mindspore/images/ir/closure.dot similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/ir/closure.dot rename to docs/note/source_zh_cn/design/mindspore/images/ir/closure.dot diff --git a/docs/source_zh_cn/design/mindspore/images/ir/closure.png b/docs/note/source_zh_cn/design/mindspore/images/ir/closure.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/ir/closure.png rename to docs/note/source_zh_cn/design/mindspore/images/ir/closure.png diff --git a/docs/source_zh_cn/design/mindspore/images/ir/hof.dot b/docs/note/source_zh_cn/design/mindspore/images/ir/hof.dot similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/ir/hof.dot rename to docs/note/source_zh_cn/design/mindspore/images/ir/hof.dot diff --git a/docs/source_zh_cn/design/mindspore/images/ir/hof.png b/docs/note/source_zh_cn/design/mindspore/images/ir/hof.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/ir/hof.png rename to docs/note/source_zh_cn/design/mindspore/images/ir/hof.png diff --git a/docs/source_zh_cn/design/mindspore/images/ir/ir.dot b/docs/note/source_zh_cn/design/mindspore/images/ir/ir.dot similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/ir/ir.dot rename to docs/note/source_zh_cn/design/mindspore/images/ir/ir.dot diff --git a/docs/source_zh_cn/design/mindspore/images/ir/ir.png b/docs/note/source_zh_cn/design/mindspore/images/ir/ir.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/ir/ir.png rename to docs/note/source_zh_cn/design/mindspore/images/ir/ir.png diff --git a/docs/source_zh_cn/design/mindspore/images/operator_split.png b/docs/note/source_zh_cn/design/mindspore/images/operator_split.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/operator_split.png rename to docs/note/source_zh_cn/design/mindspore/images/operator_split.png diff --git a/docs/source_zh_cn/design/mindspore/images/tensor_redistribution.png b/docs/note/source_zh_cn/design/mindspore/images/tensor_redistribution.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/tensor_redistribution.png rename to docs/note/source_zh_cn/design/mindspore/images/tensor_redistribution.png diff --git a/docs/source_zh_cn/design/mindspore/images/tensor_redistribution1.png b/docs/note/source_zh_cn/design/mindspore/images/tensor_redistribution1.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/tensor_redistribution1.png rename to docs/note/source_zh_cn/design/mindspore/images/tensor_redistribution1.png diff --git a/docs/source_zh_cn/design/mindspore/images/tensor_redistribution2.png b/docs/note/source_zh_cn/design/mindspore/images/tensor_redistribution2.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/tensor_redistribution2.png rename to docs/note/source_zh_cn/design/mindspore/images/tensor_redistribution2.png diff --git a/docs/source_zh_cn/design/mindspore/images/tensor_redistribution3.png b/docs/note/source_zh_cn/design/mindspore/images/tensor_redistribution3.png similarity index 100% rename from docs/source_zh_cn/design/mindspore/images/tensor_redistribution3.png rename to docs/note/source_zh_cn/design/mindspore/images/tensor_redistribution3.png diff --git a/docs/source_zh_cn/design/mindspore/ir.md b/docs/note/source_zh_cn/design/mindspore/ir.md similarity index 100% rename from docs/source_zh_cn/design/mindspore/ir.md rename to docs/note/source_zh_cn/design/mindspore/ir.md diff --git a/docs/source_zh_cn/glossary.md b/docs/note/source_zh_cn/glossary.md similarity index 100% rename from docs/source_zh_cn/glossary.md rename to docs/note/source_zh_cn/glossary.md diff --git a/lite/docs/source_zh_cn/glossary.md b/docs/note/source_zh_cn/glossary_lite.md similarity index 100% rename from lite/docs/source_zh_cn/glossary.md rename to docs/note/source_zh_cn/glossary_lite.md diff --git a/docs/source_zh_cn/help_seeking_path.md b/docs/note/source_zh_cn/help_seeking_path.md similarity index 100% rename from docs/source_zh_cn/help_seeking_path.md rename to docs/note/source_zh_cn/help_seeking_path.md diff --git a/docs/note/source_zh_cn/images/MindSpore-Lite-architecture.png b/docs/note/source_zh_cn/images/MindSpore-Lite-architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..abf28796690f5649f8bc92382dfd4c2c83187620 Binary files /dev/null and b/docs/note/source_zh_cn/images/MindSpore-Lite-architecture.png differ diff --git a/docs/source_zh_cn/images/architecture.eddx b/docs/note/source_zh_cn/images/architecture.eddx similarity index 100% rename from docs/source_zh_cn/images/architecture.eddx rename to docs/note/source_zh_cn/images/architecture.eddx diff --git a/docs/source_zh_cn/images/architecture.png b/docs/note/source_zh_cn/images/architecture.png similarity index 100% rename from docs/source_zh_cn/images/architecture.png rename to docs/note/source_zh_cn/images/architecture.png diff --git a/docs/source_zh_cn/images/help_seeking_path.png b/docs/note/source_zh_cn/images/help_seeking_path.png similarity index 100% rename from docs/source_zh_cn/images/help_seeking_path.png rename to docs/note/source_zh_cn/images/help_seeking_path.png diff --git a/docs/source_zh_cn/index.rst b/docs/note/source_zh_cn/index.rst similarity index 100% rename from docs/source_zh_cn/index.rst rename to docs/note/source_zh_cn/index.rst diff --git a/lite/docs/source_zh_cn/index.rst b/docs/note/source_zh_cn/index_lite.rst similarity index 100% rename from lite/docs/source_zh_cn/index.rst rename to docs/note/source_zh_cn/index_lite.rst diff --git a/docs/source_zh_cn/network_list.md b/docs/note/source_zh_cn/network_list.md similarity index 99% rename from docs/source_zh_cn/network_list.md rename to docs/note/source_zh_cn/network_list.md index 351a5223c2083d20655f4eac118991dd08da7400..638b9518ac39645cd505fb655f5975060760bb54 100644 --- a/docs/source_zh_cn/network_list.md +++ b/docs/note/source_zh_cn/network_list.md @@ -1,60 +1,60 @@ -# 网络支持 - -`Linux` `Ascend` `GPU` `CPU` `模型开发` `中级` `高级` - - - -- [网络支持](#网络支持) - - [Model Zoo](#model-zoo) - - [预训练模型](#预训练模型) - - - - - -## Model Zoo - -| 领域 | 子领域 | 网络 | Ascend | GPU | CPU -|:---- |:------- |:---- |:---- |:---- |:---- -|计算机视觉（CV） | 图像分类（Image Classification） | [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/alexnet/src/alexnet.py) | Supported | Supported | Doing -| 计算机视觉（CV） | 图像分类（Image Classification） | [GoogleNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/googlenet/src/googlenet.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 图像分类（Image Classification） | [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/lenet/src/lenet.py) | Supported | Supported | Supported -| 计算机视觉（CV） | 图像分类（Image Classification） | [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported | Supported | Doing -|计算机视觉（CV） | 图像分类（Image Classification） | [ResNet-101](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported |Doing | Doing -|计算机视觉（CV） | 图像分类（Image Classification） | [SE-ResNet50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported |Doing | Doing -|计算机视觉（CV） | 图像分类（Image Classification） | [ResNext50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnext50/src/image_classification.py) | Supported | Supported | Doing -| 计算机视觉（CV） | 图像分类（Image Classification） | [VGG16](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/vgg16/src/vgg.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 图像分类（Image Classification） | [InceptionV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/inceptionv3/src/inception_v3.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 移动端图像分类（Mobile Image Classification）
目标检测（Image Classification）
语义分割（Semantic Tegmentation） | [MobileNetV2](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/mobilenetv2/src/mobilenetV2.py) | Supported | Supported | Doing -| 计算机视觉（CV） | 移动端图像分类（Mobile Image Classification）
目标检测（Image Classification）
语义分割（Semantic Tegmentation） | [MobileNetV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/mobilenetv3/src/mobilenetV3.py) | Doing | Supported | Doing -|计算机视觉（CV） | 目标检测（Targets Detection） | [SSD](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/ssd/src/ssd.py) | Supported |Doing | Doing -| 计算机视觉（CV） | 目标检测（Targets Detection） | [YoloV3-ResNet18](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_resnet18/src/yolov3.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 目标检测（Targets Detection） | [YoloV3-DarkNet53](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_darknet53/src/yolo.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 目标检测（Targets Detection） | [FasterRCNN](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/faster_rcnn/src/FasterRcnn/faster_rcnn_r50.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 语义分割（Semantic Segmentation） | [DeeplabV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/deeplabv3/src/deeplabv3.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 目标检测（Targets Detection） | [WarpCTC](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/warpctc/src/warpctc.py) | Doing | Supported | Doing -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [BERT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py) | Supported | Doing | Doing -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [Transformer](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/transformer/src/transformer_model.py) | Supported | Doing | Doing -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [SentimentNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/lstm/src/lstm.py) | Doing | Supported | Supported -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [MASS](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/mass/src/transformer/transformer_for_train.py) | Supported | Doing | Doing -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [TinyBert](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/tinybert/src/tinybert_model.py) | Supported | Supported | Doing -| 推荐（Recommender） | 推荐系统、点击率预估（Recommender System, CTR prediction） | [DeepFM](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/recommend/deepfm/src/deepfm.py) | Supported | Supported | Doing -| 推荐（Recommender） | 推荐系统、搜索、排序（Recommender System, Search ranking） | [Wide&Deep](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/recommend/wide_and_deep/src/wide_and_deep.py) | Supported | Supported | Doing -| 图神经网络（GNN） | 文本分类（Text Classification） | [GCN](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/gnn/gcn/src/gcn.py) | Supported | Doing | Doing -| 图神经网络（GNN） | 文本分类（Text Classification） | [GAT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/gnn/gat/src/gat.py) | Supported | Doing | Doing - -> 你也可以使用 [MindWizard工具](https://gitee.com/mindspore/mindinsight/tree/master/mindinsight/wizard/) 快速生成经典网络脚本。 - -## 预训练模型 -*代表MindSpore已发布的版本号，支持网络训练的硬件平台有CPU、GPU和Ascend，以下表格中 ✓ 代表模型是基于选中的硬件平台训练而来。 - -| 领域 | 子领域 | 网络 |数据集 | CPU | GPU | Ascend | 0.5.0-beta* -|:---- |:----- |:---- |:---- |:---- |:---- |:---- |:------ -|计算机视觉（CV） | 图像分类（Image Classification） | [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/alexnet/src/alexnet.py) | CIFAR-10 | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/cv/alexnet/alexnet_ascend_0.5.0_cifar10_official_classification_20200716.tar.gz) -|计算机视觉（CV） | 图像分类（Image Classification）| [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/lenet/src/lenet.py)| MNIST | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/cv/lenet/lenet_ascend_0.5.0_mnist_official_classification_20200716.tar.gz) -|计算机视觉（CV） | 图像分类（Image Classification）| [VGG16](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/vgg16/src/vgg.py)|CIFAR-10 | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/cv/vgg/vgg16_ascend_0.5.0_cifar10_official_classification_20200715.tar.gz) -|计算机视觉（CV） | 图像分类（Image Classification）| [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) |CIFAR-10 | | | ✓ |[下载](http://download.mindspore.cn/model_zoo/official/cv/resnet/resnet50_v1.5_ascend_0.3.0_cifar10_official_classification_20200718.tar.gz) -|计算机视觉（CV） | 目标检测（Targets Detection）| [YoloV3-DarkNet53](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/yolov3_darknet53) |COCO 2014 | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/cv/yolo/yolov3_darknet53_ascend_0.5.0_coco2014_official_object_detection_20200717.tar.gz) -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding）| [BERT_Base](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py) | zhwiki | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/nlp/bert/bert_base_ascend_0.5.0_cn-wiki_official_nlp_20200720.tar.gz) -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding）| [BERT_NEZHA](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py)| zhwiki | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/nlp/bert/bert_nezha_ascend_0.5.0_cn-wiki_official_nlp_20200720.tar.gz) -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding）| [Transformer](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/transformer/src/transformer_model.py)|WMT English-German | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/nlp/transformer/transformer_ascend_0.5.0_wmtende_official_machine_translation_20200713.tar.gz) +# 网络支持 + +`Linux` `Ascend` `GPU` `CPU` `模型开发` `中级` `高级` + + + +- [网络支持](#网络支持) + - [Model Zoo](#model-zoo) + - [预训练模型](#预训练模型) + + + + + +## Model Zoo + +| 领域 | 子领域 | 网络 | Ascend | GPU | CPU +|:---- |:------- |:---- |:---- |:---- |:---- +|计算机视觉（CV） | 图像分类（Image Classification） | [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/alexnet/src/alexnet.py) | Supported | Supported | Doing +| 计算机视觉（CV） | 图像分类（Image Classification） | [GoogleNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/googlenet/src/googlenet.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 图像分类（Image Classification） | [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/lenet/src/lenet.py) | Supported | Supported | Supported +| 计算机视觉（CV） | 图像分类（Image Classification） | [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported | Supported | Doing +|计算机视觉（CV） | 图像分类（Image Classification） | [ResNet-101](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported |Doing | Doing +|计算机视觉（CV） | 图像分类（Image Classification） | [SE-ResNet50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) | Supported |Doing | Doing +|计算机视觉（CV） | 图像分类（Image Classification） | [ResNext50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnext50/src/image_classification.py) | Supported | Supported | Doing +| 计算机视觉（CV） | 图像分类（Image Classification） | [VGG16](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/vgg16/src/vgg.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 图像分类（Image Classification） | [InceptionV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/inceptionv3/src/inception_v3.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 移动端图像分类（Mobile Image Classification）
目标检测（Image Classification）
语义分割（Semantic Tegmentation） | [MobileNetV2](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/mobilenetv2/src/mobilenetV2.py) | Supported | Supported | Doing +| 计算机视觉（CV） | 移动端图像分类（Mobile Image Classification）
目标检测（Image Classification）
语义分割（Semantic Tegmentation） | [MobileNetV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/mobilenetv3/src/mobilenetV3.py) | Doing | Supported | Doing +|计算机视觉（CV） | 目标检测（Targets Detection） | [SSD](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/ssd/src/ssd.py) | Supported |Doing | Doing +| 计算机视觉（CV） | 目标检测（Targets Detection） | [YoloV3-ResNet18](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_resnet18/src/yolov3.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 目标检测（Targets Detection） | [YoloV3-DarkNet53](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/yolov3_darknet53/src/yolo.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 目标检测（Targets Detection） | [FasterRCNN](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/faster_rcnn/src/FasterRcnn/faster_rcnn_r50.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 语义分割（Semantic Segmentation） | [DeeplabV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/deeplabv3/src/deeplabv3.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 目标检测（Targets Detection） | [WarpCTC](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/warpctc/src/warpctc.py) | Doing | Supported | Doing +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [BERT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py) | Supported | Doing | Doing +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [Transformer](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/transformer/src/transformer_model.py) | Supported | Doing | Doing +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [SentimentNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/lstm/src/lstm.py) | Doing | Supported | Supported +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [MASS](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/mass/src/transformer/transformer_for_train.py) | Supported | Doing | Doing +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [TinyBert](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/tinybert/src/tinybert_model.py) | Supported | Supported | Doing +| 推荐（Recommender） | 推荐系统、点击率预估（Recommender System, CTR prediction） | [DeepFM](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/recommend/deepfm/src/deepfm.py) | Supported | Supported | Doing +| 推荐（Recommender） | 推荐系统、搜索、排序（Recommender System, Search ranking） | [Wide&Deep](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/recommend/wide_and_deep/src/wide_and_deep.py) | Supported | Supported | Doing +| 图神经网络（GNN） | 文本分类（Text Classification） | [GCN](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/gnn/gcn/src/gcn.py) | Supported | Doing | Doing +| 图神经网络（GNN） | 文本分类（Text Classification） | [GAT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/gnn/gat/src/gat.py) | Supported | Doing | Doing + +> 你也可以使用 [MindWizard工具](https://gitee.com/mindspore/mindinsight/tree/master/mindinsight/wizard/) 快速生成经典网络脚本。 + +## 预训练模型 +*代表MindSpore已发布的版本号，支持网络训练的硬件平台有CPU、GPU和Ascend，以下表格中 ✓ 代表模型是基于选中的硬件平台训练而来。 + +| 领域 | 子领域 | 网络 |数据集 | CPU | GPU | Ascend | 0.5.0-beta* +|:---- |:----- |:---- |:---- |:---- |:---- |:---- |:------ +|计算机视觉（CV） | 图像分类（Image Classification） | [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/alexnet/src/alexnet.py) | CIFAR-10 | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/cv/alexnet/alexnet_ascend_0.5.0_cifar10_official_classification_20200716.tar.gz) +|计算机视觉（CV） | 图像分类（Image Classification）| [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/lenet/src/lenet.py)| MNIST | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/cv/lenet/lenet_ascend_0.5.0_mnist_official_classification_20200716.tar.gz) +|计算机视觉（CV） | 图像分类（Image Classification）| [VGG16](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/vgg16/src/vgg.py)|CIFAR-10 | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/cv/vgg/vgg16_ascend_0.5.0_cifar10_official_classification_20200715.tar.gz) +|计算机视觉（CV） | 图像分类（Image Classification）| [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py) |CIFAR-10 | | | ✓ |[下载](http://download.mindspore.cn/model_zoo/official/cv/resnet/resnet50_v1.5_ascend_0.3.0_cifar10_official_classification_20200718.tar.gz) +|计算机视觉（CV） | 目标检测（Targets Detection）| [YoloV3-DarkNet53](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/yolov3_darknet53) |COCO 2014 | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/cv/yolo/yolov3_darknet53_ascend_0.5.0_coco2014_official_object_detection_20200717.tar.gz) +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding）| [BERT_Base](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py) | zhwiki | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/nlp/bert/bert_base_ascend_0.5.0_cn-wiki_official_nlp_20200720.tar.gz) +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding）| [BERT_NEZHA](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/bert/src/bert_model.py)| zhwiki | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/nlp/bert/bert_nezha_ascend_0.5.0_cn-wiki_official_nlp_20200720.tar.gz) +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding）| [Transformer](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/nlp/transformer/src/transformer_model.py)|WMT English-German | | | ✓ | [下载](http://download.mindspore.cn/model_zoo/official/nlp/transformer/transformer_ascend_0.5.0_wmtende_official_machine_translation_20200713.tar.gz) diff --git a/docs/source_zh_cn/operator_list.md b/docs/note/source_zh_cn/operator_list.md similarity index 100% rename from docs/source_zh_cn/operator_list.md rename to docs/note/source_zh_cn/operator_list.md diff --git a/lite/docs/source_zh_cn/operator_list.md b/docs/note/source_zh_cn/operator_list_lite.md similarity index 100% rename from lite/docs/source_zh_cn/operator_list.md rename to docs/note/source_zh_cn/operator_list_lite.md diff --git a/docs/source_zh_cn/roadmap.md b/docs/note/source_zh_cn/roadmap.md similarity index 100% rename from docs/source_zh_cn/roadmap.md rename to docs/note/source_zh_cn/roadmap.md diff --git a/docs/source_zh_cn/technical_white_paper.md b/docs/note/source_zh_cn/technical_white_paper.md similarity index 100% rename from docs/source_zh_cn/technical_white_paper.md rename to docs/note/source_zh_cn/technical_white_paper.md diff --git a/docs/programming_guide/Makefile b/docs/programming_guide/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..1eff8952707bdfa503c8d60c1e9a903053170ba2 --- /dev/null +++ b/docs/programming_guide/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source_zh_cn +BUILDDIR = build_zh_cn + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/programming_guide/requirements.txt b/docs/programming_guide/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..162b50040286bb9a0177801c580a31013082a360 --- /dev/null +++ b/docs/programming_guide/requirements.txt @@ -0,0 +1,6 @@ +sphinx >= 2.2.1, <= 2.4.4 +recommonmark +sphinx-markdown-tables +sphinx_rtd_theme +numpy +jieba diff --git a/api/source_zh_cn/programming_guide/api_structure.md b/docs/programming_guide/source_zh_cn/api_structure.md similarity index 100% rename from api/source_zh_cn/programming_guide/api_structure.md rename to docs/programming_guide/source_zh_cn/api_structure.md diff --git a/api/source_zh_cn/programming_guide/augmentation.md b/docs/programming_guide/source_zh_cn/augmentation.md similarity index 100% rename from api/source_zh_cn/programming_guide/augmentation.md rename to docs/programming_guide/source_zh_cn/augmentation.md diff --git a/api/source_zh_cn/programming_guide/auto_augmentation.md b/docs/programming_guide/source_zh_cn/auto_augmentation.md similarity index 100% rename from api/source_zh_cn/programming_guide/auto_augmentation.md rename to docs/programming_guide/source_zh_cn/auto_augmentation.md diff --git a/api/source_zh_cn/programming_guide/auto_parallel_context.md b/docs/programming_guide/source_zh_cn/auto_parallel_context.md similarity index 100% rename from api/source_zh_cn/programming_guide/auto_parallel_context.md rename to docs/programming_guide/source_zh_cn/auto_parallel_context.md diff --git a/api/source_zh_cn/programming_guide/callback.md b/docs/programming_guide/source_zh_cn/callback.md similarity index 100% rename from api/source_zh_cn/programming_guide/callback.md rename to docs/programming_guide/source_zh_cn/callback.md diff --git a/api/source_zh_cn/programming_guide/cell.md b/docs/programming_guide/source_zh_cn/cell.md similarity index 100% rename from api/source_zh_cn/programming_guide/cell.md rename to docs/programming_guide/source_zh_cn/cell.md diff --git a/api/source_zh_cn/programming_guide/component.md b/docs/programming_guide/source_zh_cn/component.md similarity index 100% rename from api/source_zh_cn/programming_guide/component.md rename to docs/programming_guide/source_zh_cn/component.md diff --git a/api/source_zh_cn/programming_guide/dataset_conversion.md b/docs/programming_guide/source_zh_cn/dataset_conversion.md similarity index 100% rename from api/source_zh_cn/programming_guide/dataset_conversion.md rename to docs/programming_guide/source_zh_cn/dataset_conversion.md diff --git a/api/source_zh_cn/programming_guide/dataset_loading.md b/docs/programming_guide/source_zh_cn/dataset_loading.md similarity index 100% rename from api/source_zh_cn/programming_guide/dataset_loading.md rename to docs/programming_guide/source_zh_cn/dataset_loading.md diff --git a/api/source_zh_cn/programming_guide/images/api_structure.png b/docs/programming_guide/source_zh_cn/images/api_structure.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/api_structure.png rename to docs/programming_guide/source_zh_cn/images/api_structure.png diff --git a/api/source_zh_cn/programming_guide/images/batch.png b/docs/programming_guide/source_zh_cn/images/batch.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/batch.png rename to docs/programming_guide/source_zh_cn/images/batch.png diff --git a/api/source_zh_cn/programming_guide/images/cifar.png b/docs/programming_guide/source_zh_cn/images/cifar.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/cifar.png rename to docs/programming_guide/source_zh_cn/images/cifar.png diff --git a/api/source_zh_cn/programming_guide/images/cifar2.png b/docs/programming_guide/source_zh_cn/images/cifar2.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/cifar2.png rename to docs/programming_guide/source_zh_cn/images/cifar2.png diff --git a/api/source_zh_cn/programming_guide/images/concat.png b/docs/programming_guide/source_zh_cn/images/concat.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/concat.png rename to docs/programming_guide/source_zh_cn/images/concat.png diff --git a/api/source_zh_cn/programming_guide/images/ctrans_invert.png b/docs/programming_guide/source_zh_cn/images/ctrans_invert.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/ctrans_invert.png rename to docs/programming_guide/source_zh_cn/images/ctrans_invert.png diff --git a/api/source_zh_cn/programming_guide/images/ctrans_resize.png b/docs/programming_guide/source_zh_cn/images/ctrans_resize.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/ctrans_resize.png rename to docs/programming_guide/source_zh_cn/images/ctrans_resize.png diff --git a/api/source_zh_cn/programming_guide/images/map.png b/docs/programming_guide/source_zh_cn/images/map.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/map.png rename to docs/programming_guide/source_zh_cn/images/map.png diff --git a/api/source_zh_cn/programming_guide/images/mnist.png b/docs/programming_guide/source_zh_cn/images/mnist.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/mnist.png rename to docs/programming_guide/source_zh_cn/images/mnist.png diff --git a/api/source_zh_cn/programming_guide/images/project.png b/docs/programming_guide/source_zh_cn/images/project.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/project.png rename to docs/programming_guide/source_zh_cn/images/project.png diff --git a/api/source_zh_cn/programming_guide/images/pytrans_compose.png b/docs/programming_guide/source_zh_cn/images/pytrans_compose.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/pytrans_compose.png rename to docs/programming_guide/source_zh_cn/images/pytrans_compose.png diff --git a/api/source_zh_cn/programming_guide/images/randomcrop.png b/docs/programming_guide/source_zh_cn/images/randomcrop.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/randomcrop.png rename to docs/programming_guide/source_zh_cn/images/randomcrop.png diff --git a/api/source_zh_cn/programming_guide/images/randomhorizontalflip.png b/docs/programming_guide/source_zh_cn/images/randomhorizontalflip.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/randomhorizontalflip.png rename to docs/programming_guide/source_zh_cn/images/randomhorizontalflip.png diff --git a/api/source_zh_cn/programming_guide/images/rename.png b/docs/programming_guide/source_zh_cn/images/rename.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/rename.png rename to docs/programming_guide/source_zh_cn/images/rename.png diff --git a/api/source_zh_cn/programming_guide/images/repeat.png b/docs/programming_guide/source_zh_cn/images/repeat.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/repeat.png rename to docs/programming_guide/source_zh_cn/images/repeat.png diff --git a/api/source_zh_cn/programming_guide/images/shuffle.png b/docs/programming_guide/source_zh_cn/images/shuffle.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/shuffle.png rename to docs/programming_guide/source_zh_cn/images/shuffle.png diff --git a/api/source_zh_cn/programming_guide/images/take.png b/docs/programming_guide/source_zh_cn/images/take.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/take.png rename to docs/programming_guide/source_zh_cn/images/take.png diff --git a/api/source_zh_cn/programming_guide/images/tranform_bad.png b/docs/programming_guide/source_zh_cn/images/tranform_bad.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/tranform_bad.png rename to docs/programming_guide/source_zh_cn/images/tranform_bad.png diff --git a/api/source_zh_cn/programming_guide/images/tranform_good_1.png b/docs/programming_guide/source_zh_cn/images/tranform_good_1.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/tranform_good_1.png rename to docs/programming_guide/source_zh_cn/images/tranform_good_1.png diff --git a/api/source_zh_cn/programming_guide/images/tranform_good_2.png b/docs/programming_guide/source_zh_cn/images/tranform_good_2.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/tranform_good_2.png rename to docs/programming_guide/source_zh_cn/images/tranform_good_2.png diff --git a/api/source_zh_cn/programming_guide/images/tranform_good_3.png b/docs/programming_guide/source_zh_cn/images/tranform_good_3.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/tranform_good_3.png rename to docs/programming_guide/source_zh_cn/images/tranform_good_3.png diff --git a/api/source_zh_cn/programming_guide/images/tranform_pipeline.png b/docs/programming_guide/source_zh_cn/images/tranform_pipeline.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/tranform_pipeline.png rename to docs/programming_guide/source_zh_cn/images/tranform_pipeline.png diff --git a/api/source_zh_cn/programming_guide/images/zip.png b/docs/programming_guide/source_zh_cn/images/zip.png similarity index 100% rename from api/source_zh_cn/programming_guide/images/zip.png rename to docs/programming_guide/source_zh_cn/images/zip.png diff --git a/api/source_zh_cn/programming_guide/nn.md b/docs/programming_guide/source_zh_cn/nn.md similarity index 100% rename from api/source_zh_cn/programming_guide/nn.md rename to docs/programming_guide/source_zh_cn/nn.md diff --git a/api/source_zh_cn/programming_guide/operator.md b/docs/programming_guide/source_zh_cn/operator.md similarity index 96% rename from api/source_zh_cn/programming_guide/operator.md rename to docs/programming_guide/source_zh_cn/operator.md index a62e2250259863469be22349a058727d7b8e12dc..0bf4eb35f4ae44d05979f7e17cd7dee05ddb9091 100644 --- a/api/source_zh_cn/programming_guide/operator.md +++ b/docs/programming_guide/source_zh_cn/operator.md @@ -1,454 +1,454 @@ -# 算子组件 - -算子组件指常用的算子及其操作，按功能大致可分为张量操作，网络操作，数组操作，图像操作，编码操作，调试操作，量化操作等七个模块。所有的算子在Ascend芯片或者CPU, GPU的支持情况，参见[这里](https://www.mindspore.cn/docs/zh-CN/master/operator_list.html "list") - - -这七类算子操作的相互关系见下: - - - -- [算子组件](#算子组件) - - [张量操作](#张量操作) - - [标量运算](#标量运算) - - [加法](#加法) - - [Element-wise 除法](#element-wise-除法) - - [Element-wise 乘](#element-wise-乘) - - [三角函数](#求三角函数) - - [向量运算](#向量运算) - - [Concat](#concat-算子) - - [Squeeze](#squeeze) - - [Sparse2Dense](#求sparse2dense改变tensor维度使其变稠密) - - [ScalarCast](#scalarcast) - - [矩阵运算](#矩阵运算) - - [矩阵乘法](#矩阵乘法) - - [常见范数](#常见范数) - - [广播机制](#广播机制) - - [网络操作](#网络操作) - - [特征提取](#特征提取) - - [卷积操作](#卷积操作) - - [卷积的反向传播操作](#卷积的反向传播算子操作) - - [激活函数](#激活函数) - - [LossFunction](#lossfunction) - - [L1 Loss](#l1loss) - - [优化算法](#优化算法) - - [SGD](#sgd) - - [数组操作](#数组操作) - - [DType](#dtype) - - [Cast](#cast) - - [Shape](#shape) - - [图像操作](#图像操作) - - [编码运算](#编码运算) - - [BoundingBoxEncode](#boundingboxencode) - - [BoundingBoxDecode](#boundingboxdecode) - - [IOU](#iou-计算) - - [调试操作](#调试操作) - - [Debug](#debug) - - [HookBackward](#hookbackward) - - [量化操作](#量化操作) - - [MinMaxUpdatePerLayer](#minmaxupdateperlayer) - - - - - -## 张量操作 - - -主要包括张量的结构操作和张量的数学运算。 -张量结构操作诸如：张量创建，索引切片，维度变换，合并分割。 -张量数学运算主要有：标量运算，向量运算，矩阵运算。另外我们会介绍张量运算的广播机制。 -本篇我们介绍张量的数学运算。 - - - -### 标量运算 -张量的数学运算符可以分为标量运算符、向量运算符、以及矩阵运算符。 -加减乘除乘方，以及三角函数，指数，对数等常见函数，逻辑比较运算符等都是标量运算符。 -标量运算符的特点是对张量实施逐元素运算。 -有些标量运算符对常用的数学运算符进行了重载。并且支持类似numpy的广播特性。 - -举例说明: -```python -import numpy as np -import mindspore # 导入mindspore包 -from mindspore import Tensor # 导入mindspore下的Tensor包 -import mindspore.ops.operations as P -input_x = mindspore.Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) -input_y = 3.0 -input_x**input_y -``` - -真实输入为： -```python -print(input_x) -[ 1. 8. 64.] -``` - -真实输出为： -```python -print(input_x**input_y) -[ 1. 8. 64.] -``` - -#### 加法 -```python -input_x + input_y -[4.0 5.0 7.0] -``` - -除普通加外，还有element-wise加法: -```python -net = NetAddN() -input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) -input_y = Tensor(np.array([4, 5, 6]), mindspore.float32) -net(input_x, input_y, input_x, input_y)[10.0, 14.0, 18.0] -``` - -#### Element-wise 除法 -```python -input_x = Tensor(np.array([-4.0, 5.0, 6.0]), mindspore.float32) -input_y = Tensor(np.array([3.0, 2.0, 3.0]), mindspore.float32) -div = P.Div() -div(input_x, input_y) -``` - -求FloorDiv: -```python -input_x = Tensor(np.array([2, 4, -1]), mindspore.int32)) -input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) -floor_div = P.FloorDiv() -floor_div(input_x, input_y)[0, 1, -1] -``` - -#### Element-wise 乘 -```python -input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) -input_y = Tensor(np.array([4.0, 5.0, 6.0]), mindspore.float32) -mul = P.Mul() -mul(input_x, input_y) -``` - -真实输出: -```python -[4, 10, 18] -``` - -#### 求三角函数: -```python -acos = P.ACos() -input_x = Tensor(np.array([0.74, 0.04, 0.30, 0.56]), mindspore.float32) -output = acos(input_x) -``` - -### 向量运算 -向量运算符只在一个特定轴上运算，将一个向量映射到一个标量或者另外一个向量。 - -#### Concat 算子: -```python -data1 = Tensor(np.array([[0, 1], [2, 1]]).astype(np.int32)) -data2 = Tensor(np.array([[0, 1], [2, 1]]).astype(np.int32)) -op = P.Concat() -output = op((data1, data2)) -``` - -#### Squeeze -```python -input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) -squeeze = P.Squeeze(2) -output = squeeze(input_tensor) -``` - -#### 求Sparse2Dense(改变tensor维度使其变稠密): -```python -indices = Tensor([[0, 1], [1, 2]]) -values = Tensor([1, 2], dtype=ms.float32) -dense_shape = (3, 4) -out = P.SparseToDense()(indices, values, dense_shape) -``` - -#### ScalarCast: -```python -scalar_cast = P.ScalarCast() -output = scalar_cast(255.0, mindspore.int32) -``` - -### 矩阵运算 -矩阵运算包括: 矩阵乘法，矩阵范数，矩阵行列式，矩阵求特征值，矩阵分解等运算。 - -#### 矩阵乘法: -```python -input_x = Tensor(np.ones(shape=[1, 3]), mindspore.float32) -input_y = Tensor(np.ones(shape=[3, 4]), mindspore.float32) -matmul = P.MatMul() -output = matmul(input_x, input_y) -``` - -#### 常见范数: - -```python -input_x = Tensor(np.ones([128, 64, 32, 64]), mindspore.float32) -scale = Tensor(np.ones([64]), mindspore.float32) -bias = Tensor(np.ones([64]), mindspore.float32) -mean = Tensor(np.ones([64]), mindspore.float32) -variance = Tensor(np.ones([64]), mindspore.float32) -batch_norm = P.BatchNorm() -output = batch_norm(input_x, scale, bias, mean, variance) -``` - -#### 广播机制 - -Broadcast 广播一个tensor到整个group -举例说明: -```python -from mindspore import Tensor -from mindspore.communication import init -import mindspore.nn as nn -import mindspore.ops.operations as P -init() -class Net(nn.Cell): - def __init__(self): - super(Net, self).__init__() - self.broadcast = P.Broadcast(1) - - def construct(self, x): - return self.broadcast((x,)) - -input_ = Tensor(np.ones([2, 8]).astype(np.float32)) -net = Net() -output = net(input_) -``` - -## 网络操作 - - -网络操作包括特征提取, 激活函数， LossFunction, 优化算法等： - -### 特征提取 - -#### 卷积操作 -举例说明: -```python -input = Tensor(np.ones([10, 32, 32, 32]), mindspore.float32) -weight = Tensor(np.ones([32, 32, 3, 3]), mindspore.float32)) -conv2d = P.Conv2D(out_channel=32, kernel_size=3) -conv2d(input, weight) -``` - -#### 卷积的反向传播算子操作： -输出结果: -```python -dout = Tensor(np.ones([10, 32, 30, 30]), mindspore.float32) -weight = Tensor(np.ones([32, 32, 3, 3]), mindspore.float32) -x = Tensor(np.ones([10, 32, 32, 32])) -conv2d_backprop_input = P.Conv2DBackpropInput(out_channel=32, kernel_size=3) -conv2d_backprop_input(dout, weight, F.shape(x)) -``` - -### 激活函数 -举例说明: -```python -input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) -softmax = P.Softmax() -softmax(input_x) -``` - -输出结果: -```python -[0.01165623, 0.03168492, 0.08612854, 0.23412167, 0.6364086] -``` - -### LossFunction - -#### L1Loss: -举例说明： -```python -loss = P.SmoothL1Loss() -input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) -target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) -loss(input_data, target_data) -``` - -输出结果: -```python -[0, 0, 0.5] -``` - -### 优化算法 -#### SGD: -```python -sgd = P.SGD() -parameters = Tensor(np.array([2, -0.5, 1.7, 4]), mindspore.float32) -gradient = Tensor(np.array([1, -1, 0.5, 2]), mindspore.float32) -learning_rate = Tensor(0.01, mindspore.float32) -accum = Tensor(np.array([0.1, 0.3, -0.2, -0.1]), mindspore.float32) -momentum = Tensor(0.1, mindspore.float32) -stat = Tensor(np.array([1.5, -0.3, 0.2, -0.7]), mindspore.float32) -result = sgd(parameters, gradient, learning_rate, accum, momentum, stat) -``` - -## 数组操作 - - - -数组操作指操作对象是一些数组的操作。 - -### DType -返回跟输入的数据类型一致的并且适配Mindspore的tensor变量， 常用于Mindspore 工程内。 -举例说明: -```python -input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) -type = P.DType()(input_tensor) -``` - -### Cast -转换输入的数据类型并且输出与目标数据类型相同的变量 -举例说明: -```python -input_np = np.random.randn(2, 3, 4, 5).astype(np.float32) -input_x = Tensor(input_np) -type_dst = mindspore.float16 -cast = P.Cast() -result = cast(input_x, type_dst) -``` - -### Shape -返回输入数据的形状 -举例说明: -```python -input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) -shape = P.Shape() -output = shape(input_tensor) -``` - -## 图像操作 - - -图像操作包括图像预处理操作， 如图像剪切（Crop，便于得到大量训练样本）和大小变化（Reise,用于构建图像金子塔等）： - -举例说明: -```python -class CropAndResizeNet(nn.Cell): - def __init__(self, crop_size): - super(CropAndResizeNet, self).__init__() - self.crop_and_resize = P.CropAndResize() - self.crop_size = crop_size - @ms_function - def construct(self, x, boxes, box_index): - return self.crop_and_resize(x, boxes, box_index, self.crop_size) - -BATCH_SIZE = 1 -NUM_BOXES = 5 -IMAGE_HEIGHT = 256 -IMAGE_WIDTH = 256 -CHANNELS = 3 -image = np.random.normal(size=[BATCH_SIZE, IMAGE_HEIGHT, IMAGE_WIDTH, CHANNELS]).astype(np.float32) -boxes = np.random.uniform(size=[NUM_BOXES, 4]).astype(np.float32) -box_index = np.random.uniform(size=[NUM_BOXES], low=0, high=BATCH_SIZE).astype(np.int32) -crop_size = (24, 24) -crop_and_resize = CropAndResizeNet(crop_size=crop_size) -output = crop_and_resize(Tensor(image), Tensor(boxes), Tensor(box_index)) -print(output.asnumpy()) -``` - -## 编码运算 - - -编码运算包括 BoundingBox Encoding和 BoundingBox Decoding， IOU计算等。 - -### BoundingBoxEncode -对物体所在区域方框进行编码，得到类似PCA的更精简信息，以便做后续类似特征提取，物体检测，图像恢复等任务。 - -举例说明: -```python -anchor_box = Tensor([[4,1,2,1],[2,2,2,3]],mindspore.float32) -groundtruth_box = Tensor([[3,1,2,2],[1,2,1,4]],mindspore.float32) -boundingbox_encode = P.BoundingBoxEncode(means=(0.0, 0.0, 0.0, 0.0), stds=(1.0, 1.0, 1.0, 1.0)) -boundingbox_encode(anchor_box, groundtruth_box) -``` -输出结果为: -```python -[[5.0000000e-01 5.0000000e-01 -6.5504000e+04 6.9335938e-01] - [-1.0000000e+00 2.5000000e-01 0.0000000e+00 4.0551758e-01]] -``` - -### BoundingBoxDecode -编码器对区域位置信息解码之后，用此算子进行解码。 - -举例说明: -```python -anchor_box = Tensor([[4,1,2,1],[2,2,2,3]],mindspore.float32) -deltas = Tensor([[3,1,2,2],[1,s2,1,4]],mindspore.float32) -boundingbox_decode = P.BoundingBoxDecode(means=(0.0, 0.0, 0.0, 0.0), stds=(1.0, 1.0, 1.0, 1.0), max_shape=(768, 1280), wh_ratio_clip=0.016) -boundingbox_decode(anchor_box, deltas) -``` -输出结果: -```python -[[4.1953125 0. 0. 5.1953125] - [2.140625 0. 3.859375 60.59375]] -``` - -### IOU 计算： -计算预测的物体所在方框和真实物体所在方框的交集区域与并集区域的占比大小。其常作为一种损失函数，用以优化模型。 - -举例说明: -```python -iou = P.IOU() -anchor_boxes = Tensor(np.random.randint(1.0, 5.0, [3, 4]), mindspore.float16) -gt_boxes = Tensor(np.random.randint(1.0, 5.0, [3, 4]), mindspore.float16) -``` - -## 调试操作 -调试操作指的是用于调试网络的一些常用算子及其操作， 例如Debug等 - -### Debug -输出tensor变量的数值， 方便用户随时随地打印想了解或者debug必需的某变量数值。 - -参考示例: -```python -class DebugNN(nn.Cell): - def __init__(self,): - self.debug = nn.Debug() - - def construct(self, x, y): - x = self.add(x, y) - self.debug(x) - return x -``` - -### HookBackward -打印中间变量的梯度，这一算子特别常用，遂举例在此，虽目前仅支持Pynative 形式 -参考示例: -```python -def hook_fn(grad_out): - print(grad_out) - -grad_all = GradOperation(get_all=True) -hook = P.HookBackward(hook_fn) - -def hook_test(x, y): - z = x * y - z = hook(z) - z = z * y - return z - -def backward(x, y): - return grad_all(hook_test)(x, y) - -backward(1, 2) -``` - -## 量化操作 - - -量化操作指对tensor做量化或者反量化操作。 量化操作指将浮点数用整数的加和表示，利用整数加和并行加速时速度快的优点， 实 -现在可接受精度损失下的性能提升。反量化指其反过程，其在精度要求高的地方常被用到。 - -### MinMaxUpdatePerLayer -完成在训练时的量化和反量化操作 -举例说明: -```python -input_tensor = Tensor(np.random.rand(3, 16, 5, 5), mstype.float32) -min_tensor = Tensor(np.array([-6]), mstype.float32) -max_tensor = Tensor(np.array([6]), mstype.float32) -output_tensor = FakeQuantPerLayer(num_bits=8)(input_tensor, min_tensor, max_tensor) -``` +# 算子组件 + +算子组件指常用的算子及其操作，按功能大致可分为张量操作，网络操作，数组操作，图像操作，编码操作，调试操作，量化操作等七个模块。所有的算子在Ascend芯片或者CPU, GPU的支持情况，参见[这里](https://www.mindspore.cn/docs/zh-CN/master/operator_list.html "list") + + +这七类算子操作的相互关系见下: + + + +- [算子组件](#算子组件) + - [张量操作](#张量操作) + - [标量运算](#标量运算) + - [加法](#加法) + - [Element-wise 除法](#element-wise-除法) + - [Element-wise 乘](#element-wise-乘) + - [三角函数](#求三角函数) + - [向量运算](#向量运算) + - [Concat](#concat-算子) + - [Squeeze](#squeeze) + - [Sparse2Dense](#求sparse2dense改变tensor维度使其变稠密) + - [ScalarCast](#scalarcast) + - [矩阵运算](#矩阵运算) + - [矩阵乘法](#矩阵乘法) + - [常见范数](#常见范数) + - [广播机制](#广播机制) + - [网络操作](#网络操作) + - [特征提取](#特征提取) + - [卷积操作](#卷积操作) + - [卷积的反向传播操作](#卷积的反向传播算子操作) + - [激活函数](#激活函数) + - [LossFunction](#lossfunction) + - [L1 Loss](#l1loss) + - [优化算法](#优化算法) + - [SGD](#sgd) + - [数组操作](#数组操作) + - [DType](#dtype) + - [Cast](#cast) + - [Shape](#shape) + - [图像操作](#图像操作) + - [编码运算](#编码运算) + - [BoundingBoxEncode](#boundingboxencode) + - [BoundingBoxDecode](#boundingboxdecode) + - [IOU](#iou-计算) + - [调试操作](#调试操作) + - [Debug](#debug) + - [HookBackward](#hookbackward) + - [量化操作](#量化操作) + - [MinMaxUpdatePerLayer](#minmaxupdateperlayer) + + + + + +## 张量操作 + + +主要包括张量的结构操作和张量的数学运算。 +张量结构操作诸如：张量创建，索引切片，维度变换，合并分割。 +张量数学运算主要有：标量运算，向量运算，矩阵运算。另外我们会介绍张量运算的广播机制。 +本篇我们介绍张量的数学运算。 + + + +### 标量运算 +张量的数学运算符可以分为标量运算符、向量运算符、以及矩阵运算符。 +加减乘除乘方，以及三角函数，指数，对数等常见函数，逻辑比较运算符等都是标量运算符。 +标量运算符的特点是对张量实施逐元素运算。 +有些标量运算符对常用的数学运算符进行了重载。并且支持类似numpy的广播特性。 + +举例说明: +```python +import numpy as np +import mindspore # 导入mindspore包 +from mindspore import Tensor # 导入mindspore下的Tensor包 +import mindspore.ops.operations as P +input_x = mindspore.Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) +input_y = 3.0 +input_x**input_y +``` + +真实输入为： +```python +print(input_x) +[ 1. 8. 64.] +``` + +真实输出为： +```python +print(input_x**input_y) +[ 1. 8. 64.] +``` + +#### 加法 +```python +input_x + input_y +[4.0 5.0 7.0] +``` + +除普通加外，还有element-wise加法: +```python +net = NetAddN() +input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) +input_y = Tensor(np.array([4, 5, 6]), mindspore.float32) +net(input_x, input_y, input_x, input_y)[10.0, 14.0, 18.0] +``` + +#### Element-wise 除法 +```python +input_x = Tensor(np.array([-4.0, 5.0, 6.0]), mindspore.float32) +input_y = Tensor(np.array([3.0, 2.0, 3.0]), mindspore.float32) +div = P.Div() +div(input_x, input_y) +``` + +求FloorDiv: +```python +input_x = Tensor(np.array([2, 4, -1]), mindspore.int32)) +input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) +floor_div = P.FloorDiv() +floor_div(input_x, input_y)[0, 1, -1] +``` + +#### Element-wise 乘 +```python +input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) +input_y = Tensor(np.array([4.0, 5.0, 6.0]), mindspore.float32) +mul = P.Mul() +mul(input_x, input_y) +``` + +真实输出: +```python +[4, 10, 18] +``` + +#### 求三角函数: +```python +acos = P.ACos() +input_x = Tensor(np.array([0.74, 0.04, 0.30, 0.56]), mindspore.float32) +output = acos(input_x) +``` + +### 向量运算 +向量运算符只在一个特定轴上运算，将一个向量映射到一个标量或者另外一个向量。 + +#### Concat 算子: +```python +data1 = Tensor(np.array([[0, 1], [2, 1]]).astype(np.int32)) +data2 = Tensor(np.array([[0, 1], [2, 1]]).astype(np.int32)) +op = P.Concat() +output = op((data1, data2)) +``` + +#### Squeeze +```python +input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) +squeeze = P.Squeeze(2) +output = squeeze(input_tensor) +``` + +#### 求Sparse2Dense(改变tensor维度使其变稠密): +```python +indices = Tensor([[0, 1], [1, 2]]) +values = Tensor([1, 2], dtype=ms.float32) +dense_shape = (3, 4) +out = P.SparseToDense()(indices, values, dense_shape) +``` + +#### ScalarCast: +```python +scalar_cast = P.ScalarCast() +output = scalar_cast(255.0, mindspore.int32) +``` + +### 矩阵运算 +矩阵运算包括: 矩阵乘法，矩阵范数，矩阵行列式，矩阵求特征值，矩阵分解等运算。 + +#### 矩阵乘法: +```python +input_x = Tensor(np.ones(shape=[1, 3]), mindspore.float32) +input_y = Tensor(np.ones(shape=[3, 4]), mindspore.float32) +matmul = P.MatMul() +output = matmul(input_x, input_y) +``` + +#### 常见范数: + +```python +input_x = Tensor(np.ones([128, 64, 32, 64]), mindspore.float32) +scale = Tensor(np.ones([64]), mindspore.float32) +bias = Tensor(np.ones([64]), mindspore.float32) +mean = Tensor(np.ones([64]), mindspore.float32) +variance = Tensor(np.ones([64]), mindspore.float32) +batch_norm = P.BatchNorm() +output = batch_norm(input_x, scale, bias, mean, variance) +``` + +#### 广播机制 + +Broadcast 广播一个tensor到整个group +举例说明: +```python +from mindspore import Tensor +from mindspore.communication import init +import mindspore.nn as nn +import mindspore.ops.operations as P +init() +class Net(nn.Cell): + def __init__(self): + super(Net, self).__init__() + self.broadcast = P.Broadcast(1) + + def construct(self, x): + return self.broadcast((x,)) + +input_ = Tensor(np.ones([2, 8]).astype(np.float32)) +net = Net() +output = net(input_) +``` + +## 网络操作 + + +网络操作包括特征提取, 激活函数， LossFunction, 优化算法等： + +### 特征提取 + +#### 卷积操作 +举例说明: +```python +input = Tensor(np.ones([10, 32, 32, 32]), mindspore.float32) +weight = Tensor(np.ones([32, 32, 3, 3]), mindspore.float32)) +conv2d = P.Conv2D(out_channel=32, kernel_size=3) +conv2d(input, weight) +``` + +#### 卷积的反向传播算子操作： +输出结果: +```python +dout = Tensor(np.ones([10, 32, 30, 30]), mindspore.float32) +weight = Tensor(np.ones([32, 32, 3, 3]), mindspore.float32) +x = Tensor(np.ones([10, 32, 32, 32])) +conv2d_backprop_input = P.Conv2DBackpropInput(out_channel=32, kernel_size=3) +conv2d_backprop_input(dout, weight, F.shape(x)) +``` + +### 激活函数 +举例说明: +```python +input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) +softmax = P.Softmax() +softmax(input_x) +``` + +输出结果: +```python +[0.01165623, 0.03168492, 0.08612854, 0.23412167, 0.6364086] +``` + +### LossFunction + +#### L1Loss: +举例说明： +```python +loss = P.SmoothL1Loss() +input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) +target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) +loss(input_data, target_data) +``` + +输出结果: +```python +[0, 0, 0.5] +``` + +### 优化算法 +#### SGD: +```python +sgd = P.SGD() +parameters = Tensor(np.array([2, -0.5, 1.7, 4]), mindspore.float32) +gradient = Tensor(np.array([1, -1, 0.5, 2]), mindspore.float32) +learning_rate = Tensor(0.01, mindspore.float32) +accum = Tensor(np.array([0.1, 0.3, -0.2, -0.1]), mindspore.float32) +momentum = Tensor(0.1, mindspore.float32) +stat = Tensor(np.array([1.5, -0.3, 0.2, -0.7]), mindspore.float32) +result = sgd(parameters, gradient, learning_rate, accum, momentum, stat) +``` + +## 数组操作 + + + +数组操作指操作对象是一些数组的操作。 + +### DType +返回跟输入的数据类型一致的并且适配Mindspore的tensor变量， 常用于Mindspore 工程内。 +举例说明: +```python +input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) +type = P.DType()(input_tensor) +``` + +### Cast +转换输入的数据类型并且输出与目标数据类型相同的变量 +举例说明: +```python +input_np = np.random.randn(2, 3, 4, 5).astype(np.float32) +input_x = Tensor(input_np) +type_dst = mindspore.float16 +cast = P.Cast() +result = cast(input_x, type_dst) +``` + +### Shape +返回输入数据的形状 +举例说明: +```python +input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) +shape = P.Shape() +output = shape(input_tensor) +``` + +## 图像操作 + + +图像操作包括图像预处理操作， 如图像剪切（Crop，便于得到大量训练样本）和大小变化（Reise,用于构建图像金子塔等）： + +举例说明: +```python +class CropAndResizeNet(nn.Cell): + def __init__(self, crop_size): + super(CropAndResizeNet, self).__init__() + self.crop_and_resize = P.CropAndResize() + self.crop_size = crop_size + @ms_function + def construct(self, x, boxes, box_index): + return self.crop_and_resize(x, boxes, box_index, self.crop_size) + +BATCH_SIZE = 1 +NUM_BOXES = 5 +IMAGE_HEIGHT = 256 +IMAGE_WIDTH = 256 +CHANNELS = 3 +image = np.random.normal(size=[BATCH_SIZE, IMAGE_HEIGHT, IMAGE_WIDTH, CHANNELS]).astype(np.float32) +boxes = np.random.uniform(size=[NUM_BOXES, 4]).astype(np.float32) +box_index = np.random.uniform(size=[NUM_BOXES], low=0, high=BATCH_SIZE).astype(np.int32) +crop_size = (24, 24) +crop_and_resize = CropAndResizeNet(crop_size=crop_size) +output = crop_and_resize(Tensor(image), Tensor(boxes), Tensor(box_index)) +print(output.asnumpy()) +``` + +## 编码运算 + + +编码运算包括 BoundingBox Encoding和 BoundingBox Decoding， IOU计算等。 + +### BoundingBoxEncode +对物体所在区域方框进行编码，得到类似PCA的更精简信息，以便做后续类似特征提取，物体检测，图像恢复等任务。 + +举例说明: +```python +anchor_box = Tensor([[4,1,2,1],[2,2,2,3]],mindspore.float32) +groundtruth_box = Tensor([[3,1,2,2],[1,2,1,4]],mindspore.float32) +boundingbox_encode = P.BoundingBoxEncode(means=(0.0, 0.0, 0.0, 0.0), stds=(1.0, 1.0, 1.0, 1.0)) +boundingbox_encode(anchor_box, groundtruth_box) +``` +输出结果为: +```python +[[5.0000000e-01 5.0000000e-01 -6.5504000e+04 6.9335938e-01] + [-1.0000000e+00 2.5000000e-01 0.0000000e+00 4.0551758e-01]] +``` + +### BoundingBoxDecode +编码器对区域位置信息解码之后，用此算子进行解码。 + +举例说明: +```python +anchor_box = Tensor([[4,1,2,1],[2,2,2,3]],mindspore.float32) +deltas = Tensor([[3,1,2,2],[1,s2,1,4]],mindspore.float32) +boundingbox_decode = P.BoundingBoxDecode(means=(0.0, 0.0, 0.0, 0.0), stds=(1.0, 1.0, 1.0, 1.0), max_shape=(768, 1280), wh_ratio_clip=0.016) +boundingbox_decode(anchor_box, deltas) +``` +输出结果: +```python +[[4.1953125 0. 0. 5.1953125] + [2.140625 0. 3.859375 60.59375]] +``` + +### IOU 计算： +计算预测的物体所在方框和真实物体所在方框的交集区域与并集区域的占比大小。其常作为一种损失函数，用以优化模型。 + +举例说明: +```python +iou = P.IOU() +anchor_boxes = Tensor(np.random.randint(1.0, 5.0, [3, 4]), mindspore.float16) +gt_boxes = Tensor(np.random.randint(1.0, 5.0, [3, 4]), mindspore.float16) +``` + +## 调试操作 +调试操作指的是用于调试网络的一些常用算子及其操作， 例如Debug等 + +### Debug +输出tensor变量的数值， 方便用户随时随地打印想了解或者debug必需的某变量数值。 + +参考示例: +```python +class DebugNN(nn.Cell): + def __init__(self,): + self.debug = nn.Debug() + + def construct(self, x, y): + x = self.add(x, y) + self.debug(x) + return x +``` + +### HookBackward +打印中间变量的梯度，这一算子特别常用，遂举例在此，虽目前仅支持Pynative 形式 +参考示例: +```python +def hook_fn(grad_out): + print(grad_out) + +grad_all = GradOperation(get_all=True) +hook = P.HookBackward(hook_fn) + +def hook_test(x, y): + z = x * y + z = hook(z) + z = z * y + return z + +def backward(x, y): + return grad_all(hook_test)(x, y) + +backward(1, 2) +``` + +## 量化操作 + + +量化操作指对tensor做量化或者反量化操作。 量化操作指将浮点数用整数的加和表示，利用整数加和并行加速时速度快的优点， 实 +现在可接受精度损失下的性能提升。反量化指其反过程，其在精度要求高的地方常被用到。 + +### MinMaxUpdatePerLayer +完成在训练时的量化和反量化操作 +举例说明: +```python +input_tensor = Tensor(np.random.rand(3, 16, 5, 5), mstype.float32) +min_tensor = Tensor(np.array([-6]), mstype.float32) +max_tensor = Tensor(np.array([6]), mstype.float32) +output_tensor = FakeQuantPerLayer(num_bits=8)(input_tensor, min_tensor, max_tensor) +``` diff --git a/api/source_zh_cn/programming_guide/ops.md b/docs/programming_guide/source_zh_cn/ops.md similarity index 100% rename from api/source_zh_cn/programming_guide/ops.md rename to docs/programming_guide/source_zh_cn/ops.md diff --git a/api/source_zh_cn/programming_guide/optim.md b/docs/programming_guide/source_zh_cn/optim.md similarity index 100% rename from api/source_zh_cn/programming_guide/optim.md rename to docs/programming_guide/source_zh_cn/optim.md diff --git a/api/source_zh_cn/programming_guide/parameter.md b/docs/programming_guide/source_zh_cn/parameter.md similarity index 100% rename from api/source_zh_cn/programming_guide/parameter.md rename to docs/programming_guide/source_zh_cn/parameter.md diff --git a/api/source_zh_cn/programming_guide/pipeline.md b/docs/programming_guide/source_zh_cn/pipeline.md similarity index 100% rename from api/source_zh_cn/programming_guide/pipeline.md rename to docs/programming_guide/source_zh_cn/pipeline.md diff --git a/api/source_zh_cn/programming_guide/sampler.md b/docs/programming_guide/source_zh_cn/sampler.md similarity index 100% rename from api/source_zh_cn/programming_guide/sampler.md rename to docs/programming_guide/source_zh_cn/sampler.md diff --git a/api/source_zh_cn/programming_guide/security_and_privacy.md b/docs/programming_guide/source_zh_cn/security_and_privacy.md similarity index 100% rename from api/source_zh_cn/programming_guide/security_and_privacy.md rename to docs/programming_guide/source_zh_cn/security_and_privacy.md diff --git a/api/source_zh_cn/programming_guide/tensor.md b/docs/programming_guide/source_zh_cn/tensor.md similarity index 100% rename from api/source_zh_cn/programming_guide/tensor.md rename to docs/programming_guide/source_zh_cn/tensor.md diff --git a/api/source_zh_cn/programming_guide/tokenizer.md b/docs/programming_guide/source_zh_cn/tokenizer.md similarity index 100% rename from api/source_zh_cn/programming_guide/tokenizer.md rename to docs/programming_guide/source_zh_cn/tokenizer.md diff --git a/api/source_zh_cn/programming_guide/type.md b/docs/programming_guide/source_zh_cn/type.md similarity index 100% rename from api/source_zh_cn/programming_guide/type.md rename to docs/programming_guide/source_zh_cn/type.md diff --git a/lite/docs/source_en/architecture_lite.md b/lite/docs/source_en/architecture_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..64585775720d39c365190d9f8f24c82931cf24e3 --- /dev/null +++ b/lite/docs/source_en/architecture_lite.md @@ -0,0 +1,19 @@ +# Overall Architecture + + + +The overall architecture of MindSpore Lite is as follows: + +![architecture](./images/MindSpore-Lite-architecture.png) + +- **Frontend:** generates models. You can use the model building API to build models and convert third-party models and models trained by MindSpore to MindSpore Lite models. Third-party models include TensorFlow Lite, Caffe 1.0, and ONNX models. + +- **IR:** defines the tensors, operators, and graphs of MindSpore. + +- **Backend:** optimizes graphs based on IR, including graph high level optimization (GHLO), graph low level optimization (GLLO), and quantization. GHLO is responsible for hardware-independent optimization, such as operator fusion and constant folding. GLLO is responsible for hardware-related optimization. Quantizer supports quantization methods after training, such as weight quantization and activation value quantization. + +- **Runtime:** inference runtime of intelligent devices. Sessions are responsible for session management and provide external APIs. The thread pool and parallel primitives are responsible for managing the thread pool used for graph execution. Memory allocation is responsible for memory overcommitment of each operator during graph execution. The operator library provides the CPU and GPU operators. + +- **Micro:** runtime of IoT devices, including the model generation .c file, thread pool, memory overcommitment, and operator library. + +Runtime and Micro share the underlying infrastructure layers, such as the operator library, memory allocation, thread pool, and parallel primitives. diff --git a/lite/docs/source_en/glossary_lite.md b/lite/docs/source_en/glossary_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..8b596229292567ddb8b24b7a0cc1560c763f224d --- /dev/null +++ b/lite/docs/source_en/glossary_lite.md @@ -0,0 +1,12 @@ +# Glossary + + + +| Acronym and Abbreviation | Description | +| ----- | ----- | +| MindSpore Lite | MindSpore AI engine is applied to the intelligent terminal and resource constrained scenes on the edge side. | +| MindSpore Micro | MindSpore AI engine with smaller package size for IOT devices. | +| GHLO | Graph high-level optimization. | +| GLLO | Graph low-level optimization. | +| RT | Runtime. | + diff --git a/lite/docs/source_en/index_lite.rst b/lite/docs/source_en/index_lite.rst new file mode 100644 index 0000000000000000000000000000000000000000..abecfe957e16896bca6efeb5a1cb376835251fa6 --- /dev/null +++ b/lite/docs/source_en/index_lite.rst @@ -0,0 +1,16 @@ +.. MindSpore documentation master file, created by + sphinx-quickstart on Thu Aug 17 10:00:00 2020. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +MindSpore Lite Documentation +============================ + +.. toctree:: + :glob: + :maxdepth: 1 + + architecture + apicc/apicc + operator_list + glossary diff --git a/lite/docs/source_en/operator_list_lite.md b/lite/docs/source_en/operator_list_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..6038b5c95690dd4b30378a7101d828ef9d0cda90 --- /dev/null +++ b/lite/docs/source_en/operator_list_lite.md @@ -0,0 +1,111 @@ +# Operator List + + + +> √ The checked items are the operators supported by MindSpore Lite。 + +| Operation | CPU
FP16 | CPU
FP32 | CPU
Int8 | CPU
UInt8 | GPU
FP16 | GPU
FP32 | Tensorflow
Lite op supported | Caffe
Lite op supported | Onnx
Lite op supported | +|-----------------------|----------|----------|-----------|----------|----------|------------------|----------|----------|----------| +| Abs | | √ | √ | √ | | | Abs | | Abs | +| Add | √ | √ | √ | √ | | √ | Add | | Add | +| AddN | | √ | | | | | AddN | | | +| Argmax | | √ | √ | √ | | | Argmax | ArgMax | ArgMax | +| Argmin | | √ | √ | √ | | | Argmin | | | +| AvgPool | √ | √ | √ | √ | | √ | MeanPooling| Pooling | AveragePool | +| BatchNorm | √ | √ | √ | √ | | √ | | BatchNorm | BatchNormalization | +| BatchToSpace | | √ | √ | √ | | | BatchToSpace, BatchToSpaceND | | | +| BiasAdd | | √ | √ | √ | | √ | | | BiasAdd | +| Broadcast | | √ | | | | | BroadcastTo | | Expand | +| Cast | √ | √ | | √ | | | Cast, DEQUANTIZE* | | Cast | +| Ceil | | √ | √ | √ | | | Ceil | | Ceil | +| Concat | √ | √ | √ | √ | √ | √ | Concat | Concat | Concat | +| Conv2d | √ | √ | √ | √ | √ | √ | Conv2D | Convolution | Conv | +| Conv2dTranspose | √ | √ | √ | √ | √ | √ | DeConv2D | Deconvolution | ConvTranspose | +| Cos | | √ | √ | √ | | | Cos | | Cos | +| Crop | | √ | √ | √ | | | | Crop | | +| DeDepthwiseConv2D | | √ | √ | √ | | | | Deconvolution| ConvTranspose | +| DepthToSpace | | √ | √ | √ | | | DepthToSpace| | DepthToSpace | +| DepthwiseConv2dNative | √ | √ | √ | √ | √ | √ | DepthwiseConv2D | Convolution | Convolution | +| Div | √ | √ | √ | √ | | √ | Div, RealDiv | | Div | +| Eltwise | √ | √ | | | | | | Eltwise | | +| Elu | | √ | | | | | Elu | | Elu | +| Equal | √ | √ | √ | √ | | | Equal | | Equal | +| Exp | | √ | | | | | Exp | | Exp | +| ExpandDims | | √ | | | | | | | | +| Fill | | √ | | | | | Fill | | | +| Flatten | | √ | | | | | | Flatten | | +| Floor | | √ | √ | √ | | | flOOR | | Floor | +| FloorDiv | √ | √ | | | | | FloorDiv | | | +| FloorMod | √ | √ | | | | | FloorMod | | | +| FullConnection | | √ | √ | √ | | | FullyConnected | InnerProduct | | +| GatherNd | | √ | √ | √ | | | GatherND | | | +| GatherV2 | | √ | √ | √ | | | Gather | | Gather | +| Greater | √ | √ | √ | √ | | | Greater | | Greater | +| GreaterEqual | √ | √ | √ | √ | | | GreaterEqual| | | +| Hswish | √ | √ | √ | √ | | | HardSwish | | | +| LeakyReLU | √ | √ | | | | √ | LeakyRelu | | LeakyRelu | +| Less | √ | √ | √ | √ | | | Less | | Less | +| LessEqual | √ | √ | √ | √ | | | LessEqual | | | +| LRN | | √ | | | | | LocalResponseNorm | | Lrn | +| Log | | √ | √ | √ | | | Log | | Log | +| LogicalAnd | √ | √ | | | | | LogicalAnd | | | +| LogicalNot | | √ | √ | √ | | | LogicalNot | | | +| LogicalOr | √ | √ | | | | | LogicalOr | | | +| LSTM | | √ | | | | | | | | +| MatMul | | √ | √ | √ | √ | √ | | | MatMul | +| Maximum | √ | √ | | | | | Maximum | | Max | +| MaxPool | √ | √ | √ | √ | | √ | MaxPooling | Pooling | MaxPool | +| Minimum | √ | √ | | | | | Minimum | | Min | +| Mul | √ | √ | √ | √ | | √ | Mul | | Mul | +| NotEqual | √ | √ | √ | √ | | | NotEqual | | | +| OneHot | | √ | | | | | OneHot | | | +| Pad | | √ | √ | √ | | | Pad | | Pad | +| Pow | | √ | √ | √ | | | Pow | Power | Power | +| PReLU | | √ | | | | √ | | PReLU | | +| Range | | √ | | | | | Range | | | +| Rank | | √ | | | | | Rank | | | +| ReduceMax | √ | √ | √ | √ | | | ReduceMax | | ReduceMax | +| ReduceMean | √ | √ | √ | √ | | | Mean | | ReduceMean | +| ReduceMin | √ | √ | √ | √ | | | ReduceMin | | ReduceMin | +| ReduceProd | √ | √ | √ | √ | | | ReduceProd | | | +| ReduceSum | √ | √ | √ | √ | | | Sum | | ReduceSum | +| ReduceSumSquare | √ | √ | √ | √ | | | | | | +| ReLU | √ | √ | √ | √ | | √ | Relu | ReLU | Relu | +| ReLU6 | √ | √ | √ | √ | | √ | Relu6 | ReLU6 | Clip* | +| Reshape | √ | √ | √ | √ | | √ | Reshape | Reshape | Reshape,Flatten | +| Resize | | √ | √ | √ | | | ResizeBilinear, NearestNeighbor | Interp | | +| Reverse | | √ | | | | | reverse | | | +| ReverseSequence | | √ | | | | | ReverseSequence | | | +| Round | | √ | √ | √ | | | Round | | | +| Rsqrt | | √ | √ | √ | | | Rsqrt | | | +| Scale | | √ | | | | | | Scale | | +| ScatterNd | | √ | | | | | ScatterNd | | | +| Shape | | √ | | | | | Shape | | Shape | +| Sigmoid | √ | √ | √ | √ | | √ | Logistic | Sigmoid | Sigmoid | +| Sin | | √ | √ | √ | | | Sin | | Sin | +| Slice | | √ | √ | √ | √ | √ | Slice | | Slice | +| Softmax | √ | √ | √ | √ | | √ | Softmax | Softmax | Softmax | +| SpaceToBatch | | √ | | | | | | | | +| SpaceToBatchND | | √ | | | | | SpaceToBatchND | | | +| SpaceToDepth | | √ | | | | | SpaceToDepth | | SpaceToDepth | +| SparseToDense | | √ | | | | | SpareToDense | | | +| Split | √ | √ | √ | √ | | | Split, SplitV | | | +| Sqrt | | √ | √ | √ | | | Sqrt | | Sqrt | +| Square | | √ | √ | √ | | | Square | | | +| SquaredDifference | | √ | | | | | SquaredDifference | | | +| Squeeze | | √ | √ | √ | | | Squeeze | | Squeeze | +| StridedSlice | | √ | √ | √ | | | StridedSlice| | | +| Stack | | √ | | | | | Stack | | | +| Sub | √ | √ | √ | √ | | √ | Sub | | Sub | +| Tanh | √ | √ | | | | | Tanh | TanH | | +| Tile | | √ | | | | | Tile | | Tile | +| TopK | | √ | √ | √ | | | TopKV2 | | | +| Transpose | √ | √ | | | | √ | Transpose | Permute | Transpose | +| Unique | | √ | | | | | Unique | | | +| Unsqueeze | | √ | √ | √ | | | | | Unsqueeze | +| Unstack | | √ | | | | | Unstack | | | +| Where | | √ | | | | | Where | | | +| ZerosLike | | √ | | | | | ZerosLike | | | + +* Clip: only support convert clip(0, 6) to Relu6. +* DEQUANTIZE: only support to convert fp16 to fp32. diff --git a/lite/docs/source_zh_cn/apicc/dataset.md b/lite/docs/source_zh_cn/apicc/dataset.md index 379d3e11632327b3075c0f8a56d53c852cdeae80..ef441d6ab75fc6c4f49fb1b40b471608a02aeb0c 100644 --- a/lite/docs/source_zh_cn/apicc/dataset.md +++ b/lite/docs/source_zh_cn/apicc/dataset.md @@ -91,7 +91,7 @@ bool SubStractMeanNormalize(LiteMat &src, LiteMat &dst, const float *mean, float 返回True或者False。 ``` -bool Padd(LiteMat &src, LiteMat &dst, const int top, const int bottom, const int left, const int right, const PaddBorderType pad_type, uint8_t fill_r, uint8_t fill_g, uint8_t fill_b) +bool Pad(LiteMat &src, LiteMat &dst, const int top, const int bottom, const int left, const int right, const PaddBorderType pad_type, uint8_t fill_r, uint8_t fill_g, uint8_t fill_b) ``` 填充图像，通道支持为3和1。 diff --git a/lite/docs/source_zh_cn/architecture_lite.md b/lite/docs/source_zh_cn/architecture_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..ce86fa9829a184306cde335d74d06a70117c6c63 --- /dev/null +++ b/lite/docs/source_zh_cn/architecture_lite.md @@ -0,0 +1,20 @@ +# 总体架构 + + + +MindSpore Lite框架的总体架构如下所示： + +![architecture](images/MindSpore-Lite-architecture.png) + +- **前端（Frontend）：** 负责模型生成，用户可以通过模型构建接口构建模型，将第三方模型和MindSpore训练的模型转换为MindSpore Lite模型，其中第三方模型包括TensorFlow Lite、Caffe 1.0和ONNX模型。 + +- **IR：** 负责MindSpore的Tensor定义、算子定义和图定义。 + +- **Backend：** 基于IR进行图优化，包括GHLO、GLLO和量化三部分。其中，GHLO负责和硬件无关的优化，如算子融合、常量折叠等；GLLO负责与硬件相关的优化；量化Quantizer支持权重量化、激活值量化等训练后量化手段。 + +- **Runtime：** 智能终端的推理运行时，其中session负责会话管理，提供对外接口；线程池和并行原语负责图执行使用的线程池管理，内存分配负责图执行中各个算子的内存复用，算子库提供CPU和GPU算子。 + +- **Micro：** IoT设备的运行时，包括模型生成.c文件、线程池、内存复用和算子库。 + +其中，Runtime和Micro共享底层的算子库、内存分配、线程池、并行原语等基础设施层。 + diff --git a/lite/docs/source_zh_cn/glossary_lite.md b/lite/docs/source_zh_cn/glossary_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..b9cf41a4c6908e4e75c927614335a60e9e8b0ac6 --- /dev/null +++ b/lite/docs/source_zh_cn/glossary_lite.md @@ -0,0 +1,12 @@ +# 术语 + + + +| 术语/缩略语 | 说明 | +| ----- | ----- | +| MindSpore Lite | 应用在智能终端，边缘册资源受限场景的MindSpore AI 引擎。 | +| MindSpore Micro | 应用在IoT设备的，包大小更小的MindSpore AI引擎。 | +| GHLO | Graph high-level optimization，图高层优化。 | +| GLLO | Graph low-level optimization，图底层优化。 | +| RT | Runtime运行时。 | + diff --git a/lite/docs/source_zh_cn/index_lite.rst b/lite/docs/source_zh_cn/index_lite.rst new file mode 100644 index 0000000000000000000000000000000000000000..20ecdbb72c0fe01cbc24c674bda6944504c792ff --- /dev/null +++ b/lite/docs/source_zh_cn/index_lite.rst @@ -0,0 +1,16 @@ +.. MindSpore documentation master file, created by + sphinx-quickstart on Thu Aug 17 10:00:00 2020. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +MindSpore端侧文档 +================== + +.. toctree:: + :glob: + :maxdepth: 1 + + architecture + apicc/apicc + operator_list + glossary diff --git a/lite/docs/source_zh_cn/operator_list_lite.md b/lite/docs/source_zh_cn/operator_list_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..3384d8baf91b1af92ff4758816790af7b6e241bc --- /dev/null +++ b/lite/docs/source_zh_cn/operator_list_lite.md @@ -0,0 +1,111 @@ +# 算子支持 + + + +> √勾选的项为MindSpore Lite所支持的算子。 + +| 操作名 | CPU
FP16 | CPU
FP32 | CPU
Int8 | CPU
UInt8 | GPU
FP16 | GPU
FP32 | 支持的Tensorflow
Lite op | 支持的Caffe
Lite op | 支持的Onnx
Lite op | +|-----------------------|----------|----------|----------|-----------|----------|-------------------|----------|----------|---------| +| Abs | | √ | √ | √ | | | Abs | | Abs | +| Add | √ | √ | √ | √ | | √ | Add | | Add | +| AddN | | √ | | | | | AddN | | | +| Argmax | | √ | √ | √ | | | Argmax | ArgMax | ArgMax | +| Argmin | | √ | √ | √ | | | Argmin | | | +| AvgPool | √ | √ | √ | √ | | √ | MeanPooling| Pooling | AveragePool | +| BatchNorm | √ | √ | √ | √ | | √ | | BatchNorm | BatchNormalization | +| BatchToSpace | | √ | √ | √ | | | BatchToSpace, BatchToSpaceND | | | +| BiasAdd | | √ | √ | √ | | √ | | | BiasAdd | +| Broadcast | | √ | | | | | BroadcastTo | | Expand | +| Cast | √ | √ | | √ | | | Cast, DEQUANTIZE* | | Cast | +| Ceil | | √ | √ | √ | | | Ceil | | Ceil | +| Concat | √ | √ | √ | √ | √ | √ | Concat | Concat | Concat | +| Conv2d | √ | √ | √ | √ | √ | √ | Conv2D | Convolution | Conv | +| Conv2dTranspose | √ | √ | √ | √ | √ | √ | DeConv2D | Deconvolution | ConvTranspose | +| Cos | | √ | √ | √ | | | Cos | | Cos | +| Crop | | √ | √ | √ | | | | Crop | | +| DeDepthwiseConv2D | | √ | √ | √ | | | | Deconvolution| ConvTranspose | +| DepthToSpace | | √ | √ | √ | | | DepthToSpace| | DepthToSpace | +| DepthwiseConv2dNative | √ | √ | √ | √ | √ | √ | DepthwiseConv2D | Convolution | Convolution | +| Div | √ | √ | √ | √ | | √ | Div, RealDiv | | Div | +| Eltwise | √ | √ | | | | | | Eltwise | | +| Elu | | √ | | | | | Elu | | Elu | +| Equal | √ | √ | √ | √ | | | Equal | | Equal | +| Exp | | √ | | | | | Exp | | Exp | +| ExpandDims | | √ | | | | | | | | +| Fill | | √ | | | | | Fill | | | +| Flatten | | √ | | | | | | Flatten | | +| Floor | | √ | √ | √ | | | flOOR | | Floor | +| FloorDiv | √ | √ | | | | | FloorDiv | | | +| FloorMod | √ | √ | | | | | FloorMod | | | +| FullConnection | | √ | √ | √ | | | FullyConnected | InnerProduct | | +| GatherNd | | √ | √ | √ | | | GatherND | | | +| GatherV2 | | √ | √ | √ | | | Gather | | Gather | +| Greater | √ | √ | √ | √ | | | Greater | | Greater | +| GreaterEqual | √ | √ | √ | √ | | | GreaterEqual| | | +| Hswish | √ | √ | √ | √ | | | HardSwish | | | +| LeakyReLU | √ | √ | | | | √ | LeakyRelu | | LeakyRelu | +| Less | √ | √ | √ | √ | | | Less | | Less | +| LessEqual | √ | √ | √ | √ | | | LessEqual | | | +| LRN | | √ | | | | | LocalResponseNorm | | Lrn | +| Log | | √ | √ | √ | | | Log | | Log | +| LogicalAnd | √ | √ | | | | | LogicalAnd | | | +| LogicalNot | | √ | √ | √ | | | LogicalNot | | | +| LogicalOr | √ | √ | | | | | LogicalOr | | | +| LSTM | | √ | | | | | | | | +| MatMul | | √ | √ | √ | √ | √ | | | MatMul | +| Maximum | √ | √ | | | | | Maximum | | Max | +| MaxPool | √ | √ | √ | √ | | √ | MaxPooling | Pooling | MaxPool | +| Minimum | √ | √ | | | | | Minimum | | Min | +| Mul | √ | √ | √ | √ | | √ | Mul | | Mul | +| NotEqual | √ | √ | √ | √ | | | NotEqual | | | +| OneHot | | √ | | | | | OneHot | | | +| Pad | | √ | √ | √ | | | Pad | | Pad | +| Pow | | √ | √ | √ | | | Pow | Power | Power | +| PReLU | | √ | | | | √ | | PReLU | | +| Range | | √ | | | | | Range | | | +| Rank | | √ | | | | | Rank | | | +| ReduceMax | √ | √ | √ | √ | | | ReduceMax | | ReduceMax | +| ReduceMean | √ | √ | √ | √ | | | Mean | | ReduceMean | +| ReduceMin | √ | √ | √ | √ | | | ReduceMin | | ReduceMin | +| ReduceProd | √ | √ | √ | √ | | | ReduceProd | | | +| ReduceSum | √ | √ | √ | √ | | | Sum | | ReduceSum | +| ReduceSumSquare | √ | √ | √ | √ | | | | | | +| ReLU | √ | √ | √ | √ | | √ | Relu | ReLU | Relu | +| ReLU6 | √ | √ | √ | √ | | √ | Relu6 | ReLU6 | Clip* | +| Reshape | √ | √ | √ | √ | | √ | Reshape | Reshape | Reshape,Flatten | +| Resize | | √ | √ | √ | | | ResizeBilinear, NearestNeighbor | Interp | | +| Reverse | | √ | | | | | reverse | | | +| ReverseSequence | | √ | | | | | ReverseSequence | | | +| Round | | √ | √ | √ | | | Round | | | +| Rsqrt | | √ | √ | √ | | | Rsqrt | | | +| Scale | | √ | | | | | | Scale | | +| ScatterNd | | √ | | | | | ScatterNd | | | +| Shape | | √ | | | | | Shape | | Shape | +| Sigmoid | √ | √ | √ | √ | | √ | Logistic | Sigmoid | Sigmoid | +| Sin | | √ | √ | √ | | | Sin | | Sin | +| Slice | | √ | √ | √ | √ | √ | Slice | | Slice | +| Softmax | √ | √ | √ | √ | | √ | Softmax | Softmax | Softmax | +| SpaceToBatch | | √ | | | | | | | | +| SpaceToBatchND | | √ | | | | | SpaceToBatchND | | | +| SpaceToDepth | | √ | | | | | SpaceToDepth | | SpaceToDepth | +| SparseToDense | | √ | | | | | SpareToDense | | | +| Split | √ | √ | √ | √ | | | Split, SplitV | | | +| Sqrt | | √ | √ | √ | | | Sqrt | | Sqrt | +| Square | | √ | √ | √ | | | Square | | | +| SquaredDifference | | √ | | | | | SquaredDifference | | | +| Squeeze | | √ | √ | √ | | | Squeeze | | Squeeze | +| StridedSlice | | √ | √ | √ | | | StridedSlice| | | +| Stack | | √ | | | | | Stack | | | +| Sub | √ | √ | √ | √ | | √ | Sub | | Sub | +| Tanh | √ | √ | | | | | Tanh | TanH | | +| Tile | | √ | | | | | Tile | | Tile | +| TopK | | √ | √ | √ | | | TopKV2 | | | +| Transpose | √ | √ | | | | √ | Transpose | Permute | Transpose | +| Unique | | √ | | | | | Unique | | | +| Unsqueeze | | √ | √ | √ | | | | | Unsqueeze | +| Unstack | | √ | | | | | Unstack | | | +| Where | | √ | | | | | Where | | | +| ZerosLike | | √ | | | | | ZerosLike | | | + +* Clip: 仅支持将clip(0, 6)转换为Relu6. +* DEQUANTIZE: 仅支持将fp16转换为fp32. diff --git a/tutorials/inference/Makefile b/tutorials/inference/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..1eff8952707bdfa503c8d60c1e9a903053170ba2 --- /dev/null +++ b/tutorials/inference/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source_zh_cn +BUILDDIR = build_zh_cn + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/tutorials/requirements.txt b/tutorials/inference/requirements.txt similarity index 100% rename from tutorials/requirements.txt rename to tutorials/inference/requirements.txt diff --git a/tutorials/lite/Makefile b/tutorials/lite/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..1eff8952707bdfa503c8d60c1e9a903053170ba2 --- /dev/null +++ b/tutorials/lite/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source_zh_cn +BUILDDIR = build_zh_cn + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/lite/tutorials/requirements.txt b/tutorials/lite/requirements.txt similarity index 100% rename from lite/tutorials/requirements.txt rename to tutorials/lite/requirements.txt diff --git a/lite/tutorials/source_en/_static/logo_source.png b/tutorials/lite/source_en/_static/logo_source.png similarity index 100% rename from lite/tutorials/source_en/_static/logo_source.png rename to tutorials/lite/source_en/_static/logo_source.png diff --git a/lite/tutorials/source_en/build.md b/tutorials/lite/source_en/build.md similarity index 100% rename from lite/tutorials/source_en/build.md rename to tutorials/lite/source_en/build.md diff --git a/lite/tutorials/source_en/conf.py b/tutorials/lite/source_en/conf.py similarity index 100% rename from lite/tutorials/source_en/conf.py rename to tutorials/lite/source_en/conf.py diff --git a/lite/tutorials/source_en/images/lite_quick_start_app_result.png b/tutorials/lite/source_en/images/lite_quick_start_app_result.png similarity index 100% rename from lite/tutorials/source_en/images/lite_quick_start_app_result.png rename to tutorials/lite/source_en/images/lite_quick_start_app_result.png diff --git a/lite/tutorials/source_en/images/lite_quick_start_home.png b/tutorials/lite/source_en/images/lite_quick_start_home.png similarity index 100% rename from lite/tutorials/source_en/images/lite_quick_start_home.png rename to tutorials/lite/source_en/images/lite_quick_start_home.png diff --git a/lite/tutorials/source_en/images/lite_quick_start_project_structure.png b/tutorials/lite/source_en/images/lite_quick_start_project_structure.png similarity index 100% rename from lite/tutorials/source_en/images/lite_quick_start_project_structure.png rename to tutorials/lite/source_en/images/lite_quick_start_project_structure.png diff --git a/lite/tutorials/source_en/images/lite_quick_start_run_app.PNG b/tutorials/lite/source_en/images/lite_quick_start_run_app.PNG similarity index 100% rename from lite/tutorials/source_en/images/lite_quick_start_run_app.PNG rename to tutorials/lite/source_en/images/lite_quick_start_run_app.PNG diff --git a/lite/tutorials/source_en/images/lite_quick_start_sdk.png b/tutorials/lite/source_en/images/lite_quick_start_sdk.png similarity index 100% rename from lite/tutorials/source_en/images/lite_quick_start_sdk.png rename to tutorials/lite/source_en/images/lite_quick_start_sdk.png diff --git a/lite/tutorials/source_en/images/side_infer_process.png b/tutorials/lite/source_en/images/side_infer_process.png similarity index 100% rename from lite/tutorials/source_en/images/side_infer_process.png rename to tutorials/lite/source_en/images/side_infer_process.png diff --git a/lite/tutorials/source_en/index.rst b/tutorials/lite/source_en/index.rst similarity index 100% rename from lite/tutorials/source_en/index.rst rename to tutorials/lite/source_en/index.rst diff --git a/lite/tutorials/source_en/quick_start/quick_start.md b/tutorials/lite/source_en/quick_start/quick_start.md similarity index 100% rename from lite/tutorials/source_en/quick_start/quick_start.md rename to tutorials/lite/source_en/quick_start/quick_start.md diff --git a/lite/tutorials/source_en/use/benchmark_tool.md b/tutorials/lite/source_en/use/benchmark_tool.md similarity index 100% rename from lite/tutorials/source_en/use/benchmark_tool.md rename to tutorials/lite/source_en/use/benchmark_tool.md diff --git a/lite/tutorials/source_en/use/converter_tool.md b/tutorials/lite/source_en/use/converter_tool.md similarity index 100% rename from lite/tutorials/source_en/use/converter_tool.md rename to tutorials/lite/source_en/use/converter_tool.md diff --git a/lite/tutorials/source_en/use/evaluating_the_model.rst b/tutorials/lite/source_en/use/evaluating_the_model.rst similarity index 100% rename from lite/tutorials/source_en/use/evaluating_the_model.rst rename to tutorials/lite/source_en/use/evaluating_the_model.rst diff --git a/lite/tutorials/source_en/use/runtime.md b/tutorials/lite/source_en/use/runtime.md similarity index 100% rename from lite/tutorials/source_en/use/runtime.md rename to tutorials/lite/source_en/use/runtime.md diff --git a/lite/tutorials/source_en/use/timeprofiler_tool.md b/tutorials/lite/source_en/use/timeprofiler_tool.md similarity index 100% rename from lite/tutorials/source_en/use/timeprofiler_tool.md rename to tutorials/lite/source_en/use/timeprofiler_tool.md diff --git a/lite/tutorials/source_zh_cn/_static/logo_source.png b/tutorials/lite/source_zh_cn/_static/logo_source.png similarity index 100% rename from lite/tutorials/source_zh_cn/_static/logo_source.png rename to tutorials/lite/source_zh_cn/_static/logo_source.png diff --git a/lite/tutorials/source_zh_cn/build.md b/tutorials/lite/source_zh_cn/build.md similarity index 100% rename from lite/tutorials/source_zh_cn/build.md rename to tutorials/lite/source_zh_cn/build.md diff --git a/lite/tutorials/source_zh_cn/conf.py b/tutorials/lite/source_zh_cn/conf.py similarity index 100% rename from lite/tutorials/source_zh_cn/conf.py rename to tutorials/lite/source_zh_cn/conf.py diff --git a/lite/tutorials/source_zh_cn/images/lite_quick_start_app_result.png b/tutorials/lite/source_zh_cn/images/lite_quick_start_app_result.png similarity index 100% rename from lite/tutorials/source_zh_cn/images/lite_quick_start_app_result.png rename to tutorials/lite/source_zh_cn/images/lite_quick_start_app_result.png diff --git a/lite/tutorials/source_zh_cn/images/lite_quick_start_home.png b/tutorials/lite/source_zh_cn/images/lite_quick_start_home.png similarity index 100% rename from lite/tutorials/source_zh_cn/images/lite_quick_start_home.png rename to tutorials/lite/source_zh_cn/images/lite_quick_start_home.png diff --git a/lite/tutorials/source_zh_cn/images/lite_quick_start_install.png b/tutorials/lite/source_zh_cn/images/lite_quick_start_install.png similarity index 100% rename from lite/tutorials/source_zh_cn/images/lite_quick_start_install.png rename to tutorials/lite/source_zh_cn/images/lite_quick_start_install.png diff --git a/lite/tutorials/source_zh_cn/images/lite_quick_start_project_structure.png b/tutorials/lite/source_zh_cn/images/lite_quick_start_project_structure.png similarity index 100% rename from lite/tutorials/source_zh_cn/images/lite_quick_start_project_structure.png rename to tutorials/lite/source_zh_cn/images/lite_quick_start_project_structure.png diff --git a/lite/tutorials/source_zh_cn/images/lite_quick_start_run_app.PNG b/tutorials/lite/source_zh_cn/images/lite_quick_start_run_app.PNG similarity index 100% rename from lite/tutorials/source_zh_cn/images/lite_quick_start_run_app.PNG rename to tutorials/lite/source_zh_cn/images/lite_quick_start_run_app.PNG diff --git a/lite/tutorials/source_zh_cn/images/lite_quick_start_sdk.png b/tutorials/lite/source_zh_cn/images/lite_quick_start_sdk.png similarity index 100% rename from lite/tutorials/source_zh_cn/images/lite_quick_start_sdk.png rename to tutorials/lite/source_zh_cn/images/lite_quick_start_sdk.png diff --git a/lite/tutorials/source_zh_cn/images/side_infer_process.png b/tutorials/lite/source_zh_cn/images/side_infer_process.png similarity index 100% rename from lite/tutorials/source_zh_cn/images/side_infer_process.png rename to tutorials/lite/source_zh_cn/images/side_infer_process.png diff --git a/lite/tutorials/source_zh_cn/index.rst b/tutorials/lite/source_zh_cn/index.rst similarity index 100% rename from lite/tutorials/source_zh_cn/index.rst rename to tutorials/lite/source_zh_cn/index.rst diff --git a/lite/tutorials/source_zh_cn/quick_start/quick_start.md b/tutorials/lite/source_zh_cn/quick_start/quick_start.md similarity index 100% rename from lite/tutorials/source_zh_cn/quick_start/quick_start.md rename to tutorials/lite/source_zh_cn/quick_start/quick_start.md diff --git a/lite/tutorials/source_zh_cn/use/benchmark_tool.md b/tutorials/lite/source_zh_cn/use/benchmark_tool.md similarity index 100% rename from lite/tutorials/source_zh_cn/use/benchmark_tool.md rename to tutorials/lite/source_zh_cn/use/benchmark_tool.md diff --git a/lite/tutorials/source_zh_cn/use/converter_tool.md b/tutorials/lite/source_zh_cn/use/converter_tool.md similarity index 100% rename from lite/tutorials/source_zh_cn/use/converter_tool.md rename to tutorials/lite/source_zh_cn/use/converter_tool.md diff --git a/lite/tutorials/source_zh_cn/use/evaluating_the_model.rst b/tutorials/lite/source_zh_cn/use/evaluating_the_model.rst similarity index 100% rename from lite/tutorials/source_zh_cn/use/evaluating_the_model.rst rename to tutorials/lite/source_zh_cn/use/evaluating_the_model.rst diff --git a/lite/tutorials/source_zh_cn/use/post_training_quantization.md b/tutorials/lite/source_zh_cn/use/post_training_quantization.md similarity index 100% rename from lite/tutorials/source_zh_cn/use/post_training_quantization.md rename to tutorials/lite/source_zh_cn/use/post_training_quantization.md diff --git a/lite/tutorials/source_zh_cn/use/runtime.md b/tutorials/lite/source_zh_cn/use/runtime.md similarity index 100% rename from lite/tutorials/source_zh_cn/use/runtime.md rename to tutorials/lite/source_zh_cn/use/runtime.md diff --git a/lite/tutorials/source_zh_cn/use/timeprofiler_tool.md b/tutorials/lite/source_zh_cn/use/timeprofiler_tool.md similarity index 100% rename from lite/tutorials/source_zh_cn/use/timeprofiler_tool.md rename to tutorials/lite/source_zh_cn/use/timeprofiler_tool.md diff --git a/tutorials/training/Makefile b/tutorials/training/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..1eff8952707bdfa503c8d60c1e9a903053170ba2 --- /dev/null +++ b/tutorials/training/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source_zh_cn +BUILDDIR = build_zh_cn + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/tutorials/training/requirements.txt b/tutorials/training/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..13396200381cc893c7a1ee023cbc0341eeea9f87 --- /dev/null +++ b/tutorials/training/requirements.txt @@ -0,0 +1,5 @@ +sphinx >= 2.2.1, <= 2.4.4 +recommonmark +sphinx-markdown-tables +sphinx_rtd_theme +jieba diff --git a/tutorials/source_en/_static/logo_source.png b/tutorials/training/source_en/_static/logo_source.png similarity index 100% rename from tutorials/source_en/_static/logo_source.png rename to tutorials/training/source_en/_static/logo_source.png diff --git a/tutorials/source_en/advanced_use/checkpoint_for_hybrid_parallel.md b/tutorials/training/source_en/advanced_use/checkpoint_for_hybrid_parallel.md similarity index 100% rename from tutorials/source_en/advanced_use/checkpoint_for_hybrid_parallel.md rename to tutorials/training/source_en/advanced_use/checkpoint_for_hybrid_parallel.md diff --git a/tutorials/source_en/advanced_use/computer_vision_application.md b/tutorials/training/source_en/advanced_use/computer_vision_application.md similarity index 85% rename from tutorials/source_en/advanced_use/computer_vision_application.md rename to tutorials/training/source_en/advanced_use/computer_vision_application.md index 8ead2a76fd2e0f4abca0363f9cd3947462b07221..b15b0539b66ef836dd3d9dca326648cdaaa57425 100644 --- a/tutorials/source_en/advanced_use/computer_vision_application.md +++ b/tutorials/training/source_en/advanced_use/computer_vision_application.md @@ -22,13 +22,13 @@ ## Overview -Computer vision is the most widely researched and mature technology field of deep learning, and is widely used in scenarios such as mobile phone photographing, intelligent security protection, and automated driving. Since AlexNet won the ImageNet competition in 2012, deep learning has greatly promoted the development of the computer vision field. Almost all the most advanced computer vision algorithms are related to deep learning. Deep neural network can extract image features layer by layer and retain local invariance. It is widely used in visual tasks such as classification, detection, segmentation, tracking, retrieval, recognition, promotion, and reconstruction. +Computer vision is one of the most widely researched and mature technology fields of deep learning, and is widely applied to scenarios such as mobile phone photographing, intelligent security protection, and automated driving. Since AlexNet won the ImageNet competition in 2012, deep learning has greatly promoted the development of the computer vision field. Almost all the most advanced computer vision algorithms are related to deep learning. Deep neural network can extract image features layer by layer and retain local invariance. It is widely used in visual tasks such as classification, detection, segmentation, tracking, retrieval, recognition, promotion, and reconstruction. This chapter describes how to apply MindSpore to computer vision scenarios based on image classification tasks. ## Image Classification -Image classification is the most basic computer vision application and belongs to the supervised learning category. For example, determine the class of a digital image, such as cat, dog, airplane, or car. The function is as follows: +Image classification is one of the most basic computer vision applications and belongs to the supervised learning category. For example, determine the class of a digital image, such as cat, dog, airplane, or car. The function is as follows: ```python def classify(image): @@ -49,9 +49,9 @@ MindSpore supports the following image classification networks: LeNet, AlexNet, Figure 1: CIFAR-10 dataset [1] -Figure 1 shows that the CIFAR-10 dataset contains 10 classes of 60,000 images. Each class contains 6000 images. 50,000 images are for training and 10,000 images are for testing. The size of each image is 32 x 32 pixels. +The CIFAR-10 dataset contains 10 classes of 60,000 images. Each class contains 6000 images. 50,000 images are for training and 10,000 images are for testing. The size of each image is 32 x 32 pixels. -Generally, a training indicator of image classification is accuracy, that is, a ratio of a quantity of accurately predicted examples to a total quantity of predicted examples. +Generally, a training indicator of image classification is accuracy, that is, a ratio of the quantity of accurately predicted examples to the total quantity of predicted examples. Next, let's use MindSpore to solve the image classification task. The overall process is as follows: 1. Download the CIFAR-10 dataset. @@ -61,12 +61,12 @@ Next, let's use MindSpore to solve the image classification task. The overall pr 5. Call the high-level `Model` API to train and save the model file. 6. Load the saved model for inference. -> This example is for the hardware platform of the Ascend 910 AI processor. You can find the complete executable sample code at： . +> This example uses the hardware platform of the Ascend 910 AI processor. You can find the complete executable sample code at： . The key parts of the task process code are explained below. ### Downloading the CIFAR-10 Dataset -CIFAR-10 dataset download address: [the website of Cifar-10 Dataset](https://www.cs.toronto.edu/~kriz/cifar.html) In this example, the data is in binary format. In the Linux environment, run the following command to download the dataset: +CIFAR-10 dataset download address: [the website of Cifar-10 Dataset](https://www.cs.toronto.edu/~kriz/cifar.html). In this example, the data is in binary format. In the Linux environment, run the following command to download the dataset: ```shell wget https://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz diff --git a/tutorials/source_en/advanced_use/customized_debugging_information.md b/tutorials/training/source_en/advanced_use/customized_debugging_information.md similarity index 100% rename from tutorials/source_en/advanced_use/customized_debugging_information.md rename to tutorials/training/source_en/advanced_use/customized_debugging_information.md diff --git a/tutorials/source_en/advanced_use/dashboard.md b/tutorials/training/source_en/advanced_use/dashboard.md similarity index 98% rename from tutorials/source_en/advanced_use/dashboard.md rename to tutorials/training/source_en/advanced_use/dashboard.md index 7c875e1c8151661194b42f4f8c26825cb76b6d2d..8094cd79478502e79cfe8e883fe0a377454a913c 100644 --- a/tutorials/source_en/advanced_use/dashboard.md +++ b/tutorials/training/source_en/advanced_use/dashboard.md @@ -1,202 +1,202 @@ -# Dashboard - -`Linux` `Ascend` `GPU` `CPU` `Model Optimization` `Intermediate` `Expert` - - - -- [Dashboard](#dashboard) - - [Overview](#overview) - - [Scalar Visualization](#scalar-visualization) - - [Parameter Distribution Visualization](#parameter-distribution-visualization) - - [Computational Graph Visualization](#computational-graph-visualization) - - [Dataset Graph Visualization](#dataset-graph-visualization) - - [Image Visualization](#image-visualization) - - [Tensor Visualization](#tensor-visualization) - - [Notices](#notices) - - - - - -## Overview - -Training dashboard is an important part of mindinsight's visualization component, and its tags include scalar visualization, parameter distribution visualization, computational visualization, data visualization, image visualization and tensor visualization. - -Access the Training Dashboard by selecting a specific training from the training list. - -## Scalar Visualization - -Scalar visualization is used to display the change trend of scalars during training. - -![scalar.png](./images/scalar.png) - -Figure 1: Scalar trend chart - -Figure 1 shows a change process of loss values during the neural network training. The horizontal coordinate indicates the training step, and the vertical coordinate indicates the loss value. - -Buttons from left to right in the upper right corner of the figure are used to display the chart in full screen, switch the Y-axis scale, enable or disable the rectangle selection, roll back the chart step by step, and restore the chart. - -- Full-screen display: Display the scalar curve in full screen. Click the button again to restore it. -- Switch Y-axis scale: Perform logarithmic conversion on the Y-axis coordinate. -- Enable/Disable rectangle selection: Draw a rectangle to select and zoom in a part of the chart. You can perform rectangle selection again on the zoomed-in chart. -- Step-by-step Rollback: Cancel operations step by step after continuously drawing rectangles to select and zooming in the same area. -- Restore chart: Restore a chart to the original state. - -There can set the threshold value to highlight the value or delete the threshold value in the lower right corner of the figure. As shown in the figure, the threshold set is less than 1.5, highlighted in red shows what is below the threshold, and it is intuitive to see the expected data value or some unusual value. - -![scalar_select.png](./images/scalar_select.png) - -Figure 2: Scalar visualization function area - -Figure 2 shows the scalar visualization function area, which allows you to view scalar information by selecting different tags, different dimensions of the horizontal axis, and smoothness. - -- Tag selection: Select the required tags to view the corresponding scalar information. -- Horizontal axis: Select any of Step, Relative Time, and Absolute Time as the horizontal axis of the scalar curve. -- Smoothness: adjust the smoothness to smooth the scalar curve. -- Scalar synthesis: Synthesize two scalar curves and display them in a chart to facilitate comparison between the two curves or view the synthesized chart. - -![scalar_compound.png](./images/scalar_compound.png) - -Figure 3: Scalar synthesis of Accuracy and Loss curves - -Figure 3 shows the scalar synthesis of the Accuracy and Loss curves. The function area of scalar synthesis is similar to that of scalar visualization. Different from the scalar visualization function area, the scalar synthesis function allows you to select a maximum of two tags at a time to synthesize and display their curves. - -## Parameter Distribution Visualization - -The parameter distribution in a form of a histogram displays tensors specified by a user. - -![histogram.png](./images/histogram.png) - -Figure 4: Histogram - -Figure 4 shows tensors recorded by a user in a form of a histogram. Click the upper right corner to zoom in the histogram. - -![histogram_func.png](./images/histogram_func.png) - -Figure 5: Function area of the parameter distribution histogram - -Figure 5 shows the function area of the parameter distribution histogram, including: - -- Tag selection: Select the required tags to view the corresponding histogram. -- Vertical axis: Select any of `Step`, `Relative time`, and `Absolute time` as the data displayed on the vertical axis of the histogram. -- Angle of view: Select either `Front` or `Top`. `Front` view refers to viewing the histogram from the front view. In this case, data between different steps is overlapped. `Top` view refers to viewing the histogram at an angle of 45 degrees. In this case, data between different steps can be presented. - -## Computational Graph Visualization - -Computational graph visualization is used to display the graph structure, data flow direction, and control flow direction of a computational graph. It supports visualization of summary log files and pb files generated by `save_graphs` configuration in `context`. - -![graph.png](./images/graph.png) - -Figure 6: Computational graph display area - -Figure 6 shows the network structure of a computational graph. As shown in the figure, select an operator in the area of the display area. The operator has two inputs and one outputs (the solid line indicates the data flow direction of the operator). - -![graph_sidebar.png](./images/graph_sidebar.png) - -Figure 7: Computational graph function area - -Figure 7 shows the function area of the computational graph, including: - -- File selection box: View the computational graphs of different files. -- Search box: Enter a node name and press Enter to view the node. -- Thumbnail: Display the thumbnail of the entire network structure. When viewing an extra large image structure, you can view the currently browsed area. -- Node information: Display the basic information of the selected node, including the node name, properties, input node, and output node. -- Legend: Display the meaning of each icon in the computational graph. - -## Dataset Graph Visualization - -Dataset graph visualization is used to display data processing and augmentation information of a single model training. - -![data_function.png](./images/data_function.png) - -Figure 8: Dataset graph function area - -Figure 8 shows the dataset graph function area which includes the following content: - -- Legend: Display the meaning of each icon in the data lineage graph. -- Data processing pipeline: Display the data processing pipeline used for training. Select a single node in the graph to view details. -- Node information: Display basic information about the selected node, including names and parameters of the data processing and augmentation operators. - -## Image Visualization - -Image visualization is used to display images specified by users. - -![image.png](./images/image_vi.png) - -Figure 9: Image visualization - -Figure 9 shows how to view images of different steps by sliding the Step slider. - -![image_function.png](./images/image_function.png) - -Figure 10: Image visualization function area - -Figure 10 shows the function area of image visualization. You can view image information by selecting different tags, brightness, and contrast. - -- Tag: Select the required tags to view the corresponding image information. -- Brightness adjustment: Adjust the brightness of all displayed images. -- Contrast adjustment: Adjust the contrast of all displayed images. - -## Tensor Visualization - -Tensor visualization is used to display tensors in the form of table and histogram. - -![tensor_function.png](./images/tensor_function.png) - -Figure 11: Tensor visualization function area - -Figure 11 shows the function area of tensor visualization. - -- Tag selection: Select the required tags to view the corresponding table data or histogram. -- View: Select `Table` or `Histogram` to display tensor data. In the `Histogram` view, there are the options of `Vertical axis` and `Angle of view`. -- Vertical axis: Select any of `Step`, `Relative time`, and `Absolute time` as the data displayed on the vertical axis of the histogram. -- Angle of view: Select either `Front` or `Top`. `Front` view refers to viewing the histogram from the front view. In this case, data between different steps is overlapped. `Top` view refers to viewing the histogram at an angle of 45 degrees. In this case, data between different steps can be presented. - -![tensor_table.png](./images/tensor_table.png) - -Figure 12: Table display - -Figure 12 shows tensors recorded by a user in a form of a table which includes the following function: - -- Click the small square button on the right side of the table to zoom in the table. -- The white box in the table shows the tensor data under which dimension is currently displayed, where the colon `:` represents all values of the current dimension, you can enter the corresponding index or `:` in the box and press `Enter` or click the button of tick on the back to query tensor data for specific dimensions. - Assuming a certain dimension is 32, the index range is -32 to 31. Note: tensor data from 0 to 2 dimensions can be queried. Tensor data of more than two dimensions is not supported, in other word, the query conditions of more than two colons `:` cannot be set. -- Query the tensor data of a specific step by dragging the hollow circle below the table. - -![tensor_histogram.png](./images/tensor_histogram.png) - -Figure 13: Histogram display - -Figure 13 shows tensors recorded by a user in a form of a histogram. Click the upper right corner to zoom in the histogram. - -## Notices - - -1. Currently MindSpore supports recording computational graph after operator fusion for Ascend 910 AI processor only. - -2. When using the Summary operator to collect data in training, 'HistogramSummary' operator affects performance, so please use as little as possible. - -3. To limit memory usage, MindInsight limits the number of tags and steps: - - There are 300 tags at most in each training dashboard. Total number of scalar tags, image tags, computation graph tags, parameter distribution(histogram) tags, tensor tags can not exceed 300. Specially, there are 10 computation graph tags and 6 tensor tags at most. When tags exceed limit, MindInsight preserves the most recently processed tags. - - There are 1000 steps at most for each scalar tag in each training dashboard. When steps exceed limit, MindInsight will sample steps randomly to meet this limit. - - There are 10 steps at most for each image tag in each training dashboard. When steps exceed limit, MindInsight will sample steps randomly to meet this limit. - - There are 50 steps at most for each parameter distribution(histogram) tag in each training dashboard. When steps exceed limit, MindInsight will sample steps randomly to meet this limit. - - There are 20 steps at most for each tensor tag in each training dashboard. When steps exceed limit, MindInsight will sample steps randomly to meet this limit. - -4. Since `TensorSummary` will record complete tensor data, the amount of data is usually relatively large. In order to limit memory usage and ensure performance, MindInsight make the following restrictions with the size of tensor and the number of value responsed and displayed on the front end: - - MindInsight supports loading tensor containing up to 10 million values. - - After the tensor is loaded, in the tensor-visible table view, you can view a maximum of 100,000 values. If the value obtained by the selected dimension query exceeds this limit, it cannot be displayed. - -5. Since tensor visualizatioin (`TensorSummary`) records raw tensor data, it requires a large amount of storage space. Before using `TensorSummary` and during training, please check that the system storage space is sufficient. - The storage space occupied by the tensor visualizatioin function can be reduced by the following methods: - 1) Avoid using `TensorSummary` to record larger tensor. - - 2) Reduce the number of `TensorSummary` operators in the network. - - After using the function, please clean up the training logs that are no longer needed in time to free up disk space. - - Remarks: The method of estimating the space usage of `TensorSummary` is as follows: - - The size of a `TensorSummary` data = the number of values in the tensor * 4 bytes. Assuming that the size of the tensor recorded by `TensorSummary` is 32 * 1 * 256 * 256, then a `TensorSummary` data needs about 32 * 1 * 256 * 256 * 4 bytes = 8,388,608 bytes = 8MiB. `TensorSummary` will record data of 20 steps by default. Then the required space when recording these 20 sets of data is about 20 * 8 MiB = 160MiB. It should be noted that due to the overhead of data structure and other factors, the actual storage space used will be slightly larger than 160MiB. - +# Dashboard + +`Linux` `Ascend` `GPU` `CPU` `Model Optimization` `Intermediate` `Expert` + + + +- [Dashboard](#dashboard) + - [Overview](#overview) + - [Scalar Visualization](#scalar-visualization) + - [Parameter Distribution Visualization](#parameter-distribution-visualization) + - [Computational Graph Visualization](#computational-graph-visualization) + - [Dataset Graph Visualization](#dataset-graph-visualization) + - [Image Visualization](#image-visualization) + - [Tensor Visualization](#tensor-visualization) + - [Notices](#notices) + + + + + +## Overview + +Training dashboard is an important part of mindinsight's visualization component, and its tags include scalar visualization, parameter distribution visualization, computational visualization, data visualization, image visualization and tensor visualization. + +Access the Training Dashboard by selecting a specific training from the training list. + +## Scalar Visualization + +Scalar visualization is used to display the change trend of scalars during training. + +![scalar.png](./images/scalar.png) + +Figure 1: Scalar trend chart + +Figure 1 shows a change process of loss values during the neural network training. The horizontal coordinate indicates the training step, and the vertical coordinate indicates the loss value. + +Buttons from left to right in the upper right corner of the figure are used to display the chart in full screen, switch the Y-axis scale, enable or disable the rectangle selection, roll back the chart step by step, and restore the chart. + +- Full-screen display: Display the scalar curve in full screen. Click the button again to restore it. +- Switch Y-axis scale: Perform logarithmic conversion on the Y-axis coordinate. +- Enable/Disable rectangle selection: Draw a rectangle to select and zoom in a part of the chart. You can perform rectangle selection again on the zoomed-in chart. +- Step-by-step Rollback: Cancel operations step by step after continuously drawing rectangles to select and zooming in the same area. +- Restore chart: Restore a chart to the original state. + +There can set the threshold value to highlight the value or delete the threshold value in the lower right corner of the figure. As shown in the figure, the threshold set is less than 1.5, highlighted in red shows what is below the threshold, and it is intuitive to see the expected data value or some unusual value. + +![scalar_select.png](./images/scalar_select.png) + +Figure 2: Scalar visualization function area + +Figure 2 shows the scalar visualization function area, which allows you to view scalar information by selecting different tags, different dimensions of the horizontal axis, and smoothness. + +- Tag selection: Select the required tags to view the corresponding scalar information. +- Horizontal axis: Select any of Step, Relative Time, and Absolute Time as the horizontal axis of the scalar curve. +- Smoothness: adjust the smoothness to smooth the scalar curve. +- Scalar synthesis: Synthesize two scalar curves and display them in a chart to facilitate comparison between the two curves or view the synthesized chart. + +![scalar_compound.png](./images/scalar_compound.png) + +Figure 3: Scalar synthesis of Accuracy and Loss curves + +Figure 3 shows the scalar synthesis of the Accuracy and Loss curves. The function area of scalar synthesis is similar to that of scalar visualization. Different from the scalar visualization function area, the scalar synthesis function allows you to select a maximum of two tags at a time to synthesize and display their curves. + +## Parameter Distribution Visualization + +The parameter distribution in a form of a histogram displays tensors specified by a user. + +![histogram.png](./images/histogram.png) + +Figure 4: Histogram + +Figure 4 shows tensors recorded by a user in a form of a histogram. Click the upper right corner to zoom in the histogram. + +![histogram_func.png](./images/histogram_func.png) + +Figure 5: Function area of the parameter distribution histogram + +Figure 5 shows the function area of the parameter distribution histogram, including: + +- Tag selection: Select the required tags to view the corresponding histogram. +- Vertical axis: Select any of `Step`, `Relative time`, and `Absolute time` as the data displayed on the vertical axis of the histogram. +- Angle of view: Select either `Front` or `Top`. `Front` view refers to viewing the histogram from the front view. In this case, data between different steps is overlapped. `Top` view refers to viewing the histogram at an angle of 45 degrees. In this case, data between different steps can be presented. + +## Computational Graph Visualization + +Computational graph visualization is used to display the graph structure, data flow direction, and control flow direction of a computational graph. It supports visualization of summary log files and pb files generated by `save_graphs` configuration in `context`. + +![graph.png](./images/graph.png) + +Figure 6: Computational graph display area + +Figure 6 shows the network structure of a computational graph. As shown in the figure, select an operator in the area of the display area. The operator has two inputs and one outputs (the solid line indicates the data flow direction of the operator). + +![graph_sidebar.png](./images/graph_sidebar.png) + +Figure 7: Computational graph function area + +Figure 7 shows the function area of the computational graph, including: + +- File selection box: View the computational graphs of different files. +- Search box: Enter a node name and press Enter to view the node. +- Thumbnail: Display the thumbnail of the entire network structure. When viewing an extra large image structure, you can view the currently browsed area. +- Node information: Display the basic information of the selected node, including the node name, properties, input node, and output node. +- Legend: Display the meaning of each icon in the computational graph. + +## Dataset Graph Visualization + +Dataset graph visualization is used to display data processing and augmentation information of a single model training. + +![data_function.png](./images/data_function.png) + +Figure 8: Dataset graph function area + +Figure 8 shows the dataset graph function area which includes the following content: + +- Legend: Display the meaning of each icon in the data lineage graph. +- Data processing pipeline: Display the data processing pipeline used for training. Select a single node in the graph to view details. +- Node information: Display basic information about the selected node, including names and parameters of the data processing and augmentation operators. + +## Image Visualization + +Image visualization is used to display images specified by users. + +![image.png](./images/image_vi.png) + +Figure 9: Image visualization + +Figure 9 shows how to view images of different steps by sliding the Step slider. + +![image_function.png](./images/image_function.png) + +Figure 10: Image visualization function area + +Figure 10 shows the function area of image visualization. You can view image information by selecting different tags, brightness, and contrast. + +- Tag: Select the required tags to view the corresponding image information. +- Brightness adjustment: Adjust the brightness of all displayed images. +- Contrast adjustment: Adjust the contrast of all displayed images. + +## Tensor Visualization + +Tensor visualization is used to display tensors in the form of table and histogram. + +![tensor_function.png](./images/tensor_function.png) + +Figure 11: Tensor visualization function area + +Figure 11 shows the function area of tensor visualization. + +- Tag selection: Select the required tags to view the corresponding table data or histogram. +- View: Select `Table` or `Histogram` to display tensor data. In the `Histogram` view, there are the options of `Vertical axis` and `Angle of view`. +- Vertical axis: Select any of `Step`, `Relative time`, and `Absolute time` as the data displayed on the vertical axis of the histogram. +- Angle of view: Select either `Front` or `Top`. `Front` view refers to viewing the histogram from the front view. In this case, data between different steps is overlapped. `Top` view refers to viewing the histogram at an angle of 45 degrees. In this case, data between different steps can be presented. + +![tensor_table.png](./images/tensor_table.png) + +Figure 12: Table display + +Figure 12 shows tensors recorded by a user in a form of a table which includes the following function: + +- Click the small square button on the right side of the table to zoom in the table. +- The white box in the table shows the tensor data under which dimension is currently displayed, where the colon `:` represents all values of the current dimension, you can enter the corresponding index or `:` in the box and press `Enter` or click the button of tick on the back to query tensor data for specific dimensions. + Assuming a certain dimension is 32, the index range is -32 to 31. Note: tensor data from 0 to 2 dimensions can be queried. Tensor data of more than two dimensions is not supported, in other word, the query conditions of more than two colons `:` cannot be set. +- Query the tensor data of a specific step by dragging the hollow circle below the table. + +![tensor_histogram.png](./images/tensor_histogram.png) + +Figure 13: Histogram display + +Figure 13 shows tensors recorded by a user in a form of a histogram. Click the upper right corner to zoom in the histogram. + +## Notices + + +1. Currently MindSpore supports recording computational graph after operator fusion for Ascend 910 AI processor only. + +2. When using the Summary operator to collect data in training, 'HistogramSummary' operator affects performance, so please use as little as possible. + +3. To limit memory usage, MindInsight limits the number of tags and steps: + - There are 300 tags at most in each training dashboard. Total number of scalar tags, image tags, computation graph tags, parameter distribution(histogram) tags, tensor tags can not exceed 300. Specially, there are 10 computation graph tags and 6 tensor tags at most. When tags exceed limit, MindInsight preserves the most recently processed tags. + - There are 1000 steps at most for each scalar tag in each training dashboard. When steps exceed limit, MindInsight will sample steps randomly to meet this limit. + - There are 10 steps at most for each image tag in each training dashboard. When steps exceed limit, MindInsight will sample steps randomly to meet this limit. + - There are 50 steps at most for each parameter distribution(histogram) tag in each training dashboard. When steps exceed limit, MindInsight will sample steps randomly to meet this limit. + - There are 20 steps at most for each tensor tag in each training dashboard. When steps exceed limit, MindInsight will sample steps randomly to meet this limit. + +4. Since `TensorSummary` will record complete tensor data, the amount of data is usually relatively large. In order to limit memory usage and ensure performance, MindInsight make the following restrictions with the size of tensor and the number of value responsed and displayed on the front end: + - MindInsight supports loading tensor containing up to 10 million values. + - After the tensor is loaded, in the tensor-visible table view, you can view a maximum of 100,000 values. If the value obtained by the selected dimension query exceeds this limit, it cannot be displayed. + +5. Since tensor visualizatioin (`TensorSummary`) records raw tensor data, it requires a large amount of storage space. Before using `TensorSummary` and during training, please check that the system storage space is sufficient. + The storage space occupied by the tensor visualizatioin function can be reduced by the following methods: + 1) Avoid using `TensorSummary` to record larger tensor. + + 2) Reduce the number of `TensorSummary` operators in the network. + + After using the function, please clean up the training logs that are no longer needed in time to free up disk space. + + Remarks: The method of estimating the space usage of `TensorSummary` is as follows: + + The size of a `TensorSummary` data = the number of values in the tensor * 4 bytes. Assuming that the size of the tensor recorded by `TensorSummary` is 32 * 1 * 256 * 256, then a `TensorSummary` data needs about 32 * 1 * 256 * 256 * 4 bytes = 8,388,608 bytes = 8MiB. `TensorSummary` will record data of 20 steps by default. Then the required space when recording these 20 sets of data is about 20 * 8 MiB = 160MiB. It should be noted that due to the overhead of data structure and other factors, the actual storage space used will be slightly larger than 160MiB. + 6. The training log file is large when using `TensorSummary` because the complete tensor data is recorded. MindInsight needs more time to parse the training log file, please be patient. \ No newline at end of file diff --git a/tutorials/source_en/advanced_use/debugging_in_pynative_mode.md b/tutorials/training/source_en/advanced_use/debugging_in_pynative_mode.md similarity index 100% rename from tutorials/source_en/advanced_use/debugging_in_pynative_mode.md rename to tutorials/training/source_en/advanced_use/debugging_in_pynative_mode.md diff --git a/tutorials/source_en/advanced_use/differential_privacy.md b/tutorials/training/source_en/advanced_use/differential_privacy.md similarity index 98% rename from tutorials/source_en/advanced_use/differential_privacy.md rename to tutorials/training/source_en/advanced_use/differential_privacy.md index 57bd79f4adb8ef69eb377f3b42265c972142be9a..13ea9770f5868ea259b16b07aab81c3204e9022f 100644 --- a/tutorials/source_en/advanced_use/differential_privacy.md +++ b/tutorials/training/source_en/advanced_use/differential_privacy.md @@ -1,358 +1,358 @@ -# Differential Privacy in Machine Learning - -`Linux` `Ascend` `Model Development` `Model Optimization` `Enterprise` `Expert` - - - -- [Differential Privacy in Machine Learning](#differential-privacy-in-machine-learning) - - [Overview](#overview) - - [Implementation](#implementation) - - [Importing Library Files](#importing-library-files) - - [Configuring Parameters](#configuring-parameters) - - [Preprocessing the Dataset](#preprocessing-the-dataset) - - [Creating the Model](#creating-the-model) - - [Introducing the Differential Privacy](#introducing-the-differential-privacy) - - [References](#references) - - - - - -## Overview - -Differential privacy is a mechanism for protecting user data privacy. What is privacy? Privacy refers to the attributes of individual users. Common attributes shared by a group of users may not be considered as privacy. For example, if we say "smoking people have a higher probability of getting lung cancer", it does not disclose privacy. However, if we say "Zhang San smokes and gets lung cancer", it discloses the privacy of Zhang San. Assume that there are 100 patients in a hospital and 10 of them have lung cancer. If the information of any 99 patients are known, we can infer whether the remaining one has lung cancer. This behavior of stealing privacy is called differential attack. Differential privacy is a method for preventing differential attacks. By adding noise, the query results of two datasets with only one different record are nearly indistinguishable. In the above example, after differential privacy is used, the statistic information of the 100 patients achieved by the attacker is almost the same as that of the 99 patients. Therefore, the attacker can hardly infer the information of the remaining one patient. - -**Differential privacy in machine learning** - -Machine learning algorithms usually update model parameters and learn data features based on a large amount of data. Ideally, these models can learn the common features of a class of entities and achieve good generalization, such as "smoking patients are more likely to get lung cancer" rather than models with individual features, such as "Zhang San is a smoker who gets lung cancer." However, machine learning algorithms do not distinguish between general and individual features. The published machine learning models, especially the deep neural networks, may unintentionally memorize and expose the features of individual entities in training data. This can be exploited by malicious attackers to reveal Zhang San's privacy information from the published model. Therefore, it is necessary to use differential privacy to protect machine learning models from privacy leakage. - -**Differential privacy definition** [1] - -$Pr[\mathcal{K}(D)\in S] \le e^{\epsilon} Pr[\mathcal{K}(D') \in S]+\delta$ - -For datasets $D$ and $D'$ that differ on only one record, the probability of obtaining the same result from $\mathcal{K}(D)$ and $\mathcal{K}(D')$ by using a randomized algorithm $\mathcal{K}$ must meet the preceding formula. $\epsilon$ indicates the differential privacy budget and $\delta$ indicates the perturbation. The smaller the values of $\epsilon$ and $\delta$, the closer the data distribution output by $\mathcal{K}$ on $D$ and $D'$. - -**Differential privacy measurement** - -Differential privacy can be measured using $\epsilon$ and $\delta$. - -- $\epsilon$: specifies the upper limit of the output probability that can be changed when a record is added to or deleted from the dataset. We usually hope that $\epsilon$ is a small constant. A smaller value indicates stricter differential privacy conditions. -- $\delta$: limits the probability of arbitrary model behavior change. Generally, this parameter is set to a small constant. You are advised to set this parameter to a value less than the reciprocal of the size of a training dataset. - -**Differential privacy implemented by MindArmour** - -MindArmour differential privacy module Differential-Privacy implements the differential privacy optimizer. Currently, SGD, Momentum, and Adam are supported. They are differential privacy optimizers based on the Gaussian mechanism. Gaussian noise mechanism supports both non-adaptive policy and adaptive policy The non-adaptive policy use a fixed noise parameter for each step while the adaptive policy changes the noise parameter along time or iteration step. An advantage of using the non-adaptive Gaussian noise is that a differential privacy budget $\epsilon$ can be strictly controlled. However, a disadvantage is that in a model training process, the noise amount added in each step is fixed. In the later training stage, large noise makes the model convergence difficult, and even causes the performance to decrease greatly and the model usability to be poor. Adaptive noise can solve this problem. In the initial model training stage, the amount of added noise is large. As the model converges, the amount of noise decreases gradually, and the impact of noise on model availability decreases. The disadvantage is that the differential privacy budget cannot be strictly controlled. Under the same initial value, the $\epsilon$ of the adaptive differential privacy is greater than that of the non-adaptive differential privacy. Rényi differential privacy (RDP) [2] is also provided to monitor differential privacy budgets. - -The LeNet model and MNIST dataset are used as an example to describe how to use the differential privacy optimizer to train a neural network model on MindSpore. - -> This example is for the Ascend 910 AI processor. You can download the complete sample code from . - -## Implementation - -### Importing Library Files - -The following are the required public modules, MindSpore modules, and differential privacy feature modules. - -```python -import os -from easydict import EasyDict as edict - -import mindspore.nn as nn -from mindspore import context -from mindspore.train.callback import ModelCheckpoint -from mindspore.train.callback import CheckpointConfig -from mindspore.train.callback import LossMonitor -from mindspore.nn.metrics import Accuracy -from mindspore.train.serialization import load_checkpoint, load_param_into_net -import mindspore.dataset as ds -import mindspore.dataset.vision.c_transforms as CV -import mindspore.dataset.transforms.c_transforms as C -from mindspore.dataset.vision import Inter -import mindspore.common.dtype as mstype - -from mindarmour.diff_privacy import DPModel -from mindarmour.diff_privacy import PrivacyMonitorFactory -from mindarmour.diff_privacy import NoiseMechanismsFactory -from mindarmour.diff_privacy import ClipMechanismsFactory -from mindarmour.utils.logger import LogUtil -from lenet5_net import LeNet5 -from lenet5_config import mnist_cfg as cfg - -LOGGER = LogUtil.get_instance() -LOGGER.set_level('INFO') -TAG = 'Lenet5_train' -``` - -### Configuring Parameters - -1. Set the running environment, dataset path, model training parameters, checkpoint storage parameters, and differential privacy parameters. Replace 'data_path' with you data path. For more configurations, see . - - ```python - cfg = edict({ - 'num_classes': 10, # the number of classes of model's output - 'lr': 0.01, # the learning rate of model's optimizer - 'momentum': 0.9, # the momentum value of model's optimizer - 'epoch_size': 10, # training epochs - 'batch_size': 256, # batch size for training - 'image_height': 32, # the height of training samples - 'image_width': 32, # the width of training samples - 'save_checkpoint_steps': 234, # the interval steps for saving checkpoint file of the model - 'keep_checkpoint_max': 10, # the maximum number of checkpoint files would be saved - 'device_target': 'Ascend', # device used - 'data_path': './MNIST_unzip', # the path of training and testing data set - 'dataset_sink_mode': False, # whether deliver all training data to device one time - 'micro_batches': 32, # the number of small batches split from an original batch - 'norm_bound': 1.0, # the clip bound of the gradients of model's training parameters - 'initial_noise_multiplier': 0.05, # the initial multiplication coefficient of the noise added to training - # parameters' gradients - 'noise_mechanisms': 'Gaussian', # the method of adding noise in gradients while training - 'clip_mechanisms': 'Gaussian', # the method of adaptive clipping gradients while training - 'clip_decay_policy': 'Linear', # Decay policy of adaptive clipping, decay_policy must be in ['Linear', 'Geometric']. - 'clip_learning_rate': 0.001, # Learning rate of update norm clip. - 'target_unclipped_quantile': 0.9, # Target quantile of norm clip. - 'fraction_stddev': 0.01, # The stddev of Gaussian normal which used in empirical_fraction. - 'optimizer': 'Momentum' # the base optimizer used for Differential privacy training - }) - ``` - -2. Configure necessary information, including the environment information and execution mode. - - ```python - context.set_context(mode=context.GRAPH_MODE, device_target=cfg.device_target) - ``` - - For details about the API configuration, see the `context.set_context`. - -### Preprocessing the Dataset - -Load the dataset and convert the dataset format to a MindSpore data format. - -```python -def generate_mnist_dataset(data_path, batch_size=32, repeat_size=1, - num_parallel_workers=1, sparse=True): - """ - create dataset for training or testing - """ - # define dataset - ds1 = ds.MnistDataset(data_path) - - # define operation parameters - resize_height, resize_width = 32, 32 - rescale = 1.0 / 255.0 - shift = 0.0 - - # define map operations - resize_op = CV.Resize((resize_height, resize_width), - interpolation=Inter.LINEAR) - rescale_op = CV.Rescale(rescale, shift) - hwc2chw_op = CV.HWC2CHW() - type_cast_op = C.TypeCast(mstype.int32) - - # apply map operations on images - if not sparse: - one_hot_enco = C.OneHot(10) - ds1 = ds1.map(operations=one_hot_enco, input_columns="label", - num_parallel_workers=num_parallel_workers) - type_cast_op = C.TypeCast(mstype.float32) - ds1 = ds1.map(operations=type_cast_op, input_columns="label", - num_parallel_workers=num_parallel_workers) - ds1 = ds1.map(operations=resize_op, input_columns="image", - num_parallel_workers=num_parallel_workers) - ds1 = ds1.map(operations=rescale_op, input_columns="image", - num_parallel_workers=num_parallel_workers) - ds1 = ds1.map(operations=hwc2chw_op, input_columns="image", - num_parallel_workers=num_parallel_workers) - - # apply DatasetOps - buffer_size = 10000 - ds1 = ds1.shuffle(buffer_size=buffer_size) - ds1 = ds1.batch(batch_size, drop_remainder=True) - ds1 = ds1.repeat(repeat_size) - - return ds1 -``` - -### Creating the Model - -The LeNet model is used as an example. You can also create and train your own model. - -```python -from mindspore import nn -from mindspore.common.initializer import TruncatedNormal - - -def conv(in_channels, out_channels, kernel_size, stride=1, padding=0): - weight = weight_variable() - return nn.Conv2d(in_channels, out_channels, - kernel_size=kernel_size, stride=stride, padding=padding, - weight_init=weight, has_bias=False, pad_mode="valid") - - -def fc_with_initialize(input_channels, out_channels): - weight = weight_variable() - bias = weight_variable() - return nn.Dense(input_channels, out_channels, weight, bias) - - -def weight_variable(): - return TruncatedNormal(0.05) - - -class LeNet5(nn.Cell): - """ - LeNet network - """ - def __init__(self): - super(LeNet5, self).__init__() - self.conv1 = conv(1, 6, 5) - self.conv2 = conv(6, 16, 5) - self.fc1 = fc_with_initialize(16*5*5, 120) - self.fc2 = fc_with_initialize(120, 84) - self.fc3 = fc_with_initialize(84, 10) - self.relu = nn.ReLU() - self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) - self.flatten = nn.Flatten() - - def construct(self, x): - x = self.conv1(x) - x = self.relu(x) - x = self.max_pool2d(x) - x = self.conv2(x) - x = self.relu(x) - x = self.max_pool2d(x) - x = self.flatten(x) - x = self.fc1(x) - x = self.relu(x) - x = self.fc2(x) - x = self.relu(x) - x = self.fc3(x) - return x -``` - -Load the LeNet network, define the loss function, configure the checkpoint parameters, and load data by using the `generate_mnist_dataset` function defined in the preceding information. - -```python -network = LeNet5() -net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") -config_ck = CheckpointConfig(save_checkpoint_steps=cfg.save_checkpoint_steps, - keep_checkpoint_max=cfg.keep_checkpoint_max) -ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", - directory='./trained_ckpt_file/', - config=config_ck) - -# get training dataset -ds_train = generate_mnist_dataset(os.path.join(cfg.data_path, "train"), - cfg.batch_size) -``` - -### Introducing the Differential Privacy - -1. Set parameters of a differential privacy optimizer. - - - Determine whether values of the `micro_batches` and `batch_size` parameters meet the requirements. The value of `batch_size` must be an integer multiple of `micro_batches`. - - Instantiate a differential privacy factory class. - - Set a noise mechanism for the differential privacy. Currently, the Gaussian noise mechanism with a fixed standard deviation (`Gaussian`) and the Gaussian noise mechanism with an adaptive standard deviation (`AdaGaussian`) are supported. - - Set an optimizer type. Currently, `SGD`, `Momentum`, and `Adam` are supported. - - Set up a differential privacy budget monitor RDP to observe changes in the differential privacy budget $\epsilon$ in each step. - - ```python - if cfg.micro_batches and cfg.batch_size % cfg.micro_batches != 0: - raise ValueError( - "Number of micro_batches should divide evenly batch_size") - # Create a factory class of DP noise mechanisms, this method is adding noise - # in gradients while training. Initial_noise_multiplier is suggested to be - # greater than 1.0, otherwise the privacy budget would be huge, which means - # that the privacy protection effect is weak. Mechanisms can be 'Gaussian' - # or 'AdaGaussian', in which noise would be decayed with 'AdaGaussian' - # mechanism while be constant with 'Gaussian' mechanism. - noise_mech = NoiseMechanismsFactory().create(cfg.noise_mechanisms, - norm_bound=cfg.norm_bound, - initial_noise_multiplier=cfg.initial_noise_multiplier, - decay_policy=None) - # Create a factory class of clip mechanisms, this method is to adaptive clip - # gradients while training, decay_policy support 'Linear' and 'Geometric', - # learning_rate is the learning rate to update clip_norm, - # target_unclipped_quantile is the target quantile of norm clip, - # fraction_stddev is the stddev of Gaussian normal which used in - # empirical_fraction, the formula is - # $empirical_fraction + N(0, fraction_stddev)$. - clip_mech = ClipMechanismsFactory().create(cfg.clip_mechanisms, - decay_policy=cfg.clip_decay_policy, - learning_rate=cfg.clip_learning_rate, - target_unclipped_quantile=cfg.target_unclipped_quantile, - fraction_stddev=cfg.fraction_stddev) - net_opt = nn.Momentum(params=network.trainable_params(), - learning_rate=cfg.lr, momentum=cfg.momentum) - # Create a monitor for DP training. The function of the monitor is to - # compute and print the privacy budget(eps and delta) while training. - rdp_monitor = PrivacyMonitorFactory.create('rdp', - num_samples=60000, - batch_size=cfg.batch_size, - initial_noise_multiplier=cfg.initial_noise_multiplier, - per_print_times=234, - noise_decay_mode=None) - ``` - -2. Package the LeNet model as a differential privacy model by transferring the network to `DPModel`. - - ```python - # Create the DP model for training. - model = DPModel(micro_batches=cfg.micro_batches, - norm_bound=cfg.norm_bound, - noise_mech=noise_mech, - clip_mech=clip_mech, - network=network, - loss_fn=net_loss, - optimizer=net_opt, - metrics={"Accuracy": Accuracy()}) - ``` - -3. Train and test the model. - - ```python - LOGGER.info(TAG, "============== Starting Training ==============") - model.train(cfg['epoch_size'], ds_train, - callbacks=[ckpoint_cb, LossMonitor(), rdp_monitor], - dataset_sink_mode=cfg.dataset_sink_mode) - - LOGGER.info(TAG, "============== Starting Testing ==============") - ckpt_file_name = 'trained_ckpt_file/checkpoint_lenet-10_234.ckpt' - param_dict = load_checkpoint(ckpt_file_name) - load_param_into_net(network, param_dict) - ds_eval = generate_mnist_dataset(os.path.join(cfg.data_path, 'test'), - batch_size=cfg.batch_size) - acc = model.eval(ds_eval, dataset_sink_mode=False) - LOGGER.info(TAG, "============== Accuracy: %s ==============", acc) - ``` - -4. Run the following command. - - Execute the script: - - ```bash - python lenet5_dp.py - ``` - - In the preceding command, replace `lenet5_dp.py` with the name of your script. - -5. Display the result. - - The accuracy of the LeNet model without differential privacy is 99%, and the accuracy of the LeNet model with Gaussian noise and adaptive clip differential privacy is mostly more than 95%. - ``` - ============== Starting Training ============== - ... - ============== Starting Testing ============== - ... - ============== Accuracy: 0.9698 ============== - ``` - -### References - -[1] C. Dwork and J. Lei. Differential privacy and robust statistics. In STOC, pages 371–380. ACM, 2009. - -[2] Ilya Mironov. Rényi differential privacy. In IEEE Computer Security Foundations Symposium, 2017. - -[3] Abadi, M. e. a., 2016. *Deep learning with differential privacy.* s.l.:Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. - - - +# Differential Privacy in Machine Learning + +`Linux` `Ascend` `Model Development` `Model Optimization` `Enterprise` `Expert` + + + +- [Differential Privacy in Machine Learning](#differential-privacy-in-machine-learning) + - [Overview](#overview) + - [Implementation](#implementation) + - [Importing Library Files](#importing-library-files) + - [Configuring Parameters](#configuring-parameters) + - [Preprocessing the Dataset](#preprocessing-the-dataset) + - [Creating the Model](#creating-the-model) + - [Introducing the Differential Privacy](#introducing-the-differential-privacy) + - [References](#references) + + + + + +## Overview + +Differential privacy is a mechanism for protecting user data privacy. What is privacy? Privacy refers to the attributes of individual users. Common attributes shared by a group of users may not be considered as privacy. For example, if we say "smoking people have a higher probability of getting lung cancer", it does not disclose privacy. However, if we say "Zhang San smokes and gets lung cancer", it discloses the privacy of Zhang San. Assume that there are 100 patients in a hospital and 10 of them have lung cancer. If the information of any 99 patients are known, we can infer whether the remaining one has lung cancer. This behavior of stealing privacy is called differential attack. Differential privacy is a method for preventing differential attacks. By adding noise, the query results of two datasets with only one different record are nearly indistinguishable. In the above example, after differential privacy is used, the statistic information of the 100 patients achieved by the attacker is almost the same as that of the 99 patients. Therefore, the attacker can hardly infer the information of the remaining one patient. + +**Differential privacy in machine learning** + +Machine learning algorithms usually update model parameters and learn data features based on a large amount of data. Ideally, these models can learn the common features of a class of entities and achieve good generalization, such as "smoking patients are more likely to get lung cancer" rather than models with individual features, such as "Zhang San is a smoker who gets lung cancer." However, machine learning algorithms do not distinguish between general and individual features. The published machine learning models, especially the deep neural networks, may unintentionally memorize and expose the features of individual entities in training data. This can be exploited by malicious attackers to reveal Zhang San's privacy information from the published model. Therefore, it is necessary to use differential privacy to protect machine learning models from privacy leakage. + +**Differential privacy definition** [1] + +$Pr[\mathcal{K}(D)\in S] \le e^{\epsilon} Pr[\mathcal{K}(D') \in S]+\delta$ + +For datasets $D$ and $D'$ that differ on only one record, the probability of obtaining the same result from $\mathcal{K}(D)$ and $\mathcal{K}(D')$ by using a randomized algorithm $\mathcal{K}$ must meet the preceding formula. $\epsilon$ indicates the differential privacy budget and $\delta$ indicates the perturbation. The smaller the values of $\epsilon$ and $\delta$, the closer the data distribution output by $\mathcal{K}$ on $D$ and $D'$. + +**Differential privacy measurement** + +Differential privacy can be measured using $\epsilon$ and $\delta$. + +- $\epsilon$: specifies the upper limit of the output probability that can be changed when a record is added to or deleted from the dataset. We usually hope that $\epsilon$ is a small constant. A smaller value indicates stricter differential privacy conditions. +- $\delta$: limits the probability of arbitrary model behavior change. Generally, this parameter is set to a small constant. You are advised to set this parameter to a value less than the reciprocal of the size of a training dataset. + +**Differential privacy implemented by MindArmour** + +MindArmour differential privacy module Differential-Privacy implements the differential privacy optimizer. Currently, SGD, Momentum, and Adam are supported. They are differential privacy optimizers based on the Gaussian mechanism. Gaussian noise mechanism supports both non-adaptive policy and adaptive policy The non-adaptive policy use a fixed noise parameter for each step while the adaptive policy changes the noise parameter along time or iteration step. An advantage of using the non-adaptive Gaussian noise is that a differential privacy budget $\epsilon$ can be strictly controlled. However, a disadvantage is that in a model training process, the noise amount added in each step is fixed. In the later training stage, large noise makes the model convergence difficult, and even causes the performance to decrease greatly and the model usability to be poor. Adaptive noise can solve this problem. In the initial model training stage, the amount of added noise is large. As the model converges, the amount of noise decreases gradually, and the impact of noise on model availability decreases. The disadvantage is that the differential privacy budget cannot be strictly controlled. Under the same initial value, the $\epsilon$ of the adaptive differential privacy is greater than that of the non-adaptive differential privacy. Rényi differential privacy (RDP) [2] is also provided to monitor differential privacy budgets. + +The LeNet model and MNIST dataset are used as an example to describe how to use the differential privacy optimizer to train a neural network model on MindSpore. + +> This example is for the Ascend 910 AI processor. You can download the complete sample code from . + +## Implementation + +### Importing Library Files + +The following are the required public modules, MindSpore modules, and differential privacy feature modules. + +```python +import os +from easydict import EasyDict as edict + +import mindspore.nn as nn +from mindspore import context +from mindspore.train.callback import ModelCheckpoint +from mindspore.train.callback import CheckpointConfig +from mindspore.train.callback import LossMonitor +from mindspore.nn.metrics import Accuracy +from mindspore.train.serialization import load_checkpoint, load_param_into_net +import mindspore.dataset as ds +import mindspore.dataset.vision.c_transforms as CV +import mindspore.dataset.transforms.c_transforms as C +from mindspore.dataset.vision import Inter +import mindspore.common.dtype as mstype + +from mindarmour.diff_privacy import DPModel +from mindarmour.diff_privacy import PrivacyMonitorFactory +from mindarmour.diff_privacy import NoiseMechanismsFactory +from mindarmour.diff_privacy import ClipMechanismsFactory +from mindarmour.utils.logger import LogUtil +from lenet5_net import LeNet5 +from lenet5_config import mnist_cfg as cfg + +LOGGER = LogUtil.get_instance() +LOGGER.set_level('INFO') +TAG = 'Lenet5_train' +``` + +### Configuring Parameters + +1. Set the running environment, dataset path, model training parameters, checkpoint storage parameters, and differential privacy parameters. Replace 'data_path' with you data path. For more configurations, see . + + ```python + cfg = edict({ + 'num_classes': 10, # the number of classes of model's output + 'lr': 0.01, # the learning rate of model's optimizer + 'momentum': 0.9, # the momentum value of model's optimizer + 'epoch_size': 10, # training epochs + 'batch_size': 256, # batch size for training + 'image_height': 32, # the height of training samples + 'image_width': 32, # the width of training samples + 'save_checkpoint_steps': 234, # the interval steps for saving checkpoint file of the model + 'keep_checkpoint_max': 10, # the maximum number of checkpoint files would be saved + 'device_target': 'Ascend', # device used + 'data_path': './MNIST_unzip', # the path of training and testing data set + 'dataset_sink_mode': False, # whether deliver all training data to device one time + 'micro_batches': 32, # the number of small batches split from an original batch + 'norm_bound': 1.0, # the clip bound of the gradients of model's training parameters + 'initial_noise_multiplier': 0.05, # the initial multiplication coefficient of the noise added to training + # parameters' gradients + 'noise_mechanisms': 'Gaussian', # the method of adding noise in gradients while training + 'clip_mechanisms': 'Gaussian', # the method of adaptive clipping gradients while training + 'clip_decay_policy': 'Linear', # Decay policy of adaptive clipping, decay_policy must be in ['Linear', 'Geometric']. + 'clip_learning_rate': 0.001, # Learning rate of update norm clip. + 'target_unclipped_quantile': 0.9, # Target quantile of norm clip. + 'fraction_stddev': 0.01, # The stddev of Gaussian normal which used in empirical_fraction. + 'optimizer': 'Momentum' # the base optimizer used for Differential privacy training + }) + ``` + +2. Configure necessary information, including the environment information and execution mode. + + ```python + context.set_context(mode=context.GRAPH_MODE, device_target=cfg.device_target) + ``` + + For details about the API configuration, see the `context.set_context`. + +### Preprocessing the Dataset + +Load the dataset and convert the dataset format to a MindSpore data format. + +```python +def generate_mnist_dataset(data_path, batch_size=32, repeat_size=1, + num_parallel_workers=1, sparse=True): + """ + create dataset for training or testing + """ + # define dataset + ds1 = ds.MnistDataset(data_path) + + # define operation parameters + resize_height, resize_width = 32, 32 + rescale = 1.0 / 255.0 + shift = 0.0 + + # define map operations + resize_op = CV.Resize((resize_height, resize_width), + interpolation=Inter.LINEAR) + rescale_op = CV.Rescale(rescale, shift) + hwc2chw_op = CV.HWC2CHW() + type_cast_op = C.TypeCast(mstype.int32) + + # apply map operations on images + if not sparse: + one_hot_enco = C.OneHot(10) + ds1 = ds1.map(operations=one_hot_enco, input_columns="label", + num_parallel_workers=num_parallel_workers) + type_cast_op = C.TypeCast(mstype.float32) + ds1 = ds1.map(operations=type_cast_op, input_columns="label", + num_parallel_workers=num_parallel_workers) + ds1 = ds1.map(operations=resize_op, input_columns="image", + num_parallel_workers=num_parallel_workers) + ds1 = ds1.map(operations=rescale_op, input_columns="image", + num_parallel_workers=num_parallel_workers) + ds1 = ds1.map(operations=hwc2chw_op, input_columns="image", + num_parallel_workers=num_parallel_workers) + + # apply DatasetOps + buffer_size = 10000 + ds1 = ds1.shuffle(buffer_size=buffer_size) + ds1 = ds1.batch(batch_size, drop_remainder=True) + ds1 = ds1.repeat(repeat_size) + + return ds1 +``` + +### Creating the Model + +The LeNet model is used as an example. You can also create and train your own model. + +```python +from mindspore import nn +from mindspore.common.initializer import TruncatedNormal + + +def conv(in_channels, out_channels, kernel_size, stride=1, padding=0): + weight = weight_variable() + return nn.Conv2d(in_channels, out_channels, + kernel_size=kernel_size, stride=stride, padding=padding, + weight_init=weight, has_bias=False, pad_mode="valid") + + +def fc_with_initialize(input_channels, out_channels): + weight = weight_variable() + bias = weight_variable() + return nn.Dense(input_channels, out_channels, weight, bias) + + +def weight_variable(): + return TruncatedNormal(0.05) + + +class LeNet5(nn.Cell): + """ + LeNet network + """ + def __init__(self): + super(LeNet5, self).__init__() + self.conv1 = conv(1, 6, 5) + self.conv2 = conv(6, 16, 5) + self.fc1 = fc_with_initialize(16*5*5, 120) + self.fc2 = fc_with_initialize(120, 84) + self.fc3 = fc_with_initialize(84, 10) + self.relu = nn.ReLU() + self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) + self.flatten = nn.Flatten() + + def construct(self, x): + x = self.conv1(x) + x = self.relu(x) + x = self.max_pool2d(x) + x = self.conv2(x) + x = self.relu(x) + x = self.max_pool2d(x) + x = self.flatten(x) + x = self.fc1(x) + x = self.relu(x) + x = self.fc2(x) + x = self.relu(x) + x = self.fc3(x) + return x +``` + +Load the LeNet network, define the loss function, configure the checkpoint parameters, and load data by using the `generate_mnist_dataset` function defined in the preceding information. + +```python +network = LeNet5() +net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") +config_ck = CheckpointConfig(save_checkpoint_steps=cfg.save_checkpoint_steps, + keep_checkpoint_max=cfg.keep_checkpoint_max) +ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", + directory='./trained_ckpt_file/', + config=config_ck) + +# get training dataset +ds_train = generate_mnist_dataset(os.path.join(cfg.data_path, "train"), + cfg.batch_size) +``` + +### Introducing the Differential Privacy + +1. Set parameters of a differential privacy optimizer. + + - Determine whether values of the `micro_batches` and `batch_size` parameters meet the requirements. The value of `batch_size` must be an integer multiple of `micro_batches`. + - Instantiate a differential privacy factory class. + - Set a noise mechanism for the differential privacy. Currently, the Gaussian noise mechanism with a fixed standard deviation (`Gaussian`) and the Gaussian noise mechanism with an adaptive standard deviation (`AdaGaussian`) are supported. + - Set an optimizer type. Currently, `SGD`, `Momentum`, and `Adam` are supported. + - Set up a differential privacy budget monitor RDP to observe changes in the differential privacy budget $\epsilon$ in each step. + + ```python + if cfg.micro_batches and cfg.batch_size % cfg.micro_batches != 0: + raise ValueError( + "Number of micro_batches should divide evenly batch_size") + # Create a factory class of DP noise mechanisms, this method is adding noise + # in gradients while training. Initial_noise_multiplier is suggested to be + # greater than 1.0, otherwise the privacy budget would be huge, which means + # that the privacy protection effect is weak. Mechanisms can be 'Gaussian' + # or 'AdaGaussian', in which noise would be decayed with 'AdaGaussian' + # mechanism while be constant with 'Gaussian' mechanism. + noise_mech = NoiseMechanismsFactory().create(cfg.noise_mechanisms, + norm_bound=cfg.norm_bound, + initial_noise_multiplier=cfg.initial_noise_multiplier, + decay_policy=None) + # Create a factory class of clip mechanisms, this method is to adaptive clip + # gradients while training, decay_policy support 'Linear' and 'Geometric', + # learning_rate is the learning rate to update clip_norm, + # target_unclipped_quantile is the target quantile of norm clip, + # fraction_stddev is the stddev of Gaussian normal which used in + # empirical_fraction, the formula is + # $empirical_fraction + N(0, fraction_stddev)$. + clip_mech = ClipMechanismsFactory().create(cfg.clip_mechanisms, + decay_policy=cfg.clip_decay_policy, + learning_rate=cfg.clip_learning_rate, + target_unclipped_quantile=cfg.target_unclipped_quantile, + fraction_stddev=cfg.fraction_stddev) + net_opt = nn.Momentum(params=network.trainable_params(), + learning_rate=cfg.lr, momentum=cfg.momentum) + # Create a monitor for DP training. The function of the monitor is to + # compute and print the privacy budget(eps and delta) while training. + rdp_monitor = PrivacyMonitorFactory.create('rdp', + num_samples=60000, + batch_size=cfg.batch_size, + initial_noise_multiplier=cfg.initial_noise_multiplier, + per_print_times=234, + noise_decay_mode=None) + ``` + +2. Package the LeNet model as a differential privacy model by transferring the network to `DPModel`. + + ```python + # Create the DP model for training. + model = DPModel(micro_batches=cfg.micro_batches, + norm_bound=cfg.norm_bound, + noise_mech=noise_mech, + clip_mech=clip_mech, + network=network, + loss_fn=net_loss, + optimizer=net_opt, + metrics={"Accuracy": Accuracy()}) + ``` + +3. Train and test the model. + + ```python + LOGGER.info(TAG, "============== Starting Training ==============") + model.train(cfg['epoch_size'], ds_train, + callbacks=[ckpoint_cb, LossMonitor(), rdp_monitor], + dataset_sink_mode=cfg.dataset_sink_mode) + + LOGGER.info(TAG, "============== Starting Testing ==============") + ckpt_file_name = 'trained_ckpt_file/checkpoint_lenet-10_234.ckpt' + param_dict = load_checkpoint(ckpt_file_name) + load_param_into_net(network, param_dict) + ds_eval = generate_mnist_dataset(os.path.join(cfg.data_path, 'test'), + batch_size=cfg.batch_size) + acc = model.eval(ds_eval, dataset_sink_mode=False) + LOGGER.info(TAG, "============== Accuracy: %s ==============", acc) + ``` + +4. Run the following command. + + Execute the script: + + ```bash + python lenet5_dp.py + ``` + + In the preceding command, replace `lenet5_dp.py` with the name of your script. + +5. Display the result. + + The accuracy of the LeNet model without differential privacy is 99%, and the accuracy of the LeNet model with Gaussian noise and adaptive clip differential privacy is mostly more than 95%. + ``` + ============== Starting Training ============== + ... + ============== Starting Testing ============== + ... + ============== Accuracy: 0.9698 ============== + ``` + +### References + +[1] C. Dwork and J. Lei. Differential privacy and robust statistics. In STOC, pages 371–380. ACM, 2009. + +[2] Ilya Mironov. Rényi differential privacy. In IEEE Computer Security Foundations Symposium, 2017. + +[3] Abadi, M. e. a., 2016. *Deep learning with differential privacy.* s.l.:Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. + + + diff --git a/tutorials/source_en/advanced_use/distributed_training_ascend.md b/tutorials/training/source_en/advanced_use/distributed_training_ascend.md similarity index 100% rename from tutorials/source_en/advanced_use/distributed_training_ascend.md rename to tutorials/training/source_en/advanced_use/distributed_training_ascend.md diff --git a/tutorials/source_en/advanced_use/distributed_training_tutorials.rst b/tutorials/training/source_en/advanced_use/distributed_training_tutorials.rst similarity index 100% rename from tutorials/source_en/advanced_use/distributed_training_tutorials.rst rename to tutorials/training/source_en/advanced_use/distributed_training_tutorials.rst diff --git a/tutorials/source_en/advanced_use/gradient_accumulation.md b/tutorials/training/source_en/advanced_use/gradient_accumulation.md similarity index 97% rename from tutorials/source_en/advanced_use/gradient_accumulation.md rename to tutorials/training/source_en/advanced_use/gradient_accumulation.md index acb3af83cf4afb4606793916738daf181310ba2a..0cd34505efd1d50148217316e7a666c987c7a1ea 100644 --- a/tutorials/source_en/advanced_use/gradient_accumulation.md +++ b/tutorials/training/source_en/advanced_use/gradient_accumulation.md @@ -1,266 +1,266 @@ -# Gradient Accumulation - -`Linux` `Ascend` `GPU` `Model Optimization` `Intermediate` `Expert` - - - -- [Gradient Accumulation](#gradient-accumulation) - - [Overview](#overview) - - [Creating a Gradient Accumulation Model](#creating-a-gradient-accumulation-model) - - [Importing Library Files](#importing-library-files) - - [Loading the Dataset](#loading-the-dataset) - - [Defining the Network](#defining-the-network) - - [Defining the Training Model](#defining-the-training-model) - - [Defining the Training Process](#defining-the-training-process) - - [Training and Saving the Model](#training-and-saving-the-model) - - [Experiment Result](#experiment-result) - - - - - -## Overview - -This tutorial describes the gradient accumulation training method to solve the problem that some large-scale networks cannot train large batch_size due to insufficient memory. - -In a traditional training method, after a loss and a gradient are calculated, a parameter is directly updated by using the obtained gradient. - -Different from the traditional training method, the concept of mini-batch is introduced to the gradient accumulation. The loss and gradient are computed for each mini-batch data, but the model parameters are not updated immediately. Instead, the obtained gradients are accumulated first, and then after the number (N) of mini-batches is specified, the accumulated gradient is used to update the network parameters. Before the next training, the accumulated gradients are cleared and re-accumulated. - -The ultimate objective is to achieve the same effect as training with N x mini-batch data. - -> This tutorial is applicable to GPUs and Ascend 910 AI Processors. You can download the main training sample code from . - -## Creating a Gradient Accumulation Model - -The MNIST dataset is used as an example to describe how to customize a simple model to implement gradient accumulation. - -### Importing Library Files -The following are the required public modules and MindSpore modules and library files. - -```python -import argparse -import os -from collections.abc import Iterable - -import mindspore.nn as nn -from mindspore import ParameterTuple -from mindspore import context -from mindspore.nn import Cell -from mindspore.ops import composite as C -from mindspore.ops import functional as F -from mindspore.ops import operations as P -from mindspore.train.dataset_helper import DatasetHelper -from mindspore.train.serialization import save_checkpoint -from model_zoo.official.cv.lenet.src.dataset import create_dataset -from model_zoo.official.cv.lenet.src.lenet import LeNet5 -``` - -### Loading the Dataset - -Use the `MnistDataset` API provided by the dataset of MindSpore to load the MNIST dataset. The code is imported from [dataset.py]() in the lenet directory of model_zoo. - -### Defining the Network - -The following uses the LeNet network as an example. You can also use other networks, such as ResNet-50 and BERT. The code is imported from [lenet.py]() in the lenet directory of model_zoo. - -### Defining the Training Model -The training process is divided into three parts: forward and backward training, parameter update, and accumulated gradient clearance. -- `TrainForwardBackward` calculates the loss and gradient, and uses grad_sum to implement gradient accumulation. -- `TrainOptim` updates parameters. -- `TrainClear` clears the gradient accumulation variable grad_sum. - -```python -_sum_op = C.MultitypeFuncGraph("grad_sum_op") -_clear_op = C.MultitypeFuncGraph("clear_op") - - -@_sum_op.register("Tensor", "Tensor") -def _cumulative_gard(grad_sum, grad): - """Apply gard sum to cumulative gradient.""" - add = P.AssignAdd() - return add(grad_sum, grad) - - -@_clear_op.register("Tensor", "Tensor") -def _clear_grad_sum(grad_sum, zero): - """Apply zero to clear grad_sum.""" - success = True - success = F.depend(success, F.assign(grad_sum, zero)) - return success - - -class TrainForwardBackward(Cell): - def __init__(self, network, optimizer, grad_sum, sens=1.0): - super(TrainForwardBackward, self).__init__(auto_prefix=False) - self.network = network - self.network.set_grad() - self.network.add_flags(defer_inline=True) - self.weights = ParameterTuple(network.trainable_params()) - self.optimizer = optimizer - self.grad_sum = grad_sum - self.grad = C.GradOperation(get_by_list=True, sens_param=True) - self.sens = sens - self.hyper_map = C.HyperMap() - - def construct(self, *inputs): - weights = self.weights - loss = self.network(*inputs) - sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens) - grads = self.grad(self.network, weights)(*inputs, sens) - return F.depend(loss, self.hyper_map(F.partial(_sum_op), self.grad_sum, grads)) - - -class TrainOptim(Cell): - def __init__(self, optimizer, grad_sum): - super(TrainOptim, self).__init__(auto_prefix=False) - self.optimizer = optimizer - self.grad_sum = grad_sum - - def construct(self): - return self.optimizer(self.grad_sum) - - -class TrainClear(Cell): - def __init__(self, grad_sum, zeros): - super(TrainClear, self).__init__(auto_prefix=False) - self.grad_sum = grad_sum - self.zeros = zeros - self.hyper_map = C.HyperMap() - - def construct(self): - seccess = self.hyper_map(F.partial(_clear_op), self.grad_sum, self.zeros) - return seccess -``` - -### Defining the Training Process -Each mini-batch calculates the loss and gradient through forward and backward training, and uses mini_steps to control the accumulated times before each parameter update. After the number of accumulation times is reached, the parameter is updated and the accumulated gradient variable is cleared. - - -```python -class GradientAccumulation: - def __init__(self, network, loss_fn, optimizer): - self._network = network - self._loss_fn = loss_fn - self._optimizer = optimizer - - params = self._optimizer.parameters - self._grad_sum = params.clone(prefix="grad_sum", init='zeros') - self._zeros = params.clone(prefix="zeros", init='zeros') - self._train_forward_backward = self._build_train_forward_backward_network() - self._train_optim = self._build_train_optim() - self._train_clear = self._build_train_clear() - - @staticmethod - def _transform_callbacks(callbacks): - """Transform callback to a list.""" - if callbacks is None: - return [] - - if isinstance(callbacks, Iterable): - return list(callbacks) - - return [callbacks] - - def _build_train_forward_backward_network(self): - """Build forward and backward network""" - network = self._network - network = nn.WithLossCell(network, self._loss_fn) - loss_scale = 1.0 - network = TrainForwardBackward(network, self._optimizer, self._grad_sum, loss_scale).set_train() - return network - - def _build_train_optim(self): - """Build optimizer network""" - network = TrainOptim(self._optimizer, self._grad_sum).set_train() - return network - - def _build_train_clear(self): - """Build clear network""" - network = TrainClear(self._grad_sum, self._zeros).set_train() - return network - - def train_process(self, epoch, train_dataset, mini_steps=None): - """ - Training process. The data would be passed to network directly. - """ - dataset_helper = DatasetHelper(train_dataset, dataset_sink_mode=False, epoch_num=epoch) - - for i in range(epoch): - step = 0 - for k, next_element in enumerate(dataset_helper): - loss = self._train_forward_backward(*next_element) - if (k + 1) % mini_steps == 0: - step += 1 - print("epoch:", i + 1, "step:", step, "loss is ", loss) - self._train_optim() - self._train_clear() - - train_dataset.reset() - - save_checkpoint(self._train_forward_backward, "gradient_accumulation.ckpt", ) -``` - -### Training and Saving the Model -Call the network, optimizer, and loss function, and then customize the `train_process` API of `GradientAccumulation` to train the model. - -```python -if __name__ == "__main__": - parser = argparse.ArgumentParser(description='MindSpore Gard Cumulative Example') - parser.add_argument('--device_target', type=str, default="Ascend", choices=['Ascend', 'GPU'], - help='device where the code will be implemented (default: Ascend)') - parser.add_argument('--data_path', type=str, default="./Data", - help='path where the dataset is saved') - args = parser.parse_args() - - context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target) - ds_train = create_dataset(os.path.join(args.data_path, "train"), 32) - - network = LeNet5(10) - net_loss = nn.SoftmaxCrossEntropyWithLogits(is_grad=False, sparse=True, reduction="mean") - net_opt = nn.Momentum(network.trainable_params(), 0.01, 0.9) - model = GradientAccumulation(network, net_loss, net_opt) - - print("============== Starting Training ==============") - model.train_process(10, ds_train, mini_steps=4) -``` - -## Experiment Result -After 10 epochs, the accuracy on the test set is about 96.31%. - -**Training Execution** -1. Run the training code and view the running result. - ```shell - $ python train.py --data_path=./MNIST_Data - ``` - The output is as follows. The loss value decreases during training. - - ```shell - epoch: 1 step: 27 loss is 0.3660637 - epoch: 1 step: 28 loss is 0.25238192 - ... - epoch: 3 step: 2 loss is 0.12296932 - epoch: 3 step: 3 loss is 0.15799297 - ... - epoch: 10 step: 448 loss is 0.06443884 - epoch: 10 step: 449 loss is 0.0067842817 - ``` - -2. Check the saved checkpoint files. - - The model file `gradient_accumulation.ckpt` is saved during training. - -**Model Validation** - -Use the saved checkpoint file to load the validation dataset through [eval.py]() in the lenet directory of model_zoo. - -```shell -$ python eval.py --data_path=./MNIST_Data --ckpt_path=./gradient_accumulation.ckpt -``` - -The output is as follows. The accuracy of the validation dataset is about 96.31%, which is the same as the result when the value of batch_size is 32. - -```shell -============== Starting Testing ============== -============== {'Accuracy': 0.9631730769230769} ============== +# Gradient Accumulation + +`Linux` `Ascend` `GPU` `Model Optimization` `Intermediate` `Expert` + + + +- [Gradient Accumulation](#gradient-accumulation) + - [Overview](#overview) + - [Creating a Gradient Accumulation Model](#creating-a-gradient-accumulation-model) + - [Importing Library Files](#importing-library-files) + - [Loading the Dataset](#loading-the-dataset) + - [Defining the Network](#defining-the-network) + - [Defining the Training Model](#defining-the-training-model) + - [Defining the Training Process](#defining-the-training-process) + - [Training and Saving the Model](#training-and-saving-the-model) + - [Experiment Result](#experiment-result) + + + + + +## Overview + +This tutorial describes the gradient accumulation training method to solve the problem that some large-scale networks cannot train large batch_size due to insufficient memory. + +In a traditional training method, after a loss and a gradient are calculated, a parameter is directly updated by using the obtained gradient. + +Different from the traditional training method, the concept of mini-batch is introduced to the gradient accumulation. The loss and gradient are computed for each mini-batch data, but the model parameters are not updated immediately. Instead, the obtained gradients are accumulated first, and then after the number (N) of mini-batches is specified, the accumulated gradient is used to update the network parameters. Before the next training, the accumulated gradients are cleared and re-accumulated. + +The ultimate objective is to achieve the same effect as training with N x mini-batch data. + +> This tutorial is applicable to GPUs and Ascend 910 AI Processors. You can download the main training sample code from . + +## Creating a Gradient Accumulation Model + +The MNIST dataset is used as an example to describe how to customize a simple model to implement gradient accumulation. + +### Importing Library Files +The following are the required public modules and MindSpore modules and library files. + +```python +import argparse +import os +from collections.abc import Iterable + +import mindspore.nn as nn +from mindspore import ParameterTuple +from mindspore import context +from mindspore.nn import Cell +from mindspore.ops import composite as C +from mindspore.ops import functional as F +from mindspore.ops import operations as P +from mindspore.train.dataset_helper import DatasetHelper +from mindspore.train.serialization import save_checkpoint +from model_zoo.official.cv.lenet.src.dataset import create_dataset +from model_zoo.official.cv.lenet.src.lenet import LeNet5 +``` + +### Loading the Dataset + +Use the `MnistDataset` API provided by the dataset of MindSpore to load the MNIST dataset. The code is imported from [dataset.py]() in the lenet directory of model_zoo. + +### Defining the Network + +The following uses the LeNet network as an example. You can also use other networks, such as ResNet-50 and BERT. The code is imported from [lenet.py]() in the lenet directory of model_zoo. + +### Defining the Training Model +The training process is divided into three parts: forward and backward training, parameter update, and accumulated gradient clearance. +- `TrainForwardBackward` calculates the loss and gradient, and uses grad_sum to implement gradient accumulation. +- `TrainOptim` updates parameters. +- `TrainClear` clears the gradient accumulation variable grad_sum. + +```python +_sum_op = C.MultitypeFuncGraph("grad_sum_op") +_clear_op = C.MultitypeFuncGraph("clear_op") + + +@_sum_op.register("Tensor", "Tensor") +def _cumulative_gard(grad_sum, grad): + """Apply gard sum to cumulative gradient.""" + add = P.AssignAdd() + return add(grad_sum, grad) + + +@_clear_op.register("Tensor", "Tensor") +def _clear_grad_sum(grad_sum, zero): + """Apply zero to clear grad_sum.""" + success = True + success = F.depend(success, F.assign(grad_sum, zero)) + return success + + +class TrainForwardBackward(Cell): + def __init__(self, network, optimizer, grad_sum, sens=1.0): + super(TrainForwardBackward, self).__init__(auto_prefix=False) + self.network = network + self.network.set_grad() + self.network.add_flags(defer_inline=True) + self.weights = ParameterTuple(network.trainable_params()) + self.optimizer = optimizer + self.grad_sum = grad_sum + self.grad = C.GradOperation(get_by_list=True, sens_param=True) + self.sens = sens + self.hyper_map = C.HyperMap() + + def construct(self, *inputs): + weights = self.weights + loss = self.network(*inputs) + sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens) + grads = self.grad(self.network, weights)(*inputs, sens) + return F.depend(loss, self.hyper_map(F.partial(_sum_op), self.grad_sum, grads)) + + +class TrainOptim(Cell): + def __init__(self, optimizer, grad_sum): + super(TrainOptim, self).__init__(auto_prefix=False) + self.optimizer = optimizer + self.grad_sum = grad_sum + + def construct(self): + return self.optimizer(self.grad_sum) + + +class TrainClear(Cell): + def __init__(self, grad_sum, zeros): + super(TrainClear, self).__init__(auto_prefix=False) + self.grad_sum = grad_sum + self.zeros = zeros + self.hyper_map = C.HyperMap() + + def construct(self): + seccess = self.hyper_map(F.partial(_clear_op), self.grad_sum, self.zeros) + return seccess +``` + +### Defining the Training Process +Each mini-batch calculates the loss and gradient through forward and backward training, and uses mini_steps to control the accumulated times before each parameter update. After the number of accumulation times is reached, the parameter is updated and the accumulated gradient variable is cleared. + + +```python +class GradientAccumulation: + def __init__(self, network, loss_fn, optimizer): + self._network = network + self._loss_fn = loss_fn + self._optimizer = optimizer + + params = self._optimizer.parameters + self._grad_sum = params.clone(prefix="grad_sum", init='zeros') + self._zeros = params.clone(prefix="zeros", init='zeros') + self._train_forward_backward = self._build_train_forward_backward_network() + self._train_optim = self._build_train_optim() + self._train_clear = self._build_train_clear() + + @staticmethod + def _transform_callbacks(callbacks): + """Transform callback to a list.""" + if callbacks is None: + return [] + + if isinstance(callbacks, Iterable): + return list(callbacks) + + return [callbacks] + + def _build_train_forward_backward_network(self): + """Build forward and backward network""" + network = self._network + network = nn.WithLossCell(network, self._loss_fn) + loss_scale = 1.0 + network = TrainForwardBackward(network, self._optimizer, self._grad_sum, loss_scale).set_train() + return network + + def _build_train_optim(self): + """Build optimizer network""" + network = TrainOptim(self._optimizer, self._grad_sum).set_train() + return network + + def _build_train_clear(self): + """Build clear network""" + network = TrainClear(self._grad_sum, self._zeros).set_train() + return network + + def train_process(self, epoch, train_dataset, mini_steps=None): + """ + Training process. The data would be passed to network directly. + """ + dataset_helper = DatasetHelper(train_dataset, dataset_sink_mode=False, epoch_num=epoch) + + for i in range(epoch): + step = 0 + for k, next_element in enumerate(dataset_helper): + loss = self._train_forward_backward(*next_element) + if (k + 1) % mini_steps == 0: + step += 1 + print("epoch:", i + 1, "step:", step, "loss is ", loss) + self._train_optim() + self._train_clear() + + train_dataset.reset() + + save_checkpoint(self._train_forward_backward, "gradient_accumulation.ckpt", ) +``` + +### Training and Saving the Model +Call the network, optimizer, and loss function, and then customize the `train_process` API of `GradientAccumulation` to train the model. + +```python +if __name__ == "__main__": + parser = argparse.ArgumentParser(description='MindSpore Gard Cumulative Example') + parser.add_argument('--device_target', type=str, default="Ascend", choices=['Ascend', 'GPU'], + help='device where the code will be implemented (default: Ascend)') + parser.add_argument('--data_path', type=str, default="./Data", + help='path where the dataset is saved') + args = parser.parse_args() + + context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target) + ds_train = create_dataset(os.path.join(args.data_path, "train"), 32) + + network = LeNet5(10) + net_loss = nn.SoftmaxCrossEntropyWithLogits(is_grad=False, sparse=True, reduction="mean") + net_opt = nn.Momentum(network.trainable_params(), 0.01, 0.9) + model = GradientAccumulation(network, net_loss, net_opt) + + print("============== Starting Training ==============") + model.train_process(10, ds_train, mini_steps=4) +``` + +## Experiment Result +After 10 epochs, the accuracy on the test set is about 96.31%. + +**Training Execution** +1. Run the training code and view the running result. + ```shell + $ python train.py --data_path=./MNIST_Data + ``` + The output is as follows. The loss value decreases during training. + + ```shell + epoch: 1 step: 27 loss is 0.3660637 + epoch: 1 step: 28 loss is 0.25238192 + ... + epoch: 3 step: 2 loss is 0.12296932 + epoch: 3 step: 3 loss is 0.15799297 + ... + epoch: 10 step: 448 loss is 0.06443884 + epoch: 10 step: 449 loss is 0.0067842817 + ``` + +2. Check the saved checkpoint files. + + The model file `gradient_accumulation.ckpt` is saved during training. + +**Model Validation** + +Use the saved checkpoint file to load the validation dataset through [eval.py]() in the lenet directory of model_zoo. + +```shell +$ python eval.py --data_path=./MNIST_Data --ckpt_path=./gradient_accumulation.ckpt +``` + +The output is as follows. The accuracy of the validation dataset is about 96.31%, which is the same as the result when the value of batch_size is 32. + +```shell +============== Starting Testing ============== +============== {'Accuracy': 0.9631730769230769} ============== ``` \ No newline at end of file diff --git a/tutorials/source_en/advanced_use/graph_kernel_fusion.md b/tutorials/training/source_en/advanced_use/graph_kernel_fusion.md similarity index 100% rename from tutorials/source_en/advanced_use/graph_kernel_fusion.md rename to tutorials/training/source_en/advanced_use/graph_kernel_fusion.md diff --git a/tutorials/source_en/advanced_use/host_device_training.md b/tutorials/training/source_en/advanced_use/host_device_training.md similarity index 100% rename from tutorials/source_en/advanced_use/host_device_training.md rename to tutorials/training/source_en/advanced_use/host_device_training.md diff --git a/tutorials/source_en/advanced_use/hub_tutorial.md b/tutorials/training/source_en/advanced_use/hub_tutorial.md similarity index 97% rename from tutorials/source_en/advanced_use/hub_tutorial.md rename to tutorials/training/source_en/advanced_use/hub_tutorial.md index 13e98abd3fa8aa362ba1b7613cd5bb613a82a43a..a47f655f0d60662d171f44e0500b638babd6180a 100644 --- a/tutorials/source_en/advanced_use/hub_tutorial.md +++ b/tutorials/training/source_en/advanced_use/hub_tutorial.md @@ -1,181 +1,181 @@ -## Submitting, Loading and Fine-tuning Models using MindSpore Hub - -`Ascend` `GPU` `MindSpore Hub` `Model Submission` `Model Loading` `Model Fine-tuning` `Beginner` `Intermediate` `Expert` - - - -- [Submitting, Loading and Fine-tuning Models using MindSpore Hub](#submitting-loading-and-fine-tuning-models-using-mindspore-hub) - - [Overview](#overview) - - [How to submit models](#how-to-submit-models) - - [Steps](#steps) - - [How to load models](#how-to-load-models) - - [Model Fine-tuning](#model-fine-tuning) - - - - - -### Overview - -For algorithm developers who are interested in publishing models into MindSpore Hub, this tutorial introduces the specific steps to submit models using GoogleNet as an example. It also describes how to load/fine-tune MindSpore Hub models for application developers who aim to do inference/transfer learning on new dataset. In summary, this tutorial helps the algorithm developers submit models efficiently and enables the application developers to perform inference or fine-tuning using MindSpore Hub APIs quickly. - -### How to submit models - -We accept publishing models to MindSpore Hub via PR in `hub` repo. Here we use GoogleNet as an example to list the steps of model submission to MindSpore Hub. - -#### Steps - -1. Host your pre-trained model in a storage location where we are able to access. - -2. Add a model generation python file called `mindspore_hub_conf.py` in your own repo using this [template](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/googlenet/mindspore_hub_conf.py). - -3. Create a `{model_name}_{model_version}_{dataset}.md` file in `hub/mshub_res/assets` using this [template](https://gitee.com/mindspore/hub/blob/master/mshub_res/assets/mindspore/gpu/0.6/alexnet_v1_cifar10.md). For each pre-trained model, please run the following command to obtain a hash value required at `asset-sha256` of this `.md` file: - - ```python - cd ../tools - python get_sha256.py ../googlenet.ckpt - ``` - -4. Check the format of the markdown file locally using `hub/mshub_res/tools/md_validator.py` by running the following command: - - ```python - python md_validator.py ../assets/mindspore/ascend/0.7/googlenet_v1_cifar10.md - ``` - -5. Create a PR in `mindspore/hub` repo. - -Once your PR is merged into master branch here, your model will show up in [MindSpore Hub Website](https://hub.mindspore.com/mindspore) within 24 hours. For more information, please refer to the [README](https://gitee.com/mindspore/hub/blob/master/mshub_res/README.md). - -### How to load models - -`mindspore_hub.load` API is used to load the pre-trained model in a single line of code. The main process of model loading is as follows: - -- Search the model of interest on [MindSpore Hub Website](https://hub.mindspore.com/mindspore). - - For example, if you aim to perform image classification on CIFAR-10 dataset using GoogleNet, please search on [MindSpore Hub Website](https://hub.mindspore.com/mindspore) with the keyword `GoogleNet`. Then all related models will be returned. Once you enter into the related model page, you can get the website `url`. - -- Complete the task of loading model using `url` , as shown in the example below: - -```python -import mindspore_hub as mshub -import mindspore -from mindspore import context, Tensor, nn -from mindspore.train.model import Model -from mindspore.common import dtype as mstype -from mindspore.dataset.transforms import py_transforms -from PIL import Image -import cv2 - -context.set_context(mode=context.GRAPH_MODE, - device_target="Ascend", - device_id=0) - -model = "mindspore/ascend/0.7/googlenet_v1_cifar10" - -image = Image.open('cifar10/a.jpg') -transforms = py_transforms.ComposeOp([py_transforms.ToTensor()]) - -# Initialize the number of classes based on the pre-trained model. -network = mshub.load(model, num_classes=10) -network.set_train(False) -out = network(transforms(image)) -``` - -### Model Fine-tuning - -When loading a model with `mindspore_hub.load` API, we can add an extra argument to load the feature extraction part of the model only. So we can easily add new layers to perform transfer learning. *This feature can be found in the related model page when an extra argument (e.g., include_top) has been integrated into the model construction by the algorithm engineer.* - -We use Googlenet as example to illustrate how to load a model trained on ImageNet dataset and then perform transfer learning (re-training) on specific sub-task dataset. The main steps are listed below: - -1. Search the model of interest on [MindSpore Hub Website](https://hub.mindspore.com/mindspore) and get the related `url`. - -2. Load the model from MindSpore Hub using the `url`. *Note that the parameter `include_top` is provided by the model developer*. - - ```python - import mindspore - from mindspore import nn - from mindspore import context - import mindspore_hub as mshub - - context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", - save_graphs=False) - - network = mshub.load('mindspore/ascend/0.7/googlenet_v1_cifar10', include_top=False) - network.set_train(False) - ``` - -3. Add a new classification layer into current model architecture. - - ```python - # Check MindSpore Hub website to conclude that the last output shape is 1024. - last_channel = 1024 - - # The number of classes in target task is 26. - num_classes = 26 - classification_layer = nn.Dense(last_channel, num_classes) - classification_layer.set_train(True) - - train_network = nn.SequentialCell([network, classification_layer]) - ``` - -4. Define `loss` and `optimizer` for training. - - ```python - from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits - - # Wrap the backbone network with loss. - loss_fn = SoftmaxCrossEntropyWithLogits() - loss_net = nn.WithLossCell(train_network, loss_fn) - - # Create an optimizer. - optim = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), Tensor(lr), config.momentum, config.weight_decay) - - train_net = nn.TrainOneStepCell(loss_net, optim) - ``` - -5. Create dataset and start fine-tuning. - - ```python - from src.dataset import create_dataset - from mindspore.train.serialization import _exec_save_checkpoint - - dataset = create_dataset("/ssd/data/garbage/train", do_train=True, batch_size=32) - - epoch_size = 15 - for epoch in range(epoch_size): - for i, items in enumerate(dataset): - data, label = items - data = mindspore.Tensor(data) - label = mindspore.Tensor(label) - - loss = train_net(data, label) - print(f"epoch: {epoch}, loss: {loss}") - # Save the ckpt file for each epoch. - ckpt_path = f"./ckpt/garbage_finetune_epoch{epoch}.ckpt" - _exec_save_checkpoint(train_network, ckpt_path) - ``` - -6. Eval on test set. - - ```python - from mindspore.train.serialization import load_checkpoint, load_param_into_net - - network = mshub.load('mindspore/ascend/0.7/googlenet_v1_cifar10', include_top=False) - train_network = nn.SequentialCell([network, nn.Dense(last_channel, num_classes)]) - - # Load a pre-trained ckpt file. - ckpt_path = "./ckpt/garbage_finetune_epoch15.ckpt" - trained_ckpt = load_checkpoint(ckpt_path) - load_param_into_net(train_network, trained_ckpt) - - # Define loss and create model. - loss = SoftmaxCrossEntropyWithLogits() - model = Model(network, loss_fn=loss, metrics={'acc'}) - - eval_dataset = create_dataset("/ssd/data/garbage/train", do_train=False, - batch_size=32) - - res = model.eval(eval_dataset) - print("result:", res, "ckpt=", ckpt_path) - ``` - +## Submitting, Loading and Fine-tuning Models using MindSpore Hub + +`Ascend` `GPU` `MindSpore Hub` `Model Submission` `Model Loading` `Model Fine-tuning` `Beginner` `Intermediate` `Expert` + + + +- [Submitting, Loading and Fine-tuning Models using MindSpore Hub](#submitting-loading-and-fine-tuning-models-using-mindspore-hub) + - [Overview](#overview) + - [How to submit models](#how-to-submit-models) + - [Steps](#steps) + - [How to load models](#how-to-load-models) + - [Model Fine-tuning](#model-fine-tuning) + + + + + +### Overview + +For algorithm developers who are interested in publishing models into MindSpore Hub, this tutorial introduces the specific steps to submit models using GoogleNet as an example. It also describes how to load/fine-tune MindSpore Hub models for application developers who aim to do inference/transfer learning on new dataset. In summary, this tutorial helps the algorithm developers submit models efficiently and enables the application developers to perform inference or fine-tuning using MindSpore Hub APIs quickly. + +### How to submit models + +We accept publishing models to MindSpore Hub via PR in `hub` repo. Here we use GoogleNet as an example to list the steps of model submission to MindSpore Hub. + +#### Steps + +1. Host your pre-trained model in a storage location where we are able to access. + +2. Add a model generation python file called `mindspore_hub_conf.py` in your own repo using this [template](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/googlenet/mindspore_hub_conf.py). + +3. Create a `{model_name}_{model_version}_{dataset}.md` file in `hub/mshub_res/assets` using this [template](https://gitee.com/mindspore/hub/blob/master/mshub_res/assets/mindspore/gpu/0.6/alexnet_v1_cifar10.md). For each pre-trained model, please run the following command to obtain a hash value required at `asset-sha256` of this `.md` file: + + ```python + cd ../tools + python get_sha256.py ../googlenet.ckpt + ``` + +4. Check the format of the markdown file locally using `hub/mshub_res/tools/md_validator.py` by running the following command: + + ```python + python md_validator.py ../assets/mindspore/ascend/0.7/googlenet_v1_cifar10.md + ``` + +5. Create a PR in `mindspore/hub` repo. + +Once your PR is merged into master branch here, your model will show up in [MindSpore Hub Website](https://hub.mindspore.com/mindspore) within 24 hours. For more information, please refer to the [README](https://gitee.com/mindspore/hub/blob/master/mshub_res/README.md). + +### How to load models + +`mindspore_hub.load` API is used to load the pre-trained model in a single line of code. The main process of model loading is as follows: + +- Search the model of interest on [MindSpore Hub Website](https://hub.mindspore.com/mindspore). + + For example, if you aim to perform image classification on CIFAR-10 dataset using GoogleNet, please search on [MindSpore Hub Website](https://hub.mindspore.com/mindspore) with the keyword `GoogleNet`. Then all related models will be returned. Once you enter into the related model page, you can get the website `url`. + +- Complete the task of loading model using `url` , as shown in the example below: + +```python +import mindspore_hub as mshub +import mindspore +from mindspore import context, Tensor, nn +from mindspore.train.model import Model +from mindspore.common import dtype as mstype +from mindspore.dataset.transforms import py_transforms +from PIL import Image +import cv2 + +context.set_context(mode=context.GRAPH_MODE, + device_target="Ascend", + device_id=0) + +model = "mindspore/ascend/0.7/googlenet_v1_cifar10" + +image = Image.open('cifar10/a.jpg') +transforms = py_transforms.ComposeOp([py_transforms.ToTensor()]) + +# Initialize the number of classes based on the pre-trained model. +network = mshub.load(model, num_classes=10) +network.set_train(False) +out = network(transforms(image)) +``` + +### Model Fine-tuning + +When loading a model with `mindspore_hub.load` API, we can add an extra argument to load the feature extraction part of the model only. So we can easily add new layers to perform transfer learning. *This feature can be found in the related model page when an extra argument (e.g., include_top) has been integrated into the model construction by the algorithm engineer.* + +We use Googlenet as example to illustrate how to load a model trained on ImageNet dataset and then perform transfer learning (re-training) on specific sub-task dataset. The main steps are listed below: + +1. Search the model of interest on [MindSpore Hub Website](https://hub.mindspore.com/mindspore) and get the related `url`. + +2. Load the model from MindSpore Hub using the `url`. *Note that the parameter `include_top` is provided by the model developer*. + + ```python + import mindspore + from mindspore import nn + from mindspore import context + import mindspore_hub as mshub + + context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", + save_graphs=False) + + network = mshub.load('mindspore/ascend/0.7/googlenet_v1_cifar10', include_top=False) + network.set_train(False) + ``` + +3. Add a new classification layer into current model architecture. + + ```python + # Check MindSpore Hub website to conclude that the last output shape is 1024. + last_channel = 1024 + + # The number of classes in target task is 26. + num_classes = 26 + classification_layer = nn.Dense(last_channel, num_classes) + classification_layer.set_train(True) + + train_network = nn.SequentialCell([network, classification_layer]) + ``` + +4. Define `loss` and `optimizer` for training. + + ```python + from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits + + # Wrap the backbone network with loss. + loss_fn = SoftmaxCrossEntropyWithLogits() + loss_net = nn.WithLossCell(train_network, loss_fn) + + # Create an optimizer. + optim = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), Tensor(lr), config.momentum, config.weight_decay) + + train_net = nn.TrainOneStepCell(loss_net, optim) + ``` + +5. Create dataset and start fine-tuning. + + ```python + from src.dataset import create_dataset + from mindspore.train.serialization import _exec_save_checkpoint + + dataset = create_dataset("/ssd/data/garbage/train", do_train=True, batch_size=32) + + epoch_size = 15 + for epoch in range(epoch_size): + for i, items in enumerate(dataset): + data, label = items + data = mindspore.Tensor(data) + label = mindspore.Tensor(label) + + loss = train_net(data, label) + print(f"epoch: {epoch}, loss: {loss}") + # Save the ckpt file for each epoch. + ckpt_path = f"./ckpt/garbage_finetune_epoch{epoch}.ckpt" + _exec_save_checkpoint(train_network, ckpt_path) + ``` + +6. Eval on test set. + + ```python + from mindspore.train.serialization import load_checkpoint, load_param_into_net + + network = mshub.load('mindspore/ascend/0.7/googlenet_v1_cifar10', include_top=False) + train_network = nn.SequentialCell([network, nn.Dense(last_channel, num_classes)]) + + # Load a pre-trained ckpt file. + ckpt_path = "./ckpt/garbage_finetune_epoch15.ckpt" + trained_ckpt = load_checkpoint(ckpt_path) + load_param_into_net(train_network, trained_ckpt) + + # Define loss and create model. + loss = SoftmaxCrossEntropyWithLogits() + model = Model(network, loss_fn=loss, metrics={'acc'}) + + eval_dataset = create_dataset("/ssd/data/garbage/train", do_train=False, + batch_size=32) + + res = model.eval(eval_dataset) + print("result:", res, "ckpt=", ckpt_path) + ``` + diff --git a/tutorials/source_en/advanced_use/images/adv_attack_result.png b/tutorials/training/source_en/advanced_use/images/adv_attack_result.png similarity index 100% rename from tutorials/source_en/advanced_use/images/adv_attack_result.png rename to tutorials/training/source_en/advanced_use/images/adv_attack_result.png diff --git a/tutorials/source_en/advanced_use/images/checkpoint_integration_process.jpg b/tutorials/training/source_en/advanced_use/images/checkpoint_integration_process.jpg similarity index 100% rename from tutorials/source_en/advanced_use/images/checkpoint_integration_process.jpg rename to tutorials/training/source_en/advanced_use/images/checkpoint_integration_process.jpg diff --git a/tutorials/source_en/advanced_use/images/cifar10.jpg b/tutorials/training/source_en/advanced_use/images/cifar10.jpg similarity index 100% rename from tutorials/source_en/advanced_use/images/cifar10.jpg rename to tutorials/training/source_en/advanced_use/images/cifar10.jpg diff --git a/tutorials/source_en/advanced_use/images/data_chart.png b/tutorials/training/source_en/advanced_use/images/data_chart.png similarity index 100% rename from tutorials/source_en/advanced_use/images/data_chart.png rename to tutorials/training/source_en/advanced_use/images/data_chart.png diff --git a/tutorials/source_en/advanced_use/images/data_function.png b/tutorials/training/source_en/advanced_use/images/data_function.png similarity index 100% rename from tutorials/source_en/advanced_use/images/data_function.png rename to tutorials/training/source_en/advanced_use/images/data_function.png diff --git a/tutorials/source_en/advanced_use/images/data_label.png b/tutorials/training/source_en/advanced_use/images/data_label.png similarity index 100% rename from tutorials/source_en/advanced_use/images/data_label.png rename to tutorials/training/source_en/advanced_use/images/data_label.png diff --git a/tutorials/source_en/advanced_use/images/data_op_profile.png b/tutorials/training/source_en/advanced_use/images/data_op_profile.png similarity index 100% rename from tutorials/source_en/advanced_use/images/data_op_profile.png rename to tutorials/training/source_en/advanced_use/images/data_op_profile.png diff --git a/tutorials/source_en/advanced_use/images/data_table.png b/tutorials/training/source_en/advanced_use/images/data_table.png similarity index 100% rename from tutorials/source_en/advanced_use/images/data_table.png rename to tutorials/training/source_en/advanced_use/images/data_table.png diff --git a/tutorials/source_en/advanced_use/images/gpu_activity_profiler.png b/tutorials/training/source_en/advanced_use/images/gpu_activity_profiler.png similarity index 100% rename from tutorials/source_en/advanced_use/images/gpu_activity_profiler.png rename to tutorials/training/source_en/advanced_use/images/gpu_activity_profiler.png diff --git a/tutorials/source_en/advanced_use/images/gpu_op_ui_profiler.png b/tutorials/training/source_en/advanced_use/images/gpu_op_ui_profiler.png similarity index 100% rename from tutorials/source_en/advanced_use/images/gpu_op_ui_profiler.png rename to tutorials/training/source_en/advanced_use/images/gpu_op_ui_profiler.png diff --git a/tutorials/source_en/advanced_use/images/graph.png b/tutorials/training/source_en/advanced_use/images/graph.png similarity index 100% rename from tutorials/source_en/advanced_use/images/graph.png rename to tutorials/training/source_en/advanced_use/images/graph.png diff --git a/tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png b/tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png similarity index 100% rename from tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png rename to tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png diff --git a/tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png b/tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png similarity index 100% rename from tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png rename to tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png diff --git a/tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png b/tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png similarity index 100% rename from tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png rename to tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png diff --git a/tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png b/tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png similarity index 100% rename from tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png rename to tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png diff --git a/tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png b/tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png similarity index 100% rename from tutorials/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png rename to tutorials/training/source_en/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png diff --git a/tutorials/source_en/advanced_use/images/graph_sidebar.png b/tutorials/training/source_en/advanced_use/images/graph_sidebar.png similarity index 100% rename from tutorials/source_en/advanced_use/images/graph_sidebar.png rename to tutorials/training/source_en/advanced_use/images/graph_sidebar.png diff --git a/tutorials/source_en/advanced_use/images/histogram.png b/tutorials/training/source_en/advanced_use/images/histogram.png similarity index 100% rename from tutorials/source_en/advanced_use/images/histogram.png rename to tutorials/training/source_en/advanced_use/images/histogram.png diff --git a/tutorials/source_en/advanced_use/images/histogram_func.png b/tutorials/training/source_en/advanced_use/images/histogram_func.png similarity index 100% rename from tutorials/source_en/advanced_use/images/histogram_func.png rename to tutorials/training/source_en/advanced_use/images/histogram_func.png diff --git a/tutorials/source_en/advanced_use/images/image_function.png b/tutorials/training/source_en/advanced_use/images/image_function.png similarity index 100% rename from tutorials/source_en/advanced_use/images/image_function.png rename to tutorials/training/source_en/advanced_use/images/image_function.png diff --git a/tutorials/source_en/advanced_use/images/image_vi.png b/tutorials/training/source_en/advanced_use/images/image_vi.png similarity index 100% rename from tutorials/source_en/advanced_use/images/image_vi.png rename to tutorials/training/source_en/advanced_use/images/image_vi.png diff --git a/tutorials/source_en/advanced_use/images/lineage_label.png b/tutorials/training/source_en/advanced_use/images/lineage_label.png similarity index 100% rename from tutorials/source_en/advanced_use/images/lineage_label.png rename to tutorials/training/source_en/advanced_use/images/lineage_label.png diff --git a/tutorials/source_en/advanced_use/images/lineage_model_chart.png b/tutorials/training/source_en/advanced_use/images/lineage_model_chart.png similarity index 100% rename from tutorials/source_en/advanced_use/images/lineage_model_chart.png rename to tutorials/training/source_en/advanced_use/images/lineage_model_chart.png diff --git a/tutorials/source_en/advanced_use/images/lineage_model_table.png b/tutorials/training/source_en/advanced_use/images/lineage_model_table.png similarity index 100% rename from tutorials/source_en/advanced_use/images/lineage_model_table.png rename to tutorials/training/source_en/advanced_use/images/lineage_model_table.png diff --git a/tutorials/source_en/advanced_use/images/minddata_profile.png b/tutorials/training/source_en/advanced_use/images/minddata_profile.png similarity index 100% rename from tutorials/source_en/advanced_use/images/minddata_profile.png rename to tutorials/training/source_en/advanced_use/images/minddata_profile.png diff --git a/tutorials/source_en/advanced_use/images/mix_precision.eddx b/tutorials/training/source_en/advanced_use/images/mix_precision.eddx similarity index 100% rename from tutorials/source_en/advanced_use/images/mix_precision.eddx rename to tutorials/training/source_en/advanced_use/images/mix_precision.eddx diff --git a/tutorials/source_en/advanced_use/images/mix_precision.jpg b/tutorials/training/source_en/advanced_use/images/mix_precision.jpg similarity index 100% rename from tutorials/source_en/advanced_use/images/mix_precision.jpg rename to tutorials/training/source_en/advanced_use/images/mix_precision.jpg diff --git a/tutorials/source_en/advanced_use/images/multi_scalars.png b/tutorials/training/source_en/advanced_use/images/multi_scalars.png similarity index 100% rename from tutorials/source_en/advanced_use/images/multi_scalars.png rename to tutorials/training/source_en/advanced_use/images/multi_scalars.png diff --git a/tutorials/source_en/advanced_use/images/multi_scalars_select.png b/tutorials/training/source_en/advanced_use/images/multi_scalars_select.png similarity index 100% rename from tutorials/source_en/advanced_use/images/multi_scalars_select.png rename to tutorials/training/source_en/advanced_use/images/multi_scalars_select.png diff --git a/tutorials/source_en/advanced_use/images/op_statistics.PNG b/tutorials/training/source_en/advanced_use/images/op_statistics.PNG similarity index 100% rename from tutorials/source_en/advanced_use/images/op_statistics.PNG rename to tutorials/training/source_en/advanced_use/images/op_statistics.PNG diff --git a/tutorials/source_en/advanced_use/images/op_type_statistics.PNG b/tutorials/training/source_en/advanced_use/images/op_type_statistics.PNG similarity index 100% rename from tutorials/source_en/advanced_use/images/op_type_statistics.PNG rename to tutorials/training/source_en/advanced_use/images/op_type_statistics.PNG diff --git a/tutorials/source_en/advanced_use/images/performance_overall.png b/tutorials/training/source_en/advanced_use/images/performance_overall.png similarity index 100% rename from tutorials/source_en/advanced_use/images/performance_overall.png rename to tutorials/training/source_en/advanced_use/images/performance_overall.png diff --git a/tutorials/source_en/advanced_use/images/scalar.png b/tutorials/training/source_en/advanced_use/images/scalar.png similarity index 100% rename from tutorials/source_en/advanced_use/images/scalar.png rename to tutorials/training/source_en/advanced_use/images/scalar.png diff --git a/tutorials/source_en/advanced_use/images/scalar_compound.png b/tutorials/training/source_en/advanced_use/images/scalar_compound.png similarity index 100% rename from tutorials/source_en/advanced_use/images/scalar_compound.png rename to tutorials/training/source_en/advanced_use/images/scalar_compound.png diff --git a/tutorials/source_en/advanced_use/images/scalar_select.png b/tutorials/training/source_en/advanced_use/images/scalar_select.png similarity index 100% rename from tutorials/source_en/advanced_use/images/scalar_select.png rename to tutorials/training/source_en/advanced_use/images/scalar_select.png diff --git a/tutorials/source_en/advanced_use/images/step_trace.png b/tutorials/training/source_en/advanced_use/images/step_trace.png similarity index 100% rename from tutorials/source_en/advanced_use/images/step_trace.png rename to tutorials/training/source_en/advanced_use/images/step_trace.png diff --git a/tutorials/source_en/advanced_use/images/synchronization_training_and_evaluation.png b/tutorials/training/source_en/advanced_use/images/synchronization_training_and_evaluation.png similarity index 100% rename from tutorials/source_en/advanced_use/images/synchronization_training_and_evaluation.png rename to tutorials/training/source_en/advanced_use/images/synchronization_training_and_evaluation.png diff --git a/tutorials/source_en/advanced_use/images/targets.png b/tutorials/training/source_en/advanced_use/images/targets.png similarity index 100% rename from tutorials/source_en/advanced_use/images/targets.png rename to tutorials/training/source_en/advanced_use/images/targets.png diff --git a/tutorials/source_en/advanced_use/images/tensor_function.png b/tutorials/training/source_en/advanced_use/images/tensor_function.png similarity index 100% rename from tutorials/source_en/advanced_use/images/tensor_function.png rename to tutorials/training/source_en/advanced_use/images/tensor_function.png diff --git a/tutorials/source_en/advanced_use/images/tensor_histogram.png b/tutorials/training/source_en/advanced_use/images/tensor_histogram.png similarity index 100% rename from tutorials/source_en/advanced_use/images/tensor_histogram.png rename to tutorials/training/source_en/advanced_use/images/tensor_histogram.png diff --git a/tutorials/source_en/advanced_use/images/tensor_table.png b/tutorials/training/source_en/advanced_use/images/tensor_table.png similarity index 100% rename from tutorials/source_en/advanced_use/images/tensor_table.png rename to tutorials/training/source_en/advanced_use/images/tensor_table.png diff --git a/tutorials/source_en/advanced_use/images/timeline.png b/tutorials/training/source_en/advanced_use/images/timeline.png similarity index 100% rename from tutorials/source_en/advanced_use/images/timeline.png rename to tutorials/training/source_en/advanced_use/images/timeline.png diff --git a/tutorials/source_en/advanced_use/lineage_and_scalars_comparision.md b/tutorials/training/source_en/advanced_use/lineage_and_scalars_comparision.md similarity index 98% rename from tutorials/source_en/advanced_use/lineage_and_scalars_comparision.md rename to tutorials/training/source_en/advanced_use/lineage_and_scalars_comparision.md index b120a5f0fffc6e70905efce9869d2922333af9ec..6823b61801c7e973a819ae6aa1d511b3c84670bc 100644 --- a/tutorials/source_en/advanced_use/lineage_and_scalars_comparision.md +++ b/tutorials/training/source_en/advanced_use/lineage_and_scalars_comparision.md @@ -1,110 +1,110 @@ -# Lineage and Scalars Comparision - -`Linux` `Ascend` `GPU` `CPU` `Model Optimization` `Intermediate` `Expert` - - - -- [Lineage and Scalars Comparision](#lineage-and-scalars-comparision) - - [Overview](#overview) - - [Model Lineage](#model-lineage) - - [Dataset Lineage](#dataset-lineage) - - [Scalars Comparision](#scalars-comparision) - - [Notices](#notices) - - - - - -## Overview - -Model lineage, data lineage and comparison Kanban in mindinsight are the same as training dashboard. In the visualization of training data, different scalar trend charts are observed by comparison dashboard to find problems, and then the lineage function is used to locate the problem causes, so as to give users the ability of efficient tuning in data enhancement and deep neural network. - -## Model Lineage - -Model lineage visualization is used to display the parameter information of all training models. - -![image.png](./images/lineage_label.png) - -Figure 1: Model parameter selection area - -Figure 1 shows the model parameter selection area, which lists the model parameter tags that can be viewed. You can select required tags to view the corresponding model parameters. - -![image.png](./images/lineage_model_chart.png) - -Figure 2: Model lineage function area - -Figure 2 shows the model lineage function area, which visualizes the model parameter information. You can select a specific area in the column to display the model information within the area. - -![image.png](./images/lineage_model_table.png) - -Figure 3: Model list - -Figure 3 shows all model information in groups. You can sort the model information in ascending or descending order by specified column. - -The overview page on the left shows information about optimization objective and related parameters. - -![targets.png](./images/targets.png) - -Figure 4: Overview page - -Figure 4 shows the optimization objective distribution, parameter importance, and scatter plots. - -## Dataset Lineage - -Dataset lineage visualization is used to display data processing and augmentation information of all model trainings. - -![data_label.png](./images/data_label.png) - -Figure 5: Data processing and augmentation operator selection area - -Figure 5 shows the data processing and augmentation operator selection area, which lists names of data processing and augmentation operators that can be viewed. You can select required tags to view related parameters. - -![data_chart.png](./images/data_chart.png) - -Figure 6: Dataset lineage function area - -Figure 6 shows the dataset lineage function area, which visualizes the parameter information used for data processing and augmentation. You can select a specific area in the column to display the parameter information within the area. - -![data_table.png](./images/data_table.png) - -Figure 7: Dataset lineage list - -Figure 7 shows the data processing and augmentation information of all model trainings. - -> If user filters the model lineage and then switches to the data lineage page, the line chart will show the latest filtered column in model lineage. - -## Scalars Comparision - -Scalars Comparision can be used to compare scalar curves between multiple trainings - -![multi_scalars.png](./images/multi_scalars.png) - -Figure 8: Scalars comparision curve area - -Figure 8 shows the scalar curve comparision between multiple trainings. The horizontal coordinate indicates the training step, and the vertical coordinate indicates the scalar value. - -Buttons from left to right in the upper right corner of the figure are used to display the chart in full screen, switch the Y-axis scale, enable or disable the rectangle selection, roll back the chart step by step, and restore the chart. - -- Full-screen Display: Display the scalar curve in full screen. Click the button again to restore it. -- Switch Y-axis Scale: Perform logarithmic conversion on the Y-axis coordinate. -- Enable/Disable Rectangle Selection: Draw a rectangle to select and zoom in a part of the chart. You can perform rectangle selection again on the zoomed-in chart. -- Step-by-step Rollback: Cancel operations step by step after continuously drawing rectangles to select and zooming in the same area. -- Restore Chart: Restore a chart to the original state. - -![multi_scalars_select.png](./images/multi_scalars_select.png) - -Figure 9: Scalars comparision function area - -Figure 9 shows the scalars comparision function area, which allows you to view scalar information by selecting different trainings or tags, different dimensions of the horizontal axis, and smoothness. - -- Training: Select or filter the required trainings to view the corresponding scalar information. -- Tag: Select the required tags to view the corresponding scalar information. -- Horizontal Axis: Select any of Step, Relative Time, and Absolute Time as the horizontal axis of the scalar curve. -- Smoothness: Adjust the smoothness to smooth the scalar curve. - -## Notices - -To ensure performance, MindInsight implements scalars comparision with the cache mechanism and the following restrictions: -- The scalars comparision supports only for trainings in cache. -- The maximum of 15 latest trainings (sorted by modification time) can be retained in the cache. +# Lineage and Scalars Comparision + +`Linux` `Ascend` `GPU` `CPU` `Model Optimization` `Intermediate` `Expert` + + + +- [Lineage and Scalars Comparision](#lineage-and-scalars-comparision) + - [Overview](#overview) + - [Model Lineage](#model-lineage) + - [Dataset Lineage](#dataset-lineage) + - [Scalars Comparision](#scalars-comparision) + - [Notices](#notices) + + + + + +## Overview + +Model lineage, data lineage and comparison Kanban in mindinsight are the same as training dashboard. In the visualization of training data, different scalar trend charts are observed by comparison dashboard to find problems, and then the lineage function is used to locate the problem causes, so as to give users the ability of efficient tuning in data enhancement and deep neural network. + +## Model Lineage + +Model lineage visualization is used to display the parameter information of all training models. + +![image.png](./images/lineage_label.png) + +Figure 1: Model parameter selection area + +Figure 1 shows the model parameter selection area, which lists the model parameter tags that can be viewed. You can select required tags to view the corresponding model parameters. + +![image.png](./images/lineage_model_chart.png) + +Figure 2: Model lineage function area + +Figure 2 shows the model lineage function area, which visualizes the model parameter information. You can select a specific area in the column to display the model information within the area. + +![image.png](./images/lineage_model_table.png) + +Figure 3: Model list + +Figure 3 shows all model information in groups. You can sort the model information in ascending or descending order by specified column. + +The overview page on the left shows information about optimization objective and related parameters. + +![targets.png](./images/targets.png) + +Figure 4: Overview page + +Figure 4 shows the optimization objective distribution, parameter importance, and scatter plots. + +## Dataset Lineage + +Dataset lineage visualization is used to display data processing and augmentation information of all model trainings. + +![data_label.png](./images/data_label.png) + +Figure 5: Data processing and augmentation operator selection area + +Figure 5 shows the data processing and augmentation operator selection area, which lists names of data processing and augmentation operators that can be viewed. You can select required tags to view related parameters. + +![data_chart.png](./images/data_chart.png) + +Figure 6: Dataset lineage function area + +Figure 6 shows the dataset lineage function area, which visualizes the parameter information used for data processing and augmentation. You can select a specific area in the column to display the parameter information within the area. + +![data_table.png](./images/data_table.png) + +Figure 7: Dataset lineage list + +Figure 7 shows the data processing and augmentation information of all model trainings. + +> If user filters the model lineage and then switches to the data lineage page, the line chart will show the latest filtered column in model lineage. + +## Scalars Comparision + +Scalars Comparision can be used to compare scalar curves between multiple trainings + +![multi_scalars.png](./images/multi_scalars.png) + +Figure 8: Scalars comparision curve area + +Figure 8 shows the scalar curve comparision between multiple trainings. The horizontal coordinate indicates the training step, and the vertical coordinate indicates the scalar value. + +Buttons from left to right in the upper right corner of the figure are used to display the chart in full screen, switch the Y-axis scale, enable or disable the rectangle selection, roll back the chart step by step, and restore the chart. + +- Full-screen Display: Display the scalar curve in full screen. Click the button again to restore it. +- Switch Y-axis Scale: Perform logarithmic conversion on the Y-axis coordinate. +- Enable/Disable Rectangle Selection: Draw a rectangle to select and zoom in a part of the chart. You can perform rectangle selection again on the zoomed-in chart. +- Step-by-step Rollback: Cancel operations step by step after continuously drawing rectangles to select and zooming in the same area. +- Restore Chart: Restore a chart to the original state. + +![multi_scalars_select.png](./images/multi_scalars_select.png) + +Figure 9: Scalars comparision function area + +Figure 9 shows the scalars comparision function area, which allows you to view scalar information by selecting different trainings or tags, different dimensions of the horizontal axis, and smoothness. + +- Training: Select or filter the required trainings to view the corresponding scalar information. +- Tag: Select the required tags to view the corresponding scalar information. +- Horizontal Axis: Select any of Step, Relative Time, and Absolute Time as the horizontal axis of the scalar curve. +- Smoothness: Adjust the smoothness to smooth the scalar curve. + +## Notices + +To ensure performance, MindInsight implements scalars comparision with the cache mechanism and the following restrictions: +- The scalars comparision supports only for trainings in cache. +- The maximum of 15 latest trainings (sorted by modification time) can be retained in the cache. - The maximum of 5 trainings can be selected for scalars comparision at the same time. \ No newline at end of file diff --git a/tutorials/source_en/advanced_use/mindinsight_commands.md b/tutorials/training/source_en/advanced_use/mindinsight_commands.md similarity index 100% rename from tutorials/source_en/advanced_use/mindinsight_commands.md rename to tutorials/training/source_en/advanced_use/mindinsight_commands.md diff --git a/tutorials/source_en/advanced_use/mixed_precision.md b/tutorials/training/source_en/advanced_use/mixed_precision.md similarity index 100% rename from tutorials/source_en/advanced_use/mixed_precision.md rename to tutorials/training/source_en/advanced_use/mixed_precision.md diff --git a/tutorials/source_en/advanced_use/model_security.md b/tutorials/training/source_en/advanced_use/model_security.md similarity index 100% rename from tutorials/source_en/advanced_use/model_security.md rename to tutorials/training/source_en/advanced_use/model_security.md diff --git a/tutorials/source_en/advanced_use/network_migration.md b/tutorials/training/source_en/advanced_use/network_migration.md similarity index 100% rename from tutorials/source_en/advanced_use/network_migration.md rename to tutorials/training/source_en/advanced_use/network_migration.md diff --git a/tutorials/source_en/advanced_use/nlp_application.md b/tutorials/training/source_en/advanced_use/nlp_application.md similarity index 100% rename from tutorials/source_en/advanced_use/nlp_application.md rename to tutorials/training/source_en/advanced_use/nlp_application.md diff --git a/tutorials/source_en/advanced_use/parameter_server_training.md b/tutorials/training/source_en/advanced_use/parameter_server_training.md similarity index 100% rename from tutorials/source_en/advanced_use/parameter_server_training.md rename to tutorials/training/source_en/advanced_use/parameter_server_training.md diff --git a/tutorials/source_en/advanced_use/performance_profiling.md b/tutorials/training/source_en/advanced_use/performance_profiling.md similarity index 100% rename from tutorials/source_en/advanced_use/performance_profiling.md rename to tutorials/training/source_en/advanced_use/performance_profiling.md diff --git a/tutorials/source_en/advanced_use/performance_profiling_gpu.md b/tutorials/training/source_en/advanced_use/performance_profiling_gpu.md similarity index 97% rename from tutorials/source_en/advanced_use/performance_profiling_gpu.md rename to tutorials/training/source_en/advanced_use/performance_profiling_gpu.md index d3327f4f1a557f7da90d3dfaff92893f65fef5d8..7a5745fd62640600a5d8c881b7ede502a253858b 100644 --- a/tutorials/source_en/advanced_use/performance_profiling_gpu.md +++ b/tutorials/training/source_en/advanced_use/performance_profiling_gpu.md @@ -1,120 +1,120 @@ -# Performance Profiler（GPU） - -`Linux` `GPU` `Model Optimization` `Intermediate` `Expert` - - - -- [Performance Profiler（GPU）](#performance-profiler-gpu) - - [Overview](#overview) - - [Operation Process](#operation-process) - - [Preparing the Training Script](#preparing-the-training-script) - - [Launch MindInsight](#launch-mindinsight) - - [Performance Analysis](#performance-analysis) - - [Operator Performance Analysis](#operator-performance-analysis) - - [Timeline Analysis](#timeline-analysis) - - - - - -## Overview -Performance data like operators' execution time is recorded in files and can be viewed on the web page, this can help the user optimize the performance of neural networks. - -## Operation Process - -> The GPU operation process is the same as that in Ascend chip. -> -> - -## Preparing the Training Script - -To enable the performance profiling of neural networks, MindSpore Profiler APIs should be added into the script.Only the output_path in parameters is worked in GPU now. Then, at the end of the training, `Profiler.analyse()` should be called to finish profiling and generate the perforamnce analyse results. - -> The sample code is the same as that in Ascend chip： -> -> - -Users can get profiling data by user-defined callback: - -```python -class StopAtStep(Callback): - def __init__(self, start_step, stop_step): - super(StopAtStep, self).__init__() - self.start_step = start_step - self.stop_step = stop_step - self.already_analysed = False - - def step_begin(self, run_context): - cb_params = run_context.original_args() - step_num = cb_params.cur_step_num - if step_num == self.start_step: - self.profiler = Profiler() - - def step_end(self, run_context): - cb_params = run_context.original_args() - step_num = cb_params.cur_step_num - if step_num == self.stop_step and not self.already_analysed: - self.profiler.analyse() - self.already_analysed = True - - def end(self, run_context): - if not self.already_analysed: - self.profiler.analyse() -``` - -The code above is just a example. Users should implement callback by themselves. - -## Launch MindInsight - -The MindInsight launch command can refer to [MindInsight Commands](https://www.mindspore.cn/tutorial/en/master/advanced_use/mindinsight_commands.html). - - -### Performance Analysis - -Users can access the Performance Profiler by selecting a specific training from the training list, and click the performance profiling link. And the Performance Profiler only support operation analysis and Timeline Analysis now, the others modules will publish soon. - -![performance_overall.png](./images/performance_overall.png) - -Figure 1：Overall Performance - -Figure 1 displays the overall performance of the training, including the overall data of Step Trace, Operator Performance, MindData Performance and Timeline. Operator Performance Analysis is supportted only: -- Operator Performance: It will collect the average execution time of operators and operator types. The overall performance page will show the pie graph for different operator types. - -Users can click the detail link to see the details of each components. - -#### Operator Performance Analysis - -The operator performance analysis component is used to display the execution time of the operators during MindSpore run. - -![gpu_op_ui_profiler.png](./images/gpu_op_ui_profiler.png) - -Figure 2: Statistics for Operator Types - -Figure 2 displays the statistics for the operator types, including: - -- Choose pie or bar graph to show the proportion time occupied by each operator type. The time of one operator type is calculated by accumulating the execution time of operators belong to this type. -- Display top 20 operator types with longest average execution time, show the proportion of total time and average execution time (ms) of each operator type. - -The bottom half of Figure 2 displays the statistics table for the operators' details, including: - -- Choose All: Display statistics for the operators, including operator position information, type, execution time, full scope time etc. The table will be sorted by average execution time by default. -- Choose Type: Display statistics for the operator types, including operator type name, execution time, execution frequency and proportion of total time, average execution time. Users can click on each line, querying for all the operators belong to this type. -- Search: There is a search box on the right, which can support fuzzy search for operators/operator types. - -![gpu_activity_profiler.png](./images/gpu_activity_profiler.png) - -Figure 3: Statistics for Kernel Activities - -Figure 3 displays the statistics for the Kernel, including: - -- Pie graph to show the proportion time occupied by each kernel activity. And the top 15 kernel activities with longest exection time. -- The statistical table's column include activity name, operation name, execution frequency, total time, average time. -- The search box on the right, which can support fuzzy search for activity name/operator full name. - -#### Timeline Analysis - -The usage is almost same as that in Ascend. The difference is GPU Timeline displays the operation information and CUDA activity. - -> The usage is follow as: -> +# Performance Profiler（GPU） + +`Linux` `GPU` `Model Optimization` `Intermediate` `Expert` + + + +- [Performance Profiler（GPU）](#performance-profiler-gpu) + - [Overview](#overview) + - [Operation Process](#operation-process) + - [Preparing the Training Script](#preparing-the-training-script) + - [Launch MindInsight](#launch-mindinsight) + - [Performance Analysis](#performance-analysis) + - [Operator Performance Analysis](#operator-performance-analysis) + - [Timeline Analysis](#timeline-analysis) + + + + + +## Overview +Performance data like operators' execution time is recorded in files and can be viewed on the web page, this can help the user optimize the performance of neural networks. + +## Operation Process + +> The GPU operation process is the same as that in Ascend chip. +> +> + +## Preparing the Training Script + +To enable the performance profiling of neural networks, MindSpore Profiler APIs should be added into the script.Only the output_path in parameters is worked in GPU now. Then, at the end of the training, `Profiler.analyse()` should be called to finish profiling and generate the perforamnce analyse results. + +> The sample code is the same as that in Ascend chip： +> +> + +Users can get profiling data by user-defined callback: + +```python +class StopAtStep(Callback): + def __init__(self, start_step, stop_step): + super(StopAtStep, self).__init__() + self.start_step = start_step + self.stop_step = stop_step + self.already_analysed = False + + def step_begin(self, run_context): + cb_params = run_context.original_args() + step_num = cb_params.cur_step_num + if step_num == self.start_step: + self.profiler = Profiler() + + def step_end(self, run_context): + cb_params = run_context.original_args() + step_num = cb_params.cur_step_num + if step_num == self.stop_step and not self.already_analysed: + self.profiler.analyse() + self.already_analysed = True + + def end(self, run_context): + if not self.already_analysed: + self.profiler.analyse() +``` + +The code above is just a example. Users should implement callback by themselves. + +## Launch MindInsight + +The MindInsight launch command can refer to [MindInsight Commands](https://www.mindspore.cn/tutorial/en/master/advanced_use/mindinsight_commands.html). + + +### Performance Analysis + +Users can access the Performance Profiler by selecting a specific training from the training list, and click the performance profiling link. And the Performance Profiler only support operation analysis and Timeline Analysis now, the others modules will publish soon. + +![performance_overall.png](./images/performance_overall.png) + +Figure 1：Overall Performance + +Figure 1 displays the overall performance of the training, including the overall data of Step Trace, Operator Performance, MindData Performance and Timeline. Operator Performance Analysis is supportted only: +- Operator Performance: It will collect the average execution time of operators and operator types. The overall performance page will show the pie graph for different operator types. + +Users can click the detail link to see the details of each components. + +#### Operator Performance Analysis + +The operator performance analysis component is used to display the execution time of the operators during MindSpore run. + +![gpu_op_ui_profiler.png](./images/gpu_op_ui_profiler.png) + +Figure 2: Statistics for Operator Types + +Figure 2 displays the statistics for the operator types, including: + +- Choose pie or bar graph to show the proportion time occupied by each operator type. The time of one operator type is calculated by accumulating the execution time of operators belong to this type. +- Display top 20 operator types with longest average execution time, show the proportion of total time and average execution time (ms) of each operator type. + +The bottom half of Figure 2 displays the statistics table for the operators' details, including: + +- Choose All: Display statistics for the operators, including operator position information, type, execution time, full scope time etc. The table will be sorted by average execution time by default. +- Choose Type: Display statistics for the operator types, including operator type name, execution time, execution frequency and proportion of total time, average execution time. Users can click on each line, querying for all the operators belong to this type. +- Search: There is a search box on the right, which can support fuzzy search for operators/operator types. + +![gpu_activity_profiler.png](./images/gpu_activity_profiler.png) + +Figure 3: Statistics for Kernel Activities + +Figure 3 displays the statistics for the Kernel, including: + +- Pie graph to show the proportion time occupied by each kernel activity. And the top 15 kernel activities with longest exection time. +- The statistical table's column include activity name, operation name, execution frequency, total time, average time. +- The search box on the right, which can support fuzzy search for activity name/operator full name. + +#### Timeline Analysis + +The usage is almost same as that in Ascend. The difference is GPU Timeline displays the operation information and CUDA activity. + +> The usage is follow as: +> > \ No newline at end of file diff --git a/tutorials/source_en/advanced_use/quantization_aware.md b/tutorials/training/source_en/advanced_use/quantization_aware.md similarity index 100% rename from tutorials/source_en/advanced_use/quantization_aware.md rename to tutorials/training/source_en/advanced_use/quantization_aware.md diff --git a/tutorials/source_en/advanced_use/serving.md b/tutorials/training/source_en/advanced_use/serving.md similarity index 100% rename from tutorials/source_en/advanced_use/serving.md rename to tutorials/training/source_en/advanced_use/serving.md diff --git a/tutorials/source_en/advanced_use/summary_record.md b/tutorials/training/source_en/advanced_use/summary_record.md similarity index 99% rename from tutorials/source_en/advanced_use/summary_record.md rename to tutorials/training/source_en/advanced_use/summary_record.md index 23ec33e637f881deebd02a9953dfad3d466d4c7a..7522d87ec4d5307a8c40b496e9ca175897455a73 100644 --- a/tutorials/source_en/advanced_use/summary_record.md +++ b/tutorials/training/source_en/advanced_use/summary_record.md @@ -366,4 +366,6 @@ For more parameter Settings, see the [MindInsight related commands](https://www. model.train(epoch=2, train_dataset, callbacks=[confusion_callback, summary_collector]) ``` -3. In each Summary log file directory, only one training data should be placed. If a summary log directory contains summary data from multiple training, MindInsight will overlay the summary data from these training when visualizing the data, which may not be consistent with the expected visualizations. \ No newline at end of file +3. In each Summary log file directory, only one training data should be placed. If a summary log directory contains summary data from multiple training, MindInsight will overlay the summary data from these training when visualizing the data, which may not be consistent with the expected visualizations. + +4. Currently, `SummaryCollector` and `SummaryRecord` do not support scenarios with GPU multi-card running. \ No newline at end of file diff --git a/tutorials/source_en/advanced_use/synchronization_training_and_evaluation.md b/tutorials/training/source_en/advanced_use/synchronization_training_and_evaluation.md similarity index 100% rename from tutorials/source_en/advanced_use/synchronization_training_and_evaluation.md rename to tutorials/training/source_en/advanced_use/synchronization_training_and_evaluation.md diff --git a/tutorials/source_en/advanced_use/visualization_tutorials.rst b/tutorials/training/source_en/advanced_use/visualization_tutorials.rst similarity index 95% rename from tutorials/source_en/advanced_use/visualization_tutorials.rst rename to tutorials/training/source_en/advanced_use/visualization_tutorials.rst index 17b1532bba19766e25351c89b259100f1b96d47d..e0a749ab35c8ee7a513bc08ff2964a139a1c7f2c 100644 --- a/tutorials/source_en/advanced_use/visualization_tutorials.rst +++ b/tutorials/training/source_en/advanced_use/visualization_tutorials.rst @@ -1,12 +1,12 @@ -Training Process Visualization -============================== - -.. toctree:: - :maxdepth: 1 - - summary_record - dashboard - lineage_and_scalars_comparision - performance_profiling - performance_profiling_gpu - mindinsight_commands +Training Process Visualization +============================== + +.. toctree:: + :maxdepth: 1 + + summary_record + dashboard + lineage_and_scalars_comparision + performance_profiling + performance_profiling_gpu + mindinsight_commands diff --git a/tutorials/source_en/conf.py b/tutorials/training/source_en/conf.py similarity index 100% rename from tutorials/source_en/conf.py rename to tutorials/training/source_en/conf.py diff --git a/tutorials/source_en/index.rst b/tutorials/training/source_en/index.rst similarity index 100% rename from tutorials/source_en/index.rst rename to tutorials/training/source_en/index.rst diff --git a/tutorials/source_en/quick_start/images/LeNet_5.jpg b/tutorials/training/source_en/quick_start/images/LeNet_5.jpg similarity index 100% rename from tutorials/source_en/quick_start/images/LeNet_5.jpg rename to tutorials/training/source_en/quick_start/images/LeNet_5.jpg diff --git a/tutorials/source_en/quick_start/quick_start.md b/tutorials/training/source_en/quick_start/quick_start.md similarity index 100% rename from tutorials/source_en/quick_start/quick_start.md rename to tutorials/training/source_en/quick_start/quick_start.md diff --git a/tutorials/source_en/quick_start/quick_video.md b/tutorials/training/source_en/quick_start/quick_video.md similarity index 99% rename from tutorials/source_en/quick_start/quick_video.md rename to tutorials/training/source_en/quick_start/quick_video.md index a46f6ae41dde7d4826433e6a24140039427d6ad2..b11de9ebd0d6a767ad183dc4e19fc019ee7bcc26 100644 --- a/tutorials/source_en/quick_start/quick_video.md +++ b/tutorials/training/source_en/quick_start/quick_video.md @@ -269,7 +269,7 @@ Provides video tutorials from installation to try-on, helping you quickly use Mi class="video-item-wraper" style="width: 33.3%;display: flex;justify-content: center;align-items: center;padding: 10px;box-sizing: border-box;"> - (This document contains Hands-on Tutorial Series. Gitee does not support display. Please check tutorials on the official website) - - -## 概述 - -### 背景 - -MindSpore模型并行场景下，每个实例进程只保存有本节点对应的参数数据。对于模型并行的Cell，其在每个节点上的参数数据，都是完整参数数据的一个切片。比如完整参数数据shape为[8, 8]，每个节点上的参数数据为其中的一部分，如shape[2, 8]。 - -对于自动切分的模型并行场景（Auto Parallel），切分逻辑由MindSpore自动生成，MindSpore的CheckPoint模块可以支持自动合并保存和基于合并保存的加载能力。 - -对于用户手动设置的并行场景（HyBrid Parallel），切分逻辑由用户自己实现，MindSpore在每个节点上保存相同的模型参数切分策略文件和本节点上的数据，用户需要自己实现CheckPoint文件的合并保存与加载功能。本教程用于指导用户在手动切分场景下，实现CheckPoint的合并保存与加载能力。 - -### 使用场景 - -如果你遇到如下两个场景，需要参考本教程操作，完成CheckPoint的合并保存与加载： - -场景1：多卡训练，单卡推理。 - - 以在64卡上训练，并在单卡上推理为例，整体操作流程如下： - -1. 执行训练，自动生成CheckPoint文件和模型参数切分策略文件。 - -2. 用户对保存的CheckPoint文件做合并处理。 - - 根据具体的切分逻辑，对于存在切分的具体模型参数做合并处理，生成新CheckPoint文件。 - -3. 在单卡环境加载新的CheckPoint文件，之后再根据需要调用export接口导出用于推理的模型。 - -若CheckPoint的保存环境和加载环境集群的卡数相同，比如在同一训练环境保存加载CheckPoint，或者单卡训练单卡推理，则可以不需要做合并保存和加载。 - -场景2：训练分为多阶段，每个阶段的集群大小不一样。 - -​ 以训练阶段一是64卡训练环境，阶段二是56卡训练环境为例，整体操作流程如下： - -1. 执行阶段一训练，自动生成CheckPoint文件和模型参数切分策略文件。 - -2. 用户对保存的CheckPoint文件做合并处理。 - - 根据具体的切分逻辑，对于存在切分的具体模型参数做合并处理，生成新CheckPoint文件。 - -3. 在阶段二集群上加载合并保存的CheckPoint文件。 - - 在加载过程中，用户需要根据新的训练环境配置，重新切分CheckPoint文件中的参数数据。 - -4. 执行阶段二训练。 - - - - -## 对保存的CheckPoint文件做合并处理 - -### 整体流程 - -首先，执行准备工作，按逻辑顺序将待合并处理的CheckPoint文件导入网络，获取模型全量参数并添加至列表中，再获取模型参数切分策略。对应下图中的Step1和Step2。 - -其次，更新参数列表，对涉及模型并行的参数做合并处理。对应下图中的Step3。 - -最后，将更新之后的参数列表，通过MindSpore提供的API保存到文件，生成新的CheckPoint文件。对应下图中的Step4。 - -![img](./images/checkpoint_integration_process.jpg) - -### 准备工作 - -#### 按逻辑顺序导入CheckPoint文件 - -定义网络，调用`load_checkpoint`、`load_param_into_net`接口，按逻辑顺序将CheckPoint文件导入网络，之后调用`parameters_and_names`接口获取网络里所有的参数数据。 -``` -net = Net() -opt = Momentum(learning_rate=0.01, momentum=0.9, params=net.get_parameters()) -net = TrainOneStepCell(net, opt) -param_dicts = [] -for i in range(rank_size): - file_name = os.path.join("./node"+str(i), "CKP_1-4_32.ckpt") # checkpoint file name of current node - param_dict = load_checkpoint(file_name) - load_param_into_net(net, param_dict) - param_dict = {} - for _, param in net.parameters_and_names(): - param_dict[param.name] = param - param_dicts.append(param_dict) -``` - -其中， - -- `rank_size`：之前分布式训练的节点数。 -- `load_checkpoint`：通过该接口加载CheckPoint模型参数文件，返回一个参数字典。 -- `load_param_into_net`：模型参数数据加载到网络中。 - -#### 获取模型参数切分策略 - -调用`build_searched_strategy`接口，得到模型各个参数的切分策略。 -``` -strategy = build_searched_strategy("./strategy_train.cpkt") -``` - -其中， - -- `strategy_train.ckpt`：保存的模型参数切分策略文件名称，训练网络之前由用户调用`set_auto_parallel_context`接口自定义`strategy_ckpt_save_file`参数生成。 - -### 对模型并行的参数做合并处理 - -下面以一个具体的模型参数为例，说明下参数合并处理的具体流程。 - -参数名称为"model_parallel_weight"，切分逻辑为4卡场景。 - -1. 针对涉及模型并行的参数，获取所有节点上的参数数据。 - - ``` - sliced_parameters = [] - for i in range(4): - parameter = param_dicts[i].get("model_parallel_weight") - sliced_parameters.append(parameter) - ``` - > 如果要保证参数更新速度不变，需要对优化器中保存的参数，如“moments.model_parallel_weight”，同样做合并处理。 - -2. 调用`merge_sliced_parameter`接口进行参数合并。 - - ``` - merged_parameter = merge_sliced_parameter(sliced_parameters, strategy) - ``` - -> 如果存在多个模型并行的参数，则需要重复步骤1到步骤2循环逐个处理。 - -### 保存数据生成新的CheckPoint文件 - -1. 将`param_dict`转换为list类型数据。 - - ``` - param_list = [] - for (key, value) in param_dict.items(): - each_param = {} - each_param["name"] = key - if isinstance(value.data, Tensor): - param_data = value.data - else: - param_data = Tensor(value.data) - each_param["data"] = param_data - param_list.append(each_param） - ``` - -2. 调用`save_checkpoint`接口，将参数数据写入文件，生成新的CheckPoint文件。 - ``` - save_checkpoint(param_list, “./CKP-Integrated_1-4_32.ckpt”) - ``` - 其中， - - `save_checkpoint`： 通过该接口将网络模型参数信息存入文件。 - - `CKP-Integrated_1-4_32.ckpt`： 新生成的CheckPoint模型参数文件名称。 - -## 加载合并保存的CheckPoint文件 - -### 整体流程 - -如果需要将合并保存的CheckPoint加载到多卡训练或推理中，在模型参数真正加载到网络前，需要对于涉及并行的参数数据按照新的逻辑切分。 -如下步骤在训练前脚本里实现，步骤1和3同单机CheckPoint加载的逻辑，步骤2为新增，用于涉及并行的模型参数的切分。 -对于加载到单卡训练/推理的场景，不涉及数据切分，则步骤2可省略。 - -### 步骤1：加载CheckPoint文件 - -调用`load_checkpoint`接口，从CheckPoint文件中加载模型参数数据。 - -``` -param_dict = load_checkpoint("./CKP-Integrated_1-4_32.ckpt") -``` - -- `load_checkpoint`：通过该接口加载CheckPoint模型参数文件，返回一个参数字典。 -- `CKP-Integrated_1-4_32.ckpt`：需要加载的CheckPoint模型参数文件名称。 - -### 步骤2：对模型并行参数做切分处理 - -下面以一个具体的模型参数为例，参数名称为“model_parallel_weight"， 数据值为Tensor [[1, 2, 3, 4], [5, 6, 7, 8]]，切分逻辑为2卡场景，按[2, 1]切分。 -切分后数据分布情况如下： - -| Device0 | Device1 | -| ------------------- | -------------------- | -| Value [1, 2, 3, 4] | Value [5, 6, 7, 8] | - -1. 对模型参数数据做切分。 - - 如下代码示例，在维度0上，将数据切分为两个切片。 - ``` - new_param = parameter_dict[“model_parallel_weight”] - slice_list = np.split(new_param.data.asnumpy(), 2, axis=0) - new_param_moments = parameter_dict[“moments.model_parallel_weight”] - slice_moments_list = np.split(new_param_moments.data.asnumpy(), 2, axis=0) - ``` - - 切分后的数据情况： - - slice_list[0] --- [1, 2, 3, 4] 对应device0 - slice_list[1] --- [5, 6, 7, 8] 对应device1 - - 与`slice_list`类似，`slice_moments_list` 也被切分为两个shape为[1, 4]的Tensor。 - -2. 在每个节点分别加载对应的数据切片。 - - 获取本节点的rank_id，根据rank_id加载数据。 - ``` - rank = get_rank() - tensor_slice = Tensor(slice_list[rank]) - tensor_slice_moments = Tensor(slice_moments_list[rank]) - ``` - - `get_rank`：获取当前设备在集群中的ID。 - -3. 修改模型参数数据值。 - - ``` - new_param.set_data(tensor_slice, True) - new_param_moments.set_data(tensor_slice_moments, True) - ``` - - - `set_data`：设置模型参数的值，接口参数类型为Tensor 或number。 - -### 步骤3：将修改后的参数数据加载到网络中 - -调用`load_param_into_net`接口，将模型参数数据加载到网络中。 -``` -net = Net() -opt = Momentum(learning_rate=0.01, momentum=0.9, params=parallel_net.get_parameters()) -load_param_into_net(net, param_dict) -load_param_into_net(opt, param_dict) -``` - -## 示例 - -### 示例场景说明 - -整体场景：训练分为两个阶段，两阶段的集群规模不一致，模拟FC层MatMul算子并行。 - -用户流程： - -1. 执行阶段1训练：阶段1为4卡训练环境，每卡上MatMul算子weight的shape为[2, 8]，训练过程中自动导出CheckPoint。 - -2. 执行脚本对CheckPoint文件做合并处理，根据具体的切分逻辑，对于存在切分的具体模型参数做合并处理，生成合并的CheckPoint文件。 - -3. 执行阶段2训练：阶段2为2卡训练环境，每卡上MatMul算子weight的shape为[4, 8]，从合并后的CheckPoint文件加载初始化模型参数数据，之后执行训练。 - -> 具体分布式环境配置和训练部分代码，此处不做详细说明，可以参考[分布式并行训练](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/distributed_training_ascend.html) -章节。 -> -> 本文档附上对CheckPoint文件做合并处理以及分布式训练前加载CheckPoint文件的示例代码，仅作为参考，实际请参考具体情况实现。 - -### 示例代码 - -1. 执行脚本对CheckPoint文件做合并处理。 - - 脚本执行命令： - ``` - python ./integrate_checkpoint.py "待合并的CheckPoint文件名称" "合并生成的CheckPoint文件路径&名称" "策略文件路径&名称" "节点数" - ``` - - integrate_checkpoint.py： - - ``` - import numpy as np - import os - import mindspore.nn as nn - from mindspore import Tensor, Parameter - from mindspore.ops import operations as P - from mindspore.train.serialization import save_checkpoint, load_checkpoint, build_searched_strategy, merge_sliced_parameter - - class Net(nn.Cell): - def __init__(self,weight_init): - super(Net, self).__init__() - self.weight = Parameter(Tensor(weight_init), "model_parallel_weight", layerwise_parallel=True) - self.fc = P.MatMul(transpose_b=True) - - def construct(self, x): - x = self.fc(x, self.weight1) - return x - - def integrate_ckpt_file(old_ckpt_file, new_ckpt_file, strategy_file, rank_size): - weight = np.ones([2, 8]).astype(np.float32) - net = Net(weight) - opt = Momentum(learning_rate=0.01, momentum=0.9, params=net.get_parameters()) - net = TrainOneStepCell(net, opt) - - # load CheckPoint into net in rank id order - param_dicts = [] - for i in range(rank_size): - file_name = os.path.join("./node"+str(i), old_ckpt_file) - param_dict = load_checkpoint(file_name) - load_param_into_net(net, param_dict) - param_dict = {} - for _, param in net.parameters_and_names(): - param_dict[param.name] = param - param_dicts.append(param_dict) - - strategy = build_searched_strategy(strategy_file) - param_dict = {} - - for paramname in ["model_parallel_weight", "moments.model_parallel_weight"]: - # get layer wise model parallel parameter - sliced_parameters = [] - for i in range(rank_size): - parameter = param_dicts[i].get(paramname) - sliced_parameters.append(parameter) - - # merge the parallel parameters of the model - merged_parameter = merge_sliced_parameter(sliced_parameters, strategy) - param_dict[paramname] = merged_parameter - - # convert param_dict to list type data - param_list = [] - for (key, value) in param_dict.items(): - each_param = {} - each_param["name"] = key - if isinstance(value.data, Tensor): - param_data = value.data - else: - param_data = Tensor(value.data) - each_param["data"] = param_data - param_list.append(each_param) - - # call the API to generate a new CheckPoint file - save_checkpoint(param_list, new_ckpt_file) - - return - - if __name__ == "__main__": - try: - old_ckpt_file = sys.argv[1] - new_ckpt_file = sys.argv[2] - strategy_file = sys.argv[3] - rank_size = int(sys.argv[4]) - integrate_ckpt_file(old_ckpt_file, new_ckpt_file, strategy_file, rank_size) - except: - print("Fail to integrate checkpoint file) - sys.exit(-1) - ``` - - 执行结果： - - 脚本执行前，CheckPoint文件中参数值： - ``` - device0： - name is model_parallel_weight - value is - [[0.87537426 1.0448935 0.86736983 0.8836905 0.77354026 0.69588304 0.9183654 0.7792076] - [0.87224025 0.8726848 0.771446 0.81967723 0.88974726 0.7988162 0.72919345 0.7677011]] - name is learning_rate - value is [0.01] - name is momentum - value is [0.9] - name is moments.model_weight - value is - [[0.2567724 -0.07485991 0.282002 0.2456022 0.454939 0.619168 0.18964815 0.45714882] - [0.25946522 0.24344791 0.45677605 0.3611395 0.23378398 0.41439137 0.5312468 0.4696194]] - - device1： - name is model_parallel_weight - value is - [[0.9210751 0.9050457 0.9827775 0.920396 0.9240526 0.9750359 1.0275179 1.0819869] - [0.73605865 0.84631145 0.9746683 0.9386582 0.82902765 0.83565056 0.9702136 1.0514659]] - name is learning_rate - value is [0.01] - name is momentum - value is [0.9] - name is moments.model_weight - value is - [[0.2417504 0.28193963 0.06713893 0.21510397 0.23380603 0.11424308 0.0218009 -0.11969765] - [0.45955992 0.22664294 0.01990281 0.0731914 0.27125207 0.27298513 -0.01716102 -0.15327111]] - - device2： - name is model_parallel_weight - value is - [[1.0108461 0.8689414 0.91719437 0.8805056 0.7994629 0.8999671 0.7585804 1.0287056 ] - [0.90653455 0.60146594 0.7206475 0.8306303 0.8364681 0.89625114 0.7354735 0.8447268]] - name is learning_rate - value is [0.01] - name is momentum - value is [0.9] - name is moments.model_weight - value is - [[0.03440702 0.41419312 0.24817684 0.30765256 0.48516113 0.24904746 0.57791173 0.00955463] - [0.13458519 0.6690533 0.49259356 0.28319967 0.25951773 0.16777472 0.45696738 0.24933104]] - - device3： - name is model_parallel_weight - value is - [[0.7147005 0.9168278 0.80178416 0.6258351 0.8413766 0.5909515 0.696347 0.71359116] - [0.20506378 0.03691584 0.2454556 0.12978578 0.19065076 0.23904312 0.27509746 0.34614682]] - name is learning_rate - value is [0.01] - name is momentum - value is [0.9] - name is moments.model_parallel_weight - value is - [[0.14152306 0.5040985 0.24455397 0.10907605 0.11319532 0.19538902 0.01208619 0.40430856] - [-0.7773164 -0.47611716 -0.6041424 -0.6144473 -0.2651842 -0.31909415 -0.4510405 -0.12860501]] - ``` - - 脚本执行后，CheckPoint文件中参数值： - - ``` - name is model_parallel_weight - value is - [[1.1138763 1.0962057 1.3516843 1.0812817 1.1579804 1.1078343 1.0906502 1.3207073] - [0.916671 1.0781671 1.0368758 0.9680898 1.1735439 1.0628364 0.9960786 1.0135143] - [0.8828271 0.7963984 0.90675324 0.9830291 0.89010954 0.897052 0.7890109 0.89784735] - [1.0011744 1.0840297 1.0201758 1.0882459 0.94232416 1.0775206 1.0195118 1.0528734] - [1.0053468 0.98402303 0.99762845 0.97587246 1.0259694 1.0055295 0.99420834 0.9496847] - [1.0851002 1.0295962 1.0999886 1.0958165 0.9765328 1.146529 1.0970603 1.1388365] - [0.7147005 0.9168278 0.80178416 0.6258351 0.8413766 0.5909515 0.696347 0.71359116] - [0.20506378 0.03691584 0.2454556 0.12978578 0.19065076 0.23904312 0.27509746 0.34614682]] - name is learning_rate - value is [0.01] - name is momentum - value is [0.9] - name is moments.model_parallel_weight - value is - [[0.2567724 -0.07485991 0.282002 0.2456022 0.454939 0.619168 0.18964815 0.45714882] - [0.25946522 0.24344791 0.45677605 0.3611395 0.23378398 0.41439137 0.5312468 0.4696194 ] - [0.2417504 0.28193963 0.06713893 0.21510397 0.23380603 0.11424308 0.0218009 -0.11969765] - [0.45955992 0.22664294 0.01990281 0.0731914 0.27125207 0.27298513 -0.01716102 -0.15327111] - [0.03440702 0.41419312 0.24817684 0.30765256 0.48516113 0.24904746 0.57791173 0.00955463] - [0.13458519 0.6690533 0.49259356 0.28319967 0.25951773 0.16777472 0.45696738 0.24933104] - [0.14152306 0.5040985 0.24455397 0.10907605 0.11319532 0.19538902 0.01208619 0.40430856] - [-0.7773164 -0.47611716 -0.6041424 -0.6144473 -0.2651842 -0.31909415 -0.4510405 - -0.12860501]] - ``` - - -2. 执行阶段2训练，训练前加载CheckPoint文件。其中训练代码部分，需要根据实际情况补充。 - - ``` - import numpy as np - import os - import mindspore.nn as nn - from mindspore import context - from mindspore.communication.management import init - from mindspore import Tensor, Parameter - from mindspore.ops import operations as P - from mindspore.train.serialization import load_checkpoint, load_param_into_net - - from mindspore.communication.management import init - devid = int(os.getenv('DEVICE_ID')) - context.set_context(mode=context.GRAPH_MODE,device_target='Ascend',save_graphs=True, device_id=devid) - init() - - class Net(nn.Cell): - def __init__(self,weight_init): - super(Net, self).__init__() - self.weight = Parameter(Tensor(weight_init), "model_parallel_weight", layerwise_parallel=True) - self.fc = P.MatMul(transpose_b=True) - - def construct(self, x): - x = self.fc(x, self.weight1) - return x - def train_mindspore_impl_fc(input, label, ckpt_file): - param_dict = load_checkpoint(ckpt_file) - - for paramname in ["model_parallel_weight", "moments.model_parallel_weight"]: - # get layer wise model parallel parameter - new_param = parameter_dict[paramname] - # split the model parameter data - slice_list = np.split(new_param.data.asnumpy(), 2, axis=0) - # Load the corresponding data slice - rank = get_rank() - tensor_slice = Tensor(slice_list[rank]) - # modify model parameter data values - new_param.set_data(tensor_slice, True) - - # load the modified parameter data into the network - weight = np.ones([4, 8]).astype(np.float32) - net = Net(weight) - load_param_into_net(net, param_dict) - opt = Momentum(learning_rate=0.01, momentum=0.9, params=parallel_net.get_parameters()) - load_param_into_net(opt, param_dict) - # train code - ... - - if __name__ == "__main__": - input = np.random.random((4, 8)).astype(np.float32) - print("mean = ", np.mean(input,axis=1, keepdims=True)) - label = np.random.random((4, 4)).astype(np.float32) - train_mindspore_impl_fc(input, label, weight1) - ``` - - 其中， - - - `mode=context.GRAPH_MODE`：使用分布式训练需要指定运行模式为图模式（PyNative模式不支持并行）。 - - `device_id`：卡物理序号，即卡所在机器中的实际序号。 - - `init`：完成分布式训练初始化操作。 - - 加载后的参数值： - - ``` - device0： - name is model_parallel_weight - value is - [[0.87537426 1.0448935 0.86736983 0.8836905 0.77354026 0.69588304 0.9183654 0.7792076] - [0.87224025 0.8726848 0.771446 0.81967723 0.88974726 0.7988162 0.72919345 0.7677011] - [0.8828271 0.7963984 0.90675324 0.9830291 0.89010954 0.897052 0.7890109 0.89784735] - [1.0011744 1.0840297 1.0201758 1.0882459 0.94232416 1.0775206 1.0195118 1.0528734]] - name is learning_rate - value is [0.01] - name is momentum - value is [0.9] - name is moments.model_weight - value is - [[0.2567724 -0.07485991 0.282002 0.2456022 0.454939 0.619168 0.18964815 0.45714882] - [0.25946522 0.24344791 0.45677605 0.3611395 0.23378398 0.41439137 0.5312468 0.4696194] - [0.2417504 0.28193963 0.06713893 0.21510397 0.23380603 0.11424308 0.0218009 -0.11969765] - [0.45955992 0.22664294 0.01990281 0.0731914 0.27125207 0.27298513 -0.01716102 -0.15327111]] - - device1： - name is model_parallel_weight - value is - [[1.0053468 0.98402303 0.99762845 0.97587246 1.0259694 1.0055295 0.99420834 0.9496847] - [1.0851002 1.0295962 1.0999886 1.0958165 0.9765328 1.146529 1.0970603 1.1388365] - [0.7147005 0.9168278 0.80178416 0.6258351 0.8413766 0.5909515 0.696347 0.71359116] - [0.20506378 0.03691584 0.2454556 0.12978578 0.19065076 0.23904312 0.27509746 0.34614682]] - name is learning_rate - value is [0.01] - name is momentum - value is [0.9] - name is moments.model_weight - value is - [[0.03440702 0.41419312 0.24817684 0.30765256 0.48516113 0.24904746 0.57791173 0.00955463] - [0.13458519 0.6690533 0.49259356 0.28319967 0.25951773 0.16777472 0.45696738 0.24933104] - [0.14152306 0.5040985 0.24455397 0.10907605 0.11319532 0.19538902 0.01208619 0.40430856] - [-0.7773164 -0.47611716 -0.6041424 -0.6144473 -0.2651842 -0.31909415 -0.4510405 -0.12860501]] - ``` + + +# 手动设置并行场景模型参数的保存和加载 + +`Linux` `Ascend` `GPU` `模型训练` `中级` `高级` + + + +- [手动设置并行场景模型参数的保存和加载](#手动设置并行场景模型参数的保存和加载) + - [概述](#概述) + - [背景](#背景) + - [使用场景](#使用场景) + - [对保存的CheckPoint文件做合并处理](#对保存的checkpoint文件做合并处理) + - [整体流程](#整体流程) + - [准备工作](#准备工作) + - [按逻辑顺序导入CheckPoint文件](#按逻辑顺序导入checkpoint文件) + - [获取模型参数切分策略](#获取模型参数切分策略) + - [对模型并行的参数做合并处理](#对模型并行的参数做合并处理) + - [保存数据生成新的CheckPoint文件](#保存数据生成新的checkpoint文件) + - [加载合并保存的CheckPoint文件](#加载合并保存的checkpoint文件) + - [整体流程](#整体流程-1) + - [步骤1：加载CheckPoint文件](#步骤1加载checkpoint文件) + - [步骤2：对模型并行参数做切分处理](#步骤2对模型并行参数做切分处理) + - [步骤3：将修改后的参数数据加载到网络中](#步骤3将修改后的参数数据加载到网络中) + - [示例](#示例) + - [示例场景说明](#示例场景说明) + - [示例代码](#示例代码) + + + + + +## 概述 + +### 背景 + +MindSpore模型并行场景下，每个实例进程只保存有本节点对应的参数数据。对于模型并行的Cell，其在每个节点上的参数数据，都是完整参数数据的一个切片。比如完整参数数据shape为[8, 8]，每个节点上的参数数据为其中的一部分，如shape[2, 8]。 + +对于自动切分的模型并行场景（Auto Parallel），切分逻辑由MindSpore自动生成，MindSpore的CheckPoint模块可以支持自动合并保存和基于合并保存的加载能力。 + +对于用户手动设置的并行场景（HyBrid Parallel），切分逻辑由用户自己实现，MindSpore在每个节点上保存相同的模型参数切分策略文件和本节点上的数据，用户需要自己实现CheckPoint文件的合并保存与加载功能。本教程用于指导用户在手动切分场景下，实现CheckPoint的合并保存与加载能力。 + +### 使用场景 + +如果你遇到如下两个场景，需要参考本教程操作，完成CheckPoint的合并保存与加载： + +场景1：多卡训练，单卡推理。 + + 以在64卡上训练，并在单卡上推理为例，整体操作流程如下： + +1. 执行训练，自动生成CheckPoint文件和模型参数切分策略文件。 + +2. 用户对保存的CheckPoint文件做合并处理。 + + 根据具体的切分逻辑，对于存在切分的具体模型参数做合并处理，生成新CheckPoint文件。 + +3. 在单卡环境加载新的CheckPoint文件，之后再根据需要调用export接口导出用于推理的模型。 + +若CheckPoint的保存环境和加载环境集群的卡数相同，比如在同一训练环境保存加载CheckPoint，或者单卡训练单卡推理，则可以不需要做合并保存和加载。 + +场景2：训练分为多阶段，每个阶段的集群大小不一样。 + +​ 以训练阶段一是64卡训练环境，阶段二是56卡训练环境为例，整体操作流程如下： + +1. 执行阶段一训练，自动生成CheckPoint文件和模型参数切分策略文件。 + +2. 用户对保存的CheckPoint文件做合并处理。 + + 根据具体的切分逻辑，对于存在切分的具体模型参数做合并处理，生成新CheckPoint文件。 + +3. 在阶段二集群上加载合并保存的CheckPoint文件。 + + 在加载过程中，用户需要根据新的训练环境配置，重新切分CheckPoint文件中的参数数据。 + +4. 执行阶段二训练。 + + + + +## 对保存的CheckPoint文件做合并处理 + +### 整体流程 + +首先，执行准备工作，按逻辑顺序将待合并处理的CheckPoint文件导入网络，获取模型全量参数并添加至列表中，再获取模型参数切分策略。对应下图中的Step1和Step2。 + +其次，更新参数列表，对涉及模型并行的参数做合并处理。对应下图中的Step3。 + +最后，将更新之后的参数列表，通过MindSpore提供的API保存到文件，生成新的CheckPoint文件。对应下图中的Step4。 + +![img](./images/checkpoint_integration_process.jpg) + +### 准备工作 + +#### 按逻辑顺序导入CheckPoint文件 + +定义网络，调用`load_checkpoint`、`load_param_into_net`接口，按逻辑顺序将CheckPoint文件导入网络，之后调用`parameters_and_names`接口获取网络里所有的参数数据。 +``` +net = Net() +opt = Momentum(learning_rate=0.01, momentum=0.9, params=net.get_parameters()) +net = TrainOneStepCell(net, opt) +param_dicts = [] +for i in range(rank_size): + file_name = os.path.join("./node"+str(i), "CKP_1-4_32.ckpt") # checkpoint file name of current node + param_dict = load_checkpoint(file_name) + load_param_into_net(net, param_dict) + param_dict = {} + for _, param in net.parameters_and_names(): + param_dict[param.name] = param + param_dicts.append(param_dict) +``` + +其中， + +- `rank_size`：之前分布式训练的节点数。 +- `load_checkpoint`：通过该接口加载CheckPoint模型参数文件，返回一个参数字典。 +- `load_param_into_net`：模型参数数据加载到网络中。 + +#### 获取模型参数切分策略 + +调用`build_searched_strategy`接口，得到模型各个参数的切分策略。 +``` +strategy = build_searched_strategy("./strategy_train.cpkt") +``` + +其中， + +- `strategy_train.ckpt`：保存的模型参数切分策略文件名称，训练网络之前由用户调用`set_auto_parallel_context`接口自定义`strategy_ckpt_save_file`参数生成。 + +### 对模型并行的参数做合并处理 + +下面以一个具体的模型参数为例，说明下参数合并处理的具体流程。 + +参数名称为"model_parallel_weight"，切分逻辑为4卡场景。 + +1. 针对涉及模型并行的参数，获取所有节点上的参数数据。 + + ``` + sliced_parameters = [] + for i in range(4): + parameter = param_dicts[i].get("model_parallel_weight") + sliced_parameters.append(parameter) + ``` + > 如果要保证参数更新速度不变，需要对优化器中保存的参数，如“moments.model_parallel_weight”，同样做合并处理。 + +2. 调用`merge_sliced_parameter`接口进行参数合并。 + + ``` + merged_parameter = merge_sliced_parameter(sliced_parameters, strategy) + ``` + +> 如果存在多个模型并行的参数，则需要重复步骤1到步骤2循环逐个处理。 + +### 保存数据生成新的CheckPoint文件 + +1. 将`param_dict`转换为list类型数据。 + + ``` + param_list = [] + for (key, value) in param_dict.items(): + each_param = {} + each_param["name"] = key + if isinstance(value.data, Tensor): + param_data = value.data + else: + param_data = Tensor(value.data) + each_param["data"] = param_data + param_list.append(each_param） + ``` + +2. 调用`save_checkpoint`接口，将参数数据写入文件，生成新的CheckPoint文件。 + ``` + save_checkpoint(param_list, “./CKP-Integrated_1-4_32.ckpt”) + ``` + 其中， + - `save_checkpoint`： 通过该接口将网络模型参数信息存入文件。 + - `CKP-Integrated_1-4_32.ckpt`： 新生成的CheckPoint模型参数文件名称。 + +## 加载合并保存的CheckPoint文件 + +### 整体流程 + +如果需要将合并保存的CheckPoint加载到多卡训练或推理中，在模型参数真正加载到网络前，需要对于涉及并行的参数数据按照新的逻辑切分。 +如下步骤在训练前脚本里实现，步骤1和3同单机CheckPoint加载的逻辑，步骤2为新增，用于涉及并行的模型参数的切分。 +对于加载到单卡训练/推理的场景，不涉及数据切分，则步骤2可省略。 + +### 步骤1：加载CheckPoint文件 + +调用`load_checkpoint`接口，从CheckPoint文件中加载模型参数数据。 + +``` +param_dict = load_checkpoint("./CKP-Integrated_1-4_32.ckpt") +``` + +- `load_checkpoint`：通过该接口加载CheckPoint模型参数文件，返回一个参数字典。 +- `CKP-Integrated_1-4_32.ckpt`：需要加载的CheckPoint模型参数文件名称。 + +### 步骤2：对模型并行参数做切分处理 + +下面以一个具体的模型参数为例，参数名称为“model_parallel_weight"， 数据值为Tensor [[1, 2, 3, 4], [5, 6, 7, 8]]，切分逻辑为2卡场景，按[2, 1]切分。 +切分后数据分布情况如下： + +| Device0 | Device1 | +| ------------------- | -------------------- | +| Value [1, 2, 3, 4] | Value [5, 6, 7, 8] | + +1. 对模型参数数据做切分。 + + 如下代码示例，在维度0上，将数据切分为两个切片。 + ``` + new_param = parameter_dict[“model_parallel_weight”] + slice_list = np.split(new_param.data.asnumpy(), 2, axis=0) + new_param_moments = parameter_dict[“moments.model_parallel_weight”] + slice_moments_list = np.split(new_param_moments.data.asnumpy(), 2, axis=0) + ``` + + 切分后的数据情况： + + slice_list[0] --- [1, 2, 3, 4] 对应device0 + slice_list[1] --- [5, 6, 7, 8] 对应device1 + + 与`slice_list`类似，`slice_moments_list` 也被切分为两个shape为[1, 4]的Tensor。 + +2. 在每个节点分别加载对应的数据切片。 + + 获取本节点的rank_id，根据rank_id加载数据。 + ``` + rank = get_rank() + tensor_slice = Tensor(slice_list[rank]) + tensor_slice_moments = Tensor(slice_moments_list[rank]) + ``` + - `get_rank`：获取当前设备在集群中的ID。 + +3. 修改模型参数数据值。 + + ``` + new_param.set_data(tensor_slice, True) + new_param_moments.set_data(tensor_slice_moments, True) + ``` + + - `set_data`：设置模型参数的值，接口参数类型为Tensor 或number。 + +### 步骤3：将修改后的参数数据加载到网络中 + +调用`load_param_into_net`接口，将模型参数数据加载到网络中。 +``` +net = Net() +opt = Momentum(learning_rate=0.01, momentum=0.9, params=parallel_net.get_parameters()) +load_param_into_net(net, param_dict) +load_param_into_net(opt, param_dict) +``` + +## 示例 + +### 示例场景说明 + +整体场景：训练分为两个阶段，两阶段的集群规模不一致，模拟FC层MatMul算子并行。 + +用户流程： + +1. 执行阶段1训练：阶段1为4卡训练环境，每卡上MatMul算子weight的shape为[2, 8]，训练过程中自动导出CheckPoint。 + +2. 执行脚本对CheckPoint文件做合并处理，根据具体的切分逻辑，对于存在切分的具体模型参数做合并处理，生成合并的CheckPoint文件。 + +3. 执行阶段2训练：阶段2为2卡训练环境，每卡上MatMul算子weight的shape为[4, 8]，从合并后的CheckPoint文件加载初始化模型参数数据，之后执行训练。 + +> 具体分布式环境配置和训练部分代码，此处不做详细说明，可以参考[分布式并行训练](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/distributed_training_ascend.html) +章节。 +> +> 本文档附上对CheckPoint文件做合并处理以及分布式训练前加载CheckPoint文件的示例代码，仅作为参考，实际请参考具体情况实现。 + +### 示例代码 + +1. 执行脚本对CheckPoint文件做合并处理。 + + 脚本执行命令： + ``` + python ./integrate_checkpoint.py "待合并的CheckPoint文件名称" "合并生成的CheckPoint文件路径&名称" "策略文件路径&名称" "节点数" + ``` + + integrate_checkpoint.py： + + ``` + import numpy as np + import os + import mindspore.nn as nn + from mindspore import Tensor, Parameter + from mindspore.ops import operations as P + from mindspore.train.serialization import save_checkpoint, load_checkpoint, build_searched_strategy, merge_sliced_parameter + + class Net(nn.Cell): + def __init__(self,weight_init): + super(Net, self).__init__() + self.weight = Parameter(Tensor(weight_init), "model_parallel_weight", layerwise_parallel=True) + self.fc = P.MatMul(transpose_b=True) + + def construct(self, x): + x = self.fc(x, self.weight1) + return x + + def integrate_ckpt_file(old_ckpt_file, new_ckpt_file, strategy_file, rank_size): + weight = np.ones([2, 8]).astype(np.float32) + net = Net(weight) + opt = Momentum(learning_rate=0.01, momentum=0.9, params=net.get_parameters()) + net = TrainOneStepCell(net, opt) + + # load CheckPoint into net in rank id order + param_dicts = [] + for i in range(rank_size): + file_name = os.path.join("./node"+str(i), old_ckpt_file) + param_dict = load_checkpoint(file_name) + load_param_into_net(net, param_dict) + param_dict = {} + for _, param in net.parameters_and_names(): + param_dict[param.name] = param + param_dicts.append(param_dict) + + strategy = build_searched_strategy(strategy_file) + param_dict = {} + + for paramname in ["model_parallel_weight", "moments.model_parallel_weight"]: + # get layer wise model parallel parameter + sliced_parameters = [] + for i in range(rank_size): + parameter = param_dicts[i].get(paramname) + sliced_parameters.append(parameter) + + # merge the parallel parameters of the model + merged_parameter = merge_sliced_parameter(sliced_parameters, strategy) + param_dict[paramname] = merged_parameter + + # convert param_dict to list type data + param_list = [] + for (key, value) in param_dict.items(): + each_param = {} + each_param["name"] = key + if isinstance(value.data, Tensor): + param_data = value.data + else: + param_data = Tensor(value.data) + each_param["data"] = param_data + param_list.append(each_param) + + # call the API to generate a new CheckPoint file + save_checkpoint(param_list, new_ckpt_file) + + return + + if __name__ == "__main__": + try: + old_ckpt_file = sys.argv[1] + new_ckpt_file = sys.argv[2] + strategy_file = sys.argv[3] + rank_size = int(sys.argv[4]) + integrate_ckpt_file(old_ckpt_file, new_ckpt_file, strategy_file, rank_size) + except: + print("Fail to integrate checkpoint file) + sys.exit(-1) + ``` + + 执行结果： + + 脚本执行前，CheckPoint文件中参数值： + ``` + device0： + name is model_parallel_weight + value is + [[0.87537426 1.0448935 0.86736983 0.8836905 0.77354026 0.69588304 0.9183654 0.7792076] + [0.87224025 0.8726848 0.771446 0.81967723 0.88974726 0.7988162 0.72919345 0.7677011]] + name is learning_rate + value is [0.01] + name is momentum + value is [0.9] + name is moments.model_weight + value is + [[0.2567724 -0.07485991 0.282002 0.2456022 0.454939 0.619168 0.18964815 0.45714882] + [0.25946522 0.24344791 0.45677605 0.3611395 0.23378398 0.41439137 0.5312468 0.4696194]] + + device1： + name is model_parallel_weight + value is + [[0.9210751 0.9050457 0.9827775 0.920396 0.9240526 0.9750359 1.0275179 1.0819869] + [0.73605865 0.84631145 0.9746683 0.9386582 0.82902765 0.83565056 0.9702136 1.0514659]] + name is learning_rate + value is [0.01] + name is momentum + value is [0.9] + name is moments.model_weight + value is + [[0.2417504 0.28193963 0.06713893 0.21510397 0.23380603 0.11424308 0.0218009 -0.11969765] + [0.45955992 0.22664294 0.01990281 0.0731914 0.27125207 0.27298513 -0.01716102 -0.15327111]] + + device2： + name is model_parallel_weight + value is + [[1.0108461 0.8689414 0.91719437 0.8805056 0.7994629 0.8999671 0.7585804 1.0287056 ] + [0.90653455 0.60146594 0.7206475 0.8306303 0.8364681 0.89625114 0.7354735 0.8447268]] + name is learning_rate + value is [0.01] + name is momentum + value is [0.9] + name is moments.model_weight + value is + [[0.03440702 0.41419312 0.24817684 0.30765256 0.48516113 0.24904746 0.57791173 0.00955463] + [0.13458519 0.6690533 0.49259356 0.28319967 0.25951773 0.16777472 0.45696738 0.24933104]] + + device3： + name is model_parallel_weight + value is + [[0.7147005 0.9168278 0.80178416 0.6258351 0.8413766 0.5909515 0.696347 0.71359116] + [0.20506378 0.03691584 0.2454556 0.12978578 0.19065076 0.23904312 0.27509746 0.34614682]] + name is learning_rate + value is [0.01] + name is momentum + value is [0.9] + name is moments.model_parallel_weight + value is + [[0.14152306 0.5040985 0.24455397 0.10907605 0.11319532 0.19538902 0.01208619 0.40430856] + [-0.7773164 -0.47611716 -0.6041424 -0.6144473 -0.2651842 -0.31909415 -0.4510405 -0.12860501]] + ``` + + 脚本执行后，CheckPoint文件中参数值： + + ``` + name is model_parallel_weight + value is + [[1.1138763 1.0962057 1.3516843 1.0812817 1.1579804 1.1078343 1.0906502 1.3207073] + [0.916671 1.0781671 1.0368758 0.9680898 1.1735439 1.0628364 0.9960786 1.0135143] + [0.8828271 0.7963984 0.90675324 0.9830291 0.89010954 0.897052 0.7890109 0.89784735] + [1.0011744 1.0840297 1.0201758 1.0882459 0.94232416 1.0775206 1.0195118 1.0528734] + [1.0053468 0.98402303 0.99762845 0.97587246 1.0259694 1.0055295 0.99420834 0.9496847] + [1.0851002 1.0295962 1.0999886 1.0958165 0.9765328 1.146529 1.0970603 1.1388365] + [0.7147005 0.9168278 0.80178416 0.6258351 0.8413766 0.5909515 0.696347 0.71359116] + [0.20506378 0.03691584 0.2454556 0.12978578 0.19065076 0.23904312 0.27509746 0.34614682]] + name is learning_rate + value is [0.01] + name is momentum + value is [0.9] + name is moments.model_parallel_weight + value is + [[0.2567724 -0.07485991 0.282002 0.2456022 0.454939 0.619168 0.18964815 0.45714882] + [0.25946522 0.24344791 0.45677605 0.3611395 0.23378398 0.41439137 0.5312468 0.4696194 ] + [0.2417504 0.28193963 0.06713893 0.21510397 0.23380603 0.11424308 0.0218009 -0.11969765] + [0.45955992 0.22664294 0.01990281 0.0731914 0.27125207 0.27298513 -0.01716102 -0.15327111] + [0.03440702 0.41419312 0.24817684 0.30765256 0.48516113 0.24904746 0.57791173 0.00955463] + [0.13458519 0.6690533 0.49259356 0.28319967 0.25951773 0.16777472 0.45696738 0.24933104] + [0.14152306 0.5040985 0.24455397 0.10907605 0.11319532 0.19538902 0.01208619 0.40430856] + [-0.7773164 -0.47611716 -0.6041424 -0.6144473 -0.2651842 -0.31909415 -0.4510405 + -0.12860501]] + ``` + + +2. 执行阶段2训练，训练前加载CheckPoint文件。其中训练代码部分，需要根据实际情况补充。 + + ``` + import numpy as np + import os + import mindspore.nn as nn + from mindspore import context + from mindspore.communication.management import init + from mindspore import Tensor, Parameter + from mindspore.ops import operations as P + from mindspore.train.serialization import load_checkpoint, load_param_into_net + + from mindspore.communication.management import init + devid = int(os.getenv('DEVICE_ID')) + context.set_context(mode=context.GRAPH_MODE,device_target='Ascend',save_graphs=True, device_id=devid) + init() + + class Net(nn.Cell): + def __init__(self,weight_init): + super(Net, self).__init__() + self.weight = Parameter(Tensor(weight_init), "model_parallel_weight", layerwise_parallel=True) + self.fc = P.MatMul(transpose_b=True) + + def construct(self, x): + x = self.fc(x, self.weight1) + return x + def train_mindspore_impl_fc(input, label, ckpt_file): + param_dict = load_checkpoint(ckpt_file) + + for paramname in ["model_parallel_weight", "moments.model_parallel_weight"]: + # get layer wise model parallel parameter + new_param = parameter_dict[paramname] + # split the model parameter data + slice_list = np.split(new_param.data.asnumpy(), 2, axis=0) + # Load the corresponding data slice + rank = get_rank() + tensor_slice = Tensor(slice_list[rank]) + # modify model parameter data values + new_param.set_data(tensor_slice, True) + + # load the modified parameter data into the network + weight = np.ones([4, 8]).astype(np.float32) + net = Net(weight) + load_param_into_net(net, param_dict) + opt = Momentum(learning_rate=0.01, momentum=0.9, params=parallel_net.get_parameters()) + load_param_into_net(opt, param_dict) + # train code + ... + + if __name__ == "__main__": + input = np.random.random((4, 8)).astype(np.float32) + print("mean = ", np.mean(input,axis=1, keepdims=True)) + label = np.random.random((4, 4)).astype(np.float32) + train_mindspore_impl_fc(input, label, weight1) + ``` + + 其中， + + - `mode=context.GRAPH_MODE`：使用分布式训练需要指定运行模式为图模式（PyNative模式不支持并行）。 + - `device_id`：卡物理序号，即卡所在机器中的实际序号。 + - `init`：完成分布式训练初始化操作。 + + 加载后的参数值： + + ``` + device0： + name is model_parallel_weight + value is + [[0.87537426 1.0448935 0.86736983 0.8836905 0.77354026 0.69588304 0.9183654 0.7792076] + [0.87224025 0.8726848 0.771446 0.81967723 0.88974726 0.7988162 0.72919345 0.7677011] + [0.8828271 0.7963984 0.90675324 0.9830291 0.89010954 0.897052 0.7890109 0.89784735] + [1.0011744 1.0840297 1.0201758 1.0882459 0.94232416 1.0775206 1.0195118 1.0528734]] + name is learning_rate + value is [0.01] + name is momentum + value is [0.9] + name is moments.model_weight + value is + [[0.2567724 -0.07485991 0.282002 0.2456022 0.454939 0.619168 0.18964815 0.45714882] + [0.25946522 0.24344791 0.45677605 0.3611395 0.23378398 0.41439137 0.5312468 0.4696194] + [0.2417504 0.28193963 0.06713893 0.21510397 0.23380603 0.11424308 0.0218009 -0.11969765] + [0.45955992 0.22664294 0.01990281 0.0731914 0.27125207 0.27298513 -0.01716102 -0.15327111]] + + device1： + name is model_parallel_weight + value is + [[1.0053468 0.98402303 0.99762845 0.97587246 1.0259694 1.0055295 0.99420834 0.9496847] + [1.0851002 1.0295962 1.0999886 1.0958165 0.9765328 1.146529 1.0970603 1.1388365] + [0.7147005 0.9168278 0.80178416 0.6258351 0.8413766 0.5909515 0.696347 0.71359116] + [0.20506378 0.03691584 0.2454556 0.12978578 0.19065076 0.23904312 0.27509746 0.34614682]] + name is learning_rate + value is [0.01] + name is momentum + value is [0.9] + name is moments.model_weight + value is + [[0.03440702 0.41419312 0.24817684 0.30765256 0.48516113 0.24904746 0.57791173 0.00955463] + [0.13458519 0.6690533 0.49259356 0.28319967 0.25951773 0.16777472 0.45696738 0.24933104] + [0.14152306 0.5040985 0.24455397 0.10907605 0.11319532 0.19538902 0.01208619 0.40430856] + [-0.7773164 -0.47611716 -0.6041424 -0.6144473 -0.2651842 -0.31909415 -0.4510405 -0.12860501]] + ``` diff --git a/tutorials/source_zh_cn/advanced_use/computer_vision_application.md b/tutorials/training/source_zh_cn/advanced_use/computer_vision_application.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/computer_vision_application.md rename to tutorials/training/source_zh_cn/advanced_use/computer_vision_application.md diff --git a/tutorials/source_zh_cn/advanced_use/customized_debugging_information.md b/tutorials/training/source_zh_cn/advanced_use/customized_debugging_information.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/customized_debugging_information.md rename to tutorials/training/source_zh_cn/advanced_use/customized_debugging_information.md diff --git a/tutorials/source_zh_cn/advanced_use/dashboard.md b/tutorials/training/source_zh_cn/advanced_use/dashboard.md similarity index 98% rename from tutorials/source_zh_cn/advanced_use/dashboard.md rename to tutorials/training/source_zh_cn/advanced_use/dashboard.md index 898695dfe24b58c764c62099b6188024601f250a..110dfa34fffca00127c668b1534379f5c5da9d3d 100644 --- a/tutorials/source_zh_cn/advanced_use/dashboard.md +++ b/tutorials/training/source_zh_cn/advanced_use/dashboard.md @@ -1,200 +1,200 @@ -# 训练看板 - -`Linux` `Ascend` `GPU` `CPU` `模型调优` `中级` `高级` - - - -- [训练看板](#训练看板) - - [概述](#概述) - - [标量可视化](#标量可视化) - - [参数分布图可视化](#参数分布图可视化) - - [计算图可视化](#计算图可视化) - - [数据图可视化](#数据图可视化) - - [图像可视化](#图像可视化) - - [张量可视化](#张量可视化) - - [注意事项](#注意事项) - - - -   - - -## 概述 - -训练看板是MindInsight的可视化组件的重要组成部分，而训练看板的标签包含：标量可视化、参数分布图可视化、计算图可视化、数据图可视化、图像可视化和张量可视化等。 - -用户从训练列表中选择指定的训练，进入训练看板。 - -## 标量可视化 - -标量可视化用于展示训练过程中，标量的变化趋势情况。 - -![scalar.png](./images/scalar.png) - -图1：标量趋势图 - -图1展示了神经网络在训练过程中损失值的变化过程。横坐标是训练步骤，纵坐标是损失值。 - -图中右上角有几个按钮功能，从左到右功能分别是全屏展示，切换Y轴比例，开启/关闭框选，分步回退和还原图形。 - -- 全屏展示即全屏展示该标量曲线，再点击一次即可恢复。 -- 切换Y轴比例是指可以将Y轴坐标进行对数转换。 -- 开启/关闭框选是指可以框选图中部分区域，并放大查看该区域， 可以在已放大的图形上叠加框选。 -- 分步回退是指对同一个区域连续框选并放大查看时，可以逐步撤销操作。 -- 还原图形是指进行了多次框选后，点击此按钮可以将图还原回原始状态。 - -图中右下角可以设置阈值并高亮显示或者删除阈值。如图所示，设置的阈值为小于1.5，红色高亮部分显示出超出阈值的部分，能够直观地看到预期的数据值或者一些异常的数值。 - -![scalar_select.png](./images/scalar_select.png) - -图2：标量可视化功能区 - -图2展示的标量可视化的功能区，提供了根据选择不同标签，水平轴的不同维度和平滑度来查看标量信息的功能。 - -- 标签选择：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的标量信息。 -- 水平轴：可以选择“步骤”、“相对时间”、“绝对时间”中的任意一项，来作为标量曲线的水平轴。 -- 平滑度：可以通过调整平滑度，对标量曲线进行平滑处理。 -- 标量合成：可以选中两条标量曲线进行合成并展示在一个图中，以方便对两条曲线进行对比或者查看合成后的图。 - -![scalar_compound.png](./images/scalar_compound.png) - -图3：Accuracy和Loss的标量合成图 - -图3展示Accuracy曲线和Loss曲线的标量合成图。标量合成的功能区与标量可视化的功能区相似。其中与标量可视化功能区不一样的地方，在于标签选择时，标量合成功能最多只能同时选择两个标签，将其曲线合成并展示。 - -## 参数分布图可视化 - -参数分布图用于将用户所指定的张量以直方图的形式进行展示。 - -![histogram.png](./images/histogram.png) - -图4: 直方图展示 - -图4将用户所记录的张量以直方图的形式进行展示。点击图中右上角，可以将图放大。 - -![histogram_func.png](./images/histogram_func.png) - -图5: 参数分布图功能区 - -图5展示参数分布图的功能区，包含以下内容: - -- 标签选择：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的直方图。 -- 纵轴：可以选择`步骤`、`相对时间`、`绝对时间`中的任意一项，来作为直方图纵轴显示的数据。 -- 视角：可以选择`正视`和`俯视`中的一种。`正视`是指从正面的角度查看直方图，此时不同步骤之间的数据会覆盖在一起。`俯视`是指偏移以45度角俯视直方图区域，这时可以呈现不同步骤之间数据的差异。 - -## 计算图可视化 - -计算图可视化用于展示计算图的图结构，数据流以及控制流的走向，支持展示summary日志文件与通过`context`的`save_graphs`参数导出的`pb`文件。 - -![graph.png](./images/graph.png) - -图6：计算图展示区 - -图6展示了计算图的网络结构。如图中所展示的，在展示区中，选中其中一个算子（图中圈红算子），可以看到该算子有两个输入和一个输出（实线代表算子的数据流走向）。 - -![graph_sidebar.png](./images/graph_sidebar.png) - -图7：计算图功能区 - -图7展示了计算图可视化的功能区，包含以下内容： - -- 文件选择框: 可以选择查看不同文件的计算图。 -- 搜索框：可以对节点进行搜索，输入节点名称点击回车，即可展示该节点。 -- 缩略图：展示整个网络图结构的缩略图，在查看超大图结构时，方便查看当前浏览的区域。 -- 节点信息：展示选中的节点的基本信息，包括节点的名称、属性、输入节点、输出节点等信息。 -- 图例：展示的是计算图中各个图标的含义。 - -## 数据图可视化 - -数据图可视化用于展示单次模型训练的数据处理和数据增强信息。 - -![data_function.png](./images/data_function.png) - -图8：数据图功能区 - -图8展示的数据图功能区包含以下内容： - -- 图例：展示数据溯源图中各个图标的含义。 -- 数据处理流水线：展示训练所使用的数据处理流水线，可以选择图中的单个节点查看详细信息。 -- 节点信息：展示选中的节点的基本信息，包括使用的数据处理和增强算子的名称、参数等。 - -## 图像可视化 - -图像可视化用于展示用户所指定的图片。 - -![image.png](./images/image_vi.png) - -图9：图像可视化 - -图9展示通过滑动图中“步骤”滑条，查看不同步骤的图片。 - -![image_function.png](./images/image_function.png) - -图10：图像可视化功能区 - -图10展示图像可视化的功能区，提供了选择查看不同标签，不同亮度和不同对比度来查看图片信息。 - -- 标签：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的图片信息。 -- 亮度调整：可以调整所展示的所有图片亮度。 -- 对比度调整：可以调整所展示的所有图片对比度。 - -## 张量可视化 - -张量可视化用于将张量以表格以及直方图的形式进行展示。 - -![tensor_function.png](./images/tensor_function.png) - -图11：张量可视化功能区 - -图11展示张量可视化的功能区，包含以下内容： - -- 标签选择：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的表格数据或者直方图。 -- 视图：可以选择`表格`或者`直方图`来展示tensor数据。在`直方图`视图下存在`纵轴`和`视角`的功能选择。 -- 纵轴：可以选择`步骤`、`相对时间`、`绝对时间`中的任意一项，来作为直方图纵轴显示的数据。 -- 视角：可以选择`正视`和`俯视`中的一种。`正视`是指从正面的角度查看直方图，此时不同步骤之间的数据会覆盖在一起。`俯视`是指偏移以45度角俯视直方图区域，这时可以呈现不同步骤之间数据的差异。 - -![tensor_table.png](./images/tensor_table.png) - -图12：表格展示 - -图12将用户所记录的张量以表格的形式展示，包含以下功能： - -- 点击表格右边小方框按钮，可以将表格放大。 -- 表格中白色方框显示当前展示的是哪个维度下的张量数据，其中冒号`:`表示当前维度的所有值，可以在方框输入对应的索引或者`:`后按`Enter`键或者点击后边的打勾按钮来查询特定维度的张量数据。 - 假设某维度是32，则其索引范围是-32到31。注意：可以查询0维到2维的张量数据，不支持查询超过两维的张量数据，即不能设置超过两个冒号`:`的查询条件。 -- 拖拽表格下方的空心圆圈可以查询特定步骤的张量数据。 - -![tensor_histogram.png](./images/tensor_histogram.png) - -图13: 直方图展示 - -图13将用户所记录的张量以直方图的形式进行展示。点击图中右上角，可以将图放大。 - -## 注意事项 - -1. 目前MindSpore仅支持在Ascend 910 AI处理器上导出算子融合后的计算图。 -2. 在训练中使用Summary算子收集数据时，`HistogramSummary` 算子会影响性能，所以请尽量少地使用。 -3. 为了控制内存占用，MindInsight对标签（tag）数目和步骤（step）数目进行了限制： - - 每个训练看板的最大标签数量为300个标签。标量标签、图片标签、计算图标签、参数分布图（直方图）标签、张量标签的数量总和不得超过300个。特别地，每个训练看板最多有10个计算图标签、6个张量标签。当实际标签数量超过这一限制时，将依照MindInsight的处理顺序，保留最近处理的300个标签。 - - 每个训练看板的每个标量标签最多有1000个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。 - - 每个训练看板的每个图片标签最多有10个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。 - - 每个训练看板的每个参数分布图（直方图）标签最多有50个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。 - - 每个训练看板的每个张量标签最多有20个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。 -4. 由于`TensorSummary`会记录完整Tensor数据，数据量通常会比较大，为了控制内存占用和出于性能上的考虑，MindInsight对Tensor的大小以及返回前端展示的数值个数进行以下限制： - - MindInsight最大支持加载含有1千万个数值的Tensor。 - - Tensor加载后，在张量可视的表格视图下，最大支持查看10万个数值，如果所选择的维度查询得到的数值超过这一限制，则无法显示。 - -5. 由于张量可视（`TensorSummary`）会记录原始张量数据，需要的存储空间较大。使用`TensorSummary`前和训练过程中请注意检查系统存储空间充足。 - - 通过以下方法可以降低张量可视功能的存储空间占用： - - 1）避免使用`TensorSummary`记录较大的Tensor。 - - 2）减少网络中`TensorSummary`算子的使用个数。 - - 功能使用完毕后，请及时清理不再需要的训练日志，以释放磁盘空间。 - - 备注：估算`TensorSummary`空间使用量的方法如下： - - 一个`TensorSummary数据的大小 ＝ Tensor中的数值个数 * 4 bytes`。假设使用`TensorSummary`记录的Tensor大小为`32 * 1 * 256 * 256`，则一个`TensorSummary`数据大约需要`32 * 1 * 256 * 256 * 4 bytes = 8,388,608 bytes = 8MiB`。`TensorSummary`默认会记录20个步骤的数据，则记录这20组数据需要的空间约为`20 * 8 MiB ＝ 160MiB`。需要注意的是，由于数据结构等因素的开销，实际使用的存储空间会略大于160MiB。 +# 训练看板 + +`Linux` `Ascend` `GPU` `CPU` `模型调优` `中级` `高级` + + + +- [训练看板](#训练看板) + - [概述](#概述) + - [标量可视化](#标量可视化) + - [参数分布图可视化](#参数分布图可视化) + - [计算图可视化](#计算图可视化) + - [数据图可视化](#数据图可视化) + - [图像可视化](#图像可视化) + - [张量可视化](#张量可视化) + - [注意事项](#注意事项) + + + +   + + +## 概述 + +训练看板是MindInsight的可视化组件的重要组成部分，而训练看板的标签包含：标量可视化、参数分布图可视化、计算图可视化、数据图可视化、图像可视化和张量可视化等。 + +用户从训练列表中选择指定的训练，进入训练看板。 + +## 标量可视化 + +标量可视化用于展示训练过程中，标量的变化趋势情况。 + +![scalar.png](./images/scalar.png) + +图1：标量趋势图 + +图1展示了神经网络在训练过程中损失值的变化过程。横坐标是训练步骤，纵坐标是损失值。 + +图中右上角有几个按钮功能，从左到右功能分别是全屏展示，切换Y轴比例，开启/关闭框选，分步回退和还原图形。 + +- 全屏展示即全屏展示该标量曲线，再点击一次即可恢复。 +- 切换Y轴比例是指可以将Y轴坐标进行对数转换。 +- 开启/关闭框选是指可以框选图中部分区域，并放大查看该区域， 可以在已放大的图形上叠加框选。 +- 分步回退是指对同一个区域连续框选并放大查看时，可以逐步撤销操作。 +- 还原图形是指进行了多次框选后，点击此按钮可以将图还原回原始状态。 + +图中右下角可以设置阈值并高亮显示或者删除阈值。如图所示，设置的阈值为小于1.5，红色高亮部分显示出超出阈值的部分，能够直观地看到预期的数据值或者一些异常的数值。 + +![scalar_select.png](./images/scalar_select.png) + +图2：标量可视化功能区 + +图2展示的标量可视化的功能区，提供了根据选择不同标签，水平轴的不同维度和平滑度来查看标量信息的功能。 + +- 标签选择：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的标量信息。 +- 水平轴：可以选择“步骤”、“相对时间”、“绝对时间”中的任意一项，来作为标量曲线的水平轴。 +- 平滑度：可以通过调整平滑度，对标量曲线进行平滑处理。 +- 标量合成：可以选中两条标量曲线进行合成并展示在一个图中，以方便对两条曲线进行对比或者查看合成后的图。 + +![scalar_compound.png](./images/scalar_compound.png) + +图3：Accuracy和Loss的标量合成图 + +图3展示Accuracy曲线和Loss曲线的标量合成图。标量合成的功能区与标量可视化的功能区相似。其中与标量可视化功能区不一样的地方，在于标签选择时，标量合成功能最多只能同时选择两个标签，将其曲线合成并展示。 + +## 参数分布图可视化 + +参数分布图用于将用户所指定的张量以直方图的形式进行展示。 + +![histogram.png](./images/histogram.png) + +图4: 直方图展示 + +图4将用户所记录的张量以直方图的形式进行展示。点击图中右上角，可以将图放大。 + +![histogram_func.png](./images/histogram_func.png) + +图5: 参数分布图功能区 + +图5展示参数分布图的功能区，包含以下内容: + +- 标签选择：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的直方图。 +- 纵轴：可以选择`步骤`、`相对时间`、`绝对时间`中的任意一项，来作为直方图纵轴显示的数据。 +- 视角：可以选择`正视`和`俯视`中的一种。`正视`是指从正面的角度查看直方图，此时不同步骤之间的数据会覆盖在一起。`俯视`是指偏移以45度角俯视直方图区域，这时可以呈现不同步骤之间数据的差异。 + +## 计算图可视化 + +计算图可视化用于展示计算图的图结构，数据流以及控制流的走向，支持展示summary日志文件与通过`context`的`save_graphs`参数导出的`pb`文件。 + +![graph.png](./images/graph.png) + +图6：计算图展示区 + +图6展示了计算图的网络结构。如图中所展示的，在展示区中，选中其中一个算子（图中圈红算子），可以看到该算子有两个输入和一个输出（实线代表算子的数据流走向）。 + +![graph_sidebar.png](./images/graph_sidebar.png) + +图7：计算图功能区 + +图7展示了计算图可视化的功能区，包含以下内容： + +- 文件选择框: 可以选择查看不同文件的计算图。 +- 搜索框：可以对节点进行搜索，输入节点名称点击回车，即可展示该节点。 +- 缩略图：展示整个网络图结构的缩略图，在查看超大图结构时，方便查看当前浏览的区域。 +- 节点信息：展示选中的节点的基本信息，包括节点的名称、属性、输入节点、输出节点等信息。 +- 图例：展示的是计算图中各个图标的含义。 + +## 数据图可视化 + +数据图可视化用于展示单次模型训练的数据处理和数据增强信息。 + +![data_function.png](./images/data_function.png) + +图8：数据图功能区 + +图8展示的数据图功能区包含以下内容： + +- 图例：展示数据溯源图中各个图标的含义。 +- 数据处理流水线：展示训练所使用的数据处理流水线，可以选择图中的单个节点查看详细信息。 +- 节点信息：展示选中的节点的基本信息，包括使用的数据处理和增强算子的名称、参数等。 + +## 图像可视化 + +图像可视化用于展示用户所指定的图片。 + +![image.png](./images/image_vi.png) + +图9：图像可视化 + +图9展示通过滑动图中“步骤”滑条，查看不同步骤的图片。 + +![image_function.png](./images/image_function.png) + +图10：图像可视化功能区 + +图10展示图像可视化的功能区，提供了选择查看不同标签，不同亮度和不同对比度来查看图片信息。 + +- 标签：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的图片信息。 +- 亮度调整：可以调整所展示的所有图片亮度。 +- 对比度调整：可以调整所展示的所有图片对比度。 + +## 张量可视化 + +张量可视化用于将张量以表格以及直方图的形式进行展示。 + +![tensor_function.png](./images/tensor_function.png) + +图11：张量可视化功能区 + +图11展示张量可视化的功能区，包含以下内容： + +- 标签选择：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的表格数据或者直方图。 +- 视图：可以选择`表格`或者`直方图`来展示tensor数据。在`直方图`视图下存在`纵轴`和`视角`的功能选择。 +- 纵轴：可以选择`步骤`、`相对时间`、`绝对时间`中的任意一项，来作为直方图纵轴显示的数据。 +- 视角：可以选择`正视`和`俯视`中的一种。`正视`是指从正面的角度查看直方图，此时不同步骤之间的数据会覆盖在一起。`俯视`是指偏移以45度角俯视直方图区域，这时可以呈现不同步骤之间数据的差异。 + +![tensor_table.png](./images/tensor_table.png) + +图12：表格展示 + +图12将用户所记录的张量以表格的形式展示，包含以下功能： + +- 点击表格右边小方框按钮，可以将表格放大。 +- 表格中白色方框显示当前展示的是哪个维度下的张量数据，其中冒号`:`表示当前维度的所有值，可以在方框输入对应的索引或者`:`后按`Enter`键或者点击后边的打勾按钮来查询特定维度的张量数据。 + 假设某维度是32，则其索引范围是-32到31。注意：可以查询0维到2维的张量数据，不支持查询超过两维的张量数据，即不能设置超过两个冒号`:`的查询条件。 +- 拖拽表格下方的空心圆圈可以查询特定步骤的张量数据。 + +![tensor_histogram.png](./images/tensor_histogram.png) + +图13: 直方图展示 + +图13将用户所记录的张量以直方图的形式进行展示。点击图中右上角，可以将图放大。 + +## 注意事项 + +1. 目前MindSpore仅支持在Ascend 910 AI处理器上导出算子融合后的计算图。 +2. 在训练中使用Summary算子收集数据时，`HistogramSummary` 算子会影响性能，所以请尽量少地使用。 +3. 为了控制内存占用，MindInsight对标签（tag）数目和步骤（step）数目进行了限制： + - 每个训练看板的最大标签数量为300个标签。标量标签、图片标签、计算图标签、参数分布图（直方图）标签、张量标签的数量总和不得超过300个。特别地，每个训练看板最多有10个计算图标签、6个张量标签。当实际标签数量超过这一限制时，将依照MindInsight的处理顺序，保留最近处理的300个标签。 + - 每个训练看板的每个标量标签最多有1000个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。 + - 每个训练看板的每个图片标签最多有10个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。 + - 每个训练看板的每个参数分布图（直方图）标签最多有50个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。 + - 每个训练看板的每个张量标签最多有20个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。 +4. 由于`TensorSummary`会记录完整Tensor数据，数据量通常会比较大，为了控制内存占用和出于性能上的考虑，MindInsight对Tensor的大小以及返回前端展示的数值个数进行以下限制： + - MindInsight最大支持加载含有1千万个数值的Tensor。 + - Tensor加载后，在张量可视的表格视图下，最大支持查看10万个数值，如果所选择的维度查询得到的数值超过这一限制，则无法显示。 + +5. 由于张量可视（`TensorSummary`）会记录原始张量数据，需要的存储空间较大。使用`TensorSummary`前和训练过程中请注意检查系统存储空间充足。 + + 通过以下方法可以降低张量可视功能的存储空间占用： + + 1）避免使用`TensorSummary`记录较大的Tensor。 + + 2）减少网络中`TensorSummary`算子的使用个数。 + + 功能使用完毕后，请及时清理不再需要的训练日志，以释放磁盘空间。 + + 备注：估算`TensorSummary`空间使用量的方法如下： + + 一个`TensorSummary数据的大小 ＝ Tensor中的数值个数 * 4 bytes`。假设使用`TensorSummary`记录的Tensor大小为`32 * 1 * 256 * 256`，则一个`TensorSummary`数据大约需要`32 * 1 * 256 * 256 * 4 bytes = 8,388,608 bytes = 8MiB`。`TensorSummary`默认会记录20个步骤的数据，则记录这20组数据需要的空间约为`20 * 8 MiB ＝ 160MiB`。需要注意的是，由于数据结构等因素的开销，实际使用的存储空间会略大于160MiB。 6. 当使用`TensorSummary`时，由于记录完整Tensor数据，训练日志文件较大，MindInsight需要更多时间解析训练日志文件，请耐心等待。 \ No newline at end of file diff --git a/tutorials/source_zh_cn/advanced_use/data_processing_acceleration.md b/tutorials/training/source_zh_cn/advanced_use/data_processing_acceleration.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/data_processing_acceleration.md rename to tutorials/training/source_zh_cn/advanced_use/data_processing_acceleration.md diff --git a/tutorials/source_zh_cn/advanced_use/dataset_conversion.md b/tutorials/training/source_zh_cn/advanced_use/dataset_conversion.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/dataset_conversion.md rename to tutorials/training/source_zh_cn/advanced_use/dataset_conversion.md diff --git a/tutorials/source_zh_cn/advanced_use/debugging_in_pynative_mode.md b/tutorials/training/source_zh_cn/advanced_use/debugging_in_pynative_mode.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/debugging_in_pynative_mode.md rename to tutorials/training/source_zh_cn/advanced_use/debugging_in_pynative_mode.md diff --git a/tutorials/source_zh_cn/advanced_use/deep_probability_program.md b/tutorials/training/source_zh_cn/advanced_use/deep_probability_program.md similarity index 97% rename from tutorials/source_zh_cn/advanced_use/deep_probability_program.md rename to tutorials/training/source_zh_cn/advanced_use/deep_probability_program.md index 13e92b1452da668a0c9f9632c0bcbf307681ff9c..93436976f5dff3ab0601952b319a02aad75e3d93 100644 --- a/tutorials/source_zh_cn/advanced_use/deep_probability_program.md +++ b/tutorials/training/source_zh_cn/advanced_use/deep_probability_program.md @@ -1,432 +1,432 @@ -# 深度概率编程 -`Ascend` `GPU` `全流程` `初级` `中级` `高级` - - - -- [深度概率编程](#深度概率编程) - - [概述](#概述) - - [贝叶斯神经网络](#贝叶斯神经网络) - - [处理数据集](#处理数据集) - - [定义损失函数和优化器](#定义损失函数和优化器) - - [训练网络](#训练网络) - - [变分推断](#变分推断) - - [定义变分自编码器](#定义变分自编码器) - - [定义损失函数和优化器](#定义损失函数和优化器) - - [处理数据](#处理数据) - - [训练网络](#训练网络) - - [DNN一键转换成BNN](#DNN一键转换成BNN) - - [定义DNN模型](#定义DNN模型) - - [定义损失函数和优化器](#定义损失函数和优化器) - - [功能一：转换整个模型](#功能一：转换整个模型) - - [功能二：转换指定类型的层](#功能二：转换指定类型的层) - - [不确定性估计](#不确定性估计) - - - -## 概述 -MindSpore深度概率编程（MindSpore Deep Probabilistic Programming, MDP）的目标是将深度学习和贝叶斯学习结合，并能面向不同的开发者。具体来说，对于专业的贝叶斯学习用户，提供概率采样、推理算法和模型构建库；另一方面，为不熟悉贝叶斯深度学习的用户提供了高级的API，从而不用更改深度学习编程逻辑，即可利用贝叶斯模型。 - -本章将详细介绍深度概率编程在MindSpore上的应用。 - -### 贝叶斯神经网络 -本例子利用贝叶斯神经网络实现一个简单的图片分类功能，整体流程如下： -1. 处理MNIST数据集。 -2. 定义贝叶斯LeNet网络。 -3. 定义损失函数和优化器。 -4. 加载数据集并进行训练。 - -#### 处理数据集 -本例子使用的是MNIST数据集，数据处理过程与教程中的[实现一个图片分类应用](https://www.mindspore.cn/tutorial/zh-CN/master/quick_start/quick_start.html)一致。 - -#### 定义贝叶斯神经网络 -本例子使用的是贝叶斯LeNet。利用bnn_layers构建贝叶斯神经网络的方法与构建普通的神经网络相同。值得注意的是，bnn_layers和普通的神经网络层可以互相组合。 - -``` -import mindspore.nn as nn -from mindspore.nn.probability import bnn_layers -import mindspore.ops.operations as P - -class BNNLeNet5(nn.Cell): - """ - bayesian Lenet network - - Args: - num_class (int): Num classes. Default: 10. - - Returns: - Tensor, output tensor - Examples: - >>> BNNLeNet5(num_class=10) - - """ - def __init__(self, num_class=10): - super(BNNLeNet5, self).__init__() - self.num_class = num_class - self.conv1 = bnn_layers.ConvReparam(1, 6, 5, stride=1, padding=0, has_bias=False, pad_mode="valid") - self.conv2 = bnn_layers.ConvReparam(6, 16, 5, stride=1, padding=0, has_bias=False, pad_mode="valid") - self.fc1 = bnn_layers.DenseReparam(16 * 5 * 5, 120) - self.fc2 = bnn_layers.DenseReparam(120, 84) - self.fc3 = bnn_layers.DenseReparam(84, self.num_class) - self.relu = nn.ReLU() - self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) - self.flatten = nn.Flatten() - self.reshape = P.Reshape() - - def construct(self, x): - x = self.conv1(x) - x = self.relu(x) - x = self.max_pool2d(x) - x = self.conv2(x) - x = self.relu(x) - x = self.max_pool2d(x) - x = self.flatten(x) - x = self.fc1(x) - x = self.relu(x) - x = self.fc2(x) - x = self.relu(x) - x = self.fc3(x) - return x -``` -#### 定义损失函数和优化器 -接下来需要定义损失函数（Loss）和优化器（Optimizer）。损失函数是深度学习的训练目标，也叫目标函数，可以理解为神经网络的输出（Logits）和标签(Labels)之间的距离，是一个标量数据。 -常见的损失函数包括均方误差、L2损失、Hinge损失、交叉熵等等。图像分类应用通常采用交叉熵损失（CrossEntropy）。 -优化器用于神经网络求解（训练）。由于神经网络参数规模庞大，无法直接求解，因而深度学习中采用随机梯度下降算法（SGD）及其改进算法进行求解。MindSpore封装了常见的优化器，如SGD、Adam、Momemtum等等。本例采用Adam优化器，通常需要设定两个参数，学习率（learnin _rate）和权重衰减项（weight decay）。 -MindSpore中定义损失函数和优化器的代码样例如下： - -``` -# loss function definition -criterion = SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") - -# optimization definition -optimizer = AdamWeightDecay(params=network.trainable_params(), learning_rate=0.0001) -``` - -#### 训练网络 -贝叶斯神经网络的训练过程与DNN基本相同，唯一不同的是将`WithLossCell`替换为适用于BNN的`WithBNNLossCell`。除了`backbone`和`loss_fn`两个参数之外，`WithBNNLossCell`增加了`dnn_factor`和`bnn_factor`两个参数。`dnn_factor`是由损失函数计算得到的网络整体损失的系数，`bnn_factor`是每个贝叶斯层的KL散度的系数，这两个参数是用来平衡网络整体损失和贝叶斯层的KL散度的，防止KL散度的值过大掩盖了网络整体损失。 - -``` -net_with_loss = bnn_layers.WithBNNLossCell(network, criterion, dnn_factor=60000, bnn_factor=0.000001) -train_bnn_network = TrainOneStepCell(net_with_loss, optimizer) -train_bnn_network.set_train() - -train_set = create_dataset('./mnist_data/train', 64, 1) -test_set = create_dataset('./mnist_data/test', 64, 1) - -epoch = 10 - -for i in range(epoch): - train_loss, train_acc = train_model(train_bnn_network, network, train_set) - - valid_acc = validate_model(network, test_set) - - print('Epoch: {} \tTraining Loss: {:.4f} \tTraining Accuracy: {:.4f} \tvalidation Accuracy: {:.4f}'. - format(i, train_loss, train_acc, valid_acc)) -``` -其中，`train_model`和`validate_model`在MindSpore中的代码样例如下： - -``` -def train_model(train_net, net, dataset): - accs = [] - loss_sum = 0 - for _, data in enumerate(dataset.create_dict_iterator()): - train_x = Tensor(data['image'].astype(np.float32)) - label = Tensor(data['label'].astype(np.int32)) - loss = train_net(train_x, label) - output = net(train_x) - log_output = P.LogSoftmax(axis=1)(output) - acc = np.mean(log_output.asnumpy().argmax(axis=1) == label.asnumpy()) - accs.append(acc) - loss_sum += loss.asnumpy() - - loss_sum = loss_sum / len(accs) - acc_mean = np.mean(accs) - return loss_sum, acc_mean - - -def validate_model(net, dataset): - accs = [] - for _, data in enumerate(dataset.create_dict_iterator()): - train_x = Tensor(data['image'].astype(np.float32)) - label = Tensor(data['label'].astype(np.int32)) - output = net(train_x) - log_output = P.LogSoftmax(axis=1)(output) - acc = np.mean(log_output.asnumpy().argmax(axis=1) == label.asnumpy()) - accs.append(acc) - - acc_mean = np.mean(accs) - return acc_mean -``` - -### 变分推断 -#### 定义变分自编码器 -我们只需要自定义编码器和解码器，编码器和解码器都是神经网络。 - -``` -class Encoder(nn.Cell): - def __init__(self): - super(Encoder, self).__init__() - self.fc1 = nn.Dense(1024, 800) - self.fc2 = nn.Dense(800, 400) - self.relu = nn.ReLU() - self.flatten = nn.Flatten() - - def construct(self, x): - x = self.flatten(x) - x = self.fc1(x) - x = self.relu(x) - x = self.fc2(x) - x = self.relu(x) - return x - - -class Decoder(nn.Cell): - def __init__(self): - super(Decoder, self).__init__() - self.fc1 = nn.Dense(400, 1024) - self.sigmoid = nn.Sigmoid() - self.reshape = P.Reshape() - - def construct(self, z): - z = self.fc1(z) - z = self.reshape(z, IMAGE_SHAPE) - z = self.sigmoid(z) - return z - - -encoder = Encoder() -decoder = Decoder() -vae = VAE(encoder, decoder, hidden_size=400, latent_size=20) -``` -### 定义损失函数和优化器 -接下来需要定义损失函数（Loss）和优化器（Optimizer）。本例使用的损失函数是ELBO，ELBO是变分推断专用的损失函数；本例使用的优化器是Adam。 -MindSpore中定义损失函数和优化器的代码样例如下： - -``` -# loss function definition -net_loss = ELBO(latent_prior='Normal', output_prior='Normal') - -# optimization definition -optimizer = nn.Adam(params=vae.trainable_params(), learning_rate=0.001) - -net_with_loss = nn.WithLossCell(vae, net_loss) -``` -#### 处理数据 -本例使用的是MNIST数据集，数据处理过程与教程中的[实现一个图片分类应用](https://www.mindspore.cn/tutorial/zh-CN/master/quick_start/quick_start.html)一致。 - -#### 训练网络 -使用`SVI`接口对VAE网络进行训练。 - -``` -from mindspore.nn.probability.infer import SVI - -vi = SVI(net_with_loss=net_with_loss, optimizer=optimizer) -vae = vi.run(train_dataset=ds_train, epochs=10) -trained_loss = vi.get_train_loss() -``` -通过`vi.run`可以得到训练好的网络，使用`vi.get_train_loss`可以得到训练之后的损失。 -#### 生成新样本或重构输入样本 -利用训练好的VAE网络，我们可以生成新的样本或重构输入样本。 - -``` -IMAGE_SHAPE = (-1, 1, 32, 32) -generated_sample = vae.generate_sample(64, IMAGE_SHAPE) -for sample in ds_train.create_dict_iterator(): - sample_x = Tensor(sample['image'], dtype=mstype.float32) - reconstructed_sample = vae.reconstruct_sample(sample_x) -``` - -### DNN一键转换成BNN -对于不熟悉贝叶斯模型的DNN研究人员，MDP提供了高级API`TransformToBNN`，支持DNN模型一键转换成BNN模型。 -#### 定义DNN模型 -本例使用的DNN模型是LeNet。 - -``` -from mindspore.common.initializer import TruncatedNormal -import mindspore.nn as nn -import mindspore.ops.operations as P - -def conv(in_channels, out_channels, kernel_size, stride=1, padding=0): - """weight initial for conv layer""" - weight = weight_variable() - return nn.Conv2d(in_channels, out_channels, - kernel_size=kernel_size, stride=stride, padding=padding, - weight_init=weight, has_bias=False, pad_mode="valid") - - -def fc_with_initialize(input_channels, out_channels): - """weight initial for fc layer""" - weight = weight_variable() - bias = weight_variable() - return nn.Dense(input_channels, out_channels, weight, bias) - - -def weight_variable(): - """weight initial""" - return TruncatedNormal(0.02) - - -class LeNet5(nn.Cell): - """ - Lenet network - - Args: - num_class (int): Num classes. Default: 10. - - Returns: - Tensor, output tensor - Examples: - >>> LeNet5(num_class=10) - - """ - def __init__(self, num_class=10): - super(LeNet5, self).__init__() - self.num_class = num_class - self.conv1 = conv(1, 6, 5) - self.conv2 = conv(6, 16, 5) - self.fc1 = fc_with_initialize(16 * 5 * 5, 120) - self.fc2 = fc_with_initialize(120, 84) - self.fc3 = fc_with_initialize(84, self.num_class) - self.relu = nn.ReLU() - self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) - self.flatten = nn.Flatten() - self.reshape = P.Reshape() - - def construct(self, x): - x = self.conv1(x) - x = self.relu(x) - x = self.max_pool2d(x) - x = self.conv2(x) - x = self.relu(x) - x = self.max_pool2d(x) - x = self.flatten(x) - x = self.fc1(x) - x = self.relu(x) - x = self.fc2(x) - x = self.relu(x) - x = self.fc3(x) - return x -``` -#### 定义损失函数和优化器 -接下来需要定义损失函数（Loss）和优化器（Optimizer）。本例使用交叉熵损失作为损失函数，Adam作为优化器。 - -``` -network = LeNet5() - -# loss function definition -criterion = SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") - -# optimization definition -optimizer = AdamWeightDecay(params=network.trainable_params(), learning_rate=0.0001) - -net_with_loss = WithLossCell(network, criterion) -train_network = TrainOneStepCell(net_with_loss, optimizer) -``` -#### 实例化TransformToBNN -`TransformToBNN`的`__init__`函数定义如下： - -``` -class TransformToBNN: - def __init__(self, trainable_dnn, dnn_factor=1, bnn_factor=1): - net_with_loss = trainable_dnn.network - self.optimizer = trainable_dnn.optimizer - self.backbone = net_with_loss.backbone_network - self.loss_fn = getattr(net_with_loss, "_loss_fn") - self.dnn_factor = dnn_factor - self.bnn_factor = bnn_factor - self.bnn_loss_file = None -``` -参数`trainable_bnn`是经过`TrainOneStepCell`包装的可训练DNN模型，`dnn_factor`和`bnn_factor`分别为由损失函数计算得到的网络整体损失的系数和每个贝叶斯层的KL散度的系数。 -MindSpore中实例化`TransformToBNN`的代码如下： - -``` -from mindspore.nn.probability import transforms - -bnn_transformer = transforms.TransformToBNN(train_network, 60000, 0.000001) -``` -#### 功能一：转换整个模型 -`transform_to_bnn_model`方法可以将整个DNN模型转换为BNN模型。其定义如下: - -``` - def transform_to_bnn_model(self, - get_dense_args=lambda dp: {"in_channels": dp.in_channels, "has_bias": dp.has_bias, - "out_channels": dp.out_channels, "activation": dp.activation}, - get_conv_args=lambda dp: {"in_channels": dp.in_channels, "out_channels": dp.out_channels, - "pad_mode": dp.pad_mode, "kernel_size": dp.kernel_size, - "stride": dp.stride, "has_bias": dp.has_bias, - "padding": dp.padding, "dilation": dp.dilation, - "group": dp.group}, - add_dense_args=None, - add_conv_args=None): - r""" - Transform the whole DNN model to BNN model, and wrap BNN model by TrainOneStepCell. - - Args: - get_dense_args (function): The arguments gotten from the DNN full connection layer. Default: lambda dp: - {"in_channels": dp.in_channels, "out_channels": dp.out_channels, "has_bias": dp.has_bias}. - get_conv_args (function): The arguments gotten from the DNN convolutional layer. Default: lambda dp: - {"in_channels": dp.in_channels, "out_channels": dp.out_channels, "pad_mode": dp.pad_mode, - "kernel_size": dp.kernel_size, "stride": dp.stride, "has_bias": dp.has_bias}. - add_dense_args (dict): The new arguments added to BNN full connection layer. Default: {}. - add_conv_args (dict): The new arguments added to BNN convolutional layer. Default: {}. - - Returns: - Cell, a trainable BNN model wrapped by TrainOneStepCell. - """ -``` -参数`get_dense_args`指定从DNN模型的全连接层中获取哪些参数，`get_conv_args`指定从DNN模型的卷积层中获取哪些参数，参数`add_dense_args`和`add_conv_args`分别指定了要为BNN层指定哪些新的参数值。需要注意的是，`add_dense_args`中的参数不能与`get_dense_args`重复，`add_conv_args`和`get_conv_args`也是如此。 -在MindSpore中将整个DNN模型转换成BNN模型的代码如下： - -``` -train_bnn_network = bnn_transformer.transform_to_bnn_model() -``` -#### 功能二：转换指定类型的层 -`transform_to_bnn_layer`方法可以将DNN模型中指定类型的层（nn.Dense或者nn.Conv2d）转换为对应的贝叶斯层。其定义如下: - -``` - def transform_to_bnn_layer(self, dnn_layer, bnn_layer, get_args=None, add_args=None): - r""" - Transform a specific type of layers in DNN model to corresponding BNN layer. - - Args: - dnn_layer_type (Cell): The type of DNN layer to be transformed to BNN layer. The optional values are - nn.Dense, nn.Conv2d. - bnn_layer_type (Cell): The type of BNN layer to be transformed to. The optional values are - DenseReparameterization, ConvReparameterization. - get_args (dict): The arguments gotten from the DNN layer. Default: None. - add_args (dict): The new arguments added to BNN layer. Default: None. - - Returns: - Cell, a trainable model wrapped by TrainOneStepCell, whose sprcific type of layer is transformed to the corresponding bayesian layer. - """ -``` -参数`dnn_layer`指定将哪个类型的DNN层转换成BNN层，`bnn_layer`指定DNN层将转换成哪个类型的BNN层，`get_args`和`add_args`分别指定从DNN层中获取哪些参数和要为BNN层的哪些参数重新赋值。 - -### 不确定性估计 -不确定性估计工具箱基于MindSpore Deep probability Programming (MDP)，适用于主流的深度学习模型，如回归、分类、目标检测等。在推理阶段，利用不确定性估计工具箱，开发人员只需通过训练模型和训练数据集，指定需要估计的任务和样本，即可得到任意不确定性（aleatoric uncertainty）和认知不确定性（epistemic uncertainty）。基于不确定性信息，开发人员可以更好地理解模型和数据集。 -以分类任务为例，本例中使用的模型是LeNet，数据集为MNist，数据处理过程与教程中的[实现一个图片分类应用](https://www.mindspore.cn/tutorial/zh-CN/master/quick_start/quick_start.html)一致。为了评估测试示例的不确定性，使用工具箱的方法如下: - -``` -from mindspore.nn.probability.toolbox.uncertainty_evaluation import UncertaintyEvaluation -from mindspore.train.serialization import load_checkpoint, load_param_into_net - -network = LeNet5() -param_dict = load_checkpoint('checkpoint_lenet.ckpt') -load_param_into_net(network, param_dict) -# get train and eval dataset -ds_train = create_dataset('workspace/mnist/train') -ds_eval = create_dataset('workspace/mnist/test') -evaluation = UncertaintyEvaluation(model=network, - train_dataset=ds_train, - task_type='classification', - num_classes=10, - epochs=1, - epi_uncer_model_path=None, - ale_uncer_model_path=None, - save_model=False) -for eval_data in ds_eval.create_dict_iterator(): - eval_data = Tensor(eval_data['image'], mstype.float32) - epistemic_uncertainty = evaluation.eval_epistemic_uncertainty(eval_data) - aleatoric_uncertainty = evaluation.eval_aleatoric_uncertainty(eval_data) -``` - - +# 深度概率编程 +`Ascend` `GPU` `全流程` `初级` `中级` `高级` + + + +- [深度概率编程](#深度概率编程) + - [概述](#概述) + - [贝叶斯神经网络](#贝叶斯神经网络) + - [处理数据集](#处理数据集) + - [定义损失函数和优化器](#定义损失函数和优化器) + - [训练网络](#训练网络) + - [变分推断](#变分推断) + - [定义变分自编码器](#定义变分自编码器) + - [定义损失函数和优化器](#定义损失函数和优化器) + - [处理数据](#处理数据) + - [训练网络](#训练网络) + - [DNN一键转换成BNN](#DNN一键转换成BNN) + - [定义DNN模型](#定义DNN模型) + - [定义损失函数和优化器](#定义损失函数和优化器) + - [功能一：转换整个模型](#功能一：转换整个模型) + - [功能二：转换指定类型的层](#功能二：转换指定类型的层) + - [不确定性估计](#不确定性估计) + + + +## 概述 +MindSpore深度概率编程（MindSpore Deep Probabilistic Programming, MDP）的目标是将深度学习和贝叶斯学习结合，并能面向不同的开发者。具体来说，对于专业的贝叶斯学习用户，提供概率采样、推理算法和模型构建库；另一方面，为不熟悉贝叶斯深度学习的用户提供了高级的API，从而不用更改深度学习编程逻辑，即可利用贝叶斯模型。 + +本章将详细介绍深度概率编程在MindSpore上的应用。 + +### 贝叶斯神经网络 +本例子利用贝叶斯神经网络实现一个简单的图片分类功能，整体流程如下： +1. 处理MNIST数据集。 +2. 定义贝叶斯LeNet网络。 +3. 定义损失函数和优化器。 +4. 加载数据集并进行训练。 + +#### 处理数据集 +本例子使用的是MNIST数据集，数据处理过程与教程中的[实现一个图片分类应用](https://www.mindspore.cn/tutorial/zh-CN/master/quick_start/quick_start.html)一致。 + +#### 定义贝叶斯神经网络 +本例子使用的是贝叶斯LeNet。利用bnn_layers构建贝叶斯神经网络的方法与构建普通的神经网络相同。值得注意的是，bnn_layers和普通的神经网络层可以互相组合。 + +``` +import mindspore.nn as nn +from mindspore.nn.probability import bnn_layers +import mindspore.ops.operations as P + +class BNNLeNet5(nn.Cell): + """ + bayesian Lenet network + + Args: + num_class (int): Num classes. Default: 10. + + Returns: + Tensor, output tensor + Examples: + >>> BNNLeNet5(num_class=10) + + """ + def __init__(self, num_class=10): + super(BNNLeNet5, self).__init__() + self.num_class = num_class + self.conv1 = bnn_layers.ConvReparam(1, 6, 5, stride=1, padding=0, has_bias=False, pad_mode="valid") + self.conv2 = bnn_layers.ConvReparam(6, 16, 5, stride=1, padding=0, has_bias=False, pad_mode="valid") + self.fc1 = bnn_layers.DenseReparam(16 * 5 * 5, 120) + self.fc2 = bnn_layers.DenseReparam(120, 84) + self.fc3 = bnn_layers.DenseReparam(84, self.num_class) + self.relu = nn.ReLU() + self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) + self.flatten = nn.Flatten() + self.reshape = P.Reshape() + + def construct(self, x): + x = self.conv1(x) + x = self.relu(x) + x = self.max_pool2d(x) + x = self.conv2(x) + x = self.relu(x) + x = self.max_pool2d(x) + x = self.flatten(x) + x = self.fc1(x) + x = self.relu(x) + x = self.fc2(x) + x = self.relu(x) + x = self.fc3(x) + return x +``` +#### 定义损失函数和优化器 +接下来需要定义损失函数（Loss）和优化器（Optimizer）。损失函数是深度学习的训练目标，也叫目标函数，可以理解为神经网络的输出（Logits）和标签(Labels)之间的距离，是一个标量数据。 +常见的损失函数包括均方误差、L2损失、Hinge损失、交叉熵等等。图像分类应用通常采用交叉熵损失（CrossEntropy）。 +优化器用于神经网络求解（训练）。由于神经网络参数规模庞大，无法直接求解，因而深度学习中采用随机梯度下降算法（SGD）及其改进算法进行求解。MindSpore封装了常见的优化器，如SGD、Adam、Momemtum等等。本例采用Adam优化器，通常需要设定两个参数，学习率（learnin _rate）和权重衰减项（weight decay）。 +MindSpore中定义损失函数和优化器的代码样例如下： + +``` +# loss function definition +criterion = SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") + +# optimization definition +optimizer = AdamWeightDecay(params=network.trainable_params(), learning_rate=0.0001) +``` + +#### 训练网络 +贝叶斯神经网络的训练过程与DNN基本相同，唯一不同的是将`WithLossCell`替换为适用于BNN的`WithBNNLossCell`。除了`backbone`和`loss_fn`两个参数之外，`WithBNNLossCell`增加了`dnn_factor`和`bnn_factor`两个参数。`dnn_factor`是由损失函数计算得到的网络整体损失的系数，`bnn_factor`是每个贝叶斯层的KL散度的系数，这两个参数是用来平衡网络整体损失和贝叶斯层的KL散度的，防止KL散度的值过大掩盖了网络整体损失。 + +``` +net_with_loss = bnn_layers.WithBNNLossCell(network, criterion, dnn_factor=60000, bnn_factor=0.000001) +train_bnn_network = TrainOneStepCell(net_with_loss, optimizer) +train_bnn_network.set_train() + +train_set = create_dataset('./mnist_data/train', 64, 1) +test_set = create_dataset('./mnist_data/test', 64, 1) + +epoch = 10 + +for i in range(epoch): + train_loss, train_acc = train_model(train_bnn_network, network, train_set) + + valid_acc = validate_model(network, test_set) + + print('Epoch: {} \tTraining Loss: {:.4f} \tTraining Accuracy: {:.4f} \tvalidation Accuracy: {:.4f}'. + format(i, train_loss, train_acc, valid_acc)) +``` +其中，`train_model`和`validate_model`在MindSpore中的代码样例如下： + +``` +def train_model(train_net, net, dataset): + accs = [] + loss_sum = 0 + for _, data in enumerate(dataset.create_dict_iterator()): + train_x = Tensor(data['image'].astype(np.float32)) + label = Tensor(data['label'].astype(np.int32)) + loss = train_net(train_x, label) + output = net(train_x) + log_output = P.LogSoftmax(axis=1)(output) + acc = np.mean(log_output.asnumpy().argmax(axis=1) == label.asnumpy()) + accs.append(acc) + loss_sum += loss.asnumpy() + + loss_sum = loss_sum / len(accs) + acc_mean = np.mean(accs) + return loss_sum, acc_mean + + +def validate_model(net, dataset): + accs = [] + for _, data in enumerate(dataset.create_dict_iterator()): + train_x = Tensor(data['image'].astype(np.float32)) + label = Tensor(data['label'].astype(np.int32)) + output = net(train_x) + log_output = P.LogSoftmax(axis=1)(output) + acc = np.mean(log_output.asnumpy().argmax(axis=1) == label.asnumpy()) + accs.append(acc) + + acc_mean = np.mean(accs) + return acc_mean +``` + +### 变分推断 +#### 定义变分自编码器 +我们只需要自定义编码器和解码器，编码器和解码器都是神经网络。 + +``` +class Encoder(nn.Cell): + def __init__(self): + super(Encoder, self).__init__() + self.fc1 = nn.Dense(1024, 800) + self.fc2 = nn.Dense(800, 400) + self.relu = nn.ReLU() + self.flatten = nn.Flatten() + + def construct(self, x): + x = self.flatten(x) + x = self.fc1(x) + x = self.relu(x) + x = self.fc2(x) + x = self.relu(x) + return x + + +class Decoder(nn.Cell): + def __init__(self): + super(Decoder, self).__init__() + self.fc1 = nn.Dense(400, 1024) + self.sigmoid = nn.Sigmoid() + self.reshape = P.Reshape() + + def construct(self, z): + z = self.fc1(z) + z = self.reshape(z, IMAGE_SHAPE) + z = self.sigmoid(z) + return z + + +encoder = Encoder() +decoder = Decoder() +vae = VAE(encoder, decoder, hidden_size=400, latent_size=20) +``` +### 定义损失函数和优化器 +接下来需要定义损失函数（Loss）和优化器（Optimizer）。本例使用的损失函数是ELBO，ELBO是变分推断专用的损失函数；本例使用的优化器是Adam。 +MindSpore中定义损失函数和优化器的代码样例如下： + +``` +# loss function definition +net_loss = ELBO(latent_prior='Normal', output_prior='Normal') + +# optimization definition +optimizer = nn.Adam(params=vae.trainable_params(), learning_rate=0.001) + +net_with_loss = nn.WithLossCell(vae, net_loss) +``` +#### 处理数据 +本例使用的是MNIST数据集，数据处理过程与教程中的[实现一个图片分类应用](https://www.mindspore.cn/tutorial/zh-CN/master/quick_start/quick_start.html)一致。 + +#### 训练网络 +使用`SVI`接口对VAE网络进行训练。 + +``` +from mindspore.nn.probability.infer import SVI + +vi = SVI(net_with_loss=net_with_loss, optimizer=optimizer) +vae = vi.run(train_dataset=ds_train, epochs=10) +trained_loss = vi.get_train_loss() +``` +通过`vi.run`可以得到训练好的网络，使用`vi.get_train_loss`可以得到训练之后的损失。 +#### 生成新样本或重构输入样本 +利用训练好的VAE网络，我们可以生成新的样本或重构输入样本。 + +``` +IMAGE_SHAPE = (-1, 1, 32, 32) +generated_sample = vae.generate_sample(64, IMAGE_SHAPE) +for sample in ds_train.create_dict_iterator(): + sample_x = Tensor(sample['image'], dtype=mstype.float32) + reconstructed_sample = vae.reconstruct_sample(sample_x) +``` + +### DNN一键转换成BNN +对于不熟悉贝叶斯模型的DNN研究人员，MDP提供了高级API`TransformToBNN`，支持DNN模型一键转换成BNN模型。 +#### 定义DNN模型 +本例使用的DNN模型是LeNet。 + +``` +from mindspore.common.initializer import TruncatedNormal +import mindspore.nn as nn +import mindspore.ops.operations as P + +def conv(in_channels, out_channels, kernel_size, stride=1, padding=0): + """weight initial for conv layer""" + weight = weight_variable() + return nn.Conv2d(in_channels, out_channels, + kernel_size=kernel_size, stride=stride, padding=padding, + weight_init=weight, has_bias=False, pad_mode="valid") + + +def fc_with_initialize(input_channels, out_channels): + """weight initial for fc layer""" + weight = weight_variable() + bias = weight_variable() + return nn.Dense(input_channels, out_channels, weight, bias) + + +def weight_variable(): + """weight initial""" + return TruncatedNormal(0.02) + + +class LeNet5(nn.Cell): + """ + Lenet network + + Args: + num_class (int): Num classes. Default: 10. + + Returns: + Tensor, output tensor + Examples: + >>> LeNet5(num_class=10) + + """ + def __init__(self, num_class=10): + super(LeNet5, self).__init__() + self.num_class = num_class + self.conv1 = conv(1, 6, 5) + self.conv2 = conv(6, 16, 5) + self.fc1 = fc_with_initialize(16 * 5 * 5, 120) + self.fc2 = fc_with_initialize(120, 84) + self.fc3 = fc_with_initialize(84, self.num_class) + self.relu = nn.ReLU() + self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) + self.flatten = nn.Flatten() + self.reshape = P.Reshape() + + def construct(self, x): + x = self.conv1(x) + x = self.relu(x) + x = self.max_pool2d(x) + x = self.conv2(x) + x = self.relu(x) + x = self.max_pool2d(x) + x = self.flatten(x) + x = self.fc1(x) + x = self.relu(x) + x = self.fc2(x) + x = self.relu(x) + x = self.fc3(x) + return x +``` +#### 定义损失函数和优化器 +接下来需要定义损失函数（Loss）和优化器（Optimizer）。本例使用交叉熵损失作为损失函数，Adam作为优化器。 + +``` +network = LeNet5() + +# loss function definition +criterion = SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") + +# optimization definition +optimizer = AdamWeightDecay(params=network.trainable_params(), learning_rate=0.0001) + +net_with_loss = WithLossCell(network, criterion) +train_network = TrainOneStepCell(net_with_loss, optimizer) +``` +#### 实例化TransformToBNN +`TransformToBNN`的`__init__`函数定义如下： + +``` +class TransformToBNN: + def __init__(self, trainable_dnn, dnn_factor=1, bnn_factor=1): + net_with_loss = trainable_dnn.network + self.optimizer = trainable_dnn.optimizer + self.backbone = net_with_loss.backbone_network + self.loss_fn = getattr(net_with_loss, "_loss_fn") + self.dnn_factor = dnn_factor + self.bnn_factor = bnn_factor + self.bnn_loss_file = None +``` +参数`trainable_bnn`是经过`TrainOneStepCell`包装的可训练DNN模型，`dnn_factor`和`bnn_factor`分别为由损失函数计算得到的网络整体损失的系数和每个贝叶斯层的KL散度的系数。 +MindSpore中实例化`TransformToBNN`的代码如下： + +``` +from mindspore.nn.probability import transforms + +bnn_transformer = transforms.TransformToBNN(train_network, 60000, 0.000001) +``` +#### 功能一：转换整个模型 +`transform_to_bnn_model`方法可以将整个DNN模型转换为BNN模型。其定义如下: + +``` + def transform_to_bnn_model(self, + get_dense_args=lambda dp: {"in_channels": dp.in_channels, "has_bias": dp.has_bias, + "out_channels": dp.out_channels, "activation": dp.activation}, + get_conv_args=lambda dp: {"in_channels": dp.in_channels, "out_channels": dp.out_channels, + "pad_mode": dp.pad_mode, "kernel_size": dp.kernel_size, + "stride": dp.stride, "has_bias": dp.has_bias, + "padding": dp.padding, "dilation": dp.dilation, + "group": dp.group}, + add_dense_args=None, + add_conv_args=None): + r""" + Transform the whole DNN model to BNN model, and wrap BNN model by TrainOneStepCell. + + Args: + get_dense_args (function): The arguments gotten from the DNN full connection layer. Default: lambda dp: + {"in_channels": dp.in_channels, "out_channels": dp.out_channels, "has_bias": dp.has_bias}. + get_conv_args (function): The arguments gotten from the DNN convolutional layer. Default: lambda dp: + {"in_channels": dp.in_channels, "out_channels": dp.out_channels, "pad_mode": dp.pad_mode, + "kernel_size": dp.kernel_size, "stride": dp.stride, "has_bias": dp.has_bias}. + add_dense_args (dict): The new arguments added to BNN full connection layer. Default: {}. + add_conv_args (dict): The new arguments added to BNN convolutional layer. Default: {}. + + Returns: + Cell, a trainable BNN model wrapped by TrainOneStepCell. + """ +``` +参数`get_dense_args`指定从DNN模型的全连接层中获取哪些参数，`get_conv_args`指定从DNN模型的卷积层中获取哪些参数，参数`add_dense_args`和`add_conv_args`分别指定了要为BNN层指定哪些新的参数值。需要注意的是，`add_dense_args`中的参数不能与`get_dense_args`重复，`add_conv_args`和`get_conv_args`也是如此。 +在MindSpore中将整个DNN模型转换成BNN模型的代码如下： + +``` +train_bnn_network = bnn_transformer.transform_to_bnn_model() +``` +#### 功能二：转换指定类型的层 +`transform_to_bnn_layer`方法可以将DNN模型中指定类型的层（nn.Dense或者nn.Conv2d）转换为对应的贝叶斯层。其定义如下: + +``` + def transform_to_bnn_layer(self, dnn_layer, bnn_layer, get_args=None, add_args=None): + r""" + Transform a specific type of layers in DNN model to corresponding BNN layer. + + Args: + dnn_layer_type (Cell): The type of DNN layer to be transformed to BNN layer. The optional values are + nn.Dense, nn.Conv2d. + bnn_layer_type (Cell): The type of BNN layer to be transformed to. The optional values are + DenseReparameterization, ConvReparameterization. + get_args (dict): The arguments gotten from the DNN layer. Default: None. + add_args (dict): The new arguments added to BNN layer. Default: None. + + Returns: + Cell, a trainable model wrapped by TrainOneStepCell, whose sprcific type of layer is transformed to the corresponding bayesian layer. + """ +``` +参数`dnn_layer`指定将哪个类型的DNN层转换成BNN层，`bnn_layer`指定DNN层将转换成哪个类型的BNN层，`get_args`和`add_args`分别指定从DNN层中获取哪些参数和要为BNN层的哪些参数重新赋值。 + +### 不确定性估计 +不确定性估计工具箱基于MindSpore Deep probability Programming (MDP)，适用于主流的深度学习模型，如回归、分类、目标检测等。在推理阶段，利用不确定性估计工具箱，开发人员只需通过训练模型和训练数据集，指定需要估计的任务和样本，即可得到任意不确定性（aleatoric uncertainty）和认知不确定性（epistemic uncertainty）。基于不确定性信息，开发人员可以更好地理解模型和数据集。 +以分类任务为例，本例中使用的模型是LeNet，数据集为MNist，数据处理过程与教程中的[实现一个图片分类应用](https://www.mindspore.cn/tutorial/zh-CN/master/quick_start/quick_start.html)一致。为了评估测试示例的不确定性，使用工具箱的方法如下: + +``` +from mindspore.nn.probability.toolbox.uncertainty_evaluation import UncertaintyEvaluation +from mindspore.train.serialization import load_checkpoint, load_param_into_net + +network = LeNet5() +param_dict = load_checkpoint('checkpoint_lenet.ckpt') +load_param_into_net(network, param_dict) +# get train and eval dataset +ds_train = create_dataset('workspace/mnist/train') +ds_eval = create_dataset('workspace/mnist/test') +evaluation = UncertaintyEvaluation(model=network, + train_dataset=ds_train, + task_type='classification', + num_classes=10, + epochs=1, + epi_uncer_model_path=None, + ale_uncer_model_path=None, + save_model=False) +for eval_data in ds_eval.create_dict_iterator(): + eval_data = Tensor(eval_data['image'], mstype.float32) + epistemic_uncertainty = evaluation.eval_epistemic_uncertainty(eval_data) + aleatoric_uncertainty = evaluation.eval_aleatoric_uncertainty(eval_data) +``` + + diff --git a/tutorials/source_zh_cn/advanced_use/differential_privacy.md b/tutorials/training/source_zh_cn/advanced_use/differential_privacy.md similarity index 97% rename from tutorials/source_zh_cn/advanced_use/differential_privacy.md rename to tutorials/training/source_zh_cn/advanced_use/differential_privacy.md index 6813d7f001952205689e75654ab29a043128747d..1418b34ae6785b4141c175426cc95b6509dcf9d9 100644 --- a/tutorials/source_zh_cn/advanced_use/differential_privacy.md +++ b/tutorials/training/source_zh_cn/advanced_use/differential_privacy.md @@ -1,358 +1,358 @@ -# 机器学习中的差分隐私 - -`Linux` `Ascend` `模型开发` `模型调优` `企业` `高级` - - - -- [机器学习中的差分隐私](#机器学习中的差分隐私) - - [概述](#概述) - - [实现阶段](#实现阶段) - - [导入需要的库文件](#导入需要的库文件) - - [参数配置](#参数配置) - - [预处理数据集](#预处理数据集) - - [建立模型](#建立模型) - - [引入差分隐私](#引入差分隐私) - - [引用](#引用) - - - - - -## 概述 - -差分隐私是一种保护用户数据隐私的机制。什么是隐私，隐私指的是单个用户的某些属性，一群用户的某一些属性可以不看做隐私。例如：“抽烟的人有更高的几率会得肺癌”，这个不泄露隐私，但是“张三抽烟，得了肺癌”，这个就泄露了张三的隐私。如果我们知道A医院，今天就诊的100个病人，其中有10个肺癌，并且我们知道了其中99个人的患病信息，就可以推测剩下一个人是否患有肺癌。这种窃取隐私的行为叫做差分攻击。差分隐私是防止差分攻击的方法，通过添加噪声，使得差别只有一条记录的两个数据集，通过模型推理获得相同结果的概率非常接近。也就是说，用了差分隐私后，攻击者知道的100个人的患病信息和99个人的患病信息几乎是一样的，从而无法推测出剩下1个人的患病情况。 - -**机器学习中的差分隐私** - -机器学习算法一般是用大量数据并更新模型参数，学习数据特征。在理想情况下，这些算法学习到一些泛化性较好的模型，例如“吸烟患者更容易得肺癌”，而不是特定的个体特征，例如“张三是个吸烟者，患有肺癌”。然而，机器学习算法并不会区分通用特征还是个体特征。当我们用机器学习来完成某个重要的任务，例如肺癌诊断，发布的机器学习模型，可能在无意中透露训练集中的个体特征，恶意攻击者可能从发布的模型获得关于张三的隐私信息，因此使用差分隐私技术来保护机器学习模型是十分必要的。 - -**差分隐私定义**[1]为： - -$Pr[\mathcal{K}(D)\in S] \le e^{\epsilon} Pr[\mathcal{K}(D') \in S]+\delta$ - -对于两个差别只有一条记录的数据集$D, D'$，通过随机算法$\mathcal{K}$，输出为结果集合$S$子集的概率满足上面公式，$\epsilon$为差分隐私预算，$\delta$ 为扰动，$\epsilon, \delta$越小，$\mathcal{K}$在$D, D'$上输出的数据分布越接近。 - -**差分隐私的度量** - -差分隐私可以用$\epsilon, \delta$ 度量。 - -- $\epsilon$：数据集中增加或者减少一条记录，引起的输出概率可以改变的上限。我们通常希望$\epsilon$是一个较小的常数，值越小表示差分隐私条件越严格。 -- $\delta$：用于限制模型行为任意改变的概率，通常设置为一个小的常数，推荐设置小于训练数据集大小的倒数。 - -**MindArmour实现的差分隐私** - -MindArmour的差分隐私模块Differential-Privacy，实现了差分隐私优化器。目前支持基于高斯机制的差分隐私SGD、Momentum、Adam优化器。其中，高斯噪声机制支持固定标准差的非自适应高斯噪声和随着时间或者迭代步数变化而变化的自适应高斯噪声，使用非自适应高斯噪声的优势在于可以严格控制差分隐私预算$\epsilon$，缺点是在模型训练过程中，每个Step添加的噪声量固定，在训练后期，较大的噪声使得模型收敛困难，甚至导致性能大幅下跌，模型可用性差。自适应噪声很好的解决了这个问题，在模型训练初期，添加的噪声量较大，随着模型逐渐收敛，噪声量逐渐减小，噪声对于模型可用性的影响减小。自适应噪声的缺点是不能严格控制差分隐私预算，在同样的初始值下，自适应差分隐私的$\epsilon$比非自适应的大。同时还提供RDP（R’enyi differential privacy）[2]用于监测差分隐私预算。 - -这里以LeNet模型，MNIST 数据集为例，说明如何在MindSpore上使用差分隐私优化器训练神经网络模型。 - -> 本例面向Ascend 910 AI处理器，你可以在这里下载完整的样例代码： - -## 实现阶段 - -### 导入需要的库文件 - -下列是我们需要的公共模块、MindSpore相关模块和差分隐私特性模块。 - -```python -import os -from easydict import EasyDict as edict - -import mindspore.nn as nn -from mindspore import context -from mindspore.train.callback import ModelCheckpoint -from mindspore.train.callback import CheckpointConfig -from mindspore.train.callback import LossMonitor -from mindspore.nn.metrics import Accuracy -from mindspore.train.serialization import load_checkpoint, load_param_into_net -import mindspore.dataset as ds -import mindspore.dataset.vision.c_transforms as CV -import mindspore.dataset.transforms.c_transforms as C -from mindspore.dataset.vision.import Inter -import mindspore.common.dtype as mstype - -from mindarmour.diff_privacy import DPModel -from mindarmour.diff_privacy import NoiseMechanismsFactory -from mindarmour.diff_privacy import ClipMechanismsFactory -from mindarmour.diff_privacy import PrivacyMonitorFactory -from mindarmour.utils.logger import LogUtil -from lenet5_net import LeNet5 -from lenet5_config import mnist_cfg as cfg - -LOGGER = LogUtil.get_instance() -LOGGER.set_level('INFO') -TAG = 'Lenet5_train' -``` - -### 参数配置 - -1. 设置运行环境、数据集路径、模型训练参数、checkpoint存储参数、差分隐私参数，`data_path`数据路径替换成你的数据集所在路径。更多配置可以参考。 - - ```python - cfg = edict({ - 'num_classes': 10, # the number of classes of model's output - 'lr': 0.01, # the learning rate of model's optimizer - 'momentum': 0.9, # the momentum value of model's optimizer - 'epoch_size': 10, # training epochs - 'batch_size': 256, # batch size for training - 'image_height': 32, # the height of training samples - 'image_width': 32, # the width of training samples - 'save_checkpoint_steps': 234, # the interval steps for saving checkpoint file of the model - 'keep_checkpoint_max': 10, # the maximum number of checkpoint files would be saved - 'device_target': 'Ascend', # device used - 'data_path': './MNIST_unzip', # the path of training and testing data set - 'dataset_sink_mode': False, # whether deliver all training data to device one time - 'micro_batches': 32, # the number of small batches split from an original batch - 'norm_bound': 1.0, # the clip bound of the gradients of model's training parameters - 'initial_noise_multiplier': 0.05, # the initial multiplication coefficient of the noise added to training - # parameters' gradients - 'noise_mechanisms': 'Gaussian', # the method of adding noise in gradients while training - 'clip_mechanisms': 'Gaussian', # the method of adaptive clipping gradients while training - 'clip_decay_policy': 'Linear', # Decay policy of adaptive clipping, decay_policy must be in ['Linear', 'Geometric']. - 'clip_learning_rate': 0.001, # Learning rate of update norm clip. - 'target_unclipped_quantile': 0.9, # Target quantile of norm clip. - 'fraction_stddev': 0.01, # The stddev of Gaussian normal which used in empirical_fraction. - 'optimizer': 'Momentum' # the base optimizer used for Differential privacy training - }) - ``` - -2. 配置必要的信息，包括环境信息、执行的模式。 - - ```python - context.set_context(mode=context.GRAPH_MODE, device_target=cfg.device_target) - ``` - - 详细的接口配置信息，请参见`context.set_context`接口说明。 - -### 预处理数据集 - -加载数据集并处理成MindSpore数据格式。 - -```python -def generate_mnist_dataset(data_path, batch_size=32, repeat_size=1, - num_parallel_workers=1, sparse=True): - """ - create dataset for training or testing - """ - # define dataset - ds1 = ds.MnistDataset(data_path) - - # define operation parameters - resize_height, resize_width = 32, 32 - rescale = 1.0 / 255.0 - shift = 0.0 - - # define map operations - resize_op = CV.Resize((resize_height, resize_width), - interpolation=Inter.LINEAR) - rescale_op = CV.Rescale(rescale, shift) - hwc2chw_op = CV.HWC2CHW() - type_cast_op = C.TypeCast(mstype.int32) - - # apply map operations on images - if not sparse: - one_hot_enco = C.OneHot(10) - ds1 = ds1.map(input_columns="label", operations=one_hot_enco, - num_parallel_workers=num_parallel_workers) - type_cast_op = C.TypeCast(mstype.float32) - ds1 = ds1.map(operations=type_cast_op, input_columns="label", - num_parallel_workers=num_parallel_workers) - ds1 = ds1.map(operations=resize_op, input_columns="image", - num_parallel_workers=num_parallel_workers) - ds1 = ds1.map(operations=rescale_op, input_columns="image", - num_parallel_workers=num_parallel_workers) - ds1 = ds1.map(operations=hwc2chw_op, input_columns="image", - num_parallel_workers=num_parallel_workers) - - # apply DatasetOps - buffer_size = 10000 - ds1 = ds1.shuffle(buffer_size=buffer_size) - ds1 = ds1.batch(batch_size, drop_remainder=True) - ds1 = ds1.repeat(repeat_size) - - return ds1 -``` - -### 建立模型 - -这里以LeNet模型为例，您也可以建立训练自己的模型。 - -```python -from mindspore import nn -from mindspore.common.initializer import TruncatedNormal - - -def conv(in_channels, out_channels, kernel_size, stride=1, padding=0): - weight = weight_variable() - return nn.Conv2d(in_channels, out_channels, - kernel_size=kernel_size, stride=stride, padding=padding, - weight_init=weight, has_bias=False, pad_mode="valid") - - -def fc_with_initialize(input_channels, out_channels): - weight = weight_variable() - bias = weight_variable() - return nn.Dense(input_channels, out_channels, weight, bias) - - -def weight_variable(): - return TruncatedNormal(0.05) - - -class LeNet5(nn.Cell): - """ - LeNet network - """ - def __init__(self): - super(LeNet5, self).__init__() - self.conv1 = conv(1, 6, 5) - self.conv2 = conv(6, 16, 5) - self.fc1 = fc_with_initialize(16*5*5, 120) - self.fc2 = fc_with_initialize(120, 84) - self.fc3 = fc_with_initialize(84, 10) - self.relu = nn.ReLU() - self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) - self.flatten = nn.Flatten() - - def construct(self, x): - x = self.conv1(x) - x = self.relu(x) - x = self.max_pool2d(x) - x = self.conv2(x) - x = self.relu(x) - x = self.max_pool2d(x) - x = self.flatten(x) - x = self.fc1(x) - x = self.relu(x) - x = self.fc2(x) - x = self.relu(x) - x = self.fc3(x) - return x -``` - -加载LeNet网络，定义损失函数、配置checkpoint、用上述定义的数据加载函数`generate_mnist_dataset`载入数据。 - -```python -network = LeNet5() -net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") -config_ck = CheckpointConfig(save_checkpoint_steps=cfg.save_checkpoint_steps, - keep_checkpoint_max=cfg.keep_checkpoint_max) -ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", - directory='./trained_ckpt_file/', - config=config_ck) - -# get training dataset -ds_train = generate_mnist_dataset(os.path.join(cfg.data_path, "train"), - cfg.batch_size) -``` - -### 引入差分隐私 - -1. 配置差分隐私优化器的参数。 - - - 判断`micro_batches`和`batch_size`参数是否符合要求，`batch_size`必须要整除`micro_batches`。 - - 实例化差分隐私工厂类。 - - 设置差分隐私的噪声机制，目前mechanisms支持固定标准差的高斯噪声机制：`Gaussian`和自适应调整标准差的高斯噪声机制：`AdaGaussian`。 - - 设置优化器类型，目前支持`SGD`、`Momentum`和`Adam`。 - - 设置差分隐私预算监测器RDP，用于观测每个step中的差分隐私预算$\epsilon$的变化。 - - ```python - if cfg.micro_batches and cfg.batch_size % cfg.micro_batches != 0: - raise ValueError( - "Number of micro_batches should divide evenly batch_size") - # Create a factory class of DP noise mechanisms, this method is adding noise - # in gradients while training. Initial_noise_multiplier is suggested to be - # greater than 1.0, otherwise the privacy budget would be huge, which means - # that the privacy protection effect is weak. Mechanisms can be 'Gaussian' - # or 'AdaGaussian', in which noise would be decayed with 'AdaGaussian' - # mechanism while be constant with 'Gaussian' mechanism. - noise_mech = NoiseMechanismsFactory().create(cfg.noise_mechanisms, - norm_bound=cfg.norm_bound, - initial_noise_multiplier=cfg.initial_noise_multiplier, - decay_policy=None) - # Create a factory class of clip mechanisms, this method is to adaptive clip - # gradients while training, decay_policy support 'Linear' and 'Geometric', - # learning_rate is the learning rate to update clip_norm, - # target_unclipped_quantile is the target quantile of norm clip, - # fraction_stddev is the stddev of Gaussian normal which used in - # empirical_fraction, the formula is - # $empirical_fraction + N(0, fraction_stddev)$. - clip_mech = ClipMechanismsFactory().create(cfg.clip_mechanisms, - decay_policy=cfg.clip_decay_policy, - learning_rate=cfg.clip_learning_rate, - target_unclipped_quantile=cfg.target_unclipped_quantile, - fraction_stddev=cfg.fraction_stddev) - net_opt = nn.Momentum(params=network.trainable_params(), - learning_rate=cfg.lr, momentum=cfg.momentum) - # Create a monitor for DP training. The function of the monitor is to - # compute and print the privacy budget(eps and delta) while training. - rdp_monitor = PrivacyMonitorFactory.create('rdp', - num_samples=60000, - batch_size=cfg.batch_size, - initial_noise_multiplier=cfg.initial_noise_multiplier, - per_print_times=234, - noise_decay_mode=None) - ``` - -2. 将LeNet模型包装成差分隐私模型，只需要将网络传入`DPModel`即可。 - - ```python - # Create the DP model for training. - model = DPModel(micro_batches=cfg.micro_batches, - norm_bound=cfg.norm_bound, - noise_mech=noise_mech, - clip_mech=clip_mech, - network=network, - loss_fn=net_loss, - optimizer=net_opt, - metrics={"Accuracy": Accuracy()}) - ``` - -3. 模型训练与测试。 - - ```python - LOGGER.info(TAG, "============== Starting Training ==============") - model.train(cfg['epoch_size'], ds_train, - callbacks=[ckpoint_cb, LossMonitor(), rdp_monitor], - dataset_sink_mode=cfg.dataset_sink_mode) - - LOGGER.info(TAG, "============== Starting Testing ==============") - ckpt_file_name = 'trained_ckpt_file/checkpoint_lenet-10_234.ckpt' - param_dict = load_checkpoint(ckpt_file_name) - load_param_into_net(network, param_dict) - ds_eval = generate_mnist_dataset(os.path.join(cfg.data_path, 'test'), - batch_size=cfg.batch_size) - acc = model.eval(ds_eval, dataset_sink_mode=False) - LOGGER.info(TAG, "============== Accuracy: %s ==============", acc) - ``` - -4. 运行命令。 - - 运行脚本，可在命令行输入命令： - - ```bash - python lenet_dp.py - ``` - - 其中`lenet5_dp.py`替换成你的脚本的名字。 - -5. 结果展示。 - - 不加差分隐私的LeNet模型精度稳定在99%，加了Gaussian噪声，自适应Clip的差分隐私LeNet模型收敛，精度稳定在95%左右。 - ``` - ============== Starting Training ============== - ... - ============== Starting Testing ============== - ... - ============== Accuracy: 0.9698 ============== - ``` - -### 引用 - -[1] C. Dwork and J. Lei. Differential privacy and robust statistics. In STOC, pages 371–380. ACM, 2009. - -[2] Ilya Mironov. Rényi differential privacy. In IEEE Computer Security Foundations Symposium, 2017. - -[3] Abadi, M. e. a., 2016. *Deep learning with differential privacy.* s.l.:Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. - - - +# 机器学习中的差分隐私 + +`Linux` `Ascend` `模型开发` `模型调优` `企业` `高级` + + + +- [机器学习中的差分隐私](#机器学习中的差分隐私) + - [概述](#概述) + - [实现阶段](#实现阶段) + - [导入需要的库文件](#导入需要的库文件) + - [参数配置](#参数配置) + - [预处理数据集](#预处理数据集) + - [建立模型](#建立模型) + - [引入差分隐私](#引入差分隐私) + - [引用](#引用) + + + + + +## 概述 + +差分隐私是一种保护用户数据隐私的机制。什么是隐私，隐私指的是单个用户的某些属性，一群用户的某一些属性可以不看做隐私。例如：“抽烟的人有更高的几率会得肺癌”，这个不泄露隐私，但是“张三抽烟，得了肺癌”，这个就泄露了张三的隐私。如果我们知道A医院，今天就诊的100个病人，其中有10个肺癌，并且我们知道了其中99个人的患病信息，就可以推测剩下一个人是否患有肺癌。这种窃取隐私的行为叫做差分攻击。差分隐私是防止差分攻击的方法，通过添加噪声，使得差别只有一条记录的两个数据集，通过模型推理获得相同结果的概率非常接近。也就是说，用了差分隐私后，攻击者知道的100个人的患病信息和99个人的患病信息几乎是一样的，从而无法推测出剩下1个人的患病情况。 + +**机器学习中的差分隐私** + +机器学习算法一般是用大量数据并更新模型参数，学习数据特征。在理想情况下，这些算法学习到一些泛化性较好的模型，例如“吸烟患者更容易得肺癌”，而不是特定的个体特征，例如“张三是个吸烟者，患有肺癌”。然而，机器学习算法并不会区分通用特征还是个体特征。当我们用机器学习来完成某个重要的任务，例如肺癌诊断，发布的机器学习模型，可能在无意中透露训练集中的个体特征，恶意攻击者可能从发布的模型获得关于张三的隐私信息，因此使用差分隐私技术来保护机器学习模型是十分必要的。 + +**差分隐私定义**[1]为： + +$Pr[\mathcal{K}(D)\in S] \le e^{\epsilon} Pr[\mathcal{K}(D') \in S]+\delta$ + +对于两个差别只有一条记录的数据集$D, D'$，通过随机算法$\mathcal{K}$，输出为结果集合$S$子集的概率满足上面公式，$\epsilon$为差分隐私预算，$\delta$ 为扰动，$\epsilon, \delta$越小，$\mathcal{K}$在$D, D'$上输出的数据分布越接近。 + +**差分隐私的度量** + +差分隐私可以用$\epsilon, \delta$ 度量。 + +- $\epsilon$：数据集中增加或者减少一条记录，引起的输出概率可以改变的上限。我们通常希望$\epsilon$是一个较小的常数，值越小表示差分隐私条件越严格。 +- $\delta$：用于限制模型行为任意改变的概率，通常设置为一个小的常数，推荐设置小于训练数据集大小的倒数。 + +**MindArmour实现的差分隐私** + +MindArmour的差分隐私模块Differential-Privacy，实现了差分隐私优化器。目前支持基于高斯机制的差分隐私SGD、Momentum、Adam优化器。其中，高斯噪声机制支持固定标准差的非自适应高斯噪声和随着时间或者迭代步数变化而变化的自适应高斯噪声，使用非自适应高斯噪声的优势在于可以严格控制差分隐私预算$\epsilon$，缺点是在模型训练过程中，每个Step添加的噪声量固定，在训练后期，较大的噪声使得模型收敛困难，甚至导致性能大幅下跌，模型可用性差。自适应噪声很好的解决了这个问题，在模型训练初期，添加的噪声量较大，随着模型逐渐收敛，噪声量逐渐减小，噪声对于模型可用性的影响减小。自适应噪声的缺点是不能严格控制差分隐私预算，在同样的初始值下，自适应差分隐私的$\epsilon$比非自适应的大。同时还提供RDP（R’enyi differential privacy）[2]用于监测差分隐私预算。 + +这里以LeNet模型，MNIST 数据集为例，说明如何在MindSpore上使用差分隐私优化器训练神经网络模型。 + +> 本例面向Ascend 910 AI处理器，你可以在这里下载完整的样例代码： + +## 实现阶段 + +### 导入需要的库文件 + +下列是我们需要的公共模块、MindSpore相关模块和差分隐私特性模块。 + +```python +import os +from easydict import EasyDict as edict + +import mindspore.nn as nn +from mindspore import context +from mindspore.train.callback import ModelCheckpoint +from mindspore.train.callback import CheckpointConfig +from mindspore.train.callback import LossMonitor +from mindspore.nn.metrics import Accuracy +from mindspore.train.serialization import load_checkpoint, load_param_into_net +import mindspore.dataset as ds +import mindspore.dataset.vision.c_transforms as CV +import mindspore.dataset.transforms.c_transforms as C +from mindspore.dataset.vision.import Inter +import mindspore.common.dtype as mstype + +from mindarmour.diff_privacy import DPModel +from mindarmour.diff_privacy import NoiseMechanismsFactory +from mindarmour.diff_privacy import ClipMechanismsFactory +from mindarmour.diff_privacy import PrivacyMonitorFactory +from mindarmour.utils.logger import LogUtil +from lenet5_net import LeNet5 +from lenet5_config import mnist_cfg as cfg + +LOGGER = LogUtil.get_instance() +LOGGER.set_level('INFO') +TAG = 'Lenet5_train' +``` + +### 参数配置 + +1. 设置运行环境、数据集路径、模型训练参数、checkpoint存储参数、差分隐私参数，`data_path`数据路径替换成你的数据集所在路径。更多配置可以参考。 + + ```python + cfg = edict({ + 'num_classes': 10, # the number of classes of model's output + 'lr': 0.01, # the learning rate of model's optimizer + 'momentum': 0.9, # the momentum value of model's optimizer + 'epoch_size': 10, # training epochs + 'batch_size': 256, # batch size for training + 'image_height': 32, # the height of training samples + 'image_width': 32, # the width of training samples + 'save_checkpoint_steps': 234, # the interval steps for saving checkpoint file of the model + 'keep_checkpoint_max': 10, # the maximum number of checkpoint files would be saved + 'device_target': 'Ascend', # device used + 'data_path': './MNIST_unzip', # the path of training and testing data set + 'dataset_sink_mode': False, # whether deliver all training data to device one time + 'micro_batches': 32, # the number of small batches split from an original batch + 'norm_bound': 1.0, # the clip bound of the gradients of model's training parameters + 'initial_noise_multiplier': 0.05, # the initial multiplication coefficient of the noise added to training + # parameters' gradients + 'noise_mechanisms': 'Gaussian', # the method of adding noise in gradients while training + 'clip_mechanisms': 'Gaussian', # the method of adaptive clipping gradients while training + 'clip_decay_policy': 'Linear', # Decay policy of adaptive clipping, decay_policy must be in ['Linear', 'Geometric']. + 'clip_learning_rate': 0.001, # Learning rate of update norm clip. + 'target_unclipped_quantile': 0.9, # Target quantile of norm clip. + 'fraction_stddev': 0.01, # The stddev of Gaussian normal which used in empirical_fraction. + 'optimizer': 'Momentum' # the base optimizer used for Differential privacy training + }) + ``` + +2. 配置必要的信息，包括环境信息、执行的模式。 + + ```python + context.set_context(mode=context.GRAPH_MODE, device_target=cfg.device_target) + ``` + + 详细的接口配置信息，请参见`context.set_context`接口说明。 + +### 预处理数据集 + +加载数据集并处理成MindSpore数据格式。 + +```python +def generate_mnist_dataset(data_path, batch_size=32, repeat_size=1, + num_parallel_workers=1, sparse=True): + """ + create dataset for training or testing + """ + # define dataset + ds1 = ds.MnistDataset(data_path) + + # define operation parameters + resize_height, resize_width = 32, 32 + rescale = 1.0 / 255.0 + shift = 0.0 + + # define map operations + resize_op = CV.Resize((resize_height, resize_width), + interpolation=Inter.LINEAR) + rescale_op = CV.Rescale(rescale, shift) + hwc2chw_op = CV.HWC2CHW() + type_cast_op = C.TypeCast(mstype.int32) + + # apply map operations on images + if not sparse: + one_hot_enco = C.OneHot(10) + ds1 = ds1.map(input_columns="label", operations=one_hot_enco, + num_parallel_workers=num_parallel_workers) + type_cast_op = C.TypeCast(mstype.float32) + ds1 = ds1.map(operations=type_cast_op, input_columns="label", + num_parallel_workers=num_parallel_workers) + ds1 = ds1.map(operations=resize_op, input_columns="image", + num_parallel_workers=num_parallel_workers) + ds1 = ds1.map(operations=rescale_op, input_columns="image", + num_parallel_workers=num_parallel_workers) + ds1 = ds1.map(operations=hwc2chw_op, input_columns="image", + num_parallel_workers=num_parallel_workers) + + # apply DatasetOps + buffer_size = 10000 + ds1 = ds1.shuffle(buffer_size=buffer_size) + ds1 = ds1.batch(batch_size, drop_remainder=True) + ds1 = ds1.repeat(repeat_size) + + return ds1 +``` + +### 建立模型 + +这里以LeNet模型为例，您也可以建立训练自己的模型。 + +```python +from mindspore import nn +from mindspore.common.initializer import TruncatedNormal + + +def conv(in_channels, out_channels, kernel_size, stride=1, padding=0): + weight = weight_variable() + return nn.Conv2d(in_channels, out_channels, + kernel_size=kernel_size, stride=stride, padding=padding, + weight_init=weight, has_bias=False, pad_mode="valid") + + +def fc_with_initialize(input_channels, out_channels): + weight = weight_variable() + bias = weight_variable() + return nn.Dense(input_channels, out_channels, weight, bias) + + +def weight_variable(): + return TruncatedNormal(0.05) + + +class LeNet5(nn.Cell): + """ + LeNet network + """ + def __init__(self): + super(LeNet5, self).__init__() + self.conv1 = conv(1, 6, 5) + self.conv2 = conv(6, 16, 5) + self.fc1 = fc_with_initialize(16*5*5, 120) + self.fc2 = fc_with_initialize(120, 84) + self.fc3 = fc_with_initialize(84, 10) + self.relu = nn.ReLU() + self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) + self.flatten = nn.Flatten() + + def construct(self, x): + x = self.conv1(x) + x = self.relu(x) + x = self.max_pool2d(x) + x = self.conv2(x) + x = self.relu(x) + x = self.max_pool2d(x) + x = self.flatten(x) + x = self.fc1(x) + x = self.relu(x) + x = self.fc2(x) + x = self.relu(x) + x = self.fc3(x) + return x +``` + +加载LeNet网络，定义损失函数、配置checkpoint、用上述定义的数据加载函数`generate_mnist_dataset`载入数据。 + +```python +network = LeNet5() +net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") +config_ck = CheckpointConfig(save_checkpoint_steps=cfg.save_checkpoint_steps, + keep_checkpoint_max=cfg.keep_checkpoint_max) +ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", + directory='./trained_ckpt_file/', + config=config_ck) + +# get training dataset +ds_train = generate_mnist_dataset(os.path.join(cfg.data_path, "train"), + cfg.batch_size) +``` + +### 引入差分隐私 + +1. 配置差分隐私优化器的参数。 + + - 判断`micro_batches`和`batch_size`参数是否符合要求，`batch_size`必须要整除`micro_batches`。 + - 实例化差分隐私工厂类。 + - 设置差分隐私的噪声机制，目前mechanisms支持固定标准差的高斯噪声机制：`Gaussian`和自适应调整标准差的高斯噪声机制：`AdaGaussian`。 + - 设置优化器类型，目前支持`SGD`、`Momentum`和`Adam`。 + - 设置差分隐私预算监测器RDP，用于观测每个step中的差分隐私预算$\epsilon$的变化。 + + ```python + if cfg.micro_batches and cfg.batch_size % cfg.micro_batches != 0: + raise ValueError( + "Number of micro_batches should divide evenly batch_size") + # Create a factory class of DP noise mechanisms, this method is adding noise + # in gradients while training. Initial_noise_multiplier is suggested to be + # greater than 1.0, otherwise the privacy budget would be huge, which means + # that the privacy protection effect is weak. Mechanisms can be 'Gaussian' + # or 'AdaGaussian', in which noise would be decayed with 'AdaGaussian' + # mechanism while be constant with 'Gaussian' mechanism. + noise_mech = NoiseMechanismsFactory().create(cfg.noise_mechanisms, + norm_bound=cfg.norm_bound, + initial_noise_multiplier=cfg.initial_noise_multiplier, + decay_policy=None) + # Create a factory class of clip mechanisms, this method is to adaptive clip + # gradients while training, decay_policy support 'Linear' and 'Geometric', + # learning_rate is the learning rate to update clip_norm, + # target_unclipped_quantile is the target quantile of norm clip, + # fraction_stddev is the stddev of Gaussian normal which used in + # empirical_fraction, the formula is + # $empirical_fraction + N(0, fraction_stddev)$. + clip_mech = ClipMechanismsFactory().create(cfg.clip_mechanisms, + decay_policy=cfg.clip_decay_policy, + learning_rate=cfg.clip_learning_rate, + target_unclipped_quantile=cfg.target_unclipped_quantile, + fraction_stddev=cfg.fraction_stddev) + net_opt = nn.Momentum(params=network.trainable_params(), + learning_rate=cfg.lr, momentum=cfg.momentum) + # Create a monitor for DP training. The function of the monitor is to + # compute and print the privacy budget(eps and delta) while training. + rdp_monitor = PrivacyMonitorFactory.create('rdp', + num_samples=60000, + batch_size=cfg.batch_size, + initial_noise_multiplier=cfg.initial_noise_multiplier, + per_print_times=234, + noise_decay_mode=None) + ``` + +2. 将LeNet模型包装成差分隐私模型，只需要将网络传入`DPModel`即可。 + + ```python + # Create the DP model for training. + model = DPModel(micro_batches=cfg.micro_batches, + norm_bound=cfg.norm_bound, + noise_mech=noise_mech, + clip_mech=clip_mech, + network=network, + loss_fn=net_loss, + optimizer=net_opt, + metrics={"Accuracy": Accuracy()}) + ``` + +3. 模型训练与测试。 + + ```python + LOGGER.info(TAG, "============== Starting Training ==============") + model.train(cfg['epoch_size'], ds_train, + callbacks=[ckpoint_cb, LossMonitor(), rdp_monitor], + dataset_sink_mode=cfg.dataset_sink_mode) + + LOGGER.info(TAG, "============== Starting Testing ==============") + ckpt_file_name = 'trained_ckpt_file/checkpoint_lenet-10_234.ckpt' + param_dict = load_checkpoint(ckpt_file_name) + load_param_into_net(network, param_dict) + ds_eval = generate_mnist_dataset(os.path.join(cfg.data_path, 'test'), + batch_size=cfg.batch_size) + acc = model.eval(ds_eval, dataset_sink_mode=False) + LOGGER.info(TAG, "============== Accuracy: %s ==============", acc) + ``` + +4. 运行命令。 + + 运行脚本，可在命令行输入命令： + + ```bash + python lenet_dp.py + ``` + + 其中`lenet5_dp.py`替换成你的脚本的名字。 + +5. 结果展示。 + + 不加差分隐私的LeNet模型精度稳定在99%，加了Gaussian噪声，自适应Clip的差分隐私LeNet模型收敛，精度稳定在95%左右。 + ``` + ============== Starting Training ============== + ... + ============== Starting Testing ============== + ... + ============== Accuracy: 0.9698 ============== + ``` + +### 引用 + +[1] C. Dwork and J. Lei. Differential privacy and robust statistics. In STOC, pages 371–380. ACM, 2009. + +[2] Ilya Mironov. Rényi differential privacy. In IEEE Computer Security Foundations Symposium, 2017. + +[3] Abadi, M. e. a., 2016. *Deep learning with differential privacy.* s.l.:Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. + + + diff --git a/tutorials/source_zh_cn/advanced_use/distributed_training_ascend.md b/tutorials/training/source_zh_cn/advanced_use/distributed_training_ascend.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/distributed_training_ascend.md rename to tutorials/training/source_zh_cn/advanced_use/distributed_training_ascend.md diff --git a/tutorials/source_zh_cn/advanced_use/distributed_training_gpu.md b/tutorials/training/source_zh_cn/advanced_use/distributed_training_gpu.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/distributed_training_gpu.md rename to tutorials/training/source_zh_cn/advanced_use/distributed_training_gpu.md diff --git a/tutorials/source_zh_cn/advanced_use/distributed_training_tutorials.rst b/tutorials/training/source_zh_cn/advanced_use/distributed_training_tutorials.rst similarity index 100% rename from tutorials/source_zh_cn/advanced_use/distributed_training_tutorials.rst rename to tutorials/training/source_zh_cn/advanced_use/distributed_training_tutorials.rst diff --git a/tutorials/source_zh_cn/advanced_use/fuzzer.md b/tutorials/training/source_zh_cn/advanced_use/fuzzer.md similarity index 97% rename from tutorials/source_zh_cn/advanced_use/fuzzer.md rename to tutorials/training/source_zh_cn/advanced_use/fuzzer.md index 516b9a7467383ff7e36ea4f712dcb30cf495bcd0..a1740ae53010845cdcecaa4c7f10c2631256c26b 100644 --- a/tutorials/source_zh_cn/advanced_use/fuzzer.md +++ b/tutorials/training/source_zh_cn/advanced_use/fuzzer.md @@ -1,195 +1,195 @@ -# AI模型安全测试 - -`Linux` `Ascend` `GPU` `CPU` `数据准备` `模型开发` `模型训练` `模型调优` `企业` `高级` - - - -- [AI模型安全测试](#ai模型安全测试) - - [概述](#概述) - - [实现阶段](#实现阶段) - - [导入需要的库文件](#引入相关包) - - [参数配置](#参数配置) - - [运用Fuzzer](#运用Fuzzer) - - -   - -## 概述 - -传统软件的决策逻辑由代码逻辑决定，传统软件通过代码行覆盖率来判断当前测试是否充分，理想情况下覆盖率越高，代码测试越充分。然而，对于深度神经网络而言，程序的决策逻辑由训练数据、网络模型结构和参数通过某种黑盒机制决定，代码行覆盖率已不足以评估测试的充分性。需要根据深度网络的特点选择更为适合的测试评价准则，指导神经网络进行更为充分的测试，发现更多的边缘错误用例，从而确保模型的通用性、鲁棒性。 - -MindArmour的Fuzzer模块以神经元覆盖率作为测试评价准则。神经元覆盖率，是指通过一组输入观察到的、激活的神经元数量和神经元输出值的范围。我们通过神经元覆盖率来指导输入变异，让输入能够激活更多的神经元，神经元值的分布范围更广，从而探索不同类型的模型输出结果、错误行为。 - -这里以LeNet模型，MNIST数据集为例，说明如何使用Fuzzer。 - -> 本例面向CPU、GPU、Ascend 910 AI处理器，你可以在这里下载完整的样例代码： - -## 实现阶段 - -### 导入需要的库文件 - -下列是我们需要的公共模块、MindSpore相关模块和Fuzzer特性模块，以及配置日志标签和日志等级。 - -```python -import sys - -import numpy as np -from mindspore import Model -from mindspore import context -from mindspore.train.serialization import load_checkpoint, load_param_into_net - -from lenet5_net import LeNet5 -from mindarmour.fuzzing.fuzzing import Fuzzer -from mindarmour.fuzzing.model_coverage_metrics import ModelCoverageMetrics -from mindarmour.utils.logger import LogUtil - -LOGGER = LogUtil.get_instance() -TAG = 'Fuzz_test' -LOGGER.set_level('INFO') -``` - -### 参数配置 - -配置必要的信息，包括环境信息、执行的模式。 - -```python -context.set_context(mode=context.GRAPH_MODE, device_target=cfg.device_target) -``` - -详细的接口配置信息，请参见`context.set_context`接口说明。 - -### 运用Fuzzer - -1. 建立LeNet模型，加载MNIST数据集，操作同[模型安全]() - - ```python - ... - # Lenet model - model = Model(net) - # get training data - data_list = "./MNIST_unzip/train" - batch_size = 32 - ds = generate_mnist_dataset(data_list, batch_size, sparse=False) - train_images = [] - for data in ds.create_tuple_iterator(): - images = data[0].astype(np.float32) - train_images.append(images) - train_images = np.concatenate(train_images, axis=0) - - # get test data - data_list = "./MNIST_unzip/test" - batch_size = 32 - ds = generate_mnist_dataset(data_list, batch_size, sparse=False) - test_images = [] - test_labels = [] - for data in ds.create_tuple_iterator(): - images = data[0].astype(np.float32) - labels = data[1] - test_images.append(images) - test_labels.append(labels) - test_images = np.concatenate(test_images, axis=0) - test_labels = np.concatenate(test_labels, axis=0) - ``` - -2. Fuzzer参数配置。 - - 设置数据变异方法及参数。目前支持的数据变异方法包含三类： - - - 图像仿射变换方法：Translate、Scale、Shear、Rotate。 - - 基于图像像素值变化的方法： Contrast、Brightness、Blur、Noise。 - - 基于对抗攻击的白盒、黑盒对抗样本生成方法：FGSM、PGD、MDIIM。 - - 数据变异方法一定要包含基于图像像素值变化的方法。 - - 前两种图像变化方法的可配置参数，以及推荐参数范围请参考：对应的类方法，也可以均设置为`'auto_param': True`，变异参数将在推荐范围内随机生成。 - - 基于对抗攻击方法的参数配置请参考对应的攻击方法类。 - - ```python - mutate_config = [{'method': 'Blur', - 'params': {'auto_param': True}}, - {'method': 'Contrast', - 'params': {'auto_param': True}}, - {'method': 'Translate', - 'params': {'auto_param': True}}, - {'method': 'Brightness', - 'params': {'auto_param': True}}, - {'method': 'Noise', - 'params': {'auto_param': True}}, - {'method': 'Scale', - 'params': {'auto_param': True}}, - {'method': 'Shear', - 'params': {'auto_param': True}}, - {'method': 'FGSM', - 'params': {'eps': 0.3, 'alpha': 0.1}} - ] - ``` - - 设置评价指标，目前支持5种评价指标，包括: - - 通用评价指标：accuracy。 - - 神经元覆盖率指标：kmnc， nbc，snac。 - - 对抗攻击评价指标：attack_success_rate。 - 也可以设置为‘auto’，默认使用所有评价指标。 - - ```python - eval_metrics =['accuracy', 'kmnc', 'attack_success_rate'] - ``` - -3. 初始化种子队列，种子队列中的每个种子，包含3个值：原始图片、图片标签。 - - ```python - # make initial seeds - initial_seeds = [] - for img, label in zip(test_images, test_labels): - initial_seeds.append([img, label]) - initial_seeds = initial_seeds[:100] - ``` - -4. 测试Fuzz测试前的神经元覆盖率。 - - ```python - segmented_num=1000 - neuron_num=10 - model_coverage_test = ModelCoverageMetrics(model, segmented_num, neuron_num, train_images) - model_coverage_test.calculate_coverage(np.array(test_images[:100]).astype(np.float32)) - LOGGER.info(TAG, 'KMNC of this test is : %s', model_coverage_test.get_kmnc()) - ``` - - 结果： - - ```python - KMNC of this test is : 0.0851 - ``` - -5. Fuzz测试。 - - ```python - model_fuzz_test = Fuzzer(model, train_images, neuron_num, segmented_num) - _, _, _, _, metrics = model_fuzz_test.fuzzing(mutate_config, initial_seeds, eval_metrics=eval_metrics) - ``` - -6. 实验结果。 - - ```python - if metrics: - for key in metrics: - LOGGER.info(TAG, key + ': %s', metrics[key]) - ``` - - Fuzz测试后结果如下： - - ```python - Accuracy: 0.7929 - Attack_success_rate: 0.3939 - Neural_coverage_KMNC: 0.4797 - ``` - - Fuzz测试前种子的KMNC神经元覆盖率为8.5%，Fuzz后，KMNC神经元覆盖率为47.97%，神经元覆盖率提升，样本的多样性提升。Fuzz后，模型对于Fuzz生成样本的准确率为79.29%，使用了对抗攻击方法的样本，攻击成功率为47.97%。由于初始化种子、变异方法和相应的参数均为随机选择的，结果有一定的浮动是正常的。 - - 原始图片： - - ![fuzz_seed](./images/fuzz_seed.png) - -​ Fuzz生成的变异图片： - +# AI模型安全测试 + +`Linux` `Ascend` `GPU` `CPU` `数据准备` `模型开发` `模型训练` `模型调优` `企业` `高级` + + + +- [AI模型安全测试](#ai模型安全测试) + - [概述](#概述) + - [实现阶段](#实现阶段) + - [导入需要的库文件](#引入相关包) + - [参数配置](#参数配置) + - [运用Fuzzer](#运用Fuzzer) + + +   + +## 概述 + +传统软件的决策逻辑由代码逻辑决定，传统软件通过代码行覆盖率来判断当前测试是否充分，理想情况下覆盖率越高，代码测试越充分。然而，对于深度神经网络而言，程序的决策逻辑由训练数据、网络模型结构和参数通过某种黑盒机制决定，代码行覆盖率已不足以评估测试的充分性。需要根据深度网络的特点选择更为适合的测试评价准则，指导神经网络进行更为充分的测试，发现更多的边缘错误用例，从而确保模型的通用性、鲁棒性。 + +MindArmour的Fuzzer模块以神经元覆盖率作为测试评价准则。神经元覆盖率，是指通过一组输入观察到的、激活的神经元数量和神经元输出值的范围。我们通过神经元覆盖率来指导输入变异，让输入能够激活更多的神经元，神经元值的分布范围更广，从而探索不同类型的模型输出结果、错误行为。 + +这里以LeNet模型，MNIST数据集为例，说明如何使用Fuzzer。 + +> 本例面向CPU、GPU、Ascend 910 AI处理器，你可以在这里下载完整的样例代码： + +## 实现阶段 + +### 导入需要的库文件 + +下列是我们需要的公共模块、MindSpore相关模块和Fuzzer特性模块，以及配置日志标签和日志等级。 + +```python +import sys + +import numpy as np +from mindspore import Model +from mindspore import context +from mindspore.train.serialization import load_checkpoint, load_param_into_net + +from lenet5_net import LeNet5 +from mindarmour.fuzzing.fuzzing import Fuzzer +from mindarmour.fuzzing.model_coverage_metrics import ModelCoverageMetrics +from mindarmour.utils.logger import LogUtil + +LOGGER = LogUtil.get_instance() +TAG = 'Fuzz_test' +LOGGER.set_level('INFO') +``` + +### 参数配置 + +配置必要的信息，包括环境信息、执行的模式。 + +```python +context.set_context(mode=context.GRAPH_MODE, device_target=cfg.device_target) +``` + +详细的接口配置信息，请参见`context.set_context`接口说明。 + +### 运用Fuzzer + +1. 建立LeNet模型，加载MNIST数据集，操作同[模型安全]() + + ```python + ... + # Lenet model + model = Model(net) + # get training data + data_list = "./MNIST_unzip/train" + batch_size = 32 + ds = generate_mnist_dataset(data_list, batch_size, sparse=False) + train_images = [] + for data in ds.create_tuple_iterator(): + images = data[0].astype(np.float32) + train_images.append(images) + train_images = np.concatenate(train_images, axis=0) + + # get test data + data_list = "./MNIST_unzip/test" + batch_size = 32 + ds = generate_mnist_dataset(data_list, batch_size, sparse=False) + test_images = [] + test_labels = [] + for data in ds.create_tuple_iterator(): + images = data[0].astype(np.float32) + labels = data[1] + test_images.append(images) + test_labels.append(labels) + test_images = np.concatenate(test_images, axis=0) + test_labels = np.concatenate(test_labels, axis=0) + ``` + +2. Fuzzer参数配置。 + + 设置数据变异方法及参数。目前支持的数据变异方法包含三类： + + - 图像仿射变换方法：Translate、Scale、Shear、Rotate。 + - 基于图像像素值变化的方法： Contrast、Brightness、Blur、Noise。 + - 基于对抗攻击的白盒、黑盒对抗样本生成方法：FGSM、PGD、MDIIM。 + + 数据变异方法一定要包含基于图像像素值变化的方法。 + + 前两种图像变化方法的可配置参数，以及推荐参数范围请参考：对应的类方法，也可以均设置为`'auto_param': True`，变异参数将在推荐范围内随机生成。 + + 基于对抗攻击方法的参数配置请参考对应的攻击方法类。 + + ```python + mutate_config = [{'method': 'Blur', + 'params': {'auto_param': True}}, + {'method': 'Contrast', + 'params': {'auto_param': True}}, + {'method': 'Translate', + 'params': {'auto_param': True}}, + {'method': 'Brightness', + 'params': {'auto_param': True}}, + {'method': 'Noise', + 'params': {'auto_param': True}}, + {'method': 'Scale', + 'params': {'auto_param': True}}, + {'method': 'Shear', + 'params': {'auto_param': True}}, + {'method': 'FGSM', + 'params': {'eps': 0.3, 'alpha': 0.1}} + ] + ``` + + 设置评价指标，目前支持5种评价指标，包括: + - 通用评价指标：accuracy。 + - 神经元覆盖率指标：kmnc， nbc，snac。 + - 对抗攻击评价指标：attack_success_rate。 + 也可以设置为‘auto’，默认使用所有评价指标。 + + ```python + eval_metrics =['accuracy', 'kmnc', 'attack_success_rate'] + ``` + +3. 初始化种子队列，种子队列中的每个种子，包含3个值：原始图片、图片标签。 + + ```python + # make initial seeds + initial_seeds = [] + for img, label in zip(test_images, test_labels): + initial_seeds.append([img, label]) + initial_seeds = initial_seeds[:100] + ``` + +4. 测试Fuzz测试前的神经元覆盖率。 + + ```python + segmented_num=1000 + neuron_num=10 + model_coverage_test = ModelCoverageMetrics(model, segmented_num, neuron_num, train_images) + model_coverage_test.calculate_coverage(np.array(test_images[:100]).astype(np.float32)) + LOGGER.info(TAG, 'KMNC of this test is : %s', model_coverage_test.get_kmnc()) + ``` + + 结果： + + ```python + KMNC of this test is : 0.0851 + ``` + +5. Fuzz测试。 + + ```python + model_fuzz_test = Fuzzer(model, train_images, neuron_num, segmented_num) + _, _, _, _, metrics = model_fuzz_test.fuzzing(mutate_config, initial_seeds, eval_metrics=eval_metrics) + ``` + +6. 实验结果。 + + ```python + if metrics: + for key in metrics: + LOGGER.info(TAG, key + ': %s', metrics[key]) + ``` + + Fuzz测试后结果如下： + + ```python + Accuracy: 0.7929 + Attack_success_rate: 0.3939 + Neural_coverage_KMNC: 0.4797 + ``` + + Fuzz测试前种子的KMNC神经元覆盖率为8.5%，Fuzz后，KMNC神经元覆盖率为47.97%，神经元覆盖率提升，样本的多样性提升。Fuzz后，模型对于Fuzz生成样本的准确率为79.29%，使用了对抗攻击方法的样本，攻击成功率为47.97%。由于初始化种子、变异方法和相应的参数均为随机选择的，结果有一定的浮动是正常的。 + + 原始图片： + + ![fuzz_seed](./images/fuzz_seed.png) + +​ Fuzz生成的变异图片： + ![fuzz_res](./images/fuzz_res.png) \ No newline at end of file diff --git a/tutorials/source_zh_cn/advanced_use/gradient_accumulation.md b/tutorials/training/source_zh_cn/advanced_use/gradient_accumulation.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/gradient_accumulation.md rename to tutorials/training/source_zh_cn/advanced_use/gradient_accumulation.md diff --git a/tutorials/source_zh_cn/advanced_use/graph_kernel_fusion.md b/tutorials/training/source_zh_cn/advanced_use/graph_kernel_fusion.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/graph_kernel_fusion.md rename to tutorials/training/source_zh_cn/advanced_use/graph_kernel_fusion.md diff --git a/tutorials/source_zh_cn/advanced_use/host_device_training.md b/tutorials/training/source_zh_cn/advanced_use/host_device_training.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/host_device_training.md rename to tutorials/training/source_zh_cn/advanced_use/host_device_training.md diff --git a/tutorials/source_zh_cn/advanced_use/hub_tutorial.md b/tutorials/training/source_zh_cn/advanced_use/hub_tutorial.md similarity index 97% rename from tutorials/source_zh_cn/advanced_use/hub_tutorial.md rename to tutorials/training/source_zh_cn/advanced_use/hub_tutorial.md index 5969c90d637fe8ac26124ea16dfff036b2316f8a..3467a32d8a2f12212a1a1cc22ee4aac9d6000ee1 100644 --- a/tutorials/source_zh_cn/advanced_use/hub_tutorial.md +++ b/tutorials/training/source_zh_cn/advanced_use/hub_tutorial.md @@ -1,179 +1,179 @@ -## 使用MindSpore Hub提交、加载和微调模型 - -`Ascend` `GPU` `MindSpore Hub` `模型上传` `模型加载` `模型微调` `初级` `中级` `高级` - - - -- [使用MindSpore Hub提交、加载和微调模型](#使用MindSporeHub提交加载和微调模型) - - [概述](#概述) - - [模型上传](#模型上传) - - [步骤](#步骤) - - [模型加载](#模型加载) - - [模型微调](#模型微调) - - - - - -### 概述 - -本教程以Googlenet为例，对想要将模型发布到MindSpore Hub的算法开发者介绍了模型上传步骤，也对想要使用MindSpore Hub模型进行推理或者微调的开发应用者描述了具体操作流程。总之，本教程可以帮助算法开发者有效地提交模型，并使得应用开发者利用MindSpore Hub的接口快速实现模型推理或微调。 - -### 模型上传 - -我们接收用户通过向`hub`仓提交PR的方式向MindSpore Hub发布模型。这里我们用Googlenet为例，列出将模型提交到MindSpore Hub的步骤。 - -#### 步骤 - -1. 将你的预训练模型托管在我们可以访问的存储位置。 - -2. 按照 [模板](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/googlenet/mindspore_hub_conf.py) 在你自己的代码仓中添加模型生成文件 `mindspore_hub_conf.py`。 - -3. 按照 [模板](https://gitee.com/mindspore/hub/blob/master/mshub_res/assets/mindspore/gpu/0.6/alexnet_v1_cifar10.md) 在 `hub/mshub_res/assets` 中创建`{model_name}_{model_version}_{dataset}.md` 文件。对于每个预训练模型，执行以下命令，用来获得`.md`文件`asset-sha256` 处所需的哈希值: - - ```python - cd ../tools - python get_sha256.py ../googlenet.ckpt - ``` - -4. 使用 `hub/mshub_res/tools/md_validator.py` 在本地核对`.md`文件的格式，执行的命令如下： - - ```python - python md_validator.py ../assets/mindspore/ascend/0.7/googlenet_v1_cifar10.md - ``` - -5. 在 `mindspore/hub` 仓创建PR。 - -一旦你的PR合并到 `mindspore/hub` 的master分支，你的模型将于24小时内在 [MindSpore Hub 网站](https://hub.mindspore.com/mindspore) 上显示。更多详细信息，请参考 [README](https://gitee.com/mindspore/hub/blob/master/mshub_res/README.md) 。 - -### 模型加载 - -`mindspore_hub.load` API用于加载预训练模型，可以实现一行代码加载模型。主要的模型加载流程如下： - -- 在MindSpore Hub官网上搜索感兴趣的模型。 - - 例如，想使用Googlenet对CIFAR-10数据集进行分类，可以在MindSpore Hub官网上使用关键词`GoogleNet`进行搜索。页面将会返回与Googlenet相关的所有模型。进入相关模型页面之后，获得详情页 `url`。 - -- 使用`url`完成模型的加载，示例代码如下： - - ```python - import mindspore_hub as mshub - import mindspore - from mindspore import context, Tensor, nn - from mindspore.train.model import Model - from mindspore.common import dtype as mstype - from mindspore.dataset.transforms import py_transforms - from PIL import Image - import cv2 - - context.set_context(mode=context.GRAPH_MODE, - device_target="Ascend", - device_id=0) - - model = "mindspore/ascend/0.7/googlenet_v1_cifar10" - - image = Image.open('cifar10/a.jpg') - transforms = py_transforms.ComposeOp([py_transforms.ToTensor()]) - - # Initialize the number of classes based on the pre-trained model. - network = mshub.load(model, num_classes=10) - network.set_train(False) - out = network(transforms(image)) - ``` - -### 模型微调 - -在使用 `mindspore_hub.load` 进行模型加载时，可以增加一个额外的参数项只加载神经网络的特征提取部分。这样我们就能很容易地在之后增加一些新的层进行迁移学习。*当算法工程师将额外的参数（例如 include_top)添加到模型构造中时，可以在模型的详情页中找到这个功能。* - -下面我们以GoogleNet为例，说明如何加载一个基于ImageNet的预训练模型，并在特定的子任务数据集上进行迁移学习（重训练）。主要的步骤如下： - -1. 在MindSpore Hub的官网上搜索感兴趣的模型，并从网站上获取特定的 `url`。 - -2. 使用 `url`进行MindSpore Hub模型的加载，*注意：`include_top` 参数需要模型开发者提供*。 - - ```python - import mindspore - from mindspore import nn - from mindspore import context - import mindspore_hub as mshub - - context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", - save_graphs=False) - - network = mshub.load('mindspore/ascend/0.7/googlenet_v1_cifar10', include_top=False) - network.set_train(False) - ``` - -3. 在现有模型结构基础上增加一个与新任务相关的分类层。 - - ```python - # Check MindSpore Hub website to conclude that the last output shape is 1024. - last_channel = 1024 - - # The number of classes in target task is 26. - num_classes = 26 - classification_layer = nn.Dense(last_channel, num_classes) - classification_layer.set_train(True) - - train_network = nn.SequentialCell([network, classification_layer]) - ``` - -4. 为模型训练选择损失函数和优化器。 - - ```python - from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits - - # Wrap the backbone network with loss. - loss_fn = SoftmaxCrossEntropyWithLogits() - loss_net = nn.WithLossCell(train_network, loss_fn) - - # Create an optimizer. - optim = opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), Tensor(lr), config.momentum, config.weight_decay) - train_net = nn.TrainOneStepCell(loss_net, optim) - ``` - -5. 构建数据集，开始重训练。 - - ```python - from src.dataset import create_dataset - from mindspore.train.serialization import _exec_save_checkpoint - - dataset = create_dataset("/ssd/data/garbage/train", do_train=True, batch_size=32) - - epoch_size = 15 - for epoch in range(epoch_size): - for i, items in enumerate(dataset): - data, label = items - data = mindspore.Tensor(data) - label = mindspore.Tensor(label) - - loss = train_net(data, label) - print(f"epoch: {epoch}, loss: {loss}") - # Save the ckpt file for each epoch. - ckpt_path = f"./ckpt/garbage_finetune_epoch{epoch}.ckpt" - _exec_save_checkpoint(train_network, ckpt_path) - ``` - -6. 在测试集上测试模型精度。 - - ```python - from mindspore.train.serialization import load_checkpoint, load_param_into_net - - network = mshub.load('mindspore/ascend/0.7/googlenet_v1_cifar10', include_top=False) - train_network = nn.SequentialCell([network, nn.Dense(last_channel, num_classes)]) - - # Load a pre-trained ckpt file. - ckpt_path = "./ckpt/garbage_finetune_epoch15.ckpt" - trained_ckpt = load_checkpoint(ckpt_path) - load_param_into_net(train_network, trained_ckpt) - - # Define loss and create model. - loss_fn = SoftmaxCrossEntropyWithLogits() - model = Model(network, loss_fn=loss, metrics={'acc'}) - - eval_dataset = create_dataset("/ssd/data/garbage/train", do_train=False, - batch_size=32) - - res = model.eval(eval_dataset) - print("result:", res, "ckpt=", ckpt_path) - ``` +## 使用MindSpore Hub提交、加载和微调模型 + +`Ascend` `GPU` `MindSpore Hub` `模型上传` `模型加载` `模型微调` `初级` `中级` `高级` + + + +- [使用MindSpore Hub提交、加载和微调模型](#使用MindSporeHub提交加载和微调模型) + - [概述](#概述) + - [模型上传](#模型上传) + - [步骤](#步骤) + - [模型加载](#模型加载) + - [模型微调](#模型微调) + + + + + +### 概述 + +本教程以Googlenet为例，对想要将模型发布到MindSpore Hub的算法开发者介绍了模型上传步骤，也对想要使用MindSpore Hub模型进行推理或者微调的开发应用者描述了具体操作流程。总之，本教程可以帮助算法开发者有效地提交模型，并使得应用开发者利用MindSpore Hub的接口快速实现模型推理或微调。 + +### 模型上传 + +我们接收用户通过向`hub`仓提交PR的方式向MindSpore Hub发布模型。这里我们用Googlenet为例，列出将模型提交到MindSpore Hub的步骤。 + +#### 步骤 + +1. 将你的预训练模型托管在我们可以访问的存储位置。 + +2. 按照 [模板](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/googlenet/mindspore_hub_conf.py) 在你自己的代码仓中添加模型生成文件 `mindspore_hub_conf.py`。 + +3. 按照 [模板](https://gitee.com/mindspore/hub/blob/master/mshub_res/assets/mindspore/gpu/0.6/alexnet_v1_cifar10.md) 在 `hub/mshub_res/assets` 中创建`{model_name}_{model_version}_{dataset}.md` 文件。对于每个预训练模型，执行以下命令，用来获得`.md`文件`asset-sha256` 处所需的哈希值: + + ```python + cd ../tools + python get_sha256.py ../googlenet.ckpt + ``` + +4. 使用 `hub/mshub_res/tools/md_validator.py` 在本地核对`.md`文件的格式，执行的命令如下： + + ```python + python md_validator.py ../assets/mindspore/ascend/0.7/googlenet_v1_cifar10.md + ``` + +5. 在 `mindspore/hub` 仓创建PR。 + +一旦你的PR合并到 `mindspore/hub` 的master分支，你的模型将于24小时内在 [MindSpore Hub 网站](https://hub.mindspore.com/mindspore) 上显示。更多详细信息，请参考 [README](https://gitee.com/mindspore/hub/blob/master/mshub_res/README.md) 。 + +### 模型加载 + +`mindspore_hub.load` API用于加载预训练模型，可以实现一行代码加载模型。主要的模型加载流程如下： + +- 在MindSpore Hub官网上搜索感兴趣的模型。 + + 例如，想使用Googlenet对CIFAR-10数据集进行分类，可以在MindSpore Hub官网上使用关键词`GoogleNet`进行搜索。页面将会返回与Googlenet相关的所有模型。进入相关模型页面之后，获得详情页 `url`。 + +- 使用`url`完成模型的加载，示例代码如下： + + ```python + import mindspore_hub as mshub + import mindspore + from mindspore import context, Tensor, nn + from mindspore.train.model import Model + from mindspore.common import dtype as mstype + from mindspore.dataset.transforms import py_transforms + from PIL import Image + import cv2 + + context.set_context(mode=context.GRAPH_MODE, + device_target="Ascend", + device_id=0) + + model = "mindspore/ascend/0.7/googlenet_v1_cifar10" + + image = Image.open('cifar10/a.jpg') + transforms = py_transforms.ComposeOp([py_transforms.ToTensor()]) + + # Initialize the number of classes based on the pre-trained model. + network = mshub.load(model, num_classes=10) + network.set_train(False) + out = network(transforms(image)) + ``` + +### 模型微调 + +在使用 `mindspore_hub.load` 进行模型加载时，可以增加一个额外的参数项只加载神经网络的特征提取部分。这样我们就能很容易地在之后增加一些新的层进行迁移学习。*当算法工程师将额外的参数（例如 include_top)添加到模型构造中时，可以在模型的详情页中找到这个功能。* + +下面我们以GoogleNet为例，说明如何加载一个基于ImageNet的预训练模型，并在特定的子任务数据集上进行迁移学习（重训练）。主要的步骤如下： + +1. 在MindSpore Hub的官网上搜索感兴趣的模型，并从网站上获取特定的 `url`。 + +2. 使用 `url`进行MindSpore Hub模型的加载，*注意：`include_top` 参数需要模型开发者提供*。 + + ```python + import mindspore + from mindspore import nn + from mindspore import context + import mindspore_hub as mshub + + context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", + save_graphs=False) + + network = mshub.load('mindspore/ascend/0.7/googlenet_v1_cifar10', include_top=False) + network.set_train(False) + ``` + +3. 在现有模型结构基础上增加一个与新任务相关的分类层。 + + ```python + # Check MindSpore Hub website to conclude that the last output shape is 1024. + last_channel = 1024 + + # The number of classes in target task is 26. + num_classes = 26 + classification_layer = nn.Dense(last_channel, num_classes) + classification_layer.set_train(True) + + train_network = nn.SequentialCell([network, classification_layer]) + ``` + +4. 为模型训练选择损失函数和优化器。 + + ```python + from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits + + # Wrap the backbone network with loss. + loss_fn = SoftmaxCrossEntropyWithLogits() + loss_net = nn.WithLossCell(train_network, loss_fn) + + # Create an optimizer. + optim = opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), Tensor(lr), config.momentum, config.weight_decay) + train_net = nn.TrainOneStepCell(loss_net, optim) + ``` + +5. 构建数据集，开始重训练。 + + ```python + from src.dataset import create_dataset + from mindspore.train.serialization import _exec_save_checkpoint + + dataset = create_dataset("/ssd/data/garbage/train", do_train=True, batch_size=32) + + epoch_size = 15 + for epoch in range(epoch_size): + for i, items in enumerate(dataset): + data, label = items + data = mindspore.Tensor(data) + label = mindspore.Tensor(label) + + loss = train_net(data, label) + print(f"epoch: {epoch}, loss: {loss}") + # Save the ckpt file for each epoch. + ckpt_path = f"./ckpt/garbage_finetune_epoch{epoch}.ckpt" + _exec_save_checkpoint(train_network, ckpt_path) + ``` + +6. 在测试集上测试模型精度。 + + ```python + from mindspore.train.serialization import load_checkpoint, load_param_into_net + + network = mshub.load('mindspore/ascend/0.7/googlenet_v1_cifar10', include_top=False) + train_network = nn.SequentialCell([network, nn.Dense(last_channel, num_classes)]) + + # Load a pre-trained ckpt file. + ckpt_path = "./ckpt/garbage_finetune_epoch15.ckpt" + trained_ckpt = load_checkpoint(ckpt_path) + load_param_into_net(train_network, trained_ckpt) + + # Define loss and create model. + loss_fn = SoftmaxCrossEntropyWithLogits() + model = Model(network, loss_fn=loss, metrics={'acc'}) + + eval_dataset = create_dataset("/ssd/data/garbage/train", do_train=False, + batch_size=32) + + res = model.eval(eval_dataset) + print("result:", res, "ckpt=", ckpt_path) + ``` diff --git a/tutorials/source_zh_cn/advanced_use/images/adv_attack_result.png b/tutorials/training/source_zh_cn/advanced_use/images/adv_attack_result.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/adv_attack_result.png rename to tutorials/training/source_zh_cn/advanced_use/images/adv_attack_result.png diff --git a/tutorials/source_zh_cn/advanced_use/images/auto_augmentation.png b/tutorials/training/source_zh_cn/advanced_use/images/auto_augmentation.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/auto_augmentation.png rename to tutorials/training/source_zh_cn/advanced_use/images/auto_augmentation.png diff --git a/tutorials/source_zh_cn/advanced_use/images/bert_model.PNG b/tutorials/training/source_zh_cn/advanced_use/images/bert_model.PNG similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/bert_model.PNG rename to tutorials/training/source_zh_cn/advanced_use/images/bert_model.PNG diff --git a/tutorials/source_zh_cn/advanced_use/images/checkpoint_integration_process.jpg b/tutorials/training/source_zh_cn/advanced_use/images/checkpoint_integration_process.jpg similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/checkpoint_integration_process.jpg rename to tutorials/training/source_zh_cn/advanced_use/images/checkpoint_integration_process.jpg diff --git a/tutorials/source_zh_cn/advanced_use/images/cifar10.jpg b/tutorials/training/source_zh_cn/advanced_use/images/cifar10.jpg similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/cifar10.jpg rename to tutorials/training/source_zh_cn/advanced_use/images/cifar10.jpg diff --git a/tutorials/source_zh_cn/advanced_use/images/cifar10_c_transforms.png b/tutorials/training/source_zh_cn/advanced_use/images/cifar10_c_transforms.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/cifar10_c_transforms.png rename to tutorials/training/source_zh_cn/advanced_use/images/cifar10_c_transforms.png diff --git a/tutorials/source_zh_cn/advanced_use/images/cloud_train_job1.png b/tutorials/training/source_zh_cn/advanced_use/images/cloud_train_job1.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/cloud_train_job1.png rename to tutorials/training/source_zh_cn/advanced_use/images/cloud_train_job1.png diff --git a/tutorials/source_zh_cn/advanced_use/images/cloud_train_job2.png b/tutorials/training/source_zh_cn/advanced_use/images/cloud_train_job2.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/cloud_train_job2.png rename to tutorials/training/source_zh_cn/advanced_use/images/cloud_train_job2.png diff --git a/tutorials/source_zh_cn/advanced_use/images/compose.png b/tutorials/training/source_zh_cn/advanced_use/images/compose.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/compose.png rename to tutorials/training/source_zh_cn/advanced_use/images/compose.png diff --git a/tutorials/source_zh_cn/advanced_use/images/data_chart.png b/tutorials/training/source_zh_cn/advanced_use/images/data_chart.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/data_chart.png rename to tutorials/training/source_zh_cn/advanced_use/images/data_chart.png diff --git a/tutorials/source_zh_cn/advanced_use/images/data_conversion_concept.png b/tutorials/training/source_zh_cn/advanced_use/images/data_conversion_concept.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/data_conversion_concept.png rename to tutorials/training/source_zh_cn/advanced_use/images/data_conversion_concept.png diff --git a/tutorials/source_zh_cn/advanced_use/images/data_enhancement_performance_scheme.png b/tutorials/training/source_zh_cn/advanced_use/images/data_enhancement_performance_scheme.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/data_enhancement_performance_scheme.png rename to tutorials/training/source_zh_cn/advanced_use/images/data_enhancement_performance_scheme.png diff --git a/tutorials/source_zh_cn/advanced_use/images/data_function.png b/tutorials/training/source_zh_cn/advanced_use/images/data_function.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/data_function.png rename to tutorials/training/source_zh_cn/advanced_use/images/data_function.png diff --git a/tutorials/source_zh_cn/advanced_use/images/data_label.png b/tutorials/training/source_zh_cn/advanced_use/images/data_label.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/data_label.png rename to tutorials/training/source_zh_cn/advanced_use/images/data_label.png diff --git a/tutorials/source_zh_cn/advanced_use/images/data_loading_performance_scheme.png b/tutorials/training/source_zh_cn/advanced_use/images/data_loading_performance_scheme.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/data_loading_performance_scheme.png rename to tutorials/training/source_zh_cn/advanced_use/images/data_loading_performance_scheme.png diff --git a/tutorials/source_zh_cn/advanced_use/images/data_op_profile.png b/tutorials/training/source_zh_cn/advanced_use/images/data_op_profile.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/data_op_profile.png rename to tutorials/training/source_zh_cn/advanced_use/images/data_op_profile.png diff --git a/tutorials/source_zh_cn/advanced_use/images/data_table.png b/tutorials/training/source_zh_cn/advanced_use/images/data_table.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/data_table.png rename to tutorials/training/source_zh_cn/advanced_use/images/data_table.png diff --git a/tutorials/source_zh_cn/advanced_use/images/dataset_pipeline.png b/tutorials/training/source_zh_cn/advanced_use/images/dataset_pipeline.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/dataset_pipeline.png rename to tutorials/training/source_zh_cn/advanced_use/images/dataset_pipeline.png diff --git a/tutorials/source_zh_cn/advanced_use/images/finetune.PNG b/tutorials/training/source_zh_cn/advanced_use/images/finetune.PNG similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/finetune.PNG rename to tutorials/training/source_zh_cn/advanced_use/images/finetune.PNG diff --git a/tutorials/source_zh_cn/advanced_use/images/fuzz_res.png b/tutorials/training/source_zh_cn/advanced_use/images/fuzz_res.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/fuzz_res.png rename to tutorials/training/source_zh_cn/advanced_use/images/fuzz_res.png diff --git a/tutorials/source_zh_cn/advanced_use/images/fuzz_seed.png b/tutorials/training/source_zh_cn/advanced_use/images/fuzz_seed.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/fuzz_seed.png rename to tutorials/training/source_zh_cn/advanced_use/images/fuzz_seed.png diff --git a/tutorials/source_zh_cn/advanced_use/images/gpu_activity_profiler.png b/tutorials/training/source_zh_cn/advanced_use/images/gpu_activity_profiler.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/gpu_activity_profiler.png rename to tutorials/training/source_zh_cn/advanced_use/images/gpu_activity_profiler.png diff --git a/tutorials/source_zh_cn/advanced_use/images/gpu_op_ui_profiler.png b/tutorials/training/source_zh_cn/advanced_use/images/gpu_op_ui_profiler.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/gpu_op_ui_profiler.png rename to tutorials/training/source_zh_cn/advanced_use/images/gpu_op_ui_profiler.png diff --git a/tutorials/source_zh_cn/advanced_use/images/graph.png b/tutorials/training/source_zh_cn/advanced_use/images/graph.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/graph.png rename to tutorials/training/source_zh_cn/advanced_use/images/graph.png diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png b/tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png rename to tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png b/tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png rename to tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png b/tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png rename to tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png b/tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png rename to tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png b/tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png rename to tutorials/training/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_sidebar.png b/tutorials/training/source_zh_cn/advanced_use/images/graph_sidebar.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/graph_sidebar.png rename to tutorials/training/source_zh_cn/advanced_use/images/graph_sidebar.png diff --git a/tutorials/source_zh_cn/advanced_use/images/histogram.png b/tutorials/training/source_zh_cn/advanced_use/images/histogram.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/histogram.png rename to tutorials/training/source_zh_cn/advanced_use/images/histogram.png diff --git a/tutorials/source_zh_cn/advanced_use/images/histogram_func.png b/tutorials/training/source_zh_cn/advanced_use/images/histogram_func.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/histogram_func.png rename to tutorials/training/source_zh_cn/advanced_use/images/histogram_func.png diff --git a/tutorials/source_zh_cn/advanced_use/images/image_function.png b/tutorials/training/source_zh_cn/advanced_use/images/image_function.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/image_function.png rename to tutorials/training/source_zh_cn/advanced_use/images/image_function.png diff --git a/tutorials/source_zh_cn/advanced_use/images/image_vi.png b/tutorials/training/source_zh_cn/advanced_use/images/image_vi.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/image_vi.png rename to tutorials/training/source_zh_cn/advanced_use/images/image_vi.png diff --git a/tutorials/source_zh_cn/advanced_use/images/introduce.PNG b/tutorials/training/source_zh_cn/advanced_use/images/introduce.PNG similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/introduce.PNG rename to tutorials/training/source_zh_cn/advanced_use/images/introduce.PNG diff --git a/tutorials/source_zh_cn/advanced_use/images/lineage_label.png b/tutorials/training/source_zh_cn/advanced_use/images/lineage_label.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/lineage_label.png rename to tutorials/training/source_zh_cn/advanced_use/images/lineage_label.png diff --git a/tutorials/source_zh_cn/advanced_use/images/lineage_model_chart.png b/tutorials/training/source_zh_cn/advanced_use/images/lineage_model_chart.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/lineage_model_chart.png rename to tutorials/training/source_zh_cn/advanced_use/images/lineage_model_chart.png diff --git a/tutorials/source_zh_cn/advanced_use/images/lineage_model_table.png b/tutorials/training/source_zh_cn/advanced_use/images/lineage_model_table.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/lineage_model_table.png rename to tutorials/training/source_zh_cn/advanced_use/images/lineage_model_table.png diff --git a/tutorials/source_zh_cn/advanced_use/images/minddata_profile.png b/tutorials/training/source_zh_cn/advanced_use/images/minddata_profile.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/minddata_profile.png rename to tutorials/training/source_zh_cn/advanced_use/images/minddata_profile.png diff --git a/tutorials/source_zh_cn/advanced_use/images/mindrecord.png b/tutorials/training/source_zh_cn/advanced_use/images/mindrecord.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/mindrecord.png rename to tutorials/training/source_zh_cn/advanced_use/images/mindrecord.png diff --git a/tutorials/source_zh_cn/advanced_use/images/mix_precision.eddx b/tutorials/training/source_zh_cn/advanced_use/images/mix_precision.eddx similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/mix_precision.eddx rename to tutorials/training/source_zh_cn/advanced_use/images/mix_precision.eddx diff --git a/tutorials/source_zh_cn/advanced_use/images/mix_precision.jpg b/tutorials/training/source_zh_cn/advanced_use/images/mix_precision.jpg similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/mix_precision.jpg rename to tutorials/training/source_zh_cn/advanced_use/images/mix_precision.jpg diff --git a/tutorials/source_zh_cn/advanced_use/images/multi_scalars.png b/tutorials/training/source_zh_cn/advanced_use/images/multi_scalars.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/multi_scalars.png rename to tutorials/training/source_zh_cn/advanced_use/images/multi_scalars.png diff --git a/tutorials/source_zh_cn/advanced_use/images/multi_scalars_select.png b/tutorials/training/source_zh_cn/advanced_use/images/multi_scalars_select.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/multi_scalars_select.png rename to tutorials/training/source_zh_cn/advanced_use/images/multi_scalars_select.png diff --git a/tutorials/source_zh_cn/advanced_use/images/op_statistics.PNG b/tutorials/training/source_zh_cn/advanced_use/images/op_statistics.PNG similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/op_statistics.PNG rename to tutorials/training/source_zh_cn/advanced_use/images/op_statistics.PNG diff --git a/tutorials/source_zh_cn/advanced_use/images/op_type_statistics.PNG b/tutorials/training/source_zh_cn/advanced_use/images/op_type_statistics.PNG similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/op_type_statistics.PNG rename to tutorials/training/source_zh_cn/advanced_use/images/op_type_statistics.PNG diff --git a/tutorials/source_zh_cn/advanced_use/images/operator_fusion.png b/tutorials/training/source_zh_cn/advanced_use/images/operator_fusion.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/operator_fusion.png rename to tutorials/training/source_zh_cn/advanced_use/images/operator_fusion.png diff --git a/tutorials/source_zh_cn/advanced_use/images/performance_overall.png b/tutorials/training/source_zh_cn/advanced_use/images/performance_overall.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/performance_overall.png rename to tutorials/training/source_zh_cn/advanced_use/images/performance_overall.png diff --git a/tutorials/source_zh_cn/advanced_use/images/pipeline.png b/tutorials/training/source_zh_cn/advanced_use/images/pipeline.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/pipeline.png rename to tutorials/training/source_zh_cn/advanced_use/images/pipeline.png diff --git a/tutorials/source_zh_cn/advanced_use/images/scalar.png b/tutorials/training/source_zh_cn/advanced_use/images/scalar.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/scalar.png rename to tutorials/training/source_zh_cn/advanced_use/images/scalar.png diff --git a/tutorials/source_zh_cn/advanced_use/images/scalar_compound.png b/tutorials/training/source_zh_cn/advanced_use/images/scalar_compound.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/scalar_compound.png rename to tutorials/training/source_zh_cn/advanced_use/images/scalar_compound.png diff --git a/tutorials/source_zh_cn/advanced_use/images/scalar_select.png b/tutorials/training/source_zh_cn/advanced_use/images/scalar_select.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/scalar_select.png rename to tutorials/training/source_zh_cn/advanced_use/images/scalar_select.png diff --git a/tutorials/source_zh_cn/advanced_use/images/shuffle.png b/tutorials/training/source_zh_cn/advanced_use/images/shuffle.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/shuffle.png rename to tutorials/training/source_zh_cn/advanced_use/images/shuffle.png diff --git a/tutorials/source_zh_cn/advanced_use/images/shuffle_performance_scheme.png b/tutorials/training/source_zh_cn/advanced_use/images/shuffle_performance_scheme.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/shuffle_performance_scheme.png rename to tutorials/training/source_zh_cn/advanced_use/images/shuffle_performance_scheme.png diff --git a/tutorials/source_zh_cn/advanced_use/images/step_trace.png b/tutorials/training/source_zh_cn/advanced_use/images/step_trace.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/step_trace.png rename to tutorials/training/source_zh_cn/advanced_use/images/step_trace.png diff --git a/tutorials/source_zh_cn/advanced_use/images/synchronization_training_and_evaluation.png b/tutorials/training/source_zh_cn/advanced_use/images/synchronization_training_and_evaluation.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/synchronization_training_and_evaluation.png rename to tutorials/training/source_zh_cn/advanced_use/images/synchronization_training_and_evaluation.png diff --git a/tutorials/source_zh_cn/advanced_use/images/targets.png b/tutorials/training/source_zh_cn/advanced_use/images/targets.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/targets.png rename to tutorials/training/source_zh_cn/advanced_use/images/targets.png diff --git a/tutorials/source_zh_cn/advanced_use/images/tensor_function.png b/tutorials/training/source_zh_cn/advanced_use/images/tensor_function.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/tensor_function.png rename to tutorials/training/source_zh_cn/advanced_use/images/tensor_function.png diff --git a/tutorials/source_zh_cn/advanced_use/images/tensor_histogram.png b/tutorials/training/source_zh_cn/advanced_use/images/tensor_histogram.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/tensor_histogram.png rename to tutorials/training/source_zh_cn/advanced_use/images/tensor_histogram.png diff --git a/tutorials/source_zh_cn/advanced_use/images/tensor_table.png b/tutorials/training/source_zh_cn/advanced_use/images/tensor_table.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/tensor_table.png rename to tutorials/training/source_zh_cn/advanced_use/images/tensor_table.png diff --git a/tutorials/source_zh_cn/advanced_use/images/timeline.png b/tutorials/training/source_zh_cn/advanced_use/images/timeline.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/timeline.png rename to tutorials/training/source_zh_cn/advanced_use/images/timeline.png diff --git a/tutorials/source_zh_cn/advanced_use/images/train_log_1_Ascend.png b/tutorials/training/source_zh_cn/advanced_use/images/train_log_1_Ascend.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/train_log_1_Ascend.png rename to tutorials/training/source_zh_cn/advanced_use/images/train_log_1_Ascend.png diff --git a/tutorials/source_zh_cn/advanced_use/images/train_log_8_Ascend.png b/tutorials/training/source_zh_cn/advanced_use/images/train_log_8_Ascend.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/train_log_8_Ascend.png rename to tutorials/training/source_zh_cn/advanced_use/images/train_log_8_Ascend.png diff --git a/tutorials/source_zh_cn/advanced_use/images/train_log_8_Ascend_clu.png b/tutorials/training/source_zh_cn/advanced_use/images/train_log_8_Ascend_clu.png similarity index 100% rename from tutorials/source_zh_cn/advanced_use/images/train_log_8_Ascend_clu.png rename to tutorials/training/source_zh_cn/advanced_use/images/train_log_8_Ascend_clu.png diff --git a/tutorials/source_zh_cn/advanced_use/lineage_and_scalars_comparision.md b/tutorials/training/source_zh_cn/advanced_use/lineage_and_scalars_comparision.md similarity index 97% rename from tutorials/source_zh_cn/advanced_use/lineage_and_scalars_comparision.md rename to tutorials/training/source_zh_cn/advanced_use/lineage_and_scalars_comparision.md index eca4a19f995063e9419649d78e74a9da1187c3c0..3022aa5b4d8bb7f5ae3de707df3e408a5bded5b3 100644 --- a/tutorials/source_zh_cn/advanced_use/lineage_and_scalars_comparision.md +++ b/tutorials/training/source_zh_cn/advanced_use/lineage_and_scalars_comparision.md @@ -1,111 +1,111 @@ -# 溯源和对比看板 - -`Linux` `Ascend` `GPU` `CPU` `模型调优` `中级` `高级` - - - -- [溯源和对比看板](#溯源和对比看板) - - [概述](#概述) - - [模型溯源](#模型溯源) - - [数据溯源](#数据溯源) - - [对比看板](#对比看板) - - [注意事项](#注意事项) - - - -   - - -## 概述 - -MindInsight中的模型溯源、数据溯源和对比看板同训练看板一样属于可视化组件中的重要组成部分，在对训练数据的可视化中，通过对比看板观察不同标量趋势图发现问题，再使用溯源功能定位问题原因，给用户在数据增强和深度神经网络中提供高效调优的能力。 - -## 模型溯源 - -模型溯源可视化用于展示所有训练的模型参数信息。 - -![image.png](./images/lineage_label.png) - -图1：模型参数选择区 - -图1展示的模型参数选择区，列举了可供查看的模型参数标签。用户可以通过勾选所需的标签，查看相应的模型参数。 - -![image.png](./images/lineage_model_chart.png) - -图2：模型溯源功能区 - -图2展示的模型溯源功能区，图像化展示了模型的参数信息。用户可以通过选择列的特定区域，展示区域范围内的模型信息。 - -![image.png](./images/lineage_model_table.png) - -图3：模型列表 - -图3分组展示所有模型信息，用户可以按指定列进行升序或降序展示模型信息。 - -左侧概览页展示优化目标和相关参数的信息。 - -![targets.png](./images/targets.png) - -图4：概览页 - -图4展示的是优化目标分布、参数重要性和散点图。 - -## 数据溯源 - -数据溯源可视化用于展示所有训练的数据处理和数据增强信息。 - -![data_label.png](./images/data_label.png) - -图5：数据处理和增强算子选择区 - -图5展示的数据处理和数据增强算子选择区，列举了可供查看的数据处理和增强算子的名称。用户可以通过勾选所需的标签，查看相应的参数等信息。 - -![data_chart.png](./images/data_chart.png) - -图6：数据溯源功能区 - -图6展示的数据溯源功能区，图像化展示了数据处理和数据增强使用的参数信息。用户可以通过选择列的特定区域，展示区域范围内的参数信息。 - -![data_table.png](./images/data_table.png) - -图7：数据溯源列表 - -图7展示所有模型训练的数据处理和数据增强信息。 - -> 如果用户筛选模型溯源随后切换到数据溯源页面时，折线图将展示最新一次筛选过的模型溯源列。 - -## 对比看板 - -对比看板可视用于多个训练之间的标量曲线对比。 - -![multi_scalars.png](./images/multi_scalars.png) - -图8: 标量对比曲线图 - -图8展示了多个训练之间的标量曲线对比效果，横坐标是训练步骤，纵坐标是标量值。 - -图中右上角有几个按钮功能，从左到右功能分别是全屏展示，切换Y轴比例，开启/关闭框选，分步回退和还原图形。 - -- 全屏展示即全屏展示该标量曲线，再点击一次即可恢复。 -- 切换Y轴比例是指可以将Y轴坐标进行对数转换。 -- 开启/关闭框选是指可以框选图中部分区域，并放大查看该区域， 可以在已放大的图形上叠加框选。 -- 分步回退是指对同一个区域连续框选并放大查看时，可以逐步撤销操作。 -- 还原图形是指进行了多次框选后，点击此按钮可以将图还原回原始状态。 - -![multi_scalars_select.png](./images/multi_scalars_select.png) - -图9：对比看板可视功能区 - -图9展示的对比看板可视的功能区，提供了根据选择不同训练或标签，水平轴的不同维度和平滑度来进行标量对比的功能。 - -- 训练: 提供了对所有训练进行多项选择的功能，用户可以通过勾选或关键字筛选所需的训练。 -- 标签：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的标量信息。 -- 水平轴：可以选择“步骤”、“相对时间”、“绝对时间”中的任意一项，来作为标量曲线的水平轴。 -- 平滑度：可以通过调整平滑度，对标量曲线进行平滑处理。 - -## 注意事项 - -出于性能上的考虑，MindInsight对比看板使用缓存机制加载训练的标量曲线数据，并进行以下限制： -- 对比看板只支持在缓存中的训练进行比较标量曲线对比。 -- 缓存最多保留最新（按修改时间排列）的15个训练。 +# 溯源和对比看板 + +`Linux` `Ascend` `GPU` `CPU` `模型调优` `中级` `高级` + + + +- [溯源和对比看板](#溯源和对比看板) + - [概述](#概述) + - [模型溯源](#模型溯源) + - [数据溯源](#数据溯源) + - [对比看板](#对比看板) + - [注意事项](#注意事项) + + + +   + + +## 概述 + +MindInsight中的模型溯源、数据溯源和对比看板同训练看板一样属于可视化组件中的重要组成部分，在对训练数据的可视化中，通过对比看板观察不同标量趋势图发现问题，再使用溯源功能定位问题原因，给用户在数据增强和深度神经网络中提供高效调优的能力。 + +## 模型溯源 + +模型溯源可视化用于展示所有训练的模型参数信息。 + +![image.png](./images/lineage_label.png) + +图1：模型参数选择区 + +图1展示的模型参数选择区，列举了可供查看的模型参数标签。用户可以通过勾选所需的标签，查看相应的模型参数。 + +![image.png](./images/lineage_model_chart.png) + +图2：模型溯源功能区 + +图2展示的模型溯源功能区，图像化展示了模型的参数信息。用户可以通过选择列的特定区域，展示区域范围内的模型信息。 + +![image.png](./images/lineage_model_table.png) + +图3：模型列表 + +图3分组展示所有模型信息，用户可以按指定列进行升序或降序展示模型信息。 + +左侧概览页展示优化目标和相关参数的信息。 + +![targets.png](./images/targets.png) + +图4：概览页 + +图4展示的是优化目标分布、参数重要性和散点图。 + +## 数据溯源 + +数据溯源可视化用于展示所有训练的数据处理和数据增强信息。 + +![data_label.png](./images/data_label.png) + +图5：数据处理和增强算子选择区 + +图5展示的数据处理和数据增强算子选择区，列举了可供查看的数据处理和增强算子的名称。用户可以通过勾选所需的标签，查看相应的参数等信息。 + +![data_chart.png](./images/data_chart.png) + +图6：数据溯源功能区 + +图6展示的数据溯源功能区，图像化展示了数据处理和数据增强使用的参数信息。用户可以通过选择列的特定区域，展示区域范围内的参数信息。 + +![data_table.png](./images/data_table.png) + +图7：数据溯源列表 + +图7展示所有模型训练的数据处理和数据增强信息。 + +> 如果用户筛选模型溯源随后切换到数据溯源页面时，折线图将展示最新一次筛选过的模型溯源列。 + +## 对比看板 + +对比看板可视用于多个训练之间的标量曲线对比。 + +![multi_scalars.png](./images/multi_scalars.png) + +图8: 标量对比曲线图 + +图8展示了多个训练之间的标量曲线对比效果，横坐标是训练步骤，纵坐标是标量值。 + +图中右上角有几个按钮功能，从左到右功能分别是全屏展示，切换Y轴比例，开启/关闭框选，分步回退和还原图形。 + +- 全屏展示即全屏展示该标量曲线，再点击一次即可恢复。 +- 切换Y轴比例是指可以将Y轴坐标进行对数转换。 +- 开启/关闭框选是指可以框选图中部分区域，并放大查看该区域， 可以在已放大的图形上叠加框选。 +- 分步回退是指对同一个区域连续框选并放大查看时，可以逐步撤销操作。 +- 还原图形是指进行了多次框选后，点击此按钮可以将图还原回原始状态。 + +![multi_scalars_select.png](./images/multi_scalars_select.png) + +图9：对比看板可视功能区 + +图9展示的对比看板可视的功能区，提供了根据选择不同训练或标签，水平轴的不同维度和平滑度来进行标量对比的功能。 + +- 训练: 提供了对所有训练进行多项选择的功能，用户可以通过勾选或关键字筛选所需的训练。 +- 标签：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的标量信息。 +- 水平轴：可以选择“步骤”、“相对时间”、“绝对时间”中的任意一项，来作为标量曲线的水平轴。 +- 平滑度：可以通过调整平滑度，对标量曲线进行平滑处理。 + +## 注意事项 + +出于性能上的考虑，MindInsight对比看板使用缓存机制加载训练的标量曲线数据，并进行以下限制： +- 对比看板只支持在缓存中的训练进行比较标量曲线对比。 +- 缓存最多保留最新（按修改时间排列）的15个训练。 - 用户最多同时对比5个训练的标量曲线。 \ No newline at end of file diff --git a/tutorials/source_zh_cn/advanced_use/membership_inference.md b/tutorials/training/source_zh_cn/advanced_use/membership_inference.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/membership_inference.md rename to tutorials/training/source_zh_cn/advanced_use/membership_inference.md diff --git a/tutorials/source_zh_cn/advanced_use/mindinsight_commands.md b/tutorials/training/source_zh_cn/advanced_use/mindinsight_commands.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/mindinsight_commands.md rename to tutorials/training/source_zh_cn/advanced_use/mindinsight_commands.md diff --git a/tutorials/source_zh_cn/advanced_use/mixed_precision.md b/tutorials/training/source_zh_cn/advanced_use/mixed_precision.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/mixed_precision.md rename to tutorials/training/source_zh_cn/advanced_use/mixed_precision.md diff --git a/tutorials/source_zh_cn/advanced_use/mobilenetv2_incremental_learning.md b/tutorials/training/source_zh_cn/advanced_use/mobilenetv2_incremental_learning.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/mobilenetv2_incremental_learning.md rename to tutorials/training/source_zh_cn/advanced_use/mobilenetv2_incremental_learning.md diff --git a/tutorials/source_zh_cn/advanced_use/model_scripts_transformation.md b/tutorials/training/source_zh_cn/advanced_use/model_scripts_transformation.md similarity index 80% rename from tutorials/source_zh_cn/advanced_use/model_scripts_transformation.md rename to tutorials/training/source_zh_cn/advanced_use/model_scripts_transformation.md index 20fbfb3570d388b110dafc7b0edcb580c7b2b3cf..604349df91ee9ab7c7a9ac2c2583db4fe0679194 100644 --- a/tutorials/source_zh_cn/advanced_use/model_scripts_transformation.md +++ b/tutorials/training/source_zh_cn/advanced_use/model_scripts_transformation.md @@ -59,10 +59,10 @@ optional arguments: `--project_path` when use graph based schema. ``` -MindConverter提供两种模型脚本迁移方案： +**MindConverter提供两种模型脚本迁移方案：** -1. 基于抽象语法树（Abstract syntax tree, AST）的脚本转换：指定`--in_file`的值，将使用基于AST的脚本转换方案； -2. 基于图结构的脚本生成：指定`--model_file`与`--shape`将使用基于图结构的脚本生成方案。 +1. **基于抽象语法树(Abstract syntax tree, AST)的脚本转换**：指定`--in_file`的值，将使用基于AST的脚本转换方案； +2. **基于图结构的脚本生成**：指定`--model_file`与`--shape`将使用基于图结构的脚本生成方案。 > 若同时指定了`--in_file`，`--model_file`将默认使用AST方案进行脚本迁移。 @@ -70,8 +70,9 @@ MindConverter提供两种模型脚本迁移方案： 其中，`--output`与`--report`参数可省略。若省略，MindConverter将在当前工作目录（Working directory）下自动创建`output`目录，将生成的脚本、转换报告输出至该目录。 -另外，当使用基于图结构的脚本生成方案时，请确保原PyTorch项目已在PYTHONPATH中，可通过CLI进入Python交互式命令行，通过import的方式判断是否已满足；若未加入，可通过`--project_path`命令手动将项目路径传入，以确保MindConverter可引用到原PyTorch脚本。 +另外，当使用基于图结构的脚本生成方案时，请确保原PyTorch项目已在Python包搜索路径中，可通过CLI进入Python交互式命令行，通过import的方式判断是否已满足；若未加入，可通过`--project_path`命令手动将项目路径传入，以确保MindConverter可引用到原PyTorch脚本。 +> 假设用户项目目录为`/home/user/project/model_training`，用户可通过如下命令手动项目添加至包搜索路径中：`export PYTHONPATH=/home/user/project/model_training:$PYTHONPATH` > 此处MindConverter需要引用原PyTorch脚本，是因为PyTorch模型反向序列化过程中会引用原脚本。 @@ -86,6 +87,11 @@ MindConverter提供两种技术方案，以应对不同脚本迁移场景： 对于上述第二种场景，推荐用户使用基于图结构的脚本生成方案，计算图作为一种标准的模型描述语言，可以消除用户代码风格多样导致的脚本转换率不稳定的问题。在已支持算子的情况下，该方案可提供优于AST方案的转换率。 +目前已基于典型图像分类网络(Resnet, VGG)对图结构的脚本转换方案进行测试。 + +> 1. 基于图结构的脚本生成方案，目前仅支持单输入、单输出模型，对于多输入模型暂不支持； +> 2. 基于图结构的脚本生成方案，由于要基于推理模式加载PyTorch模型，会导致转换后网络中Dropout算子丢失，需要用户手动补齐； +> 3. 基于图结构的脚本生成方案持续优化中。 ## 使用示例 @@ -94,7 +100,9 @@ MindConverter提供两种技术方案，以应对不同脚本迁移场景： 若用户希望使用基于AST的方案进行脚本迁移，假设原PyTorch脚本路径为`/home/user/model.py`，希望将脚本输出至`/home/user/output`，转换报告输出至`/home/user/output/report`，则脚本转换命令为： ```bash -mindconverter --in_file /home/user/model.py --output /home/user/output --report /home/user/output/report +mindconverter --in_file /home/user/model.py \ + --output /home/user/output \ + --report /home/user/output/report ``` 转换报告中，对于未转换的代码行形式为如下，其中x, y指明的是原PyTorch脚本中代码的行、列号。对于未成功转换的算子，可参考[MindSporeAPI映射查询功能](https://www.mindspore.cn/docs/zh-CN/master/index.html#operator_api) 手动对代码进行迁移。对于工具无法迁移的算子，会保留原脚本中的代码。 @@ -114,7 +122,7 @@ line x:y: [UnConvert] 'operator' didn't convert. ... [Convert Over] ``` -对于部分未成功转换的算子，报告中会提供修改建议，如`line 157:23`，会建议将`torch.nn.AdaptiveAvgPool2d`替换为`mindspore.ops.operations.ReduceMean`。 +对于部分未成功转换的算子，报告中会提供修改建议，如`line 157:23`，MindConverter建议将`torch.nn.AdaptiveAvgPool2d`替换为`mindspore.ops.operations.ReduceMean`。 ### 基于图结构的脚本生成示例 @@ -122,7 +130,10 @@ line x:y: [UnConvert] 'operator' didn't convert. ... 若用户已将PyTorch模型保存为.pth格式，假设模型绝对路径为`/home/user/model.pth`，该模型期望的输入样本shape为(3, 224, 224)，原PyTorch脚本位于`/home/user/project/model_training`，希望将脚本输出至`/home/user/output`，转换报告输出至`/home/user/output/report`，则脚本生成命令为： ```bash -mindconverter --model_file /home/user/model.pth --shape 3,224,224 --output /home/user/output --report /home/user/output/report --project_path /home/user/project/model_training +mindconverter --model_file /home/user/model.pth --shape 3,224,224 \ + --output /home/user/output \ + --report /home/user/output/report \ + --project_path /home/user/project/model_training ``` 执行该命令，MindSpore代码文件、转换报告生成至相应目录。 @@ -139,5 +150,5 @@ mindconverter --model_file /home/user/model.pth --shape 3,224,224 --output /home ## 注意事项 1. PyTorch不作为MindInsight明确声明的依赖库，但若想使用基于图结构的脚本生成工具，需要用户手动安装与生成PyTorch模型版本一致的PyTorch库； -2. MindConverter目前仅支持单输入模型，对于多输入模型暂不支持； -3. 脚本转换工具本质上为算子驱动，对于MindConverter未维护的PyTorch或ONNX算子与MindSpore算子映射，将会出现相应的算子无法转换的问题，对于该类算子，用户可手动修改，或基于MindConverter实现映射关系，向MindInsight仓库贡献。 \ No newline at end of file +2. 脚本转换工具本质上为算子驱动，对于MindConverter未维护的PyTorch或ONNX算子与MindSpore算子映射，将会出现相应的算子无法转换的问题，对于该类算子，用户可手动修改，或基于MindConverter实现映射关系，向MindInsight仓库贡献。 + diff --git a/tutorials/source_zh_cn/advanced_use/model_security.md b/tutorials/training/source_zh_cn/advanced_use/model_security.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/model_security.md rename to tutorials/training/source_zh_cn/advanced_use/model_security.md diff --git a/tutorials/source_zh_cn/advanced_use/network_migration.md b/tutorials/training/source_zh_cn/advanced_use/network_migration.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/network_migration.md rename to tutorials/training/source_zh_cn/advanced_use/network_migration.md diff --git a/tutorials/source_zh_cn/advanced_use/nlp_application.md b/tutorials/training/source_zh_cn/advanced_use/nlp_application.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/nlp_application.md rename to tutorials/training/source_zh_cn/advanced_use/nlp_application.md diff --git a/tutorials/source_zh_cn/advanced_use/optimize_the_performance_of_data_preparation.md b/tutorials/training/source_zh_cn/advanced_use/optimize_the_performance_of_data_preparation.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/optimize_the_performance_of_data_preparation.md rename to tutorials/training/source_zh_cn/advanced_use/optimize_the_performance_of_data_preparation.md diff --git a/tutorials/source_zh_cn/advanced_use/parameter_server_training.md b/tutorials/training/source_zh_cn/advanced_use/parameter_server_training.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/parameter_server_training.md rename to tutorials/training/source_zh_cn/advanced_use/parameter_server_training.md diff --git a/tutorials/source_zh_cn/advanced_use/performance_profiling.md b/tutorials/training/source_zh_cn/advanced_use/performance_profiling.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/performance_profiling.md rename to tutorials/training/source_zh_cn/advanced_use/performance_profiling.md diff --git a/tutorials/source_zh_cn/advanced_use/performance_profiling_gpu.md b/tutorials/training/source_zh_cn/advanced_use/performance_profiling_gpu.md similarity index 97% rename from tutorials/source_zh_cn/advanced_use/performance_profiling_gpu.md rename to tutorials/training/source_zh_cn/advanced_use/performance_profiling_gpu.md index 68a7ef86cb87e5c2e7fee8d6178850b34a168718..bc7002b089d6ea9becd8837a8bbeec8aad63efc0 100644 --- a/tutorials/source_zh_cn/advanced_use/performance_profiling_gpu.md +++ b/tutorials/training/source_zh_cn/advanced_use/performance_profiling_gpu.md @@ -1,121 +1,121 @@ -# 性能调试（GPU） - -`Linux` `GPU` `模型调优` `中级` `高级` - - - -- [性能调试（GPU）](#性能调试-gpu) - - [概述](#概述) - - [操作流程](#操作流程) - - [准备训练脚本](#准备训练脚本) - - [启动MindInsight](#启动mindinsight) - - [性能分析](#性能分析) - - [算子性能分析](#算子性能分析) - - [Timeline分析](#timeline分析) - - - - - -## 概述 -将训练过程中的算子耗时等信息记录到文件中，通过可视化界面供用户查看分析，帮助用户更高效地调试神经网络性能。 - -## 操作流程 - -> 操作流程可以参考Ascend 910上profiler的操作： -> -> - -## 准备训练脚本 - -为了收集神经网络的性能数据，需要在训练脚本中添加MindSpore Profiler相关接口。 -- `set_context`之后，需要初始化MindSpore `Profiler`对象，GPU场景下初始化Profiler对象时只有output_path参数有效。 -- 在训练结束后，调用`Profiler.analyse()`停止性能数据收集并生成性能分析结果。 - -> 样例代码与Ascend使用方式一致可以参考： -> -> - -GPU场景下还可以用自定义callback的方式收集性能数据，示例如下： - -```python -class StopAtStep(Callback): - def __init__(self, start_step, stop_step): - super(StopAtStep, self).__init__() - self.start_step = start_step - self.stop_step = stop_step - self.already_analysed = False - - def step_begin(self, run_context): - cb_params = run_context.original_args() - step_num = cb_params.cur_step_num - if step_num == self.start_step: - self.profiler = Profiler() - - def step_end(self, run_context): - cb_params = run_context.original_args() - step_num = cb_params.cur_step_num - if step_num == self.stop_step and not self.already_analysed: - self.profiler.analyse() - self.already_analysed = True - - def end(self, run_context): - if not self.already_analysed: - self.profiler.analyse() -``` - -以上代码仅供参考，用户可根据所需场景自由实现。 - -## 启动MindInsight - -启动命令请参考[MindInsight相关命令](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/mindinsight_commands.html)。 - - -### 性能分析 - -用户从训练列表中选择指定的训练，点击性能调试，可以查看该次训练的性能数据（目前GPU场景只支持算子耗时排名统计和Timeline功能，其他功能敬请期待）。 - -![performance_overall.png](./images/performance_overall.png) - -图1：性能数据总览 - -图1展示了性能数据总览页面，包含了迭代轨迹（Step Trace）、算子性能、MindData性能和Timeline等组件的数据总体呈现。目前GPU场景下只支持算子性能统计功能： -- 算子性能：统计单算子以及各算子类型的执行时间，进行排序展示；总览页中展示了各算子类型平均执行时间占比的饼状图。 - -用户可以点击查看详情链接，进入组件页面进行详细分析。 - -#### 算子性能分析 - -使用算子性能分析组件可以对MindSpore运行过程中的各个算子的执行时间进行统计展示。 - -![gpu_op_ui_profiler.png](./images/gpu_op_ui_profiler.png) - -图2：算子类别统计分析 - -图2展示了按算子类别进行统计分析的结果，包含以下内容： -- 可以选择饼图/柱状图展示各算子类别的时间占比，每个算子类别的执行时间会统计属于该类别的算子执行时间总和以及平均执行时间。 -- 统计前20个平均执行时间最长的算子类别。 - -图2下半部分展示了算子性能统计表，包含以下内容： -- 选择全部：按单个算子的统计结果进行排序展示，展示维度包括算子位置（Device/Host）、算子类型、算子执行时间、算子全名等；默认按算子平均执行时间排序。 -- 选择分类：按算子类别的统计结果进行排序展示，展示维度包括算子分类名称、算子类别执行时间、执行频次、执行总时间的比例、平均执行时间。点击每个算子类别，可以进一步查看该类别下所有单个算子的统计信息。 -- 搜索：在右侧搜索框中输入字符串，支持对算子名称/类别进行模糊搜索。 - -![gpu_activity_profiler.png](./images/gpu_activity_profiler.png) - -图3：内核信息分析 - -图3展示了CUDA activity信息统计，包含以下内容： - -- 统计图表：展示了各个kernel activity的占比以及前15个算子的耗时信息。 -- 内核信息列表：信息列表展示activity的名称、所属算子名称、执行次数、总时间、平均时间等信息。 -- 搜索：可以通过name(activity名称)以及op_full_name（所属算子名称）进程部分匹配的搜索。 - -#### Timeline分析 - -GPU场景下，Timeline分析的使用方法和Ascend场景相同，不同之处是，GPU Timeline展示的是算子信息和CUDA activity的信息。使用方法参考： - -> 样例代码与Ascend使用方式一致可以参考： -> -> - +# 性能调试（GPU） + +`Linux` `GPU` `模型调优` `中级` `高级` + + + +- [性能调试（GPU）](#性能调试-gpu) + - [概述](#概述) + - [操作流程](#操作流程) + - [准备训练脚本](#准备训练脚本) + - [启动MindInsight](#启动mindinsight) + - [性能分析](#性能分析) + - [算子性能分析](#算子性能分析) + - [Timeline分析](#timeline分析) + + + + + +## 概述 +将训练过程中的算子耗时等信息记录到文件中，通过可视化界面供用户查看分析，帮助用户更高效地调试神经网络性能。 + +## 操作流程 + +> 操作流程可以参考Ascend 910上profiler的操作： +> +> + +## 准备训练脚本 + +为了收集神经网络的性能数据，需要在训练脚本中添加MindSpore Profiler相关接口。 +- `set_context`之后，需要初始化MindSpore `Profiler`对象，GPU场景下初始化Profiler对象时只有output_path参数有效。 +- 在训练结束后，调用`Profiler.analyse()`停止性能数据收集并生成性能分析结果。 + +> 样例代码与Ascend使用方式一致可以参考： +> +> + +GPU场景下还可以用自定义callback的方式收集性能数据，示例如下： + +```python +class StopAtStep(Callback): + def __init__(self, start_step, stop_step): + super(StopAtStep, self).__init__() + self.start_step = start_step + self.stop_step = stop_step + self.already_analysed = False + + def step_begin(self, run_context): + cb_params = run_context.original_args() + step_num = cb_params.cur_step_num + if step_num == self.start_step: + self.profiler = Profiler() + + def step_end(self, run_context): + cb_params = run_context.original_args() + step_num = cb_params.cur_step_num + if step_num == self.stop_step and not self.already_analysed: + self.profiler.analyse() + self.already_analysed = True + + def end(self, run_context): + if not self.already_analysed: + self.profiler.analyse() +``` + +以上代码仅供参考，用户可根据所需场景自由实现。 + +## 启动MindInsight + +启动命令请参考[MindInsight相关命令](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/mindinsight_commands.html)。 + + +### 性能分析 + +用户从训练列表中选择指定的训练，点击性能调试，可以查看该次训练的性能数据（目前GPU场景只支持算子耗时排名统计和Timeline功能，其他功能敬请期待）。 + +![performance_overall.png](./images/performance_overall.png) + +图1：性能数据总览 + +图1展示了性能数据总览页面，包含了迭代轨迹（Step Trace）、算子性能、MindData性能和Timeline等组件的数据总体呈现。目前GPU场景下只支持算子性能统计功能： +- 算子性能：统计单算子以及各算子类型的执行时间，进行排序展示；总览页中展示了各算子类型平均执行时间占比的饼状图。 + +用户可以点击查看详情链接，进入组件页面进行详细分析。 + +#### 算子性能分析 + +使用算子性能分析组件可以对MindSpore运行过程中的各个算子的执行时间进行统计展示。 + +![gpu_op_ui_profiler.png](./images/gpu_op_ui_profiler.png) + +图2：算子类别统计分析 + +图2展示了按算子类别进行统计分析的结果，包含以下内容： +- 可以选择饼图/柱状图展示各算子类别的时间占比，每个算子类别的执行时间会统计属于该类别的算子执行时间总和以及平均执行时间。 +- 统计前20个平均执行时间最长的算子类别。 + +图2下半部分展示了算子性能统计表，包含以下内容： +- 选择全部：按单个算子的统计结果进行排序展示，展示维度包括算子位置（Device/Host）、算子类型、算子执行时间、算子全名等；默认按算子平均执行时间排序。 +- 选择分类：按算子类别的统计结果进行排序展示，展示维度包括算子分类名称、算子类别执行时间、执行频次、执行总时间的比例、平均执行时间。点击每个算子类别，可以进一步查看该类别下所有单个算子的统计信息。 +- 搜索：在右侧搜索框中输入字符串，支持对算子名称/类别进行模糊搜索。 + +![gpu_activity_profiler.png](./images/gpu_activity_profiler.png) + +图3：内核信息分析 + +图3展示了CUDA activity信息统计，包含以下内容： + +- 统计图表：展示了各个kernel activity的占比以及前15个算子的耗时信息。 +- 内核信息列表：信息列表展示activity的名称、所属算子名称、执行次数、总时间、平均时间等信息。 +- 搜索：可以通过name(activity名称)以及op_full_name（所属算子名称）进程部分匹配的搜索。 + +#### Timeline分析 + +GPU场景下，Timeline分析的使用方法和Ascend场景相同，不同之处是，GPU Timeline展示的是算子信息和CUDA activity的信息。使用方法参考： + +> 样例代码与Ascend使用方式一致可以参考： +> +> + diff --git a/tutorials/source_zh_cn/advanced_use/quantization_aware.md b/tutorials/training/source_zh_cn/advanced_use/quantization_aware.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/quantization_aware.md rename to tutorials/training/source_zh_cn/advanced_use/quantization_aware.md diff --git a/tutorials/source_zh_cn/advanced_use/second_order_optimizer_for_resnet50_application.md b/tutorials/training/source_zh_cn/advanced_use/second_order_optimizer_for_resnet50_application.md similarity index 97% rename from tutorials/source_zh_cn/advanced_use/second_order_optimizer_for_resnet50_application.md rename to tutorials/training/source_zh_cn/advanced_use/second_order_optimizer_for_resnet50_application.md index b63c08ab952bac78d5067b479dbd5a3aea66ad01..35dd89dd17b27479e5b5c6bfd4da4c87aa73e0a4 100644 --- a/tutorials/source_zh_cn/advanced_use/second_order_optimizer_for_resnet50_application.md +++ b/tutorials/training/source_zh_cn/advanced_use/second_order_optimizer_for_resnet50_application.md @@ -1,464 +1,464 @@ -# ResNet-50二阶优化实践 - -`Linux` `Ascend` `GPU` `模型开发` `模型调优` `高级` - - - -- [ResNet-50二阶优化实践](#resnet-50二阶优化实践) - - [概述](#概述) - - [准备环节](#准备环节) - - [准备数据集](#准备数据集) - - [配置分布式环境变量](#配置分布式环境变量) - - [加载处理数据集](#加载处理数据集) - - [定义网络](#定义网络) - - [定义损失函数及THOR优化器](#定义损失函数及thor优化器) - - [定义损失函数](#定义损失函数) - - [定义优化器](#定义优化器) - - [训练网络](#训练网络) - - [配置模型保存](#配置模型保存) - - [配置训练网络](#配置训练网络) - - [运行脚本](#运行脚本) - - [模型推理](#模型推理) - - -   - -## 概述 - -常见的优化算法可分为一阶优化算法和二阶优化算法。经典的一阶优化算法如SGD等，计算量小、计算速度快，但是收敛的速度慢，所需的迭代次数多。而二阶优化算法使用目标函数的二阶导数来加速收敛，能更快地收敛到模型最优值，所需要的迭代次数少，但由于二阶优化算法过高的计算成本，导致其总体执行时间仍然慢于一阶，故目前在深度神经网络训练中二阶优化算法的应用并不普遍。二阶优化算法的主要计算成本在于二阶信息矩阵（Hessian矩阵、[FIM矩阵](https://arxiv.org/pdf/1808.07172.pdf)等）的求逆运算，时间复杂度约为$O(n^3)$。 - -MindSpore开发团队在现有的自然梯度算法的基础上，对FIM矩阵采用近似、切分等优化加速手段，极大的降低了逆矩阵的计算复杂度，开发出了可用的二阶优化器THOR。使用8块Ascend 910 AI处理器，THOR可以在72min内完成ResNet50-v1.5网络和ImageNet数据集的训练，相比于SGD+Momentum速度提升了近一倍。 - - -本篇教程将主要介绍如何在Ascend 910 以及GPU上，使用MindSpore提供的二阶优化器THOR训练ResNet50-v1.5网络和ImageNet数据集。 -> 你可以在这里下载完整的示例代码： - 。 - -### 示例代码目录结构 - -```shell -├── resnet_thor - ├── README.md - ├── scripts - ├── run_distribute_train.sh # launch distributed training for Ascend 910 - └── run_eval.sh # launch inference for Ascend 910 - ├── run_distribute_train_gpu.sh # launch distributed training for GPU - └── run_eval_gpu.sh # launch inference for GPU - ├── src - ├── crossentropy.py # CrossEntropy loss function - ├── config.py # parameter configuration - ├── dataset_helper.py # dataset helper for minddata dataset - ├── grad_reducer_thor.py # grad reduce for thor - ├── model_thor.py # model for train - ├── resnet_thor.py # resnet50_thor backone - ├── thor.py # thor optimizer - ├── thor_layer.py # thor layer - └── dataset.py # data preprocessing - ├── eval.py # infer script - └── train.py # train script - -``` - -整体执行流程如下： -1. 准备ImageNet数据集，处理需要的数据集； -2. 定义ResNet50网络； -3. 定义损失函数和THOR优化器； -4. 加载数据集并进行训练，训练完成后，查看结果及保存模型文件； -5. 加载保存的模型，进行推理。 - - -## 准备环节 - -实践前，确保已经正确安装MindSpore。如果没有，可以通过[MindSpore安装页面](https://www.mindspore.cn/install)安装MindSpore。 - -### 准备数据集 - -下载完整的ImageNet2012数据集，将数据集解压分别存放到本地工作区的`ImageNet2012/ilsvrc`、`ImageNet2012/ilsvrc_eval`路径下。 - -目录结构如下： - -``` -└─ImageNet2012 - ├─ilsvrc - │ n03676483/ - │ n04067472/ - │ n01622779/ - │ ...... - └─ilsvrc_eval - │ n03018349/ - │ n02504013 - │ n07871810 - │ ...... - -``` -### 配置分布式环境变量 -#### Ascend 910 -Ascend 910 AI处理器的分布式环境变量配置参考[分布式并行训练 (Ascend)](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/distributed_training_ascend.html#id4)。 - -#### GPU -GPU的分布式环境配置参考[分布式并行训练 (GPU)](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/distributed_training_gpu.html#id4)。 - - -## 加载处理数据集 - -分布式训练时，通过并行的方式加载数据集，同时通过MindSpore提供的数据增强接口对数据集进行处理。加载处理数据集的脚本在源码的`src/dataset.py`脚本中。 -```python -import os -import mindspore.common.dtype as mstype -import mindspore.dataset.engine as de -import mindspore.dataset.vision.c_transforms as C -import mindspore.dataset.transforms.c_transforms as C2 -from mindspore.communication.management import init, get_rank, get_group_size - -def create_dataset(dataset_path, do_train, repeat_num=1, batch_size=32, target="Ascend"): - if target == "Ascend": - device_num, rank_id = _get_rank_info() - else: - init() - rank_id = get_rank() - device_num = get_group_size() - if device_num == 1: - ds = de.ImageFolderDataset(dataset_path, num_parallel_workers=8, shuffle=True) - else: - ds = de.ImageFolderDataset(dataset_path, num_parallel_workers=8, shuffle=True, - num_shards=device_num, shard_id=rank_id) - - image_size = 224 - mean = [0.485 * 255, 0.456 * 255, 0.406 * 255] - std = [0.229 * 255, 0.224 * 255, 0.225 * 255] - # define map operations - if do_train: - trans = [ - C.RandomCropDecodeResize(image_size, scale=(0.08, 1.0), ratio=(0.75, 1.333)), - C.RandomHorizontalFlip(prob=0.5), - C.Normalize(mean=mean, std=std), - C.HWC2CHW() - ] - else: - trans = [ - C.Decode(), - C.Resize(256), - C.CenterCrop(image_size), - C.Normalize(mean=mean, std=std), - C.HWC2CHW() - ] - type_cast_op = C2.TypeCast(mstype.int32) - ds = ds.map(operations=trans, input_columns="image", num_parallel_workers=8) - ds = ds.map(operations=type_cast_op, input_columns="label", num_parallel_workers=8) - - # apply batch operations - ds = ds.batch(batch_size, drop_remainder=True) - - # apply dataset repeat operation - ds = ds.repeat(repeat_num) - - return ds -``` - -> MindSpore支持进行多种数据处理和增强的操作，各种操作往往组合使用，具体可以参考[数据处理与数据增强](https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/data_processing_and_augmentation.html)章节。 - - -## 定义网络 -本示例中使用的网络模型为ResNet50-v1.5，先定义[ResNet50网络](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py)，然后使用二阶优化器自定义的算子替换`Conv2d`和 -和`Dense`算子。定义好的网络模型在在源码`src/resnet_thor.py`脚本中，自定义的算子`Conv2d_thor`和`Dense_thor`在`src/thor_layer.py`脚本中。 - -- 使用`Conv2d_thor`替换原网络模型中的`Conv2d` -- 使用`Dense_thor`替换原网络模型中的`Dense` - -> 使用THOR自定义的算子`Conv2d_thor`和`Dense_thor`是为了保存模型训练中的二阶矩阵信息，新定义的网络与原网络模型的backbone一致。 - -网络构建完成以后，在`__main__`函数中调用定义好的ResNet50： -```python -... -from src.resnet_thor import resnet50 -... -f __name__ == "__main__": - ... - # define the net - net = resnet50(class_num=config.class_num, damping=damping, loss_scale=config.loss_scale, - frequency=config.frequency, batch_size=config.batch_size) - ... -``` - - -## 定义损失函数及THOR优化器 - - -### 定义损失函数 - -MindSpore支持的损失函数有`SoftmaxCrossEntropyWithLogits`、`L1Loss`、`MSELoss`等。THOR优化器需要使用`SoftmaxCrossEntropyWithLogits`损失函数。 - -损失函数的实现步骤在`src/crossentropy.py`脚本中。这里使用了深度网络模型训练中的一个常用trick：label smoothing，通过对真实标签做平滑处理，提高模型对分类错误标签的容忍度，从而可以增加模型的泛化能力。 -```python -class CrossEntropy(_Loss): - """CrossEntropy""" - def __init__(self, smooth_factor=0., num_classes=1000): - super(CrossEntropy, self).__init__() - self.onehot = P.OneHot() - self.on_value = Tensor(1.0 - smooth_factor, mstype.float32) - self.off_value = Tensor(1.0 * smooth_factor / (num_classes - 1), mstype.float32) - self.ce = nn.SoftmaxCrossEntropyWithLogits() - self.mean = P.ReduceMean(False) - - def construct(self, logit, label): - one_hot_label = self.onehot(label, F.shape(logit)[1], self.on_value, self.off_value) - loss = self.ce(logit, one_hot_label) - loss = self.mean(loss, 0) - return loss -``` -在`__main__`函数中调用定义好的损失函数： - -```python -... -from src.crossentropy import CrossEntropy -... -if __name__ == "__main__": - ... - # define the loss function - if not config.use_label_smooth: - config.label_smooth_factor = 0.0 - loss = CrossEntropy(smooth_factor=config.label_smooth_factor, num_classes=config.class_num) - ... -``` - -### 定义优化器 - -THOR优化器的参数更新公式如下： - -$$ \theta^{t+1} = \theta^t + \alpha F^{-1}\nabla E$$ - -参数更新公式中各参数的含义如下： -- $\theta$：网络中的可训参数； -- $t$：迭代次数； -- $\alpha$：学习率值，参数的更新步长； -- $F^{-1}$：FIM矩阵，在网络中计算获得； -- $\nabla E$：一阶梯度值。 - -从参数更新公式中可以看出，THOR优化器需要额外计算的是每一层的FIM矩阵，每一层的FIM矩阵就是之前在自定义的网络模型中计算获得的。FIM矩阵可以对每一层参数更新的步长和方向进行自适应的调整，加速收敛的同时可以降低调参的复杂度。 - -```python -... -if args_opt.device_target == "Ascend": - from src.thor import THOR -else: - from src.thor import THOR_GPU as THOR -... - -if __name__ == "__main__": - ... - # learning rate setting - lr = get_model_lr(0, config.lr_init, config.lr_decay, config.lr_end_epoch, step_size, decay_epochs=39) - # define the optimizer - opt = THOR(filter(lambda x: x.requires_grad, net.get_parameters()), Tensor(lr), config.momentum, - filter(lambda x: 'matrix_A' in x.name, net.get_parameters()), - filter(lambda x: 'matrix_G' in x.name, net.get_parameters()), - filter(lambda x: 'A_inv_max' in x.name, net.get_parameters()), - filter(lambda x: 'G_inv_max' in x.name, net.get_parameters()), - config.weight_decay, config.loss_scale) - ... -``` - -## 训练网络 - -### 配置模型保存 - -MindSpore提供了callback机制，可以在训练过程中执行自定义逻辑，这里使用框架提供的`ModelCheckpoint`函数。 -`ModelCheckpoint`可以保存网络模型和参数，以便进行后续的fine-tuning操作。 -`TimeMonitor`、`LossMonitor`是MindSpore官方提供的callback函数，可以分别用于监控训练过程中单步迭代时间和`loss`值的变化。 - -```python -... -from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, TimeMonitor, LossMonitor -... -if __name__ == "__main__": - ... - # define callbacks - time_cb = TimeMonitor(data_size=step_size) - loss_cb = LossMonitor() - cb = [time_cb, loss_cb] - if config.save_checkpoint: - config_ck = CheckpointConfig(save_checkpoint_steps=config.save_checkpoint_epochs * step_size, - keep_checkpoint_max=config.keep_checkpoint_max) - ckpt_cb = ModelCheckpoint(prefix="resnet", directory=ckpt_save_dir, config=config_ck) - cb += [ckpt_cb] - ... -``` - -### 配置训练网络 - -通过MindSpore提供的`model.train`接口可以方便地进行网络的训练。THOR优化器通过降低二阶矩阵更新频率，来减少计算量，提升计算速度，故重新定义一个Model_Thor类，继承MindSpore提供的Model类。在Model_Thor类中增加二阶矩阵更新频率控制参数，用户可以通过调整该参数，优化整体的性能。 - - -```python -... -from mindspore.train.loss_scale_manager import FixedLossScaleManager -from src.model_thor import Model_Thor as Model -... - -if __name__ == "__main__": - ... - loss_scale = FixedLossScaleManager(config.loss_scale, drop_overflow_update=False) - if target == "Ascend": - model = Model(net, loss_fn=loss, optimizer=opt, amp_level='O2', loss_scale_manager=loss_scale, - keep_batchnorm_fp32=False, metrics={'acc'}, frequency=config.frequency) - else: - model = Model(net, loss_fn=loss, optimizer=opt, loss_scale_manager=loss_scale, metrics={'acc'}, - amp_level="O2", keep_batchnorm_fp32=True, frequency=config.frequency) - ... -``` - -### 运行脚本 -训练脚本定义完成之后，调`scripts`目录下的shell脚本，启动分布式训练进程。 -#### Ascend 910 -目前MindSpore分布式在Ascend上执行采用单卡单进程运行方式，即每张卡上运行1个进程，进程数量与使用的卡的数量一致。其中，0卡在前台执行，其他卡放在后台执行。每个进程创建1个目录，目录名称为`train_parallel`+ `device_id`，用来保存日志信息，算子编译信息以及训练的checkpoint文件。下面以使用8张卡的分布式训练脚本为例，演示如何运行脚本： - -使用以下命令运行脚本： -``` -sh run_distribute_train.sh [RANK_TABLE_FILE] [DATASET_PATH] [DEVICE_NUM] -``` -脚本需要传入变量`RANK_TABLE_FILE`、`DATASET_PATH`和`DEVICE_NUM`，其中： -- `RANK_TABLE_FILE`：组网信息文件的路径。 -- `DATASET_PATH`：训练数据集路径。 -- `DEVICE_NUM`: 实际的运行卡数。 -其余环境变量请参考安装教程中的配置项。 - -训练过程中loss打印示例如下： - -```bash -... -epoch: 1 step: 5004, loss is 4.4182425 -epoch: 2 step: 5004, loss is 3.740064 -epoch: 3 step: 5004, loss is 4.0546017 -epoch: 4 step: 5004, loss is 3.7598825 -epoch: 5 step: 5004, loss is 3.3744206 -...... -epoch: 40 step: 5004, loss is 1.6907625 -epoch: 41 step: 5004, loss is 1.8217756 -epoch: 42 step: 5004, loss is 1.6453942 -... -``` - -训练完后，每张卡训练产生的checkpoint文件保存在各自训练目录下，`device_0`产生的checkpoint文件示例如下: - -```bash -└─train_parallel0 - ├─resnet-1_5004.ckpt - ├─resnet-2_5004.ckpt - │ ...... - ├─resnet-42_5004.ckpt -``` - -其中， -`*.ckpt`：指保存的模型参数文件。checkpoint文件名称具体含义：*网络名称*-*epoch数*_*step数*.ckpt。 - -#### GPU -在GPU硬件平台上，MindSpore采用OpenMPI的`mpirun`进行分布式训练，进程创建1个目录，目录名称为`train_parallel`，用来保存日志信息和训练的checkpoint文件。下面以使用8张卡的分布式训练脚本为例，演示如何运行脚本： -``` -sh run_distribute_train_gpu.sh [DATASET_PATH] [DEVICE_NUM] -``` -脚本需要传入变量`DATASET_PATH`和`DEVICE_NUM`，其中： -- `DATASET_PATH`：训练数据集路径。 -- `DEVICE_NUM`: 实际的运行卡数。 - -在GPU训练时，无需设置`DEVICE_ID`环境变量，因此在主训练脚本中不需要调用`int(os.getenv('DEVICE_ID'))`来获取卡的物理序号，同时`context`中也无需传入`device_id`。我们需要将device_target设置为GPU，并需要调用`init()`来使能NCCL。 - -训练过程中loss打印示例如下： -```bash -... -epoch: 1 step: 5004, loss is 4.2546034 -epoch: 2 step: 5004, loss is 4.0819564 -epoch: 3 step: 5004, loss is 3.7005644 -epoch: 4 step: 5004, loss is 3.2668946 -epoch: 5 step: 5004, loss is 3.023509 -...... -epoch: 36 step: 5004, loss is 1.645802 -... -``` - -训练完后，保存的模型文件示例如下: - -```bash -└─train_parallel - ├─ckpt_0 - ├─resnet-1_5004.ckpt - ├─resnet-2_5004.ckpt - │ ...... - ├─resnet-36_5004.ckpt - ...... - ├─ckpt_7 - ├─resnet-1_5004.ckpt - ├─resnet-2_5004.ckpt - │ ...... - ├─resnet-36_5004.ckpt - -``` - -## 模型推理 - -使用训练过程中保存的checkpoint文件进行推理，验证模型的泛化能力。首先通过`load_checkpoint`接口加载模型文件，然后调用`Model`的`eval`接口对输入图片类别作出预测，再与输入图片的真实类别做比较，得出最终的预测精度值。 - -### 定义推理网络 - -1. 使用`load_checkpoint`接口加载模型文件。 -2. 使用`model.eval`接口读入测试数据集，进行推理。 -3. 计算得出预测精度值。 - -```python -... -from mindspore.train.serialization import load_checkpoint, load_param_into_net -... - -if __name__ == "__main__": - ... - # define net - net = resnet(class_num=config.class_num) - net.add_flags_recursive(thor=False) - - # load checkpoint - param_dict = load_checkpoint(args_opt.checkpoint_path) - keys = list(param_dict.keys()) - for key in keys: - if "damping" in key: - param_dict.pop(key) - load_param_into_net(net, param_dict) - net.set_train(False) - - # define model - model = Model(net, loss_fn=loss, metrics={'top_1_accuracy', 'top_5_accuracy'}) - - # eval model - res = model.eval(dataset) - print("result:", res, "ckpt=", args_opt.checkpoint_path) -``` - -### 执行推理 -推理网络定义完成之后，调用`scripts`目录下的shell脚本，进行推理。 -#### Ascend 910 -在Ascend 910硬件平台上，推理的执行命令如下： -``` -sh run_eval.sh [DATASET_PATH] [CHECKPOINT_PATH] -``` -脚本需要传入变量`DATASET_PATH`和`CHECKPOINT_PATH`，其中： -- `DATASET_PATH`：推理数据集路径。 -- `CHECKPOINT_PATH`: 保存的checkpoint路径。 - -目前推理使用的是单卡（默认device 0）进行推理，推理的结果如下： -``` -result: {'top_5_accuracy': 0.9295574583866837, 'top_1_accuracy': 0.761443661971831} ckpt=train_parallel0/resnet-42_5004.ckpt -``` -- `top_5_accuracy`：对于一个输入图片，如果预测概率排名前五的标签中包含真实标签，即认为分类正确； -- `top_1_accuracy`：对于一个输入图片，如果预测概率最大的标签与真实标签相同，即认为分类正确。 -#### GPU - -在GPU硬件平台上，推理的执行命令如下： -``` -sh run_eval_gpu.sh [DATASET_PATH] [CHECKPOINT_PATH] -``` -脚本需要传入变量`DATASET_PATH`和`CHECKPOINT_PATH`，其中： -- `DATASET_PATH`：推理数据集路径。 -- `CHECKPOINT_PATH`: 保存的checkpoint路径。 - -推理的结果如下： -``` -result: {'top_5_accuracy': 0.9287972151088348, 'top_1_accuracy': 0.7597031049935979} ckpt=train_parallel/resnet-36_5004.ckpt -``` +# ResNet-50二阶优化实践 + +`Linux` `Ascend` `GPU` `模型开发` `模型调优` `高级` + + + +- [ResNet-50二阶优化实践](#resnet-50二阶优化实践) + - [概述](#概述) + - [准备环节](#准备环节) + - [准备数据集](#准备数据集) + - [配置分布式环境变量](#配置分布式环境变量) + - [加载处理数据集](#加载处理数据集) + - [定义网络](#定义网络) + - [定义损失函数及THOR优化器](#定义损失函数及thor优化器) + - [定义损失函数](#定义损失函数) + - [定义优化器](#定义优化器) + - [训练网络](#训练网络) + - [配置模型保存](#配置模型保存) + - [配置训练网络](#配置训练网络) + - [运行脚本](#运行脚本) + - [模型推理](#模型推理) + + +   + +## 概述 + +常见的优化算法可分为一阶优化算法和二阶优化算法。经典的一阶优化算法如SGD等，计算量小、计算速度快，但是收敛的速度慢，所需的迭代次数多。而二阶优化算法使用目标函数的二阶导数来加速收敛，能更快地收敛到模型最优值，所需要的迭代次数少，但由于二阶优化算法过高的计算成本，导致其总体执行时间仍然慢于一阶，故目前在深度神经网络训练中二阶优化算法的应用并不普遍。二阶优化算法的主要计算成本在于二阶信息矩阵（Hessian矩阵、[FIM矩阵](https://arxiv.org/pdf/1808.07172.pdf)等）的求逆运算，时间复杂度约为$O(n^3)$。 + +MindSpore开发团队在现有的自然梯度算法的基础上，对FIM矩阵采用近似、切分等优化加速手段，极大的降低了逆矩阵的计算复杂度，开发出了可用的二阶优化器THOR。使用8块Ascend 910 AI处理器，THOR可以在72min内完成ResNet50-v1.5网络和ImageNet数据集的训练，相比于SGD+Momentum速度提升了近一倍。 + + +本篇教程将主要介绍如何在Ascend 910 以及GPU上，使用MindSpore提供的二阶优化器THOR训练ResNet50-v1.5网络和ImageNet数据集。 +> 你可以在这里下载完整的示例代码： + 。 + +### 示例代码目录结构 + +```shell +├── resnet_thor + ├── README.md + ├── scripts + ├── run_distribute_train.sh # launch distributed training for Ascend 910 + └── run_eval.sh # launch inference for Ascend 910 + ├── run_distribute_train_gpu.sh # launch distributed training for GPU + └── run_eval_gpu.sh # launch inference for GPU + ├── src + ├── crossentropy.py # CrossEntropy loss function + ├── config.py # parameter configuration + ├── dataset_helper.py # dataset helper for minddata dataset + ├── grad_reducer_thor.py # grad reduce for thor + ├── model_thor.py # model for train + ├── resnet_thor.py # resnet50_thor backone + ├── thor.py # thor optimizer + ├── thor_layer.py # thor layer + └── dataset.py # data preprocessing + ├── eval.py # infer script + └── train.py # train script + +``` + +整体执行流程如下： +1. 准备ImageNet数据集，处理需要的数据集； +2. 定义ResNet50网络； +3. 定义损失函数和THOR优化器； +4. 加载数据集并进行训练，训练完成后，查看结果及保存模型文件； +5. 加载保存的模型，进行推理。 + + +## 准备环节 + +实践前，确保已经正确安装MindSpore。如果没有，可以通过[MindSpore安装页面](https://www.mindspore.cn/install)安装MindSpore。 + +### 准备数据集 + +下载完整的ImageNet2012数据集，将数据集解压分别存放到本地工作区的`ImageNet2012/ilsvrc`、`ImageNet2012/ilsvrc_eval`路径下。 + +目录结构如下： + +``` +└─ImageNet2012 + ├─ilsvrc + │ n03676483/ + │ n04067472/ + │ n01622779/ + │ ...... + └─ilsvrc_eval + │ n03018349/ + │ n02504013 + │ n07871810 + │ ...... + +``` +### 配置分布式环境变量 +#### Ascend 910 +Ascend 910 AI处理器的分布式环境变量配置参考[分布式并行训练 (Ascend)](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/distributed_training_ascend.html#id4)。 + +#### GPU +GPU的分布式环境配置参考[分布式并行训练 (GPU)](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/distributed_training_gpu.html#id4)。 + + +## 加载处理数据集 + +分布式训练时，通过并行的方式加载数据集，同时通过MindSpore提供的数据增强接口对数据集进行处理。加载处理数据集的脚本在源码的`src/dataset.py`脚本中。 +```python +import os +import mindspore.common.dtype as mstype +import mindspore.dataset.engine as de +import mindspore.dataset.vision.c_transforms as C +import mindspore.dataset.transforms.c_transforms as C2 +from mindspore.communication.management import init, get_rank, get_group_size + +def create_dataset(dataset_path, do_train, repeat_num=1, batch_size=32, target="Ascend"): + if target == "Ascend": + device_num, rank_id = _get_rank_info() + else: + init() + rank_id = get_rank() + device_num = get_group_size() + if device_num == 1: + ds = de.ImageFolderDataset(dataset_path, num_parallel_workers=8, shuffle=True) + else: + ds = de.ImageFolderDataset(dataset_path, num_parallel_workers=8, shuffle=True, + num_shards=device_num, shard_id=rank_id) + + image_size = 224 + mean = [0.485 * 255, 0.456 * 255, 0.406 * 255] + std = [0.229 * 255, 0.224 * 255, 0.225 * 255] + # define map operations + if do_train: + trans = [ + C.RandomCropDecodeResize(image_size, scale=(0.08, 1.0), ratio=(0.75, 1.333)), + C.RandomHorizontalFlip(prob=0.5), + C.Normalize(mean=mean, std=std), + C.HWC2CHW() + ] + else: + trans = [ + C.Decode(), + C.Resize(256), + C.CenterCrop(image_size), + C.Normalize(mean=mean, std=std), + C.HWC2CHW() + ] + type_cast_op = C2.TypeCast(mstype.int32) + ds = ds.map(operations=trans, input_columns="image", num_parallel_workers=8) + ds = ds.map(operations=type_cast_op, input_columns="label", num_parallel_workers=8) + + # apply batch operations + ds = ds.batch(batch_size, drop_remainder=True) + + # apply dataset repeat operation + ds = ds.repeat(repeat_num) + + return ds +``` + +> MindSpore支持进行多种数据处理和增强的操作，各种操作往往组合使用，具体可以参考[数据处理与数据增强](https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/data_processing_and_augmentation.html)章节。 + + +## 定义网络 +本示例中使用的网络模型为ResNet50-v1.5，先定义[ResNet50网络](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/official/cv/resnet/src/resnet.py)，然后使用二阶优化器自定义的算子替换`Conv2d`和 +和`Dense`算子。定义好的网络模型在在源码`src/resnet_thor.py`脚本中，自定义的算子`Conv2d_thor`和`Dense_thor`在`src/thor_layer.py`脚本中。 + +- 使用`Conv2d_thor`替换原网络模型中的`Conv2d` +- 使用`Dense_thor`替换原网络模型中的`Dense` + +> 使用THOR自定义的算子`Conv2d_thor`和`Dense_thor`是为了保存模型训练中的二阶矩阵信息，新定义的网络与原网络模型的backbone一致。 + +网络构建完成以后，在`__main__`函数中调用定义好的ResNet50： +```python +... +from src.resnet_thor import resnet50 +... +f __name__ == "__main__": + ... + # define the net + net = resnet50(class_num=config.class_num, damping=damping, loss_scale=config.loss_scale, + frequency=config.frequency, batch_size=config.batch_size) + ... +``` + + +## 定义损失函数及THOR优化器 + + +### 定义损失函数 + +MindSpore支持的损失函数有`SoftmaxCrossEntropyWithLogits`、`L1Loss`、`MSELoss`等。THOR优化器需要使用`SoftmaxCrossEntropyWithLogits`损失函数。 + +损失函数的实现步骤在`src/crossentropy.py`脚本中。这里使用了深度网络模型训练中的一个常用trick：label smoothing，通过对真实标签做平滑处理，提高模型对分类错误标签的容忍度，从而可以增加模型的泛化能力。 +```python +class CrossEntropy(_Loss): + """CrossEntropy""" + def __init__(self, smooth_factor=0., num_classes=1000): + super(CrossEntropy, self).__init__() + self.onehot = P.OneHot() + self.on_value = Tensor(1.0 - smooth_factor, mstype.float32) + self.off_value = Tensor(1.0 * smooth_factor / (num_classes - 1), mstype.float32) + self.ce = nn.SoftmaxCrossEntropyWithLogits() + self.mean = P.ReduceMean(False) + + def construct(self, logit, label): + one_hot_label = self.onehot(label, F.shape(logit)[1], self.on_value, self.off_value) + loss = self.ce(logit, one_hot_label) + loss = self.mean(loss, 0) + return loss +``` +在`__main__`函数中调用定义好的损失函数： + +```python +... +from src.crossentropy import CrossEntropy +... +if __name__ == "__main__": + ... + # define the loss function + if not config.use_label_smooth: + config.label_smooth_factor = 0.0 + loss = CrossEntropy(smooth_factor=config.label_smooth_factor, num_classes=config.class_num) + ... +``` + +### 定义优化器 + +THOR优化器的参数更新公式如下： + +$$ \theta^{t+1} = \theta^t + \alpha F^{-1}\nabla E$$ + +参数更新公式中各参数的含义如下： +- $\theta$：网络中的可训参数； +- $t$：迭代次数； +- $\alpha$：学习率值，参数的更新步长； +- $F^{-1}$：FIM矩阵，在网络中计算获得； +- $\nabla E$：一阶梯度值。 + +从参数更新公式中可以看出，THOR优化器需要额外计算的是每一层的FIM矩阵，每一层的FIM矩阵就是之前在自定义的网络模型中计算获得的。FIM矩阵可以对每一层参数更新的步长和方向进行自适应的调整，加速收敛的同时可以降低调参的复杂度。 + +```python +... +if args_opt.device_target == "Ascend": + from src.thor import THOR +else: + from src.thor import THOR_GPU as THOR +... + +if __name__ == "__main__": + ... + # learning rate setting + lr = get_model_lr(0, config.lr_init, config.lr_decay, config.lr_end_epoch, step_size, decay_epochs=39) + # define the optimizer + opt = THOR(filter(lambda x: x.requires_grad, net.get_parameters()), Tensor(lr), config.momentum, + filter(lambda x: 'matrix_A' in x.name, net.get_parameters()), + filter(lambda x: 'matrix_G' in x.name, net.get_parameters()), + filter(lambda x: 'A_inv_max' in x.name, net.get_parameters()), + filter(lambda x: 'G_inv_max' in x.name, net.get_parameters()), + config.weight_decay, config.loss_scale) + ... +``` + +## 训练网络 + +### 配置模型保存 + +MindSpore提供了callback机制，可以在训练过程中执行自定义逻辑，这里使用框架提供的`ModelCheckpoint`函数。 +`ModelCheckpoint`可以保存网络模型和参数，以便进行后续的fine-tuning操作。 +`TimeMonitor`、`LossMonitor`是MindSpore官方提供的callback函数，可以分别用于监控训练过程中单步迭代时间和`loss`值的变化。 + +```python +... +from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, TimeMonitor, LossMonitor +... +if __name__ == "__main__": + ... + # define callbacks + time_cb = TimeMonitor(data_size=step_size) + loss_cb = LossMonitor() + cb = [time_cb, loss_cb] + if config.save_checkpoint: + config_ck = CheckpointConfig(save_checkpoint_steps=config.save_checkpoint_epochs * step_size, + keep_checkpoint_max=config.keep_checkpoint_max) + ckpt_cb = ModelCheckpoint(prefix="resnet", directory=ckpt_save_dir, config=config_ck) + cb += [ckpt_cb] + ... +``` + +### 配置训练网络 + +通过MindSpore提供的`model.train`接口可以方便地进行网络的训练。THOR优化器通过降低二阶矩阵更新频率，来减少计算量，提升计算速度，故重新定义一个Model_Thor类，继承MindSpore提供的Model类。在Model_Thor类中增加二阶矩阵更新频率控制参数，用户可以通过调整该参数，优化整体的性能。 + + +```python +... +from mindspore.train.loss_scale_manager import FixedLossScaleManager +from src.model_thor import Model_Thor as Model +... + +if __name__ == "__main__": + ... + loss_scale = FixedLossScaleManager(config.loss_scale, drop_overflow_update=False) + if target == "Ascend": + model = Model(net, loss_fn=loss, optimizer=opt, amp_level='O2', loss_scale_manager=loss_scale, + keep_batchnorm_fp32=False, metrics={'acc'}, frequency=config.frequency) + else: + model = Model(net, loss_fn=loss, optimizer=opt, loss_scale_manager=loss_scale, metrics={'acc'}, + amp_level="O2", keep_batchnorm_fp32=True, frequency=config.frequency) + ... +``` + +### 运行脚本 +训练脚本定义完成之后，调`scripts`目录下的shell脚本，启动分布式训练进程。 +#### Ascend 910 +目前MindSpore分布式在Ascend上执行采用单卡单进程运行方式，即每张卡上运行1个进程，进程数量与使用的卡的数量一致。其中，0卡在前台执行，其他卡放在后台执行。每个进程创建1个目录，目录名称为`train_parallel`+ `device_id`，用来保存日志信息，算子编译信息以及训练的checkpoint文件。下面以使用8张卡的分布式训练脚本为例，演示如何运行脚本： + +使用以下命令运行脚本： +``` +sh run_distribute_train.sh [RANK_TABLE_FILE] [DATASET_PATH] [DEVICE_NUM] +``` +脚本需要传入变量`RANK_TABLE_FILE`、`DATASET_PATH`和`DEVICE_NUM`，其中： +- `RANK_TABLE_FILE`：组网信息文件的路径。 +- `DATASET_PATH`：训练数据集路径。 +- `DEVICE_NUM`: 实际的运行卡数。 +其余环境变量请参考安装教程中的配置项。 + +训练过程中loss打印示例如下： + +```bash +... +epoch: 1 step: 5004, loss is 4.4182425 +epoch: 2 step: 5004, loss is 3.740064 +epoch: 3 step: 5004, loss is 4.0546017 +epoch: 4 step: 5004, loss is 3.7598825 +epoch: 5 step: 5004, loss is 3.3744206 +...... +epoch: 40 step: 5004, loss is 1.6907625 +epoch: 41 step: 5004, loss is 1.8217756 +epoch: 42 step: 5004, loss is 1.6453942 +... +``` + +训练完后，每张卡训练产生的checkpoint文件保存在各自训练目录下，`device_0`产生的checkpoint文件示例如下: + +```bash +└─train_parallel0 + ├─resnet-1_5004.ckpt + ├─resnet-2_5004.ckpt + │ ...... + ├─resnet-42_5004.ckpt +``` + +其中， +`*.ckpt`：指保存的模型参数文件。checkpoint文件名称具体含义：*网络名称*-*epoch数*_*step数*.ckpt。 + +#### GPU +在GPU硬件平台上，MindSpore采用OpenMPI的`mpirun`进行分布式训练，进程创建1个目录，目录名称为`train_parallel`，用来保存日志信息和训练的checkpoint文件。下面以使用8张卡的分布式训练脚本为例，演示如何运行脚本： +``` +sh run_distribute_train_gpu.sh [DATASET_PATH] [DEVICE_NUM] +``` +脚本需要传入变量`DATASET_PATH`和`DEVICE_NUM`，其中： +- `DATASET_PATH`：训练数据集路径。 +- `DEVICE_NUM`: 实际的运行卡数。 + +在GPU训练时，无需设置`DEVICE_ID`环境变量，因此在主训练脚本中不需要调用`int(os.getenv('DEVICE_ID'))`来获取卡的物理序号，同时`context`中也无需传入`device_id`。我们需要将device_target设置为GPU，并需要调用`init()`来使能NCCL。 + +训练过程中loss打印示例如下： +```bash +... +epoch: 1 step: 5004, loss is 4.2546034 +epoch: 2 step: 5004, loss is 4.0819564 +epoch: 3 step: 5004, loss is 3.7005644 +epoch: 4 step: 5004, loss is 3.2668946 +epoch: 5 step: 5004, loss is 3.023509 +...... +epoch: 36 step: 5004, loss is 1.645802 +... +``` + +训练完后，保存的模型文件示例如下: + +```bash +└─train_parallel + ├─ckpt_0 + ├─resnet-1_5004.ckpt + ├─resnet-2_5004.ckpt + │ ...... + ├─resnet-36_5004.ckpt + ...... + ├─ckpt_7 + ├─resnet-1_5004.ckpt + ├─resnet-2_5004.ckpt + │ ...... + ├─resnet-36_5004.ckpt + +``` + +## 模型推理 + +使用训练过程中保存的checkpoint文件进行推理，验证模型的泛化能力。首先通过`load_checkpoint`接口加载模型文件，然后调用`Model`的`eval`接口对输入图片类别作出预测，再与输入图片的真实类别做比较，得出最终的预测精度值。 + +### 定义推理网络 + +1. 使用`load_checkpoint`接口加载模型文件。 +2. 使用`model.eval`接口读入测试数据集，进行推理。 +3. 计算得出预测精度值。 + +```python +... +from mindspore.train.serialization import load_checkpoint, load_param_into_net +... + +if __name__ == "__main__": + ... + # define net + net = resnet(class_num=config.class_num) + net.add_flags_recursive(thor=False) + + # load checkpoint + param_dict = load_checkpoint(args_opt.checkpoint_path) + keys = list(param_dict.keys()) + for key in keys: + if "damping" in key: + param_dict.pop(key) + load_param_into_net(net, param_dict) + net.set_train(False) + + # define model + model = Model(net, loss_fn=loss, metrics={'top_1_accuracy', 'top_5_accuracy'}) + + # eval model + res = model.eval(dataset) + print("result:", res, "ckpt=", args_opt.checkpoint_path) +``` + +### 执行推理 +推理网络定义完成之后，调用`scripts`目录下的shell脚本，进行推理。 +#### Ascend 910 +在Ascend 910硬件平台上，推理的执行命令如下： +``` +sh run_eval.sh [DATASET_PATH] [CHECKPOINT_PATH] +``` +脚本需要传入变量`DATASET_PATH`和`CHECKPOINT_PATH`，其中： +- `DATASET_PATH`：推理数据集路径。 +- `CHECKPOINT_PATH`: 保存的checkpoint路径。 + +目前推理使用的是单卡（默认device 0）进行推理，推理的结果如下： +``` +result: {'top_5_accuracy': 0.9295574583866837, 'top_1_accuracy': 0.761443661971831} ckpt=train_parallel0/resnet-42_5004.ckpt +``` +- `top_5_accuracy`：对于一个输入图片，如果预测概率排名前五的标签中包含真实标签，即认为分类正确； +- `top_1_accuracy`：对于一个输入图片，如果预测概率最大的标签与真实标签相同，即认为分类正确。 +#### GPU + +在GPU硬件平台上，推理的执行命令如下： +``` +sh run_eval_gpu.sh [DATASET_PATH] [CHECKPOINT_PATH] +``` +脚本需要传入变量`DATASET_PATH`和`CHECKPOINT_PATH`，其中： +- `DATASET_PATH`：推理数据集路径。 +- `CHECKPOINT_PATH`: 保存的checkpoint路径。 + +推理的结果如下： +``` +result: {'top_5_accuracy': 0.9287972151088348, 'top_1_accuracy': 0.7597031049935979} ckpt=train_parallel/resnet-36_5004.ckpt +``` diff --git a/tutorials/source_zh_cn/advanced_use/serving.md b/tutorials/training/source_zh_cn/advanced_use/serving.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/serving.md rename to tutorials/training/source_zh_cn/advanced_use/serving.md diff --git a/tutorials/source_zh_cn/advanced_use/summary_record.md b/tutorials/training/source_zh_cn/advanced_use/summary_record.md similarity index 99% rename from tutorials/source_zh_cn/advanced_use/summary_record.md rename to tutorials/training/source_zh_cn/advanced_use/summary_record.md index 65913760f539fa01f49aca6b54babb8dd29a1955..da92ca826ede8f714d8757d25dc3ed163dcf53a3 100644 --- a/tutorials/source_zh_cn/advanced_use/summary_record.md +++ b/tutorials/training/source_zh_cn/advanced_use/summary_record.md @@ -372,4 +372,6 @@ mindinsight stop model.train(epoch=2, train_dataset, callbacks=[confusion_callback, summary_collector]) ``` -3. 每个summary日志文件目录中，应该只放置一次训练的数据。一个summary日志目录中如果存放了多次训练的summary数据，MindInsight在可视化数据时会将这些训练的summary数据进行叠加展示，可能会与预期可视化效果不相符。 \ No newline at end of file +3. 每个summary日志文件目录中，应该只放置一次训练的数据。一个summary日志目录中如果存放了多次训练的summary数据，MindInsight在可视化数据时会将这些训练的summary数据进行叠加展示，可能会与预期可视化效果不相符。 + +4. 当前 `SummaryCollector` 和 `SummaryRecord` 不支持GPU多卡运行的场景。 \ No newline at end of file diff --git a/tutorials/source_zh_cn/advanced_use/synchronization_training_and_evaluation.md b/tutorials/training/source_zh_cn/advanced_use/synchronization_training_and_evaluation.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/synchronization_training_and_evaluation.md rename to tutorials/training/source_zh_cn/advanced_use/synchronization_training_and_evaluation.md diff --git a/tutorials/source_zh_cn/advanced_use/use_on_the_cloud.md b/tutorials/training/source_zh_cn/advanced_use/use_on_the_cloud.md similarity index 100% rename from tutorials/source_zh_cn/advanced_use/use_on_the_cloud.md rename to tutorials/training/source_zh_cn/advanced_use/use_on_the_cloud.md diff --git a/tutorials/source_zh_cn/advanced_use/visualization_tutorials.rst b/tutorials/training/source_zh_cn/advanced_use/visualization_tutorials.rst similarity index 100% rename from tutorials/source_zh_cn/advanced_use/visualization_tutorials.rst rename to tutorials/training/source_zh_cn/advanced_use/visualization_tutorials.rst diff --git a/tutorials/source_zh_cn/conf.py b/tutorials/training/source_zh_cn/conf.py similarity index 100% rename from tutorials/source_zh_cn/conf.py rename to tutorials/training/source_zh_cn/conf.py diff --git a/tutorials/source_zh_cn/index.rst b/tutorials/training/source_zh_cn/index.rst similarity index 100% rename from tutorials/source_zh_cn/index.rst rename to tutorials/training/source_zh_cn/index.rst diff --git a/tutorials/source_zh_cn/quick_start/images/LeNet_5.jpg b/tutorials/training/source_zh_cn/quick_start/images/LeNet_5.jpg similarity index 100% rename from tutorials/source_zh_cn/quick_start/images/LeNet_5.jpg rename to tutorials/training/source_zh_cn/quick_start/images/LeNet_5.jpg diff --git a/tutorials/source_zh_cn/quick_start/images/linear_regression.gif b/tutorials/training/source_zh_cn/quick_start/images/linear_regression.gif similarity index 100% rename from tutorials/source_zh_cn/quick_start/images/linear_regression.gif rename to tutorials/training/source_zh_cn/quick_start/images/linear_regression.gif diff --git a/tutorials/source_zh_cn/quick_start/images/linear_regression_eval_datasets.png b/tutorials/training/source_zh_cn/quick_start/images/linear_regression_eval_datasets.png similarity index 100% rename from tutorials/source_zh_cn/quick_start/images/linear_regression_eval_datasets.png rename to tutorials/training/source_zh_cn/quick_start/images/linear_regression_eval_datasets.png diff --git a/tutorials/source_zh_cn/quick_start/images/model_net_and_eval_datasets.png b/tutorials/training/source_zh_cn/quick_start/images/model_net_and_eval_datasets.png similarity index 100% rename from tutorials/source_zh_cn/quick_start/images/model_net_and_eval_datasets.png rename to tutorials/training/source_zh_cn/quick_start/images/model_net_and_eval_datasets.png diff --git a/tutorials/source_zh_cn/quick_start/linear_regression.md b/tutorials/training/source_zh_cn/quick_start/linear_regression.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/linear_regression.md rename to tutorials/training/source_zh_cn/quick_start/linear_regression.md diff --git a/tutorials/source_zh_cn/quick_start/quick_start.md b/tutorials/training/source_zh_cn/quick_start/quick_start.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_start.md rename to tutorials/training/source_zh_cn/quick_start/quick_start.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video.md b/tutorials/training/source_zh_cn/quick_start/quick_video.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video.md rename to tutorials/training/source_zh_cn/quick_start/quick_video.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/ascend910.md b/tutorials/training/source_zh_cn/quick_start/quick_video/ascend910.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/ascend910.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/ascend910.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/community.md b/tutorials/training/source_zh_cn/quick_start/quick_video/community.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/community.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/community.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/cpu_operator_development.md b/tutorials/training/source_zh_cn/quick_start/quick_video/cpu_operator_development.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/cpu_operator_development.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/cpu_operator_development.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/cpu_ubuntu.md b/tutorials/training/source_zh_cn/quick_start/quick_video/cpu_ubuntu.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/cpu_ubuntu.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/cpu_ubuntu.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/cpu_windows.md b/tutorials/training/source_zh_cn/quick_start/quick_video/cpu_windows.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/cpu_windows.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/cpu_windows.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/customized_debugging.md b/tutorials/training/source_zh_cn/quick_start/quick_video/customized_debugging.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/customized_debugging.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/customized_debugging.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/gpu.md b/tutorials/training/source_zh_cn/quick_start/quick_video/gpu.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/gpu.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/gpu.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/gpu_operator_development.md b/tutorials/training/source_zh_cn/quick_start/quick_video/gpu_operator_development.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/gpu_operator_development.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/gpu_operator_development.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/loading_the_dataset_and_converting_data_format.md b/tutorials/training/source_zh_cn/quick_start/quick_video/loading_the_dataset_and_converting_data_format.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/loading_the_dataset_and_converting_data_format.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/loading_the_dataset_and_converting_data_format.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/mindInsight_dashboard.md b/tutorials/training/source_zh_cn/quick_start/quick_video/mindInsight_dashboard.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/mindInsight_dashboard.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/mindInsight_dashboard.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/mindInsight_installation_and_common_commands.md b/tutorials/training/source_zh_cn/quick_start/quick_video/mindInsight_installation_and_common_commands.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/mindInsight_installation_and_common_commands.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/mindInsight_installation_and_common_commands.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/mindInsight_lineage_and_scalars_comparision.md b/tutorials/training/source_zh_cn/quick_start/quick_video/mindInsight_lineage_and_scalars_comparision.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/mindInsight_lineage_and_scalars_comparision.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/mindInsight_lineage_and_scalars_comparision.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/quick_start_video.md b/tutorials/training/source_zh_cn/quick_start/quick_video/quick_start_video.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/quick_start_video.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/quick_start_video.md diff --git a/tutorials/source_zh_cn/quick_start/quick_video/saving_and_loading_model_parameters.md b/tutorials/training/source_zh_cn/quick_start/quick_video/saving_and_loading_model_parameters.md similarity index 100% rename from tutorials/source_zh_cn/quick_start/quick_video/saving_and_loading_model_parameters.md rename to tutorials/training/source_zh_cn/quick_start/quick_video/saving_and_loading_model_parameters.md diff --git a/tutorials/source_zh_cn/use/custom_operator.md b/tutorials/training/source_zh_cn/use/custom_operator.md similarity index 100% rename from tutorials/source_zh_cn/use/custom_operator.md rename to tutorials/training/source_zh_cn/use/custom_operator.md diff --git a/tutorials/source_zh_cn/use/data_preparation.rst b/tutorials/training/source_zh_cn/use/data_preparation.rst similarity index 100% rename from tutorials/source_zh_cn/use/data_preparation.rst rename to tutorials/training/source_zh_cn/use/data_preparation.rst diff --git a/tutorials/source_zh_cn/use/defining_the_network.rst b/tutorials/training/source_zh_cn/use/defining_the_network.rst similarity index 100% rename from tutorials/source_zh_cn/use/defining_the_network.rst rename to tutorials/training/source_zh_cn/use/defining_the_network.rst diff --git a/tutorials/source_zh_cn/use/image_loading.md b/tutorials/training/source_zh_cn/use/image_loading.md similarity index 100% rename from tutorials/source_zh_cn/use/image_loading.md rename to tutorials/training/source_zh_cn/use/image_loading.md diff --git a/tutorials/source_zh_cn/use/images/dataset_pipeline.eddx b/tutorials/training/source_zh_cn/use/images/dataset_pipeline.eddx similarity index 100% rename from tutorials/source_zh_cn/use/images/dataset_pipeline.eddx rename to tutorials/training/source_zh_cn/use/images/dataset_pipeline.eddx diff --git a/tutorials/source_zh_cn/use/images/map.eddx b/tutorials/training/source_zh_cn/use/images/map.eddx similarity index 100% rename from tutorials/source_zh_cn/use/images/map.eddx rename to tutorials/training/source_zh_cn/use/images/map.eddx diff --git a/tutorials/source_zh_cn/use/images/mnist_5.png b/tutorials/training/source_zh_cn/use/images/mnist_5.png similarity index 100% rename from tutorials/source_zh_cn/use/images/mnist_5.png rename to tutorials/training/source_zh_cn/use/images/mnist_5.png diff --git a/tutorials/source_zh_cn/use/images/mnist_5_resize_crop.png b/tutorials/training/source_zh_cn/use/images/mnist_5_resize_crop.png similarity index 100% rename from tutorials/source_zh_cn/use/images/mnist_5_resize_crop.png rename to tutorials/training/source_zh_cn/use/images/mnist_5_resize_crop.png diff --git a/tutorials/source_zh_cn/use/multi_platform_inference.md b/tutorials/training/source_zh_cn/use/multi_platform_inference.md similarity index 100% rename from tutorials/source_zh_cn/use/multi_platform_inference.md rename to tutorials/training/source_zh_cn/use/multi_platform_inference.md diff --git a/tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md b/tutorials/training/source_zh_cn/use/saving_and_loading_model_parameters.md similarity index 100% rename from tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md rename to tutorials/training/source_zh_cn/use/saving_and_loading_model_parameters.md diff --git a/tutorials/source_zh_cn/use/text_loading.md b/tutorials/training/source_zh_cn/use/text_loading.md similarity index 100% rename from tutorials/source_zh_cn/use/text_loading.md rename to tutorials/training/source_zh_cn/use/text_loading.md