diff --git a/api/source_en/conf.py b/api/source_en/conf.py index 78844f9f262ffaf5cf8bb4df1789e6ea999c43f0..5a88b942e0200aecacdaa026cda2bc05759d1992 100644 --- a/api/source_en/conf.py +++ b/api/source_en/conf.py @@ -29,7 +29,7 @@ copyright = '2020, MindSpore' author = 'MindSpore' # The full version, including alpha/beta/rc tags -release = 'master' +release = 'r0.5' # -- General configuration --------------------------------------------------- diff --git a/api/source_en/index.rst b/api/source_en/index.rst index da3c2911b739089a0ab4e818cb1599723c3ff8d5..44d9b0f294bf34b24fe2000e116257c16761e5ea 100644 --- a/api/source_en/index.rst +++ b/api/source_en/index.rst @@ -47,4 +47,4 @@ MindSpore API :maxdepth: 1 :caption: C++ API - predict + predict diff --git a/api/source_zh_cn/conf.py b/api/source_zh_cn/conf.py index ec00aa84a8818fda2090b11bcae334dcdece19b4..96158f425267f5dd59cba36a07d6349c75e44f83 100644 --- a/api/source_zh_cn/conf.py +++ b/api/source_zh_cn/conf.py @@ -29,7 +29,7 @@ copyright = '2020, MindSpore' author = 'MindSpore' # The full version, including alpha/beta/rc tags -release = 'master' +release = 'r0.5' # -- General configuration --------------------------------------------------- diff --git a/api/source_zh_cn/index.rst b/api/source_zh_cn/index.rst index c82309f980aa9ae8bb6aa3285978995646a4b145..7d6f36c93b1fe3b35b64135a42586003f835aaf3 100644 --- a/api/source_zh_cn/index.rst +++ b/api/source_zh_cn/index.rst @@ -47,4 +47,4 @@ MindSpore API :maxdepth: 1 :caption: C++ API - predict + predict diff --git a/docs/source_en/architecture.md b/docs/source_en/architecture.md index cd30baf77b5b436e4761e78c2f87545ad6b36c34..0f95ac41174fee3a85c4cb00805dc320670e0244 100644 --- a/docs/source_en/architecture.md +++ b/docs/source_en/architecture.md @@ -8,7 +8,7 @@ This document describes the overall architecture of MindSpore. - + The MindSpore framework consists of the Frontend Expression layer, Graph Engine layer, and Backend Runtime layer. diff --git a/docs/source_en/benchmark.md b/docs/source_en/benchmark.md index 51fd7faaef4c4e91382a85d00ca102fcc764e04d..9ab4b2c157867518acd6ac1011706c795d64bc44 100644 --- a/docs/source_en/benchmark.md +++ b/docs/source_en/benchmark.md @@ -1,19 +1,19 @@ # Benchmarks - + This document describes the MindSpore benchmarks. -For details about the MindSpore pre-trained model, see [Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/mindspore/model_zoo). +For details about the MindSpore pre-trained model, see [Model Zoo](https://gitee.com/mindspore/mindspore/tree/r0.5/model_zoo). ## Training Performance ### ResNet -| Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | +| Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| ResNet-50 v1.5 | CNN | ImageNet2012 | 0.2.0-alpha | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 32 | 1787 images/sec | - | -| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 32 | 13689 images/sec | 0.95 | -| | | | | Ascend: 16 * Ascend 910
CPU：384 Cores | Mixed | 32 | 27090 images/sec | 0.94 | +| ResNet-50 v1.5 | CNN | ImageNet2012 | 0.5.0-beta | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 256 | 2115 images/sec | - | +| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 256 | 16600 images/sec | 0.98 | +| | | | | Ascend: 16 * Ascend 910
CPU：384 Cores | Mixed | 256 | 32768 images/sec | 0.96 | 1. The preceding performance is obtained based on ModelArts, the HUAWEI CLOUD AI development platform. It is the average performance obtained by the Ascend 910 AI processor during the overall training process. 2. For details about other open source frameworks, see [ResNet-50 v1.5 for TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/Classification/RN50v1.5#nvidia-dgx-2-16x-v100-32g). @@ -22,8 +22,8 @@ For details about the MindSpore pre-trained model, see [Model Zoo](https://gitee | Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| BERT-Large | Attention | zhwiki | 0.2.0-alpha | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 96 | 210 sentences/sec | - | -| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 96 | 1613 sentences/sec | 0.96 | +| BERT-Large | Attention | zhwiki | 0.5.0-beta | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 96 | 269 sentences/sec | - | +| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 96 | 2069 sentences/sec | 0.96 | 1. The preceding performance is obtained based on ModelArts, the HUAWEI CLOUD AI development platform. The network contains 24 hidden layers, the sequence length is 128 tokens, and the vocabulary contains 21128 tokens. 2. For details about other open source frameworks, see [BERT For TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT). \ No newline at end of file diff --git a/docs/source_en/conf.py b/docs/source_en/conf.py index 6db42071de7116aaf62bbb7f0b2c09825b13c69b..101a88e0b71397e6b07b9ac5f5252eece006b63a 100644 --- a/docs/source_en/conf.py +++ b/docs/source_en/conf.py @@ -20,7 +20,7 @@ copyright = '2020, MindSpore' author = 'MindSpore' # The full version, including alpha/beta/rc tags -release = 'master' +release = 'r0.5' # -- General configuration --------------------------------------------------- diff --git a/docs/source_en/constraints_on_network_construction.md b/docs/source_en/constraints_on_network_construction.md index 69345b9878d01d5ecdcea9a647e883cb71b00a16..d55d176e1696fe31477df2c2c662302f0c2f92bd 100644 --- a/docs/source_en/constraints_on_network_construction.md +++ b/docs/source_en/constraints_on_network_construction.md @@ -23,7 +23,7 @@ - + ## Overview MindSpore can compile user source code based on the Python syntax into computational graphs, and can convert common functions or instances inherited from nn.Cell into computational graphs. Currently, MindSpore does not support conversion of any Python source code into computational graphs. Therefore, there are constraints on source code compilation, including syntax constraints and network definition constraints. As MindSpore evolves, the constraints may change. @@ -148,8 +148,8 @@ Currently, the following syntax is not supported in network constructors: ## Network Definition Constraints ### Instance Types on the Entire Network -* Common Python function with the [@ms_function](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.html#mindspore.ms_function) decorator. -* Cell subclass inherited from [nn.Cell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell). +* Common Python function with the [@ms_function](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.html#mindspore.ms_function) decorator. +* Cell subclass inherited from [nn.Cell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell). ### Network Input Type * The training data input parameters of the entire network must be of the Tensor type. @@ -162,13 +162,13 @@ Currently, the following syntax is not supported in network constructors: | Category | Content | :----------- |:-------- -| `Cell` instance |[mindspore/nn/*](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html), and custom [Cell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell). +| `Cell` instance |[mindspore/nn/*](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html), and custom [Cell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell). | Member function of a `Cell` instance | Member functions of other classes in the construct function of Cell can be called. | Function | Custom Python functions and system functions listed in the preceding content. | Dataclass instance | Class decorated with @dataclass. -| Primitive operator |[mindspore/ops/operations/*](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html). -| Composite operator |[mindspore/ops/composite/*](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.composite.html). -| Operator generated by constexpr |Uses the value generated by [@constexpr](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.html#mindspore.ops.constexpr) to calculate operators. +| Primitive operator |[mindspore/ops/operations/*](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html). +| Composite operator |[mindspore/ops/composite/*](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.composite.html). +| Operator generated by constexpr |Uses the value generated by [@constexpr](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.html#mindspore.ops.constexpr) to calculate operators. ### Other Constraints diff --git a/docs/source_en/glossary.md b/docs/source_en/glossary.md index 073a76f32a361f3227f9ab16005b811c152fe0a3..3f0f4b1471ae7a358e7e92c7041aad57b3be51d3 100644 --- a/docs/source_en/glossary.md +++ b/docs/source_en/glossary.md @@ -6,7 +6,7 @@ - + | Acronym and Abbreviation | Description | | ----- | ----- | diff --git a/docs/source_en/network_list.md b/docs/source_en/network_list.md index e8aef86babfa22792e62f8a1dbe9f99672676f31..1678b352c0cba49335838a09999790a49e85b876 100644 --- a/docs/source_en/network_list.md +++ b/docs/source_en/network_list.md @@ -1,18 +1,28 @@ # Network List - + | Domain | Sub Domain | Network | Ascend | GPU | CPU |:------ |:------| :----------- |:------ |:------ |:----- -|Computer Version (CV) | Image Classification | [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/alexnet/src/alexnet.py) | Supported | Supported | Doing -| Computer Version (CV) | Image Classification | [GoogleNet](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/googlenet.py) | Supported | Doing | Doing -| Computer Version (CV) | Image Classification | [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/lenet/src/lenet.py) | Supported | Supported | Supported -| Computer Version (CV) | Image Classification | [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/resnet.py) | Supported | Doing | Doing -|Computer Version (CV) | Image Classification | [ResNet-101](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/resnet.py) | Supported |Doing | Doing -| Computer Version (CV) | Image Classification | [VGG16](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/vgg.py) | Supported | Doing | Doing -| Computer Version (CV) | Mobile Image Classification
Image Classification
Semantic Tegmentation | [MobileNetV2](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/mobilenetv2/src/mobilenetV2.py) | Supported | Doing | Doing -|Computer Version (CV) | Targets Detection | [SSD](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/ssd/src/ssd.py) | Supported |Doing | Doing -| Computer Version (CV) | Targets Detection | [YoloV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/yolov3/src/yolov3.py) | Supported | Doing | Doing -| Computer Version (CV) | Semantic Segmentation | [Deeplabv3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/deeplabv3/src/deeplabv3.py) | Supported | Doing | Doing -| Natural Language Processing (NLP) | Natural Language Understanding | [BERT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/bert/src/bert_model.py) | Supported | Doing | Doing -| Natural Language Processing (NLP) | Natural Language Understanding | [SentimentNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/lstm/src/lstm.py) | Doing | Supported | Supported +|Computer Version (CV) | Image Classification | [AlexNet](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/alexnet/src/alexnet.py) | Supported | Supported | Doing +| Computer Version (CV) | Image Classification | [GoogleNet](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/googlenet/src/googlenet.py) | Supported | Doing | Doing +| Computer Version (CV) | Image Classification | [LeNet](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/lenet/src/lenet.py) | Supported | Supported | Supported +| Computer Version (CV) | Image Classification | [ResNet-50](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/resnet/src/resnet.py) | Supported | Doing | Doing +|Computer Version (CV) | Image Classification | [ResNet-101](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/resnet/src/resnet.py) | Supported |Doing | Doing +| Computer Version (CV) | Image Classification | [VGG16](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/vgg16/src/vgg.py) | Supported | Doing | Doing +| Computer Version (CV) | Mobile Image Classification
Image Classification
Semantic Tegmentation | [MobileNetV2](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/mobilenetv2/src/mobilenetV2.py) | Supported | Supported | Doing +| Computer Version (CV) | Mobile Image Classification
Image Classification
Semantic Tegmentation | [MobileNetV3](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/mobilenetv3/src/mobilenetV3.py) | Doing | Supported | Doing +|Computer Version (CV) | Targets Detection | [SSD](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/ssd/src/ssd.py) | Supported |Doing | Doing +| Computer Version (CV) | Targets Detection | [YoloV3](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/yolov3/src/yolov3.py) | Supported | Doing | Doing +| Computer Version (CV) | Targets Detection | [FasterRCNN](https://gitee.com/mindspore/mindspore/tree/r0.5/model_zoo/faster_rcnn/src/FasterRcnn) | Supported | Doing | Doing +| Computer Version (CV) | Semantic Segmentation | [Deeplabv3](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/deeplabv3/src/deeplabv3.py) | Supported | Doing | Doing +| Natural Language Processing (NLP) | Natural Language Understanding | [BERT](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/bert/src/bert_model.py) | Supported | Doing | Doing +| Natural Language Processing (NLP) | Natural Language Understanding | [Transformer](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/Transformer/src/transformer_model.py) | Supported | Doing | Doing +| Natural Language Processing (NLP) | Natural Language Understanding | [SentimentNet](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/lstm/src/lstm.py) | Doing | Supported | Supported +| Recommender | Recommender System, CTR prediction | [DeepFM](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/deepfm/src/deepfm.py) | Supported | Doing | Doing +| Recommender | Recommender System, Search ranking | [Wide&Deep](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/wide_and_deep/src/wide_and_deep.py) | Supported | Doing | Doing +| Graph Neural Networks（GNN）| Text Classification | [GCN](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/gcn/src/gcn.py) | Supported | Doing | Doing +| Graph Neural Networks（GNN）| Text Classification | [GAT](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/gat/src/gat.py) | Supported | Doing | Doing + +## Pre-trained Models +Coming soon. diff --git a/docs/source_en/operator_list.md b/docs/source_en/operator_list.md index 528884e8945edc8ae7dfc6963642e06860eb5c82..cb7f8555c798653e9871d5a838c01bfb042a97fd 100644 --- a/docs/source_en/operator_list.md +++ b/docs/source_en/operator_list.md @@ -8,286 +8,291 @@ - + ## mindspore.nn | Operation | Ascend | GPU | CPU |Operator Type | :----------- |:------ |:------ |:-----|:--- -| [mindspore.nn.Softmax](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Softmax) | Supported | Supported | Supported |layer/activation -| [mindspore.nn.LogSoftmax](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LogSoftmax) | Supported | Supported | Doing |layer/activation -| [mindspore.nn.ReLU](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ReLU) | Supported | Supported | Supported |layer/activation -| [mindspore.nn.ReLU6](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ReLU6) |Supported | Supported | Doing |layer/activation -| [mindspore.nn.HSwish](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.HSwish) | Doing | Supported | Doing |layer/activation -| [mindspore.nn.HSigmoid](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.HSigmoid) | Doing | Supported | Doing |layer/activation -| [mindspore.nn.LeakyReLU](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LeakyReLU) | Supported |Doing | Doing |layer/activation -| [mindspore.nn.Tanh](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Tanh) | Supported | Supported | Doing |layer/activation -| [mindspore.nn.GELU](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.GELU) | Supported | Supported | Doing |layer/activation -| [mindspore.nn.Sigmoid](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Sigmoid) | Supported |Doing | Doing |layer/activation -| [mindspore.nn.PReLU](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.PReLU) | Supported |Doing | Doing |layer/activation -| [mindspore.nn.Dropout](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Dropout) |Supported | Supported | Doing |layer/basic -| [mindspore.nn.Flatten](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Flatten) |Supported | Supported | Doing |layer/basic -| [mindspore.nn.Dense](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Dense) |Supported | Supported | Doing |layer/basic -| [mindspore.nn.DenseBnAct](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.DenseBnAct) |Supported | Supported | Supported |layer/basic -| [mindspore.nn.ClipByNorm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ClipByNorm) |Doing | Supported | Doing |layer/basic -| [mindspore.nn.Norm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Norm) |Doing | Supported | Doing |layer/basic -| [mindspore.nn.OneHot](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.OneHot) | Supported | Supported | Doing |layer/basic -| [mindspore.nn.Range](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Range) | Supported | Doing | Doing |layer/basic -| [mindspore.nn.SequentialCell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SequentialCell) |Doing | Supported | Doing |layer/container -| [mindspore.nn.CellList](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.CellList) | Supported | Supported | Doing |layer/container -| [mindspore.nn.Conv2d](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2d) | Supported | Supported | Supported |layer/conv -| [mindspore.nn.Conv2dTranspose](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2dTranspose) | Supported | Supported | Doing |layer/conv -| [mindspore.nn.Conv2dBnAct](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2dBnAct) | Supported | Supported | Supported |layer/conv -| [mindspore.nn.Embedding](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Embedding) |Doing | Supported | Doing |layer/embedding -| [mindspore.nn.ImageGradients](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ImageGradients) | Doing |Doing | Doing |layer/image -| [mindspore.nn.SSIM](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SSIM) | Doing |Doing | Doing |layer/image -| [mindspore.nn.PSNR](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.PSNR) | Doing |Doing | Doing |layer/image -| [mindspore.nn.CentralCrop](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.CentralCrop) | Supported |Doing | Doing |layer/image -| [mindspore.nn.LSTM](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LSTM) | Doing | Supported | Supported |layer/lstm -| [mindspore.nn.GlobalBatchNorm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.GlobalBatchNorm) | Supported |Doing | Doing |layer/normalization -| [mindspore.nn.BatchNorm1d](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm1d) | Supported |Doing | Doing |layer/normalization -| [mindspore.nn.BatchNorm2d](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d) | Supported | Supported | Doing |layer/normalization -| [mindspore.nn.GroupNorm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.GroupNorm) | Supported | Doing | Doing |layer/normalization -| [mindspore.nn.LayerNorm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LayerNorm) | Supported | Supported | Doing |layer/normalization -| [mindspore.nn.MaxPool2d](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.MaxPool2d) | Supported | Supported | Supported |layer/pooling -| [mindspore.nn.AvgPool2d](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AvgPool2d) |Doing | Supported | Doing |layer/pooling -| [mindspore.nn.L1Loss](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.L1Loss) |Doing |Doing | Doing |loss/loss -| [mindspore.nn.MSELoss](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.MSELoss) | Supported |Doing | Doing |loss/loss -| [mindspore.nn.SmoothL1Loss](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SmoothL1Loss) |Supported |Doing | Doing |loss/loss -| [mindspore.nn.SoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SoftmaxCrossEntropyWithLogits) | Supported | Supported | Doing |loss/loss -| [mindspore.nn.SoftmaxCrossEntropyExpand](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SoftmaxCrossEntropyExpand) | Supported |Doing | Doing |loss/loss -| [mindspore.nn.ProximalAdagrad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ProximalAdagrad) | Supported | Doing | Doing |optim/ProximalAdagrad -| [mindspore.nn.LazyAdam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LazyAdam) | Supported | Doing | Doing |optim/lazyadam -| [mindspore.nn.Adam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Adam) | Supported |Doing | Doing |optim/adam -| [mindspore.nn.AdamWeightDecay](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecay) | Supported | Supported | Doing |optim/adam -| [mindspore.nn.AdamWeightDecayDynamicLR](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecayDynamicLR) | Supported | Supported | Doing |optim/adam -| [mindspore.nn.Lamb](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Lamb) | Supported | Supported | Doing |optim/lamb -| [mindspore.nn.LARS](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LARS) |Doing |Doing | Doing |optim/lars -| [mindspore.nn.Momentum](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Momentum) | Supported | Supported | Doing |optim/momentum -| [mindspore.nn.Optimizer](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Optimizer) | Supported | Supported | Doing |optim/optimizer -| [mindspore.nn.RMSProp](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.RMSProp) | Supported | Support | Doing |optim/optimizer -| [mindspore.nn.SGD](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SGD) |Supported |Doing | Doing |optim/sgd -| [mindspore.nn.WithLossCell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithLossCell) | Supported | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.WithGradCell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithGradCell) | Supported | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.TrainOneStepCell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.TrainOneStepCell) | Supported | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.DataWrapper](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.DataWrapper) |Doing | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.GetNextSingleOp](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.GetNextSingleOp) |Doing | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.WithEvalCell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithEvalCell) | Supported | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.ParameterUpdate](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ParameterUpdate) | Supported |Doing | Doing |wrap/cell_wrapper -| [mindspore.nn.DistributedGradReducer](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.DistributedGradReducer) | Supported |Doing | Doing |wrap/grad_reducer -| [mindspore.nn.DynamicLossScaleUpdateCell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.DynamicLossScaleUpdateCell) | Doing |Doing | Doing |wrap/loss_scale -| [mindspore.nn.FixedLossScaleUpdateCell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.FixedLossScaleUpdateCell) | Doing |Doing | Doing |wrap/loss_scale -| [mindspore.nn.TrainOneStepWithLossScaleCell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.TrainOneStepWithLossScaleCell) | Doing |Doing | Doing |wrap/loss_scale -| [mindspore.nn.Cell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell) | Supported | Supported | Supported |cell +| [mindspore.nn.Softmax](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Softmax) | Supported | Supported | Supported |layer/activation +| [mindspore.nn.LogSoftmax](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LogSoftmax) | Supported | Supported | Doing |layer/activation +| [mindspore.nn.ReLU](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ReLU) | Supported | Supported | Supported |layer/activation +| [mindspore.nn.ReLU6](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ReLU6) |Supported | Supported | Doing |layer/activation +| [mindspore.nn.HSwish](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.HSwish) | Doing | Supported | Doing |layer/activation +| [mindspore.nn.HSigmoid](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.HSigmoid) | Doing | Supported | Doing |layer/activation +| [mindspore.nn.LeakyReLU](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LeakyReLU) | Supported |Doing | Doing |layer/activation +| [mindspore.nn.Tanh](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Tanh) | Supported | Supported | Doing |layer/activation +| [mindspore.nn.GELU](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.GELU) | Supported | Supported | Doing |layer/activation +| [mindspore.nn.Sigmoid](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Sigmoid) | Supported |Supported | Doing |layer/activation +| [mindspore.nn.PReLU](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.PReLU) | Supported |Doing | Doing |layer/activation +| [mindspore.nn.Dropout](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Dropout) |Supported | Supported | Doing |layer/basic +| [mindspore.nn.Flatten](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Flatten) |Supported | Supported | Doing |layer/basic +| [mindspore.nn.Dense](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Dense) |Supported | Supported | Doing |layer/basic +| [mindspore.nn.DenseBnAct](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.DenseBnAct) |Supported | Supported | Supported |layer/basic +| [mindspore.nn.ClipByNorm](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ClipByNorm) |Doing | Supported | Doing |layer/basic +| [mindspore.nn.Norm](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Norm) |Doing | Supported | Doing |layer/basic +| [mindspore.nn.OneHot](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.OneHot) | Supported | Supported | Doing |layer/basic +| [mindspore.nn.Range](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Range) | Supported | Doing | Doing |layer/basic +| [mindspore.nn.SequentialCell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SequentialCell) |Doing | Supported | Doing |layer/container +| [mindspore.nn.CellList](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.CellList) | Supported | Supported | Doing |layer/container +| [mindspore.nn.Conv2d](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2d) | Supported | Supported | Supported |layer/conv +| [mindspore.nn.Conv2dTranspose](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2dTranspose) | Supported | Supported | Doing |layer/conv +| [mindspore.nn.Conv2dBnAct](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2dBnAct) | Supported | Supported | Supported |layer/conv +| [mindspore.nn.Embedding](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Embedding) |Doing | Supported | Doing |layer/embedding +| [mindspore.nn.ImageGradients](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ImageGradients) | Doing |Doing | Doing |layer/image +| [mindspore.nn.SSIM](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SSIM) | Doing |Doing | Doing |layer/image +| [mindspore.nn.PSNR](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.PSNR) | Doing |Doing | Doing |layer/image +| [mindspore.nn.CentralCrop](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.CentralCrop) | Supported |Doing | Doing |layer/image +| [mindspore.nn.LSTM](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LSTM) | Doing | Supported | Supported |layer/lstm +| [mindspore.nn.GlobalBatchNorm](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.GlobalBatchNorm) | Supported |Doing | Doing |layer/normalization +| [mindspore.nn.BatchNorm1d](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm1d) | Supported |Doing | Doing |layer/normalization +| [mindspore.nn.BatchNorm2d](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d) | Supported | Supported | Doing |layer/normalization +| [mindspore.nn.GroupNorm](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.GroupNorm) | Supported | Doing | Doing |layer/normalization +| [mindspore.nn.LayerNorm](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LayerNorm) | Supported | Supported | Doing |layer/normalization +| [mindspore.nn.MatrixDiag](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MatrixDiag) | Supported | Doing | Doing | layer/normalization +| [mindspore.nn.MatrixDiagPart](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MatrixDiagPart) | Supported | Doing | Doing | layer/normalization +| [mindspore.nn.MatrixSetDiag](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MatrixSetDiag) | Supported | Doing | Doing | layer/normalization +| [mindspore.nn.LinSpace](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LinSpace) | Supported | Doing | Doing | layer/normalization +| [mindspore.nn.MaxPool2d](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MaxPool2d) | Supported | Supported | Supported |layer/pooling +| [mindspore.nn.AvgPool2d](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.AvgPool2d) |Doing | Supported | Doing |layer/pooling +| [mindspore.nn.L1Loss](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.L1Loss) |Doing |Doing | Doing |loss/loss +| [mindspore.nn.MSELoss](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MSELoss) | Supported |Doing | Doing |loss/loss +| [mindspore.nn.SmoothL1Loss](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SmoothL1Loss) |Supported |Doing | Doing |loss/loss +| [mindspore.nn.SoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SoftmaxCrossEntropyWithLogits) | Supported | Supported | Doing |loss/loss +| [mindspore.nn.SoftmaxCrossEntropyExpand](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SoftmaxCrossEntropyExpand) | Supported |Doing | Doing |loss/loss +| [mindspore.nn.ProximalAdagrad](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ProximalAdagrad) | Supported | Doing | Doing |optim/ProximalAdagrad +| [mindspore.nn.LazyAdam](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LazyAdam) | Supported | Doing | Doing |optim/lazyadam +| [mindspore.nn.Adam](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Adam) | Supported |Doing | Doing |optim/adam +| [mindspore.nn.AdamWeightDecay](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecay) | Supported | Supported | Doing |optim/adam +| [mindspore.nn.AdamWeightDecayDynamicLR](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecayDynamicLR) | Supported | Supported | Doing |optim/adam +| [mindspore.nn.Lamb](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Lamb) | Supported | Supported | Doing |optim/lamb +| [mindspore.nn.LARS](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LARS) |Doing |Doing | Doing |optim/lars +| [mindspore.nn.Momentum](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Momentum) | Supported | Supported | Doing |optim/momentum +| [mindspore.nn.Optimizer](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Optimizer) | Supported | Supported | Doing |optim/optimizer +| [mindspore.nn.RMSProp](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.RMSProp) | Supported | Support | Doing |optim/optimizer +| [mindspore.nn.SGD](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SGD) |Supported |Doing | Doing |optim/sgd +| [mindspore.nn.WithLossCell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithLossCell) | Supported | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.WithGradCell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithGradCell) | Supported | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.TrainOneStepCell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.TrainOneStepCell) | Supported | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.DataWrapper](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.DataWrapper) |Doing | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.GetNextSingleOp](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.GetNextSingleOp) |Doing | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.WithEvalCell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithEvalCell) | Supported | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.ParameterUpdate](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ParameterUpdate) | Supported |Doing | Doing |wrap/cell_wrapper +| [mindspore.nn.DistributedGradReducer](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.DistributedGradReducer) | Supported |Doing | Doing |wrap/grad_reducer +| [mindspore.nn.DynamicLossScaleUpdateCell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.DynamicLossScaleUpdateCell) | Doing |Doing | Doing |wrap/loss_scale +| [mindspore.nn.FixedLossScaleUpdateCell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.FixedLossScaleUpdateCell) | Doing |Doing | Doing |wrap/loss_scale +| [mindspore.nn.TrainOneStepWithLossScaleCell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.TrainOneStepWithLossScaleCell) | Doing |Doing | Doing |wrap/loss_scale +| [mindspore.nn.Cell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell) | Supported | Supported | Supported |cell ## mindspore.ops.operations | Operation | Ascend | GPU | CPU |Operator Type | :----------- |:------ |:------ |:-----|:--- -| [mindspore.ops.operations.Flatten](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Flatten) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.Softmax](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Softmax) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.Acosh](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Acosh) | Doing | Doing | Doing | nn_ops -| [mindspore.ops.operations.FloorMod](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloorMod) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.Elu](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Elu) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.MirrorPad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MirrorPad) | Doing | Doing | Doing | nn_ops -| [mindspore.ops.operations.Unpack](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Unpack) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.Pack](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pack) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.L2Loss](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.L2Loss) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.CTCLoss](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CTCLoss) | Doing | Doing | Doing | nn_ops -| [mindspore.ops.operations.LogSoftmax](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogSoftmax) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.Softplus](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Softplus) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.ReLU](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReLU) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.ReLU6](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReLU6) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.HSwish](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HSwish) | Doing | Supported |Doing | nn_ops -| [mindspore.ops.operations.HSigmoid](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HSigmoid) | Doing | Supported |Doing | nn_ops -| [mindspore.ops.operations.Sigmoid](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sigmoid) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.Tanh](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Tanh) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.BatchNorm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchNorm) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.Conv2D](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Conv2D) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.DepthwiseConv2dNative](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNative) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.DepthwiseConv2dNativeBackpropInput](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNativeBackpropInput) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.DepthwiseConv2dNativeiBackpropFilter](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNativeBackpropFilter) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.MaxPoolWithArgmax](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MaxPoolWithArgmax) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.MaxPool](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MaxPool) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.AvgPool](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AvgPool) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.Conv2DBackpropInput](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Conv2DBackpropInput) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.BiasAdd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BiasAdd) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.TopK](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TopK) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.SoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SoftmaxCrossEntropyWithLogits) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.SparseSoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseSoftmaxCrossEntropyWithLogits) | Doing | Supported | Supported | nn_ops -| [mindspore.ops.operations.ApplyMomentum](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyMomentum) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.ApplyRMSProp](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyRMSProp) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.ApplyCenteredRMSProp](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyCenteredRMSProp) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.SparseApplyAdagrad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdagrad) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.SparseApplyProximalAdagrad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyProximalAdagrad) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.ApplyProximalAdagrad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyProximalAdagrad) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.SparseApplyLazyAdam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyLazyAdam) | Doing | Doing | Supported | nn_ops -| [mindspore.ops.operations.SparseApplyAdam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdam) | Doing | Doing | Supported | nn_ops -| [mindspore.ops.operations.SmoothL1Loss](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SmoothL1Loss) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.SGD](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SGD) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.LayerNorm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LayerNorm) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.L2Normalize](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.L2Normalize) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.DropoutGenMask](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DropoutGenMask) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.DropoutDoMask](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DropoutDoMask) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.ResizeBilinear](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ResizeBilinear) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.OneHot](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.OneHot) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.Gelu](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Gelu) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.GetNext](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GetNext) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.PReLU](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.PReLU) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.LSTM](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LSTM) | Doing | Supported | Supported | nn_ops -| [mindspore.ops.operations.BasicLSTMCell](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BasicLSTMCell) | Doing | Doing | Doing | nn_ops -| [mindspore.ops.operations.SigmoidCrossEntropyWithLogits](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SigmoidCrossEntropyWithLogits) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.Pad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pad) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.ROIAlign](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ROIAlign) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.Adam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Adam) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.BinaryCrossEntropy](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BinaryCrossEntropy) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.LARSUpdate](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LARSUpdate) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.TensorAdd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TensorAdd) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.AssignAdd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AssignAdd) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.AssignSub](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AssignSub) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ReduceMean](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMean) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.ReduceSum](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceSum) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.ReduceAll](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceAll) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ReduceMax](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMax) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.ReduceMin](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMin) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ReduceProd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceProd) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.CumProd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CumProd) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.MatMul](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MatMul) | Supported | Supported | Supported | math_ops -| [mindspore.ops.operations.BatchMatMul](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchMatMul) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.CumSum](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CumSum) | Supported | Doing| Doing | math_ops -| [mindspore.ops.operations.AddN](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AddN) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Neg](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Neg) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Sub](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sub) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Mul](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Mul) | Supported | Supported | Supported | math_ops -| [mindspore.ops.operations.Square](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Square) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.SquareSumAll](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SquareSumAll) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Rsqrt](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Rsqrt) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Sqrt](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sqrt) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Reciprocal](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Reciprocal) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Pow](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pow) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Exp](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Exp) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Log](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Log) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Log1p](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Log1p) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Minimum](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Minimum) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Maximum](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Maximum) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.RealDiv](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.RealDiv) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Div](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Div) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.DivNoNan](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DivNoNan) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.FloorDiv](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloorDiv) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Floor](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Floor) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Equal](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Equal) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.EqualCount](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.EqualCount) | Doing | Supported | Supported | math_ops -| [mindspore.ops.operations.NotEqual](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NotEqual) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Greater](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Greater) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.GreaterEqual](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GreaterEqual) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Less](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Less) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Atan2](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Atan2) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.LessEqual](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LessEqual) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.LogicalNot](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalNot) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.LogicalAnd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalAnd) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.LogicalOr](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalOr) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.BitwiseAnd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseAnd) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.BitwiseOr](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseOr) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Ceil](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Ceil) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Inv](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Inv) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Invert](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Invert) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.BitwiseXor](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseXor) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.NPUAllocFloatStatus](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUAllocFloatStatus) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.NPUGetFloatStatus](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUGetFloatStatus) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.NPUClearFloatStatus](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUClearFloatStatus) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.FloatStatus](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloatStatus) | Doing | Supported | Doing | math_ops -| [mindspore.ops.operations.Cos](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cos) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Cosh](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cosh) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ACos](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ACos) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.BesselI0e](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BesselI0e) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.BesselI1e](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BesselI1e) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Asin](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Asin) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Asinh](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Asinh) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Erf](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Erf) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Erfc](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Erfc) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Sin](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sin) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Sinh](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sinh) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Expm1](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Expm1) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.NMSWithMask](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NMSWithMask) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Abs](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Abs) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Sign](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sign) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Round](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Round) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ApproximateEqual](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApproximateEqual) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.InplaceAdd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceAdd) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.InplaceSub](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceSub) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ExpandDims](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ExpandDims) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.DType](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DType) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.SameTypeShape](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SameTypeShape) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Cast](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cast) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.IsSubClass](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsSubClass) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.IsInstance](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsInstance) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Reshape](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Reshape) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Shape](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Shape) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Squeeze](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Squeeze) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Transpose](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Transpose) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.GatherV2](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherV2) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Split](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Split) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.Rank](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Rank) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.TruncatedNormal](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TruncatedNormal) | Doing | Doing | Doing | array_ops -| [mindspore.ops.operations.Size](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Size) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Fill](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Fill) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.OnesLike](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.OnesLike) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ZerosLike](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ZerosLike) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.TupleToArray](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TupleToArray) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.ScalarToArray](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarToArray) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.ScalarToTensor](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarToTensor) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.InvertPermutation](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InvertPermutation) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.Argmax](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Argmax) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Argmin](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Argmin) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ArgMaxWithValue](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ArgMaxWithValue) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.ArgMinWithValue](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ArgMinWithValue) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.Tile](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Tile) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.UnsortedSegmentSum](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.UnsortedSegmentSum) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.UnsortedSegmentMin](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.UnsortedSegmentMin) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.Concat](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Concat) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Slice](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Slice) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Select](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Select) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.StridedSlice](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.StridedSlice) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Diag](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Diag) | Doing | Doing | Doing | array_ops -| [mindspore.ops.operations.DiagPart](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DiagPart) | Doing | Doing | Doing | array_ops -| [mindspore.ops.operations.Eye](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Eye) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.ScatterNd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNd) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ResizeNearestNeighbor](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ResizeNearestNeighbor) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.GatherNd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherNd) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ApplyFtrl](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyFtrl) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.SparseApplyFtrl](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyFtrl) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ScatterNdUpdate](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNdUpdate) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ScatterUpdate](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterUpdate) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.SpaceToDepth](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToDepth) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.DepthToSpace](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthToSpace) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.SpaceToBatch](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToBatch) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.SpaceToBatchND](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToBatchND) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.BatchToSpace](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchToSpace) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.BatchToSpaceND](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchToSpaceND) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.IsFinite](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsFinite) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.InplaceUpdate](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceUpdate) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ReduceOp](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceOp) | Supported | Supported | Doing | comm_ops -| [mindspore.ops.operations.AllReduce](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AllReduce) | Supported | Supported | Doing | comm_ops -| [mindspore.ops.operations.AllGather](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AllGather) | Supported | Supported | Doing | comm_ops -| [mindspore.ops.operations.HostAllGather](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HostAllGather) | Doing | Doing | Supported | comm_ops -| [mindspore.ops.operations.ReduceScatter](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceScatter) | Doing | Supported | Doing | comm_ops -| [mindspore.ops.operations.HostReduceScatter](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HostReduceScatter) | Doing | Doing | Supported | comm_ops -| [mindspore.ops.operations.Broadcast](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Broadcast) | Supported | Doing | Doing | comm_ops -| [mindspore.ops.operations.ControlDepend](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ControlDepend) | Supported | Supported | Supported | control_ops -| [mindspore.ops.operations.GeSwitch](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GeSwitch) | Doing | Doing | Doing | control_ops -| [mindspore.ops.operations.Merge](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Merge) | Doing | Doing | Doing | control_ops -| [mindspore.ops.operations.ScalarSummary](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarSummary) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.ImageSummary](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ImageSummary) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.TensorSummary](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TensorSummary) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.HistogramSummary](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HistogramSummary) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.InsertGradientOf](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InsertGradientOf) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.Print](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Print) | Supported | Doing | Doing | debug_ops -| [mindspore.ops.operations.Assign](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Assign) | Supported | Supported | Doing | other_ops -| [mindspore.ops.operations.BoundingBoxEncode](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BoundingBoxEncode) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.BoundingBoxDecode](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BoundingBoxDecode) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.CheckValid](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CheckValid) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.IOU](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IOU) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.MakeRefKey](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MakeRefKey) | Supported | Supported | Supported | other_ops -| [mindspore.ops.operations.InTopK](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InTopK) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.RandomChoiceWithMask](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.RandomChoiceWithMask) | Doing| Doing | Doing | random_ops -| [mindspore.ops.operations.ScalarCast](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarCast) | Supported | Supported | Supported | inner_ops +| [mindspore.ops.operations.Flatten](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Flatten) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.Softmax](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Softmax) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.Acosh](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Acosh) | Doing | Doing | Doing | nn_ops +| [mindspore.ops.operations.FloorMod](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloorMod) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.Elu](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Elu) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.MirrorPad](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MirrorPad) | Doing | Doing | Doing | nn_ops +| [mindspore.ops.operations.Unpack](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Unpack) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.Pack](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pack) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.L2Loss](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.L2Loss) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.CTCLoss](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CTCLoss) | Doing | Doing | Doing | nn_ops +| [mindspore.ops.operations.LogSoftmax](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogSoftmax) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.Softplus](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Softplus) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.ReLU](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReLU) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.ReLU6](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReLU6) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.HSwish](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HSwish) | Doing | Supported |Doing | nn_ops +| [mindspore.ops.operations.HSigmoid](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HSigmoid) | Doing | Supported |Doing | nn_ops +| [mindspore.ops.operations.Sigmoid](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sigmoid) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.Tanh](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Tanh) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.BatchNorm](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchNorm) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.LRN](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LRN) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.Conv2D](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Conv2D) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.DepthwiseConv2dNative](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNative) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.DepthwiseConv2dNativeBackpropInput](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNativeBackpropInput) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.DepthwiseConv2dNativeiBackpropFilter](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNativeBackpropFilter) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.MaxPoolWithArgmax](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MaxPoolWithArgmax) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.MaxPool](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MaxPool) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.AvgPool](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AvgPool) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.Conv2DBackpropInput](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Conv2DBackpropInput) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.BiasAdd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BiasAdd) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.TopK](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TopK) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.SoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SoftmaxCrossEntropyWithLogits) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.SparseSoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseSoftmaxCrossEntropyWithLogits) | Doing | Supported | Supported | nn_ops +| [mindspore.ops.operations.ApplyMomentum](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyMomentum) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.ApplyRMSProp](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyRMSProp) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.ApplyCenteredRMSProp](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyCenteredRMSProp) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.SparseApplyAdagrad](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdagrad) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.SparseApplyProximalAdagrad](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyProximalAdagrad) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.ApplyProximalAdagrad](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyProximalAdagrad) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.SparseApplyLazyAdam](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyLazyAdam) | Doing | Doing | Supported | nn_ops +| [mindspore.ops.operations.SparseApplyAdam](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdam) | Doing | Doing | Supported | nn_ops +| [mindspore.ops.operations.SmoothL1Loss](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SmoothL1Loss) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.SGD](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SGD) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.LayerNorm](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LayerNorm) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.L2Normalize](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.L2Normalize) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.DropoutGenMask](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DropoutGenMask) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.DropoutDoMask](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DropoutDoMask) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.ResizeBilinear](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ResizeBilinear) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.OneHot](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.OneHot) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.Gelu](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Gelu) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.GetNext](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GetNext) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.PReLU](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.PReLU) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.LSTM](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LSTM) | Doing | Supported | Supported | nn_ops +| [mindspore.ops.operations.BasicLSTMCell](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BasicLSTMCell) | Doing | Doing | Doing | nn_ops +| [mindspore.ops.operations.SigmoidCrossEntropyWithLogits](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SigmoidCrossEntropyWithLogits) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.Pad](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pad) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.ROIAlign](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ROIAlign) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.Adam](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Adam) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.BinaryCrossEntropy](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BinaryCrossEntropy) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.LARSUpdate](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LARSUpdate) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.TensorAdd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TensorAdd) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.AssignAdd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AssignAdd) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.AssignSub](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AssignSub) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ReduceMean](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMean) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.ReduceSum](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceSum) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.ReduceAll](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceAll) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ReduceMax](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMax) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.ReduceMin](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMin) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ReduceProd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceProd) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.CumProd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CumProd) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.MatMul](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MatMul) | Supported | Supported | Supported | math_ops +| [mindspore.ops.operations.BatchMatMul](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchMatMul) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.CumSum](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CumSum) | Supported | Doing| Doing | math_ops +| [mindspore.ops.operations.AddN](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AddN) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Neg](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Neg) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Sub](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sub) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Mul](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Mul) | Supported | Supported | Supported | math_ops +| [mindspore.ops.operations.Square](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Square) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.SquareSumAll](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SquareSumAll) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Rsqrt](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Rsqrt) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Sqrt](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sqrt) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Reciprocal](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Reciprocal) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Pow](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pow) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Exp](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Exp) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Log](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Log) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Log1p](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Log1p) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Minimum](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Minimum) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Maximum](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Maximum) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.RealDiv](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.RealDiv) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Div](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Div) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.DivNoNan](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DivNoNan) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.FloorDiv](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloorDiv) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Floor](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Floor) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Equal](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Equal) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.EqualCount](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.EqualCount) | Doing | Supported | Supported | math_ops +| [mindspore.ops.operations.NotEqual](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NotEqual) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Greater](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Greater) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.GreaterEqual](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GreaterEqual) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Less](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Less) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Atan2](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Atan2) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.LessEqual](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LessEqual) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.LogicalNot](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalNot) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.LogicalAnd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalAnd) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.LogicalOr](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalOr) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.BitwiseAnd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseAnd) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.BitwiseOr](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseOr) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Ceil](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Ceil) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Inv](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Inv) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Invert](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Invert) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.BitwiseXor](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseXor) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.NPUAllocFloatStatus](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUAllocFloatStatus) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.NPUGetFloatStatus](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUGetFloatStatus) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.NPUClearFloatStatus](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUClearFloatStatus) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.FloatStatus](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloatStatus) | Doing | Supported | Doing | math_ops +| [mindspore.ops.operations.Cos](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cos) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Cosh](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cosh) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ACos](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ACos) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.BesselI0e](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BesselI0e) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.BesselI1e](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BesselI1e) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Asin](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Asin) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Asinh](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Asinh) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Erf](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Erf) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Erfc](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Erfc) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Sin](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sin) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Sinh](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sinh) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Expm1](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Expm1) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.NMSWithMask](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NMSWithMask) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Abs](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Abs) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Sign](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sign) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Round](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Round) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ApproximateEqual](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApproximateEqual) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.InplaceAdd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceAdd) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.InplaceSub](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceSub) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ExpandDims](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ExpandDims) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.DType](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DType) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.SameTypeShape](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SameTypeShape) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Cast](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cast) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.IsSubClass](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsSubClass) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.IsInstance](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsInstance) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Reshape](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Reshape) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Shape](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Shape) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Squeeze](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Squeeze) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Transpose](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Transpose) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.GatherV2](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherV2) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Split](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Split) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.Rank](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Rank) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.TruncatedNormal](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TruncatedNormal) | Doing | Doing | Doing | array_ops +| [mindspore.ops.operations.Size](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Size) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Fill](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Fill) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.OnesLike](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.OnesLike) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ZerosLike](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ZerosLike) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.TupleToArray](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TupleToArray) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.ScalarToArray](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarToArray) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.ScalarToTensor](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarToTensor) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.InvertPermutation](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InvertPermutation) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.Argmax](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Argmax) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Argmin](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Argmin) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ArgMaxWithValue](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ArgMaxWithValue) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.ArgMinWithValue](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ArgMinWithValue) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.Tile](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Tile) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.UnsortedSegmentSum](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.UnsortedSegmentSum) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.UnsortedSegmentMin](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.UnsortedSegmentMin) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.Concat](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Concat) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Slice](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Slice) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Select](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Select) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.StridedSlice](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.StridedSlice) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Diag](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Diag) | Doing | Doing | Doing | array_ops +| [mindspore.ops.operations.DiagPart](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DiagPart) | Doing | Doing | Doing | array_ops +| [mindspore.ops.operations.Eye](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Eye) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.ScatterNd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNd) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ResizeNearestNeighbor](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ResizeNearestNeighbor) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.GatherNd](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherNd) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ApplyFtrl](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyFtrl) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.SparseApplyFtrl](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyFtrl) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ScatterNdUpdate](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNdUpdate) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ScatterUpdate](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterUpdate) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.SpaceToDepth](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToDepth) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.DepthToSpace](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthToSpace) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.SpaceToBatch](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToBatch) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.SpaceToBatchND](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToBatchND) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.BatchToSpace](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchToSpace) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.BatchToSpaceND](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchToSpaceND) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.IsFinite](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsFinite) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.InplaceUpdate](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceUpdate) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ReduceOp](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceOp) | Supported | Supported | Doing | comm_ops +| [mindspore.ops.operations.AllReduce](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AllReduce) | Supported | Supported | Doing | comm_ops +| [mindspore.ops.operations.AllGather](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AllGather) | Supported | Supported | Doing | comm_ops +| [mindspore.ops.operations.HostAllGather](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HostAllGather) | Doing | Doing | Supported | comm_ops +| [mindspore.ops.operations.ReduceScatter](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceScatter) | Doing | Supported | Doing | comm_ops +| [mindspore.ops.operations.HostReduceScatter](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HostReduceScatter) | Doing | Doing | Supported | comm_ops +| [mindspore.ops.operations.Broadcast](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Broadcast) | Supported | Doing | Doing | comm_ops +| [mindspore.ops.operations.ControlDepend](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ControlDepend) | Supported | Supported | Supported | control_ops +| [mindspore.ops.operations.GeSwitch](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GeSwitch) | Doing | Doing | Doing | control_ops +| [mindspore.ops.operations.Merge](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Merge) | Doing | Doing | Doing | control_ops +| [mindspore.ops.operations.ScalarSummary](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarSummary) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.ImageSummary](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ImageSummary) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.TensorSummary](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TensorSummary) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.HistogramSummary](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HistogramSummary) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.InsertGradientOf](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InsertGradientOf) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.Print](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Print) | Supported | Doing | Doing | debug_ops +| [mindspore.ops.operations.Assign](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Assign) | Supported | Supported | Doing | other_ops +| [mindspore.ops.operations.BoundingBoxEncode](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BoundingBoxEncode) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.BoundingBoxDecode](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BoundingBoxDecode) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.CheckValid](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CheckValid) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.IOU](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IOU) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.MakeRefKey](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MakeRefKey) | Supported | Supported | Supported | other_ops +| [mindspore.ops.operations.InTopK](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InTopK) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.RandomChoiceWithMask](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.RandomChoiceWithMask) | Doing| Doing | Doing | random_ops +| [mindspore.ops.operations.ScalarCast](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarCast) | Supported | Supported | Supported | inner_ops diff --git a/docs/source_en/roadmap.md b/docs/source_en/roadmap.md index 5c89ddfb0d26a29a7f515ee9e6485bf752359c89..aeec98616e8afaeee3558fa964a1a31d99b6680c 100644 --- a/docs/source_en/roadmap.md +++ b/docs/source_en/roadmap.md @@ -14,7 +14,7 @@ MindSpore's top priority plans in the year are displayed as follows. We will con - + In general, we will make continuous improvements in the following aspects: 1. Support more preset models. diff --git a/docs/source_zh_cn/architecture.md b/docs/source_zh_cn/architecture.md index 27205cde10022548ae9d25911746b1308a61956c..7f05d732c2bf31c033edd3c6a57fbad055f80f41 100644 --- a/docs/source_zh_cn/architecture.md +++ b/docs/source_zh_cn/architecture.md @@ -8,7 +8,7 @@ - + MindSpore框架架构总体分为MindSpore前端表示层、MindSpore计算图引擎和MindSpore后端运行时三层。 diff --git a/docs/source_zh_cn/benchmark.md b/docs/source_zh_cn/benchmark.md index 264a5f6d69fa784a8c41f9105eb6035fbf835b2a..02e3db37d23aeb640ce40fa2d106d9d65a5ca276 100644 --- a/docs/source_zh_cn/benchmark.md +++ b/docs/source_zh_cn/benchmark.md @@ -1,18 +1,18 @@ # 基准性能 - + -本文介绍MindSpore的基准性能。MindSpore预训练模型可参考[Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/mindspore/model_zoo)。 +本文介绍MindSpore的基准性能。MindSpore预训练模型可参考[Model Zoo](https://gitee.com/mindspore/mindspore/tree/r0.5/model_zoo)。 ## 训练性能 ### ResNet -| Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | +| Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| ResNet-50 v1.5 | CNN | ImageNet2012 | 0.2.0-alpha | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 32 | 1787 images/sec | - | -| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 32 | 13689 images/sec | 0.95 | -| | | | | Ascend: 16 * Ascend 910
CPU：384 Cores | Mixed | 32 | 27090 images/sec | 0.94 | +| ResNet-50 v1.5 | CNN | ImageNet2012 | 0.5.0-beta | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 256 | 2115 images/sec | - | +| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 256 | 16600 images/sec | 0.98 | +| | | | | Ascend: 16 * Ascend 910
CPU：384 Cores | Mixed | 256 | 32768 images/sec | 0.96 | 1. 以上数据基于华为云AI开发平台ModelArts测试获得，是训练过程整体下沉至Ascend 910 AI处理器执行所得的平均性能。 2. 业界其他开源框架数据可参考：[ResNet-50 v1.5 for TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/Classification/RN50v1.5#nvidia-dgx-2-16x-v100-32g)。 @@ -21,8 +21,8 @@ | Network | Network Type | Dataset | MindSpore Version | Resource                 | Precision | Batch Size | Throughput | Speedup | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| BERT-Large | Attention | zhwiki | 0.2.0-alpha | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 96 | 210 sentences/sec | - | -| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 96 | 1613 sentences/sec | 0.96 | +| BERT-Large | Attention | zhwiki | 0.5.0-beta | Ascend: 1 * Ascend 910
CPU：24 Cores | Mixed | 96 | 269 sentences/sec | - | +| | | | | Ascend: 8 * Ascend 910
CPU：192 Cores | Mixed | 96 | 2069 sentences/sec | 0.96 | 1. 以上数据基于华为云AI开发平台ModelArts测试获得，其中网络包含24个隐藏层，句长为128个token，字典表包含21128个token。 2. 业界其他开源框架数据可参考：[BERT For TensorFlow](https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT)。 \ No newline at end of file diff --git a/docs/source_zh_cn/conf.py b/docs/source_zh_cn/conf.py index f58451f3fafe89fa2a734d62f19c94424250757b..66bfcc4be4f22188b4de1a82452ef64c2cd6f47b 100644 --- a/docs/source_zh_cn/conf.py +++ b/docs/source_zh_cn/conf.py @@ -20,7 +20,7 @@ copyright = '2020, MindSpore' author = 'MindSpore' # The full version, including alpha/beta/rc tags -release = 'master' +release = 'r0.5' # -- General configuration --------------------------------------------------- diff --git a/docs/source_zh_cn/constraints_on_network_construction.md b/docs/source_zh_cn/constraints_on_network_construction.md index ff628c798272c2a83e0e7ff0aee2da47dc71c65e..6cdf89695101d1e3514c329281ed8c8925c14a51 100644 --- a/docs/source_zh_cn/constraints_on_network_construction.md +++ b/docs/source_zh_cn/constraints_on_network_construction.md @@ -23,7 +23,7 @@ - + ## 概述 MindSpore完成从用户源码到计算图的编译，用户源码基于Python语法编写，当前MindSpore支持将普通函数或者继承自nn.Cell的实例转换生成计算图，暂不支持将任意Python源码转换成计算图，所以对于用户源码支持的写法有所限制，主要包括语法约束和网络定义约束两方面。随着MindSpore的演进，这些约束可能会发生变化。 @@ -143,8 +143,8 @@ ## 网络定义约束 ### 整网实例类型 -* 带[@ms_function](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.html#mindspore.ms_function)装饰器的普通Python函数。 -* 继承自[nn.Cell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell)的Cell子类。 +* 带[@ms_function](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.html#mindspore.ms_function)装饰器的普通Python函数。 +* 继承自[nn.Cell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell)的Cell子类。 ### 网络输入类型 * 整网的训练数据输入参数只能是Tensor类型。 @@ -157,13 +157,13 @@ | 类别 | 内容 | :----------- |:-------- -| `Cell`实例 |[mindspore/nn/*](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html)、自定义[Cell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell)。 +| `Cell`实例 |[mindspore/nn/*](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html)、自定义[Cell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell)。 | `Cell`实例的成员函数 | Cell的construct中可以调用其他类成员函数。 | 函数 | 自定义Python函数、前文中列举的系统函数。 | dataclass实例 | 使用@dataclass装饰的类。 -| Primitive算子 |[mindspore/ops/operations/*](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html) -| Composite算子 |[mindspore/ops/composite/*](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.composite.html) -| constexpr生成算子 |使用[@constexpr](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.html#mindspore.ops.constexpr)生成的值计算算子。 +| Primitive算子 |[mindspore/ops/operations/*](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html) +| Composite算子 |[mindspore/ops/composite/*](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.composite.html) +| constexpr生成算子 |使用[@constexpr](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.html#mindspore.ops.constexpr)生成的值计算算子。 ### 其他约束 diff --git a/docs/source_zh_cn/glossary.md b/docs/source_zh_cn/glossary.md index eb1c2cdd6ace20ac52d24438031efaaed935f64d..20aee2b801113e63c630aedc4a5ab12d521208a6 100644 --- a/docs/source_zh_cn/glossary.md +++ b/docs/source_zh_cn/glossary.md @@ -6,7 +6,7 @@ - + | 术语/缩略语 | 说明 | | ----- | ----- | diff --git a/docs/source_zh_cn/index.rst b/docs/source_zh_cn/index.rst index 3ce1555906e8fd0b38c6661ec21d748b6d8c4089..8e68b121f8549e85df45b59096585031b0784450 100644 --- a/docs/source_zh_cn/index.rst +++ b/docs/source_zh_cn/index.rst @@ -13,8 +13,9 @@ MindSpore文档 architecture roadmap benchmark + technical_white_paper network_list operator_list constraints_on_network_construction glossary - community + community \ No newline at end of file diff --git a/docs/source_zh_cn/network_list.md b/docs/source_zh_cn/network_list.md index 09e12befc6b672f1fb689e3b7cbcbf4e608fdff6..57770cee2cc4da0543d0a6ec3c23b877173490b9 100644 --- a/docs/source_zh_cn/network_list.md +++ b/docs/source_zh_cn/network_list.md @@ -1,18 +1,29 @@ # 网络支持 - + | 领域 | 子领域 | 网络 | Ascend | GPU | CPU |:------ |:------| :----------- |:------ |:------ |:----- -|计算机视觉（CV） | 图像分类（Image Classification） | [AlexNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/alexnet/src/alexnet.py) | Supported | Supported | Doing -| 计算机视觉（CV） | 图像分类（Image Classification） | [GoogleNet](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/googlenet.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 图像分类（Image Classification） | [LeNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/lenet/src/lenet.py) | Supported | Supported | Supported -| 计算机视觉（CV） | 图像分类（Image Classification） | [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/resnet.py) | Supported | Doing | Doing -|计算机视觉（CV） | 图像分类（Image Classification） | [ResNet-101](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/resnet.py) | Supported |Doing | Doing -| 计算机视觉（CV） | 图像分类（Image Classification） | [VGG16](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/vgg.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 移动端图像分类（Mobile Image Classification）
目标检测（Image Classification）
语义分割（Semantic Tegmentation） | [MobileNetV2](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/mobilenetv2/src/mobilenetV2.py) | Supported | Doing | Doing -|计算机视觉（CV） | 目标检测（Targets Detection） | [SSD](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/ssd/src/ssd.py) | Supported |Doing | Doing -| 计算机视觉（CV） | 目标检测（Targets Detection） | [YoloV3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/yolov3/src/yolov3.py) | Supported | Doing | Doing -| 计算机视觉（CV） | 语义分割（Semantic Segmentation） | [Deeplabv3](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/deeplabv3/src/deeplabv3.py) | Supported | Doing | Doing -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [BERT](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/bert/src/bert_model.py) | Supported | Doing | Doing -| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [SentimentNet](https://gitee.com/mindspore/mindspore/blob/master/model_zoo/lstm/src/lstm.py) | Doing | Supported | Supported +|计算机视觉（CV） | 图像分类（Image Classification） | [AlexNet](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/alexnet/src/alexnet.py) | Supported | Supported | Doing +| 计算机视觉（CV） | 图像分类（Image Classification） | [GoogleNet](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/googlenet/src/googlenet.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 图像分类（Image Classification） | [LeNet](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/lenet/src/lenet.py) | Supported | Supported | Supported +| 计算机视觉（CV） | 图像分类（Image Classification） | [ResNet-50](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/resnet/src/resnet.py) | Supported | Doing | Doing +|计算机视觉（CV） | 图像分类（Image Classification） | [ResNet-101](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/resnet/src/resnet.py) | Supported |Doing | Doing +| 计算机视觉（CV） | 图像分类（Image Classification） | [VGG16](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/vgg16/src/vgg.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 移动端图像分类（Mobile Image Classification）
目标检测（Image Classification）
语义分割（Semantic Tegmentation） | [MobileNetV2](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/mobilenetv2/src/mobilenetV2.py) | Supported | Supported | Doing +| 计算机视觉（CV） | 移动端图像分类（Mobile Image Classification）
目标检测（Image Classification）
语义分割（Semantic Tegmentation） | [MobileNetV3](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/mobilenetv3/src/mobilenetV3.py) | Doing | Supported | Doing +|计算机视觉（CV） | 目标检测（Targets Detection） | [SSD](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/ssd/src/ssd.py) | Supported |Doing | Doing +| 计算机视觉（CV） | 目标检测（Targets Detection） | [YoloV3](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/yolov3/src/yolov3.py) | Supported | Doing | Doing +| 计算机视觉（CV） | 目标检测（Targets Detection） | [FasterRCNN](https://gitee.com/mindspore/mindspore/tree/r0.5/model_zoo/faster_rcnn/src/FasterRcnn) | Supported | Doing | Doing +| 计算机视觉（CV） | 语义分割（Semantic Segmentation） | [Deeplabv3](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/deeplabv3/src/deeplabv3.py) | Supported | Doing | Doing +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [BERT](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/bert/src/bert_model.py) | Supported | Doing | Doing +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [Transformer](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/Transformer/src/transformer_model.py) | Supported | Doing | Doing +| 自然语言处理（NLP） | 自然语言理解（Natural Language Understanding） | [SentimentNet](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/lstm/src/lstm.py) | Doing | Supported | Supported +| 推荐（Recommender） | 推荐系统、点击率预估（Recommender System, CTR prediction） | [DeepFM](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/deepfm/src/deepfm.py) | Supported | Doing | Doing +| 推荐（Recommender） | 推荐系统、搜索、排序（Recommender System, Search ranking） | [Wide&Deep](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/wide_and_deep/src/wide_and_deep.py) | Supported | Doing | Doing +| 图神经网络（GNN） | 文本分类（Text Classification） | [GCN](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/gcn/src/gcn.py) | Supported | Doing | Doing +| 图神经网络（GNN） | 文本分类（Text Classification） | [GAT](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/gat/src/gat.py) | Supported | Doing | Doing + + +## 预训练模型 +建设中，即将上线 diff --git a/docs/source_zh_cn/operator_list.md b/docs/source_zh_cn/operator_list.md index 7c1c98cba1aae4dbf1431d636a795874db83fbaa..d0fdb4808593938d1e3f812ebd25b9afa4ebe444 100644 --- a/docs/source_zh_cn/operator_list.md +++ b/docs/source_zh_cn/operator_list.md @@ -8,286 +8,291 @@ - + ## mindspore.nn | 操作名 | Ascend | GPU | CPU |算子类别 | :----------- |:------ |:------ |:-----|:--- -| [mindspore.nn.Softmax](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Softmax) | Supported | Supported | Supported |layer/activation -| [mindspore.nn.LogSoftmax](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LogSoftmax) | Supported | Supported | Doing |layer/activation -| [mindspore.nn.ReLU](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ReLU) | Supported | Supported | Supported |layer/activation -| [mindspore.nn.ReLU6](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ReLU6) |Supported | Supported | Doing |layer/activation -| [mindspore.nn.HSwish](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.HSwish) | Doing | Supported | Doing |layer/activation -| [mindspore.nn.HSigmoid](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.HSigmoid) | Doing | Supported | Doing |layer/activation -| [mindspore.nn.LeakyReLU](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LeakyReLU) | Supported |Doing | Doing |layer/activation -| [mindspore.nn.Tanh](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Tanh) | Supported | Supported | Doing |layer/activation -| [mindspore.nn.GELU](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.GELU) | Supported | Supported | Doing |layer/activation -| [mindspore.nn.Sigmoid](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Sigmoid) | Supported |Doing | Doing |layer/activation -| [mindspore.nn.PReLU](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.PReLU) | Supported |Doing | Doing |layer/activation -| [mindspore.nn.Dropout](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Dropout) |Supported | Supported | Doing |layer/basic -| [mindspore.nn.Flatten](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Flatten) |Supported | Supported | Doing |layer/basic -| [mindspore.nn.Dense](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Dense) |Supported | Supported | Doing |layer/basic -| [mindspore.nn.DenseBnAct](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.DenseBnAct) |Supported | Supported | Supported |layer/basic -| [mindspore.nn.ClipByNorm](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ClipByNorm) |Doing | Supported | Doing |layer/basic -| [mindspore.nn.Norm](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Norm) |Doing | Supported | Doing |layer/basic -| [mindspore.nn.OneHot](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.OneHot) | Supported | Supported | Doing |layer/basic -| [mindspore.nn.Range](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Range) | Supported | Doing | Doing |layer/basic -| [mindspore.nn.SequentialCell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SequentialCell) |Doing | Supported | Doing |layer/container -| [mindspore.nn.CellList](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.CellList) | Supported | Supported | Doing |layer/container -| [mindspore.nn.Conv2d](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2d) | Supported | Supported | Supported |layer/conv -| [mindspore.nn.Conv2dTranspose](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2dTranspose) | Supported | Supported | Doing |layer/conv -| [mindspore.nn.Conv2dBnAct](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2dBnAct) | Supported | Supported | Supported |layer/conv -| [mindspore.nn.Embedding](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Embedding) |Doing | Supported | Doing |layer/embedding -| [mindspore.nn.ImageGradients](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ImageGradients) | Doing |Doing | Doing |layer/image -| [mindspore.nn.SSIM](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SSIM) | Doing |Doing | Doing |layer/image -| [mindspore.nn.PSNR](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.PSNR) | Doing |Doing | Doing |layer/image -| [mindspore.nn.CentralCrop](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.CentralCrop) | Supported |Doing | Doing |layer/image -| [mindspore.nn.LSTM](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LSTM) | Doing | Supported | Supported |layer/lstm -| [mindspore.nn.GlobalBatchNorm](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.GlobalBatchNorm) | Supported |Doing | Doing |layer/normalization -| [mindspore.nn.BatchNorm1d](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm1d) | Supported |Doing | Doing |layer/normalization -| [mindspore.nn.BatchNorm2d](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d) | Supported | Supported | Doing |layer/normalization -| [mindspore.nn.GroupNorm](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.GroupNorm) | Supported | Doing | Doing |layer/normalization -| [mindspore.nn.LayerNorm](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LayerNorm) | Supported | Supported | Doing |layer/normalization -| [mindspore.nn.MaxPool2d](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.MaxPool2d) | Supported | Supported | Supported |layer/pooling -| [mindspore.nn.AvgPool2d](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AvgPool2d) |Doing | Supported | Doing |layer/pooling -| [mindspore.nn.L1Loss](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.L1Loss) |Doing |Doing | Doing |loss/loss -| [mindspore.nn.MSELoss](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.MSELoss) | Supported |Doing | Doing |loss/loss -| [mindspore.nn.SmoothL1Loss](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SmoothL1Loss) | Supported |Doing | Doing |loss/loss -| [mindspore.nn.SoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SoftmaxCrossEntropyWithLogits) | Supported | Supported | Doing |loss/loss -| [mindspore.nn.SoftmaxCrossEntropyExpand](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SoftmaxCrossEntropyExpand) | Supported |Doing | Doing |loss/loss -| [mindspore.nn.ProximalAdagrad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ProximalAdagrad) | Supported |Doing | Doing |optim/ProximalAdagrad -| [mindspore.nn.LazyAdam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LazyAdam) | Supported |Doing | Doing |optim/lazyadam -| [mindspore.nn.Adam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Adam) | Supported |Doing | Doing |optim/adam -| [mindspore.nn.AdamWeightDecay](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecay) | Supported | Supported | Doing |optim/adam -| [mindspore.nn.AdamWeightDecayDynamicLR](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecayDynamicLR) | Supported | Supported | Doing |optim/adam -| [mindspore.nn.Lamb](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Lamb) | Supported | Supported | Doing |optim/lamb -| [mindspore.nn.LARS](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LARS) |Doing |Doing | Doing |optim/lars -| [mindspore.nn.Momentum](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Momentum) | Supported | Supported | Doing |optim/momentum -| [mindspore.nn.Optimizer](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Optimizer) | Supported | Supported | Doing |optim/optimizer -| [mindspore.nn.RMSProp](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.RMSProp) | Supported | Supported | Doing |optim/optimizer -| [mindspore.nn.SGD](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.SGD) |Supported |Doing | Doing |optim/sgd -| [mindspore.nn.WithLossCell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithLossCell) | Supported | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.WithGradCell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithGradCell) | Supported | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.TrainOneStepCell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.TrainOneStepCell) | Supported | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.DataWrapper](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.DataWrapper) |Doing | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.GetNextSingleOp](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.GetNextSingleOp) |Doing | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.WithEvalCell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithEvalCell) | Supported | Supported | Doing |wrap/cell_wrapper -| [mindspore.nn.ParameterUpdate](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.ParameterUpdate) | Supported |Doing | Doing |wrap/cell_wrapper -| [mindspore.nn.DistributedGradReducer](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.DistributedGradReducer) | Supported |Doing | Doing |wrap/grad_reducer -| [mindspore.nn.DynamicLossScaleUpdateCell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.DynamicLossScaleUpdateCell) | Doing |Doing | Doing |wrap/loss_scale -| [mindspore.nn.FixedLossScaleUpdateCell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.FixedLossScaleUpdateCell) | Doing |Doing | Doing |wrap/loss_scale -| [mindspore.nn.TrainOneStepWithLossScaleCell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.TrainOneStepWithLossScaleCell) | Doing |Doing | Doing |wrap/loss_scale -| [mindspore.nn.Cell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell) | Supported | Supported | Supported |cell +| [mindspore.nn.Softmax](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Softmax) | Supported | Supported | Supported |layer/activation +| [mindspore.nn.LogSoftmax](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LogSoftmax) | Supported | Supported | Doing |layer/activation +| [mindspore.nn.ReLU](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ReLU) | Supported | Supported | Supported |layer/activation +| [mindspore.nn.ReLU6](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ReLU6) |Supported | Supported | Doing |layer/activation +| [mindspore.nn.HSwish](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.HSwish) | Doing | Supported | Doing |layer/activation +| [mindspore.nn.HSigmoid](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.HSigmoid) | Doing | Supported | Doing |layer/activation +| [mindspore.nn.LeakyReLU](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LeakyReLU) | Supported |Doing | Doing |layer/activation +| [mindspore.nn.Tanh](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Tanh) | Supported | Supported | Doing |layer/activation +| [mindspore.nn.GELU](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.GELU) | Supported | Supported | Doing |layer/activation +| [mindspore.nn.Sigmoid](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Sigmoid) | Supported |Supported | Doing |layer/activation +| [mindspore.nn.PReLU](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.PReLU) | Supported |Doing | Doing |layer/activation +| [mindspore.nn.Dropout](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Dropout) |Supported | Supported | Doing |layer/basic +| [mindspore.nn.Flatten](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Flatten) |Supported | Supported | Doing |layer/basic +| [mindspore.nn.Dense](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Dense) |Supported | Supported | Doing |layer/basic +| [mindspore.nn.DenseBnAct](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.DenseBnAct) |Supported | Supported | Supported |layer/basic +| [mindspore.nn.ClipByNorm](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ClipByNorm) |Doing | Supported | Doing |layer/basic +| [mindspore.nn.Norm](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Norm) |Doing | Supported | Doing |layer/basic +| [mindspore.nn.OneHot](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.OneHot) | Supported | Supported | Doing |layer/basic +| [mindspore.nn.Range](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Range) | Supported | Doing | Doing |layer/basic +| [mindspore.nn.SequentialCell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SequentialCell) |Doing | Supported | Doing |layer/container +| [mindspore.nn.CellList](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.CellList) | Supported | Supported | Doing |layer/container +| [mindspore.nn.Conv2d](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2d) | Supported | Supported | Supported |layer/conv +| [mindspore.nn.Conv2dTranspose](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2dTranspose) | Supported | Supported | Doing |layer/conv +| [mindspore.nn.Conv2dBnAct](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2dBnAct) | Supported | Supported | Supported |layer/conv +| [mindspore.nn.Embedding](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Embedding) |Doing | Supported | Doing |layer/embedding +| [mindspore.nn.ImageGradients](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ImageGradients) | Doing |Doing | Doing |layer/image +| [mindspore.nn.SSIM](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SSIM) | Doing |Doing | Doing |layer/image +| [mindspore.nn.PSNR](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.PSNR) | Doing |Doing | Doing |layer/image +| [mindspore.nn.CentralCrop](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.CentralCrop) | Supported |Doing | Doing |layer/image +| [mindspore.nn.LSTM](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LSTM) | Doing | Supported | Supported |layer/lstm +| [mindspore.nn.GlobalBatchNorm](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.GlobalBatchNorm) | Supported |Doing | Doing |layer/normalization +| [mindspore.nn.BatchNorm1d](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm1d) | Supported |Doing | Doing |layer/normalization +| [mindspore.nn.BatchNorm2d](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d) | Supported | Supported | Doing |layer/normalization +| [mindspore.nn.GroupNorm](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.GroupNorm) | Supported | Doing | Doing |layer/normalization +| [mindspore.nn.LayerNorm](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LayerNorm) | Supported | Supported | Doing |layer/normalization +| [mindspore.nn.MatrixDiag](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MatrixDiag) | Supported | Doing | Doing | layer/normalization +| [mindspore.nn.MatrixDiagPart](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MatrixDiagPart) | Supported | Doing | Doing | layer/normalization +| [mindspore.nn.MatrixSetDiag](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MatrixSetDiag) | Supported | Doing | Doing | layer/normalization +| [mindspore.nn.LinSpace](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LinSpace) | Supported | Doing | Doing | layer/normalization +| [mindspore.nn.MaxPool2d](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MaxPool2d) | Supported | Supported | Supported |layer/pooling +| [mindspore.nn.AvgPool2d](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.AvgPool2d) |Doing | Supported | Doing |layer/pooling +| [mindspore.nn.L1Loss](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.L1Loss) |Doing |Doing | Doing |loss/loss +| [mindspore.nn.MSELoss](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.MSELoss) | Supported |Doing | Doing |loss/loss +| [mindspore.nn.SmoothL1Loss](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SmoothL1Loss) | Supported |Doing | Doing |loss/loss +| [mindspore.nn.SoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SoftmaxCrossEntropyWithLogits) | Supported | Supported | Doing |loss/loss +| [mindspore.nn.SoftmaxCrossEntropyExpand](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SoftmaxCrossEntropyExpand) | Supported |Doing | Doing |loss/loss +| [mindspore.nn.ProximalAdagrad](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ProximalAdagrad) | Supported |Doing | Doing |optim/ProximalAdagrad +| [mindspore.nn.LazyAdam](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LazyAdam) | Supported |Doing | Doing |optim/lazyadam +| [mindspore.nn.Adam](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Adam) | Supported |Doing | Doing |optim/adam +| [mindspore.nn.AdamWeightDecay](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecay) | Supported | Supported | Doing |optim/adam +| [mindspore.nn.AdamWeightDecayDynamicLR](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecayDynamicLR) | Supported | Supported | Doing |optim/adam +| [mindspore.nn.Lamb](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Lamb) | Supported | Supported | Doing |optim/lamb +| [mindspore.nn.LARS](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.LARS) |Doing |Doing | Doing |optim/lars +| [mindspore.nn.Momentum](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Momentum) | Supported | Supported | Doing |optim/momentum +| [mindspore.nn.Optimizer](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Optimizer) | Supported | Supported | Doing |optim/optimizer +| [mindspore.nn.RMSProp](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.RMSProp) | Supported | Supported | Doing |optim/optimizer +| [mindspore.nn.SGD](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.SGD) |Supported |Doing | Doing |optim/sgd +| [mindspore.nn.WithLossCell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithLossCell) | Supported | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.WithGradCell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithGradCell) | Supported | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.TrainOneStepCell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.TrainOneStepCell) | Supported | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.DataWrapper](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.DataWrapper) |Doing | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.GetNextSingleOp](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.GetNextSingleOp) |Doing | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.WithEvalCell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.WithEvalCell) | Supported | Supported | Doing |wrap/cell_wrapper +| [mindspore.nn.ParameterUpdate](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.ParameterUpdate) | Supported |Doing | Doing |wrap/cell_wrapper +| [mindspore.nn.DistributedGradReducer](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.DistributedGradReducer) | Supported |Doing | Doing |wrap/grad_reducer +| [mindspore.nn.DynamicLossScaleUpdateCell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.DynamicLossScaleUpdateCell) | Doing |Doing | Doing |wrap/loss_scale +| [mindspore.nn.FixedLossScaleUpdateCell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.FixedLossScaleUpdateCell) | Doing |Doing | Doing |wrap/loss_scale +| [mindspore.nn.TrainOneStepWithLossScaleCell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.TrainOneStepWithLossScaleCell) | Doing |Doing | Doing |wrap/loss_scale +| [mindspore.nn.Cell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Cell) | Supported | Supported | Supported |cell ## mindspore.ops.operations | 操作名 | Ascend | GPU | CPU |算子类别 | :----------- |:------ |:------ |:-----|:--- -| [mindspore.ops.operations.Flatten](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Flatten) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.Softmax](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Softmax) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.Acosh](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Acosh) | Doing | Doing | Doing | nn_ops -| [mindspore.ops.operations.FloorMod](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloorMod) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.Elu](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Elu) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.MirrorPad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MirrorPad) | Doing | Doing | Doing | nn_ops -| [mindspore.ops.operations.Unpack](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Unpack) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.Pack](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pack) | Supported| Doing | Doing | nn_ops -| [mindspore.ops.operations.L2Loss](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.L2Loss) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.CTCLoss](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CTCLoss) | Doing | Doing | Doing | nn_ops -| [mindspore.ops.operations.LogSoftmax](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogSoftmax) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.Softplus](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Softplus) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.ReLU](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReLU) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.ReLU6](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReLU6) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.HSwish](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HSwish) | Doing | Supported |Doing | nn_ops -| [mindspore.ops.operations.HSigmoid](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HSigmoid) | Doing | Supported |Doing | nn_ops -| [mindspore.ops.operations.Sigmoid](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sigmoid) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.Tanh](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Tanh) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.BatchNorm](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchNorm) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.Conv2D](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Conv2D) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.DepthwiseConv2dNative](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNative) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.DepthwiseConv2dNativeBackpropInput](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNativeBackpropInput) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.DepthwiseConv2dNativeiBackpropFilter](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNativeBackpropFilter) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.MaxPoolWithArgmax](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MaxPoolWithArgmax) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.MaxPool](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MaxPool) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.AvgPool](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AvgPool) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.Conv2DBackpropInput](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Conv2DBackpropInput) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.BiasAdd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BiasAdd) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.TopK](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TopK) | Supported | Doing |Doing | nn_ops -| [mindspore.ops.operations.SoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SoftmaxCrossEntropyWithLogits) | Supported | Supported |Doing | nn_ops -| [mindspore.ops.operations.SparseSoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseSoftmaxCrossEntropyWithLogits) | Doing | Supported | Supported | nn_ops -| [mindspore.ops.operations.ApplyMomentum](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyMomentum) | Supported | Supported | Supported | nn_ops -| [mindspore.ops.operations.ApplyRMSProp](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyRMSProp) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.ApplyCenteredRMSProp](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyCenteredRMSProp) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.SparseApplyAdagrad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdagrad) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.SparseApplyProximalAdagrad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyProximalAdagrad) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.ApplyProximalAdagrad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyProximalAdagrad) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.SparseApplyLazyAdam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyLazyAdam) | Doing | Doing | Supported | nn_ops -| [mindspore.ops.operations.SparseApplyAdam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdam) | Doing | Doing | Supported | nn_ops -| [mindspore.ops.operations.SmoothL1Loss](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SmoothL1Loss) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.SGD](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SGD) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.LayerNorm](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LayerNorm) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.L2Normalize](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.L2Normalize) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.DropoutGenMask](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DropoutGenMask) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.DropoutDoMask](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DropoutDoMask) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.ResizeBilinear](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ResizeBilinear) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.OneHot](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.OneHot) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.Gelu](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Gelu) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.GetNext](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GetNext) | Supported | Supported | Doing | nn_ops -| [mindspore.ops.operations.PReLU](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.PReLU) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.LSTM](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LSTM) | Doing | Supported | Supported | nn_ops -| [mindspore.ops.operations.BasicLSTMCell](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BasicLSTMCell) | Doing | Doing | Doing | nn_ops -| [mindspore.ops.operations.SigmoidCrossEntropyWithLogits](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SigmoidCrossEntropyWithLogits) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.Pad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pad) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.ROIAlign](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ROIAlign) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.Adam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Adam) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.BinaryCrossEntropy](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BinaryCrossEntropy) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.LARSUpdate](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LARSUpdate) | Supported | Doing | Doing | nn_ops -| [mindspore.ops.operations.TensorAdd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TensorAdd) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.AssignAdd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AssignAdd) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.AssignSub](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AssignSub) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ReduceMean](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMean) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.ReduceSum](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceSum) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.ReduceAll](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceAll) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ReduceMax](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMax) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.ReduceMin](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMin) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ReduceProd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceProd) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.CumProd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CumProd) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.MatMul](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MatMul) | Supported | Supported | Supported | math_ops -| [mindspore.ops.operations.BatchMatMul](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchMatMul) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.CumSum](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CumSum) | Supported | Doing| Doing | math_ops -| [mindspore.ops.operations.AddN](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AddN) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Neg](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Neg) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Sub](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sub) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Mul](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Mul) | Supported | Supported | Supported | math_ops -| [mindspore.ops.operations.Square](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Square) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.SquareSumAll](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SquareSumAll) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Rsqrt](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Rsqrt) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Sqrt](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sqrt) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Reciprocal](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Reciprocal) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Pow](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pow) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Exp](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Exp) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Log](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Log) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Log1p](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Log1p) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Minimum](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Minimum) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Maximum](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Maximum) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.RealDiv](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.RealDiv) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Div](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Div) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.DivNoNan](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DivNoNan) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.FloorDiv](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloorDiv) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Floor](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Floor) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Equal](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Equal) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.EqualCount](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.EqualCount) | Doing | Supported | Supported | math_ops -| [mindspore.ops.operations.NotEqual](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NotEqual) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Greater](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Greater) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.GreaterEqual](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GreaterEqual) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Less](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Less) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.Atan2](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Atan2) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.LessEqual](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LessEqual) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.LogicalNot](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalNot) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.LogicalAnd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalAnd) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.LogicalOr](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalOr) | Supported | Supported | Doing | math_ops -| [mindspore.ops.operations.BitwiseAnd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseAnd) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.BitwiseOr](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseOr) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.BitwiseXor](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseXor) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Ceil](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Ceil) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Inv](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Inv) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Invert](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Invert) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.NPUAllocFloatStatus](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUAllocFloatStatus) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.NPUGetFloatStatus](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUGetFloatStatus) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.NPUClearFloatStatus](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUClearFloatStatus) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.FloatStatus](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloatStatus) | Doing | Supported | Doing | math_ops -| [mindspore.ops.operations.Cos](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cos) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Cosh](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cosh) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ACos](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ACos) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.BesselI0e](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BesselI0e) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.BesselI1e](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BesselI1e) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Asin](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Asin) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Asinh](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Asinh) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Erf](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Erf) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Erfc](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Erfc) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Sin](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sin) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Sinh](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sinh) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Expm1](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Expm1) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.NMSWithMask](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NMSWithMask) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Abs](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Abs) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Sign](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sign) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.Round](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Round) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ApproximateEqual](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApproximateEqual) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.InplaceAdd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceAdd) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.InplaceSub](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceSub) | Supported | Doing | Doing | math_ops -| [mindspore.ops.operations.ExpandDims](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ExpandDims) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.DType](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DType) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.SameTypeShape](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SameTypeShape) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Cast](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cast) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.IsSubClass](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsSubClass) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.IsInstance](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsInstance) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Reshape](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Reshape) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Shape](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Shape) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Squeeze](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Squeeze) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Transpose](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Transpose) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.GatherV2](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherV2) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Split](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Split) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.Rank](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Rank) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.TruncatedNormal](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TruncatedNormal) | Doing | Doing | Doing | array_ops -| [mindspore.ops.operations.Size](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Size) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Fill](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Fill) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.OnesLike](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.OnesLike) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ZerosLike](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ZerosLike) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.TupleToArray](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TupleToArray) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.ScalarToArray](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarToArray) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.ScalarToTensor](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarToTensor) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.InvertPermutation](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InvertPermutation) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.Argmax](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Argmax) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.Argmin](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Argmin) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ArgMaxWithValue](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ArgMaxWithValue) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.ArgMinWithValue](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ArgMinWithValue) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.Tile](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Tile) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.UnsortedSegmentSum](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.UnsortedSegmentSum) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.UnsortedSegmentMin](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.UnsortedSegmentMin) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.Concat](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Concat) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Slice](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Slice) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Select](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Select) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.StridedSlice](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.StridedSlice) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.Diag](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Diag) | Doing | Doing | Doing | array_ops -| [mindspore.ops.operations.DiagPart](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DiagPart) | Doing | Doing | Doing | array_ops -| [mindspore.ops.operations.Eye](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Eye) | Supported | Supported | Supported | array_ops -| [mindspore.ops.operations.ScatterNd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNd) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ResizeNearestNeighbor](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ResizeNearestNeighbor) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.GatherNd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherNd) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ApplyFtrl](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyFtrl) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.SparseApplyFtrl](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyFtrl) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ScatterNdUpdate](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNdUpdate) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ScatterUpdate](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterUpdate) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.SpaceToDepth](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToDepth) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.DepthToSpace](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthToSpace) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.SpaceToBatch](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToBatch) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.SpaceToBatchND](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToBatchND) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.BatchToSpace](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchToSpace) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.BatchToSpaceND](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchToSpaceND) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.IsFinite](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsFinite) | Supported | Supported | Doing | array_ops -| [mindspore.ops.operations.InplaceUpdate](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceUpdate) | Supported | Doing | Doing | array_ops -| [mindspore.ops.operations.ReduceOp](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceOp) | Supported | Supported | Doing | comm_ops -| [mindspore.ops.operations.AllReduce](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AllReduce) | Supported | Supported | Doing | comm_ops -| [mindspore.ops.operations.AllGather](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AllGather) | Supported | Supported | Doing | comm_ops -| [mindspore.ops.operations.HostAllGather](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HostAllGather) | Doing | Doing | Supported | comm_ops -| [mindspore.ops.operations.ReduceScatter](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceScatter) | Doing | Supported | Doing | comm_ops -| [mindspore.ops.operations.HostReduceScatter](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HostReduceScatter) | Doing | Doing | Supported | comm_ops -| [mindspore.ops.operations.Broadcast](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Broadcast) | Supported | Doing | Doing | comm_ops -| [mindspore.ops.operations.ControlDepend](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ControlDepend) | Supported | Supported | Supported | control_ops -| [mindspore.ops.operations.GeSwitch](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GeSwitch) | Doing | Doing | Doing | control_ops -| [mindspore.ops.operations.Merge](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Merge) | Doing | Doing | Doing | control_ops -| [mindspore.ops.operations.ScalarSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarSummary) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.ImageSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ImageSummary) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.TensorSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TensorSummary) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.HistogramSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HistogramSummary) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.InsertGradientOf](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InsertGradientOf) | Supported | Supported | Supported | debug_ops -| [mindspore.ops.operations.Print](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Print) | Supported | Doing | Doing | debug_ops -| [mindspore.ops.operations.Assign](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Assign) | Supported | Supported | Doing | other_ops -| [mindspore.ops.operations.BoundingBoxEncode](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BoundingBoxEncode) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.BoundingBoxDecode](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BoundingBoxDecode) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.CheckValid](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CheckValid) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.IOU](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IOU) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.MakeRefKey](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MakeRefKey) | Supported | Supported | Supported | other_ops -| [mindspore.ops.operations.InTopK](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InTopK) | Supported | Doing | Doing | other_ops -| [mindspore.ops.operations.RandomChoiceWithMask](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.RandomChoiceWithMask) | Doing| Doing | Doing | random_ops -| [mindspore.ops.operations.ScalarCast](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarCast) | Supported | Supported | Supported | inner_ops +| [mindspore.ops.operations.Flatten](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Flatten) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.Softmax](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Softmax) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.Acosh](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Acosh) | Doing | Doing | Doing | nn_ops +| [mindspore.ops.operations.FloorMod](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloorMod) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.Elu](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Elu) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.MirrorPad](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MirrorPad) | Doing | Doing | Doing | nn_ops +| [mindspore.ops.operations.Unpack](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Unpack) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.Pack](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pack) | Supported| Doing | Doing | nn_ops +| [mindspore.ops.operations.L2Loss](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.L2Loss) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.CTCLoss](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CTCLoss) | Doing | Doing | Doing | nn_ops +| [mindspore.ops.operations.LogSoftmax](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogSoftmax) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.Softplus](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Softplus) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.ReLU](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReLU) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.ReLU6](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReLU6) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.HSwish](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HSwish) | Doing | Supported |Doing | nn_ops +| [mindspore.ops.operations.HSigmoid](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HSigmoid) | Doing | Supported |Doing | nn_ops +| [mindspore.ops.operations.Sigmoid](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sigmoid) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.Tanh](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Tanh) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.BatchNorm](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchNorm) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.LRN](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LRN) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.Conv2D](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Conv2D) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.DepthwiseConv2dNative](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNative) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.DepthwiseConv2dNativeBackpropInput](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNativeBackpropInput) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.DepthwiseConv2dNativeiBackpropFilter](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthwiseConv2dNativeBackpropFilter) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.MaxPoolWithArgmax](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MaxPoolWithArgmax) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.MaxPool](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MaxPool) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.AvgPool](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AvgPool) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.Conv2DBackpropInput](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Conv2DBackpropInput) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.BiasAdd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BiasAdd) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.TopK](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TopK) | Supported | Doing |Doing | nn_ops +| [mindspore.ops.operations.SoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SoftmaxCrossEntropyWithLogits) | Supported | Supported |Doing | nn_ops +| [mindspore.ops.operations.SparseSoftmaxCrossEntropyWithLogits](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseSoftmaxCrossEntropyWithLogits) | Doing | Supported | Supported | nn_ops +| [mindspore.ops.operations.ApplyMomentum](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyMomentum) | Supported | Supported | Supported | nn_ops +| [mindspore.ops.operations.ApplyRMSProp](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyRMSProp) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.ApplyCenteredRMSProp](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyCenteredRMSProp) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.SparseApplyAdagrad](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdagrad) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.SparseApplyProximalAdagrad](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyProximalAdagrad) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.ApplyProximalAdagrad](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyProximalAdagrad) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.SparseApplyLazyAdam](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyLazyAdam) | Doing | Doing | Supported | nn_ops +| [mindspore.ops.operations.SparseApplyAdam](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdam) | Doing | Doing | Supported | nn_ops +| [mindspore.ops.operations.SmoothL1Loss](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SmoothL1Loss) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.SGD](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SGD) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.LayerNorm](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LayerNorm) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.L2Normalize](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.L2Normalize) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.DropoutGenMask](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DropoutGenMask) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.DropoutDoMask](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DropoutDoMask) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.ResizeBilinear](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ResizeBilinear) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.OneHot](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.OneHot) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.Gelu](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Gelu) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.GetNext](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GetNext) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.PReLU](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.PReLU) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.LSTM](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LSTM) | Doing | Supported | Supported | nn_ops +| [mindspore.ops.operations.BasicLSTMCell](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BasicLSTMCell) | Doing | Doing | Doing | nn_ops +| [mindspore.ops.operations.SigmoidCrossEntropyWithLogits](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SigmoidCrossEntropyWithLogits) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.Pad](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pad) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.ROIAlign](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ROIAlign) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.Adam](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Adam) | Supported | Supported | Doing | nn_ops +| [mindspore.ops.operations.BinaryCrossEntropy](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BinaryCrossEntropy) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.LARSUpdate](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LARSUpdate) | Supported | Doing | Doing | nn_ops +| [mindspore.ops.operations.TensorAdd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TensorAdd) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.AssignAdd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AssignAdd) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.AssignSub](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AssignSub) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ReduceMean](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMean) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.ReduceSum](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceSum) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.ReduceAll](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceAll) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ReduceMax](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMax) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.ReduceMin](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceMin) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ReduceProd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceProd) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.CumProd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CumProd) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.MatMul](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MatMul) | Supported | Supported | Supported | math_ops +| [mindspore.ops.operations.BatchMatMul](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchMatMul) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.CumSum](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CumSum) | Supported | Doing| Doing | math_ops +| [mindspore.ops.operations.AddN](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AddN) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Neg](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Neg) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Sub](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sub) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Mul](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Mul) | Supported | Supported | Supported | math_ops +| [mindspore.ops.operations.Square](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Square) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.SquareSumAll](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SquareSumAll) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Rsqrt](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Rsqrt) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Sqrt](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sqrt) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Reciprocal](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Reciprocal) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Pow](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Pow) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Exp](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Exp) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Log](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Log) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Log1p](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Log1p) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Minimum](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Minimum) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Maximum](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Maximum) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.RealDiv](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.RealDiv) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Div](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Div) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.DivNoNan](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DivNoNan) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.FloorDiv](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloorDiv) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Floor](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Floor) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Equal](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Equal) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.EqualCount](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.EqualCount) | Doing | Supported | Supported | math_ops +| [mindspore.ops.operations.NotEqual](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NotEqual) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Greater](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Greater) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.GreaterEqual](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GreaterEqual) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Less](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Less) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.Atan2](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Atan2) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.LessEqual](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LessEqual) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.LogicalNot](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalNot) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.LogicalAnd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalAnd) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.LogicalOr](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LogicalOr) | Supported | Supported | Doing | math_ops +| [mindspore.ops.operations.BitwiseAnd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseAnd) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.BitwiseOr](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseOr) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.BitwiseXor](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BitwiseXor) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Ceil](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Ceil) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Inv](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Inv) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Invert](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Invert) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.NPUAllocFloatStatus](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUAllocFloatStatus) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.NPUGetFloatStatus](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUGetFloatStatus) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.NPUClearFloatStatus](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NPUClearFloatStatus) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.FloatStatus](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FloatStatus) | Doing | Supported | Doing | math_ops +| [mindspore.ops.operations.Cos](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cos) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Cosh](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cosh) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ACos](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ACos) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.BesselI0e](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BesselI0e) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.BesselI1e](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BesselI1e) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Asin](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Asin) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Asinh](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Asinh) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Erf](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Erf) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Erfc](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Erfc) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Sin](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sin) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Sinh](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sinh) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Expm1](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Expm1) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.NMSWithMask](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.NMSWithMask) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Abs](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Abs) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Sign](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Sign) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.Round](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Round) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ApproximateEqual](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApproximateEqual) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.InplaceAdd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceAdd) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.InplaceSub](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceSub) | Supported | Doing | Doing | math_ops +| [mindspore.ops.operations.ExpandDims](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ExpandDims) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.DType](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DType) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.SameTypeShape](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SameTypeShape) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Cast](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Cast) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.IsSubClass](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsSubClass) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.IsInstance](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsInstance) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Reshape](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Reshape) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Shape](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Shape) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Squeeze](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Squeeze) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Transpose](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Transpose) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.GatherV2](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherV2) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Split](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Split) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.Rank](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Rank) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.TruncatedNormal](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TruncatedNormal) | Doing | Doing | Doing | array_ops +| [mindspore.ops.operations.Size](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Size) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Fill](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Fill) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.OnesLike](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.OnesLike) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ZerosLike](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ZerosLike) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.TupleToArray](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TupleToArray) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.ScalarToArray](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarToArray) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.ScalarToTensor](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarToTensor) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.InvertPermutation](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InvertPermutation) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.Argmax](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Argmax) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.Argmin](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Argmin) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ArgMaxWithValue](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ArgMaxWithValue) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.ArgMinWithValue](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ArgMinWithValue) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.Tile](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Tile) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.UnsortedSegmentSum](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.UnsortedSegmentSum) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.UnsortedSegmentMin](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.UnsortedSegmentMin) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.Concat](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Concat) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Slice](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Slice) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Select](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Select) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.StridedSlice](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.StridedSlice) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.Diag](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Diag) | Doing | Doing | Doing | array_ops +| [mindspore.ops.operations.DiagPart](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DiagPart) | Doing | Doing | Doing | array_ops +| [mindspore.ops.operations.Eye](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Eye) | Supported | Supported | Supported | array_ops +| [mindspore.ops.operations.ScatterNd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNd) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ResizeNearestNeighbor](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ResizeNearestNeighbor) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.GatherNd](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherNd) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ApplyFtrl](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyFtrl) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.SparseApplyFtrl](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyFtrl) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ScatterNdUpdate](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNdUpdate) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ScatterUpdate](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterUpdate) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.SpaceToDepth](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToDepth) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.DepthToSpace](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.DepthToSpace) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.SpaceToBatch](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToBatch) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.SpaceToBatchND](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SpaceToBatchND) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.BatchToSpace](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchToSpace) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.BatchToSpaceND](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BatchToSpaceND) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.IsFinite](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IsFinite) | Supported | Supported | Doing | array_ops +| [mindspore.ops.operations.InplaceUpdate](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InplaceUpdate) | Supported | Doing | Doing | array_ops +| [mindspore.ops.operations.ReduceOp](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceOp) | Supported | Supported | Doing | comm_ops +| [mindspore.ops.operations.AllReduce](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AllReduce) | Supported | Supported | Doing | comm_ops +| [mindspore.ops.operations.AllGather](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.AllGather) | Supported | Supported | Doing | comm_ops +| [mindspore.ops.operations.HostAllGather](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HostAllGather) | Doing | Doing | Supported | comm_ops +| [mindspore.ops.operations.ReduceScatter](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ReduceScatter) | Doing | Supported | Doing | comm_ops +| [mindspore.ops.operations.HostReduceScatter](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HostReduceScatter) | Doing | Doing | Supported | comm_ops +| [mindspore.ops.operations.Broadcast](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Broadcast) | Supported | Doing | Doing | comm_ops +| [mindspore.ops.operations.ControlDepend](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ControlDepend) | Supported | Supported | Supported | control_ops +| [mindspore.ops.operations.GeSwitch](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GeSwitch) | Doing | Doing | Doing | control_ops +| [mindspore.ops.operations.Merge](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Merge) | Doing | Doing | Doing | control_ops +| [mindspore.ops.operations.ScalarSummary](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarSummary) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.ImageSummary](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ImageSummary) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.TensorSummary](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.TensorSummary) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.HistogramSummary](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.HistogramSummary) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.InsertGradientOf](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InsertGradientOf) | Supported | Supported | Supported | debug_ops +| [mindspore.ops.operations.Print](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Print) | Supported | Doing | Doing | debug_ops +| [mindspore.ops.operations.Assign](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.Assign) | Supported | Supported | Doing | other_ops +| [mindspore.ops.operations.BoundingBoxEncode](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BoundingBoxEncode) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.BoundingBoxDecode](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.BoundingBoxDecode) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.CheckValid](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.CheckValid) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.IOU](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.IOU) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.MakeRefKey](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.MakeRefKey) | Supported | Supported | Supported | other_ops +| [mindspore.ops.operations.InTopK](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.InTopK) | Supported | Doing | Doing | other_ops +| [mindspore.ops.operations.RandomChoiceWithMask](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.RandomChoiceWithMask) | Doing| Doing | Doing | random_ops +| [mindspore.ops.operations.ScalarCast](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScalarCast) | Supported | Supported | Supported | inner_ops diff --git a/docs/source_zh_cn/roadmap.md b/docs/source_zh_cn/roadmap.md index 528182d2e3ec4224b3f07df433ea93609e939016..772a7092a1c27d4ef89dba0b26304a57eed36c17 100644 --- a/docs/source_zh_cn/roadmap.md +++ b/docs/source_zh_cn/roadmap.md @@ -23,7 +23,7 @@ - + ## 预置模型 * CV：目标检测、GAN、图像分割、姿态识别等场景经典模型。 diff --git a/docs/source_zh_cn/technical_white_paper.md b/docs/source_zh_cn/technical_white_paper.md new file mode 100644 index 0000000000000000000000000000000000000000..4e34a3725679ae2105bea39aa768378483720ce1 --- /dev/null +++ b/docs/source_zh_cn/technical_white_paper.md @@ -0,0 +1,11 @@ +# 技术白皮书 + +## 引言 +深度学习研究和应用在近几十年得到了爆炸式的发展，掀起了人工智能的第三次浪潮，并且在图像识别、语音识别与合成、无人驾驶、机器视觉等方面取得了巨大的成功。这也对算法的应用以及依赖的框架有了更高级的要求。深度学习框架的不断发展使得在大型数据集上训练神经网络模型时，可以方便地使用大量的计算资源。 + +深度学习是使用多层结构从原始数据中自动学习并提取高层次特征的一类机器学习算法。通常，从原始数据中提取高层次、抽象的特征是非常困难的。目前有两种主流的深度学习框架：一种是在执行之前构造一个静态图，定义所有操作和网络结构，典型代表是TensorFlow，这种方法以牺牲易用性为代价，来提高训练期间的性能；另一种是立即执行的动态图计算，典型代表是PyTorch。通过比较可以发现，动态图更灵活、更易调试，但会牺牲性能。因此，现有深度学习框架难以同时满足易开发、高效执行的要求。 + +## 简介 +MindSpore作为新一代深度学习框架，是源于全产业的最佳实践，最佳匹配昇腾处理器算力，支持终端、边缘、云全场景灵活部署，开创全新的AI编程范式，降低AI开发门槛。MindSpore是一种全新的深度学习计算框架，旨在实现易开发、高效执行、全场景覆盖三大目标。为了实现易开发的目标，MindSpore采用基于源码转换（Source Code Transformation，SCT）的自动微分（Automatic Differentiation，AD）机制，该机制可以用控制流表示复杂的组合。函数被转换成函数中间表达（Intermediate Representation，IR），中间表达构造出一个能够在不同设备上解析和执行的计算图。在执行前，计算图上应用了多种软硬件协同优化技术，以提升端、边、云等不同场景下的性能和效率。MindSpore支持动态图，更易于检查运行模式。由于采用了基于源码转换的自动微分机制，所以动态图和静态图之间的模式切换非常简单。为了在大型数据集上有效训练大模型，通过高级手动配置策略，MindSpore可以支持数据并行、模型并行和混合并行训练，具有很强的灵活性。此外，MindSpore还有“自动并行”能力，它通过在庞大的策略空间中进行高效搜索来找到一种快速的并行策略。MindSpore框架的具体优势，请查看详细介绍。 + +[查看技术白皮书](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com:443/%E7%99%BD%E7%9A%AE%E4%B9%A6/MindSpore%EF%BC%9A%E4%B8%80%E7%A7%8D%E5%85%A8%E5%9C%BA%E6%99%AF%E8%A6%86%E7%9B%96%E7%9A%84%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%AE%A1%E7%AE%97%E6%A1%86%E6%9E%B6.pdf) \ No newline at end of file diff --git a/install/mindspore_cpu_install.md b/install/mindspore_cpu_install.md index ba4c8e689aac880bca8386d7e95ebcaa659d078a..f2bd38f3978a3ce9b3c775072311bcf1b06fe7c9 100644 --- a/install/mindspore_cpu_install.md +++ b/install/mindspore_cpu_install.md @@ -21,9 +21,9 @@ | 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | | ---- | :--- | :--- | :--- | -| MindSpore master | Ubuntu 16.04（及以上） x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindspore/blob/master/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
**安装依赖：**
与可执行文件安装依赖相同 | +| MindSpore 0.5.0-beta | Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindspore/blob/r0.5/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
**安装依赖：**
与可执行文件安装依赖相同 | -- Ubuntu版本为18.04时，GCC 7.3.0可以直接通过apt命令安装。 +- GCC 7.3.0可以直接通过apt命令安装。 - 在联网状态下，安装whl包时会自动下载`requirements.txt`中的依赖项，其余情况需自行安装。 ### Conda安装（可选） @@ -62,17 +62,17 @@ 1. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 2. 在源码根目录下执行如下命令编译MindSpore。 ```bash - bash build.sh -e cpu -z -j4 + bash build.sh -e cpu -j4 ``` > - 在执行上述命令前，需保证可执行文件cmake和patch所在路径已加入环境变量PATH中。 > - `build.sh`中会执行`git clone`获取第三方依赖库的代码，请提前确保git的网络设置正确可用。 - > - 如果编译机性能较好，可在执行中增加-j{线程数}来增加线程数量。如`bash build.sh -e cpu -z -j12`。 + > - 如果编译机性能较好，可在执行中增加-j{线程数}来增加线程数量。如`bash build.sh -e cpu -j12`。 3. 执行如下命令安装MindSpore。 @@ -97,7 +97,7 @@ | 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | | ---------------------- | :------------------ | :----------------------------------------------------------- | :----------------------- | -| MindArmour master | Ubuntu 16.04（及以上） x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- 其他依赖项参见[setup.py](https://gitee.com/mindspore/mindarmour/blob/master/setup.py) | 与可执行文件安装依赖相同 | +| MindArmour 0.5.0-beta | Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- 其他依赖项参见[setup.py](https://gitee.com/mindspore/mindarmour/blob/r0.5/setup.py) | 与可执行文件安装依赖相同 | - 在联网状态下，安装whl包时会自动下载`setup.py`中的依赖项，其余情况需自行安装。 @@ -122,7 +122,7 @@ 1. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindarmour.git + git clone https://gitee.com/mindspore/mindarmour.git -b r0.5 ``` 2. 在源码根目录下，执行如下命令编译并安装MindArmour。 diff --git a/install/mindspore_cpu_install_en.md b/install/mindspore_cpu_install_en.md index 6170254ae2690bce51e2fe5db63bbe4497088440..a2af197e52b5010bad45b6b1cc6d63ef49999366 100644 --- a/install/mindspore_cpu_install_en.md +++ b/install/mindspore_cpu_install_en.md @@ -21,9 +21,9 @@ This document describes how to quickly install MindSpore on a Ubuntu system with | Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | | ---- | :--- | :--- | :--- | -| MindSpore master | Ubuntu 16.04 or later x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindspore/blob/master/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
same as the executable file installation dependencies. | +| MindSpore 0.5.0-beta | Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindspore/blob/r0.5/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
same as the executable file installation dependencies. | -- When Ubuntu version is 18.04, GCC 7.3.0 can be installed by using apt command. +- GCC 7.3.0 can be installed by using apt command. - When the network is connected, dependency items in the `requirements.txt` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items. ### (Optional) Installing Conda @@ -62,17 +62,17 @@ This document describes how to quickly install MindSpore on a Ubuntu system with 1. Download the source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 2. Run the following command in the root directory of the source code to compile MindSpore: ```bash - bash build.sh -e cpu -z -j4 + bash build.sh -e cpu -j4 ``` > - Before running the preceding command, ensure that the paths where the executable files cmake and patch store have been added to the environment variable PATH. > - In the `build.sh` script, the `git clone` command will be executed to obtain the code in the third-party dependency database. Ensure that the network settings of Git are correct. - > - If the compiler performance is strong, you can add -j{Number of threads} in to script to increase the number of threads. For example, `bash build.sh -e cpu -z -j12`. + > - If the compiler performance is strong, you can add -j{Number of threads} in to script to increase the number of threads. For example, `bash build.sh -e cpu -j12`. 3. Run the following command to install MindSpore: @@ -97,7 +97,7 @@ If you need to conduct AI model security research or enhance the security of the | Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | | ---- | :--- | :--- | :--- | -| MindArmour master | Ubuntu 16.04 or later x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- For details about other dependency items, see [setup.py](https://gitee.com/mindspore/mindarmour/blob/master/setup.py). | Same as the executable file installation dependencies. | +| MindArmour 0.5.0-beta | Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- For details about other dependency items, see [setup.py](https://gitee.com/mindspore/mindarmour/blob/r0.5/setup.py). | Same as the executable file installation dependencies. | - When the network is connected, dependency items in the `setup.py` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items. @@ -122,7 +122,7 @@ If you need to conduct AI model security research or enhance the security of the 1. Download the source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindarmour.git + git clone https://gitee.com/mindspore/mindarmour.git -b r0.5 ``` 2. Run the following command in the root directory of the source code to compile and install MindArmour: diff --git a/install/mindspore_cpu_win_install.md b/install/mindspore_cpu_win_install.md index 5b33ddb3c8567ac9182297fb91ba4b93f3173497..d851edeb3b90a62ad2ccdc01b836b0afe3c0c5b5 100644 --- a/install/mindspore_cpu_win_install.md +++ b/install/mindspore_cpu_win_install.md @@ -4,7 +4,7 @@ -- [Windows系统安装MindSpore](#windows系统安装mindspore) +- [安装MindSpore](#安装mindspore) - [环境要求](#环境要求) - [系统要求和软件依赖](#系统要求和软件依赖) - [Conda安装（可选）](#conda安装可选) @@ -20,7 +20,7 @@ | 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | | ---- | :--- | :--- | :--- | -| MindSpore master | Windows 10 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindspore/blob/master/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [MinGW-W64 GCC-7.3.0](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/7.3.0/threads-posix/seh/x86_64-7.3.0-release-posix-seh-rt_v5-rev0.7z) x86_64-posix-seh
- [ActivePerl](http://downloads.activestate.com/ActivePerl/releases/5.24.3.2404/ActivePerl-5.24.3.2404-MSWin32-x64-404865.exe) 5.24.3.2404
- [CMake](https://cmake.org/download/) 3.14.1
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
**安装依赖：**
与可执行文件安装依赖相同 | +| MindSpore 0.5.0-beta | Windows 10 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindspore/blob/r0.5/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [MinGW-W64 GCC-7.3.0](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/7.3.0/threads-posix/seh/x86_64-7.3.0-release-posix-seh-rt_v5-rev0.7z) x86_64-posix-seh
- [ActivePerl](http://downloads.activestate.com/ActivePerl/releases/5.24.3.2404/ActivePerl-5.24.3.2404-MSWin32-x64-404865.exe) 5.24.3.2404
- [CMake](https://cmake.org/download/) 3.14.1
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
**安装依赖：**
与可执行文件安装依赖相同 | - 在联网状态下，安装whl包时会自动下载`requirements.txt`中的依赖项，其余情况需自行安装。 @@ -62,7 +62,7 @@ 1. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 2. 在源码根目录下执行如下命令编译MindSpore。 diff --git a/install/mindspore_cpu_win_install_en.md b/install/mindspore_cpu_win_install_en.md index 2f1fe17739a318d9ebbc63c5d79ea46409ec662d..1800a6dfe948145256bb4dbcd9a515bed592798d 100644 --- a/install/mindspore_cpu_win_install_en.md +++ b/install/mindspore_cpu_win_install_en.md @@ -4,7 +4,7 @@ This document describes how to quickly install MindSpore on a Windows system wit -- [MindSpore Installation Guide on Windows](#mindspore-installation-guide-on-windows) +- [MindSpore Installation Guide](#mindspore-installation-guide) - [Environment Requirements](#environment-requirements) - [System Requirements and Software Dependencies](#system-requirements-and-software-dependencies) - [(Optional) Installing Conda](#optional-installing-conda) @@ -20,7 +20,7 @@ This document describes how to quickly install MindSpore on a Windows system wit | Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | | ---- | :--- | :--- | :--- | -| MindSpore master | Windows 10 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindspore/blob/master/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [MinGW-W64 GCC-7.3.0](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/7.3.0/threads-posix/seh/x86_64-7.3.0-release-posix-seh-rt_v5-rev0.7z) x86_64-posix-seh
- [ActivePerl](http://downloads.activestate.com/ActivePerl/releases/5.24.3.2404/ActivePerl-5.24.3.2404-MSWin32-x64-404865.exe) 5.24.3.2404
- [CMake](https://cmake.org/download/) 3.14.1
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
**Installation dependencies:**
same as the executable file installation dependencies. | +| MindSpore 0.5.0-beta | Windows 10 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindspore/blob/r0.5/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [MinGW-W64 GCC-7.3.0](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/7.3.0/threads-posix/seh/x86_64-7.3.0-release-posix-seh-rt_v5-rev0.7z) x86_64-posix-seh
- [ActivePerl](http://downloads.activestate.com/ActivePerl/releases/5.24.3.2404/ActivePerl-5.24.3.2404-MSWin32-x64-404865.exe) 5.24.3.2404
- [CMake](https://cmake.org/download/) 3.14.1
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
**Installation dependencies:**
same as the executable file installation dependencies. | - When the network is connected, dependency items in the `requirements.txt` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items. @@ -62,7 +62,7 @@ This document describes how to quickly install MindSpore on a Windows system wit 1. Download the source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 2. Run the following command in the root directory of the source code to compile MindSpore: diff --git a/install/mindspore_d_install.md b/install/mindspore_d_install.md index ddd4c1c9b88ba603323c552997a4bceb586a33d4..602339400d18875e21cc601fa38fae97256d5917 100644 --- a/install/mindspore_d_install.md +++ b/install/mindspore_d_install.md @@ -33,10 +33,10 @@ | 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | | ---- | :--- | :--- | :--- | -| MindSpore master | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- Ascend 910 AI处理器配套软件包（对应版本Atlas Data Center Solution V100R020C00T100）
- [gmp](https://gmplib.org/download/gmp/) 6.1.2
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindspore/blob/master/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- Ascend 910 AI处理器配套软件包（对应版本Atlas Data Center Solution V100R020C00T100）
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
- [gmp](https://gmplib.org/download/gmp/) 6.1.2
**安装依赖：**
与可执行文件安装依赖相同 | +| MindSpore 0.5.0-beta | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- Ascend 910 AI处理器配套软件包（对应版本Atlas Data Center Solution V100R020C00）
- [gmp](https://gmplib.org/download/gmp/) 6.1.2
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindspore/blob/r0.5/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- Ascend 910 AI处理器配套软件包（对应版本Atlas Data Center Solution V100R020C00）
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
- [gmp](https://gmplib.org/download/gmp/) 6.1.2
**安装依赖：**
与可执行文件安装依赖相同 | -- 确认当前用户有权限访问Ascend 910 AI处理器配套软件包（对应版本Atlas Data Center Solution V100R020C00T100）的安装路径`/usr/local/Ascend`，若无权限，需要root用户将当前用户添加到`/usr/local/Ascend`所在的用户组，具体配置请详见配套软件包的说明文档。 -- Ubuntu版本为18.04时，GCC 7.3.0可以直接通过apt命令安装。 +- 确认当前用户有权限访问Ascend 910 AI处理器配套软件包（对应版本Atlas Data Center Solution V100R020C00）的安装路径`/usr/local/Ascend`，若无权限，需要root用户将当前用户添加到`/usr/local/Ascend`所在的用户组，具体配置请详见配套软件包的说明文档。 +- GCC 7.3.0可以直接通过apt命令安装。 - 在联网状态下，安装whl包时会自动下载`requirements.txt`中的依赖项，其余情况需自行安装。 ### Conda安装（可选） @@ -57,7 +57,7 @@ ### 配套软件包依赖配置 - - 安装Ascend 910 AI处理器配套软件包（对应版本Atlas Data Center Solution V100R020C00T100）提供的whl包，whl包随配套软件包发布，升级配套软件包之后需要重新安装。 + - 安装Ascend 910 AI处理器配套软件包（对应版本Atlas Data Center Solution V100R020C00）提供的whl包，whl包随配套软件包发布，升级配套软件包之后需要重新安装。 ```bash pip install /usr/local/Ascend/fwkacllib/lib64/topi-{version}-py3-none-any.whl @@ -82,17 +82,17 @@ 1. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 2. 在源码根目录下，执行如下命令编译MindSpore。 ```bash - bash build.sh -e d -z + bash build.sh -e ascend ``` > - 在执行上述命令前，需保证可执行文件`cmake`和`patch`所在路径已加入环境变量PATH中。 > - `build.sh`中会执行`git clone`获取第三方依赖库的代码，请提前确保git的网络设置正确可用。 - > - `build.sh`中默认的编译线程数为8，如果编译机性能较差可能会出现编译错误，可在执行中增加-j{线程数}来减少线程数量。如`bash build.sh -e d -z -j4`。 + > - `build.sh`中默认的编译线程数为8，如果编译机性能较差可能会出现编译错误，可在执行中增加-j{线程数}来减少线程数量。如`bash build.sh -e ascend -j4`。 3. 执行如下命令安装MindSpore。 @@ -103,19 +103,40 @@ ## 配置环境变量 -- 安装好MindSpore之后，需要导出Runtime相关环境变量。 +- EulerOS操作系统，安装好MindSpore之后，需要导出Runtime相关环境变量。 ```bash # control log level. 0-DEBUG, 1-INFO, 2-WARNING, 3-ERROR, default level is WARNING. export GLOG_v=2 + # Conda environmental options LOCAL_ASCEND=/usr/local/Ascend # the root directory of run package + # lib libraries that the run package depends on export LD_LIBRARY_PATH=${LOCAL_ASCEND}/add-ons/:${LOCAL_ASCEND}/fwkacllib/lib64:${LD_LIBRARY_PATH} + + # Environment variables that must be configured + export TBE_IMPL_PATH=${LOCAL_ASCEND}/opp/op_impl/built-in/ai_core/tbe # TBE operator implementation tool path + export PATH=${LOCAL_ASCEND}/fwkacllib/ccec_compiler/bin/:${PATH} # TBE operator compilation tool path + export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH} # Python library that TBE implementation depends on + ``` + +- Ubuntu操作系统，安装好MindSpore之后，需要导出Runtime相关环境变量，注意：需要将如下配置中{version}替换为环境上真实的版本号。 + + ```bash + # control log level. 0-DEBUG, 1-INFO, 2-WARNING, 3-ERROR, default level is WARNING. + export GLOG_v=2 + + # Conda environmental options + LOCAL_ASCEND=/usr/local/Ascend # the root directory of run package + + # lib libraries that the run package depends on + export LD_LIBRARY_PATH=${LOCAL_ASCEND}/add-ons/:${LOCAL_ASCEND}/ascend-toolkit/{version}/fwkacllib/lib64:${LOCAL_ASCEND}/driver/lib64:${LD_LIBRARY_PATH} + # Environment variables that must be configured - export TBE_IMPL_PATH=${LOCAL_ASCEND}/opp/op_impl/built-in/ai_core/tbe # TBE operator implementation tool path - export PATH=${LOCAL_ASCEND}/fwkacllib/ccec_compiler/bin/:${PATH} # TBE operator compilation tool path - export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH} # Python library that TBE implementation depends on + export TBE_IMPL_PATH=${LOCAL_ASCEND}/ascend-toolkit/{version}/opp/op_impl/built-in/ai_core/tbe # TBE operator implementation tool path + export PATH=${LOCAL_ASCEND}/ascend-toolkit/{version}/fwkacllib/ccec_compiler/bin/:${PATH} # TBE operator compilation tool path + export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH} # Python library that TBE implementation depends on ``` ## 安装验证 @@ -160,7 +181,7 @@ | 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | | ---- | :--- | :--- | :--- | -| MindInsight master | - Ubuntu 18.04 aarch64
- 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64
| - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindinsight/blob/master/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**安装依赖：**
与可执行文件安装依赖相同 | +| MindInsight 0.5.0-beta | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64
| - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindinsight/blob/r0.5/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**安装依赖：**
与可执行文件安装依赖相同 | - 在联网状态下，安装whl包时会自动下载`requirements.txt`中的依赖项，其余情况需自行安装。 @@ -185,7 +206,7 @@ 1. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindinsight.git + git clone https://gitee.com/mindspore/mindinsight.git -b r0.5 ``` > **不能**直接在仓库主页下载zip包获取源码。 @@ -225,7 +246,7 @@ | 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | | ---- | :--- | :--- | :--- | -| MindArmour master | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64
| - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- 其他依赖项参见[setup.py](https://gitee.com/mindspore/mindarmour/blob/master/setup.py) | 与可执行文件安装依赖相同 | +| MindArmour 0.5.0-beta | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64
| - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- 其他依赖项参见[setup.py](https://gitee.com/mindspore/mindarmour/blob/r0.5/setup.py) | 与可执行文件安装依赖相同 | - 在联网状态下，安装whl包时会自动下载`setup.py`中的依赖项，其余情况需自行安装。 @@ -250,7 +271,7 @@ 1. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindarmour.git + git clone https://gitee.com/mindspore/mindarmour.git -b r0.5 ``` 2. 在源码根目录下，执行如下命令编译并安装MindArmour。 diff --git a/install/mindspore_d_install_en.md b/install/mindspore_d_install_en.md index bc6292f557f8502fb7f001543f70e6e8acb2ec2a..1b34561f7db256ff17038a17d36bf748b27f1259 100644 --- a/install/mindspore_d_install_en.md +++ b/install/mindspore_d_install_en.md @@ -32,10 +32,10 @@ This document describes how to quickly install MindSpore on an Ascend AI process | Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | | ---- | :--- | :--- | :--- | -| MindSpore master | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- Ascend 910 AI processor software package(Version:Atlas Data Center Solution V100R020C00T100)
- [gmp](https://gmplib.org/download/gmp/) 6.1.2
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindspore/blob/master/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- Ascend 910 AI processor software package(Version:Atlas Data Center Solution V100R020C00T100)
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
- [gmp](https://gmplib.org/download/gmp/) 6.1.2
**Installation dependencies:**
same as the executable file installation dependencies. | +| MindSpore 0.5.0-beta | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- Ascend 910 AI processor software package(Version:Atlas Data Center Solution V100R020C00)
- [gmp](https://gmplib.org/download/gmp/) 6.1.2
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindspore/blob/r0.5/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- Ascend 910 AI processor software package(Version:Atlas Data Center Solution V100R020C00)
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
- [gmp](https://gmplib.org/download/gmp/) 6.1.2
**Installation dependencies:**
same as the executable file installation dependencies. | -- Confirm that the current user has the right to access the installation path `/usr/local/Ascend `of Ascend 910 AI processor software package(Version:Atlas Data Center Solution V100R020C00T100). If not, the root user needs to add the current user to the user group where `/usr/local/Ascend` is located. For the specific configuration, please refer to the software package instruction document. -- When Ubuntu version is 18.04, GCC 7.3.0 can be installed by using apt command. +- Confirm that the current user has the right to access the installation path `/usr/local/Ascend `of Ascend 910 AI processor software package(Version:Atlas Data Center Solution V100R020C00). If not, the root user needs to add the current user to the user group where `/usr/local/Ascend` is located. For the specific configuration, please refer to the software package instruction document. +- GCC 7.3.0 can be installed by using apt command. - When the network is connected, dependency items in the `requirements.txt` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items. ### (Optional) Installing Conda @@ -56,7 +56,7 @@ This document describes how to quickly install MindSpore on an Ascend AI process ### Configuring software package Dependencies - - Install the .whl package provided in Ascend 910 AI processor software package(Version:Atlas Data Center Solution V100R020C00T100). The .whl package is released with the software package. After software package is upgraded, reinstall the .whl package. + - Install the .whl package provided in Ascend 910 AI processor software package(Version:Atlas Data Center Solution V100R020C00). The .whl package is released with the software package. After software package is upgraded, reinstall the .whl package. ```bash pip install /usr/local/Ascend/fwkacllib/lib64/topi-{version}-py3-none-any.whl @@ -81,17 +81,17 @@ The compilation and installation must be performed on the Ascend 910 AI processo 1. Download the source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 2. Run the following command in the root directory of the source code to compile MindSpore: ```bash - bash build.sh -e d -z + bash build.sh -e ascend ``` > - Before running the preceding command, ensure that the paths where the executable files `cmake` and `patch` store have been added to the environment variable PATH. > - In the `build.sh` script, the `git clone` command will be executed to obtain the code in the third-party dependency database. Ensure that the network settings of Git are correct. - > - In the `build.sh` script, the default number of compilation threads is 8. If the compiler performance is poor, compilation errors may occur. You can add -j{Number of threads} in to script to reduce the number of threads. For example, `bash build.sh -e d -z -j4`. + > - In the `build.sh` script, the default number of compilation threads is 8. If the compiler performance is poor, compilation errors may occur. You can add -j{Number of threads} in to script to reduce the number of threads. For example, `bash build.sh -e ascend -j4`. 3. Run the following command to install MindSpore: @@ -102,19 +102,40 @@ The compilation and installation must be performed on the Ascend 910 AI processo ## Configuring Environment Variables -- After MindSpore is installed, export runtime-related environment variables. +- In EulerOS, after MindSpore is installed, export runtime-related environment variables. ```bash # control log level. 0-DEBUG, 1-INFO, 2-WARNING, 3-ERROR, default level is WARNING. export GLOG_v=2 + # Conda environmental options LOCAL_ASCEND=/usr/local/Ascend # the root directory of run package + # lib libraries that the run package depends on export LD_LIBRARY_PATH=${LOCAL_ASCEND}/add-ons/:${LOCAL_ASCEND}/fwkacllib/lib64:${LD_LIBRARY_PATH} + + # Environment variables that must be configured + export TBE_IMPL_PATH=${LOCAL_ASCEND}/opp/op_impl/built-in/ai_core/tbe # TBE operator implementation tool path + export PATH=${LOCAL_ASCEND}/fwkacllib/ccec_compiler/bin/:${PATH} # TBE operator compilation tool path + export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH} # Python library that TBE implementation depends on + ``` + +- In Ubuntu, after MindSpore is installed, export runtime-related environment variables. Note: you need to replace {version} in the following configuration with the actual version number on the environment. + + ```bash + # control log level. 0-DEBUG, 1-INFO, 2-WARNING, 3-ERROR, default level is WARNING. + export GLOG_v=2 + + # Conda environmental options + LOCAL_ASCEND=/usr/local/Ascend # the root directory of run package + + # lib libraries that the run package depends on + export LD_LIBRARY_PATH=${LOCAL_ASCEND}/add-ons/:${LOCAL_ASCEND}/ascend-toolkit/{version}/fwkacllib/lib64:${LOCAL_ASCEND}/driver/lib64:${LD_LIBRARY_PATH} + # Environment variables that must be configured - export TBE_IMPL_PATH=${LOCAL_ASCEND}/opp/op_impl/built-in/ai_core/tbe # TBE operator implementation tool path - export PATH=${LOCAL_ASCEND}/fwkacllib/ccec_compiler/bin/:${PATH} # TBE operator compilation tool path - export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH} # Python library that TBE implementation depends on + export TBE_IMPL_PATH=${LOCAL_ASCEND}/ascend-toolkit/{version}/opp/op_impl/built-in/ai_core/tbe # TBE operator implementation tool path + export PATH=${LOCAL_ASCEND}/ascend-toolkit/{version}/fwkacllib/ccec_compiler/bin/:${PATH} # TBE operator compilation tool path + export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH} # Python library that TBE implementation depends on ``` ## Installation Verification @@ -159,7 +180,7 @@ If you need to analyze information such as model scalars, graphs, and model trac | Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | | ---- | :--- | :--- | :--- | -| MindInsight master | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64
| - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindinsight/blob/master/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**Installation dependencies:**
same as the executable file installation dependencies. | +| MindInsight 0.5.0-beta | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64
| - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindinsight/blob/r0.5/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**Installation dependencies:**
same as the executable file installation dependencies. | - When the network is connected, dependency items in the `requirements.txt` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items. @@ -184,7 +205,7 @@ If you need to analyze information such as model scalars, graphs, and model trac 1. Download the source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindinsight.git + git clone https://gitee.com/mindspore/mindinsight.git -b r0.5 ``` > You are **not** supposed to obtain the source code from the zip package downloaded from the repository homepage. @@ -226,7 +247,7 @@ If you need to conduct AI model security research or enhance the security of the | Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | | ---- | :--- | :--- | :--- | -| MindArmour master | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64
| - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- For details about other dependency items, see [setup.py](https://gitee.com/mindspore/mindarmour/blob/master/setup.py). | Same as the executable file installation dependencies. | +| MindArmour 0.5.0-beta | - Ubuntu 18.04 aarch64
- Ubuntu 18.04 x86_64
- EulerOS 2.8 aarch64
- EulerOS 2.5 x86_64
| - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- For details about other dependency items, see [setup.py](https://gitee.com/mindspore/mindarmour/blob/r0.5/setup.py). | Same as the executable file installation dependencies. | - When the network is connected, dependency items in the `setup.py` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items. @@ -251,7 +272,7 @@ If you need to conduct AI model security research or enhance the security of the 1. Download the source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindarmour.git + git clone https://gitee.com/mindspore/mindarmour.git -b r0.5 ``` 2. Run the following command in the root directory of the source code to compile and install MindArmour: diff --git a/install/mindspore_gpu_install.md b/install/mindspore_gpu_install.md index 6fec7a71c51708a188ec475332e4169fa737f47d..7ebea9626fbf564625e410bbf7a852bdc6369cd2 100644 --- a/install/mindspore_gpu_install.md +++ b/install/mindspore_gpu_install.md @@ -28,9 +28,8 @@ | 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | | ---- | :--- | :--- | :--- | -| MindSpore master | Ubuntu 16.04（及以上） x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- [CUDA 9.2](https://developer.nvidia.com/cuda-92-download-archive) / [CUDA 10.1](https://developer.nvidia.com/cuda-10.1-download-archive-base)
- [CuDNN](https://developer.nvidia.com/rdp/cudnn-archive) >= 7.6
- [OpenMPI](https://www.open-mpi.org/faq/?category=building#easy-build) 3.1.5 （可选，单机多卡/多机多卡训练需要）
- [NCCL](https://docs.nvidia.com/deeplearning/sdk/nccl-install-guide/index.html#debian) 2.4.8-1 （可选，单机多卡/多机多卡训练需要）
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindspore/blob/master/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
- [Autoconf](https://www.gnu.org/software/autoconf) >= 2.69
- [Libtool](https://www.gnu.org/software/libtool) >= 2.4.6-29.fc30
- [Automake](https://www.gnu.org/software/automake) >= 1.15.1
- [CUDA 9.2](https://developer.nvidia.com/cuda-92-download-archive) / [CUDA 10.1](https://developer.nvidia.com/cuda-10.1-download-archive-base)
- [CuDNN](https://developer.nvidia.com/rdp/cudnn-archive) >= 7.6
**安装依赖：**
与可执行文件安装依赖相同 | +| MindSpore 0.5.0-beta | Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- [CUDA 10.1](https://developer.nvidia.com/cuda-10.1-download-archive-base)
- [CuDNN](https://developer.nvidia.com/rdp/cudnn-archive) >= 7.6
- [OpenMPI](https://www.open-mpi.org/faq/?category=building#easy-build) 3.1.5 （可选，单机多卡/多机多卡训练需要）
- [NCCL](https://docs.nvidia.com/deeplearning/sdk/nccl-install-guide/index.html#debian) 2.4.8-1 （可选，单机多卡/多机多卡训练需要）
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindspore/blob/r0.5/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
- [Autoconf](https://www.gnu.org/software/autoconf) >= 2.69
- [Libtool](https://www.gnu.org/software/libtool) >= 2.4.6-29.fc30
- [Automake](https://www.gnu.org/software/automake) >= 1.15.1
- [CUDA 9.2](https://developer.nvidia.com/cuda-92-download-archive) / [CUDA 10.1](https://developer.nvidia.com/cuda-10.1-download-archive-base)
- [CuDNN](https://developer.nvidia.com/rdp/cudnn-archive) >= 7.6
**安装依赖：**
与可执行文件安装依赖相同 | -- Ubuntu版本为18.04时，GCC 7.3.0可以直接通过apt命令安装。 - 在联网状态下，安装whl包时会自动下载`requirements.txt`中的依赖项，其余情况需自行安装。 - 为了方便用户使用，MindSpore降低了对Autoconf、Libtool、Automake版本的依赖，可以使用系统自带版本。 @@ -64,16 +63,16 @@ 1. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 2. 在源码根目录下执行如下命令编译MindSpore。 ```bash - bash build.sh -e gpu -M on -z + bash build.sh -e gpu ``` > - 在执行上述命令前，需保证可执行文件`cmake`和`patch`所在路径已加入环境变量PATH中。 > - `build.sh`中会执行`git clone`获取第三方依赖库的代码，请提前确保git的网络设置正确可用。 - > - `build.sh`中默认的编译线程数为8，如果编译机性能较差可能会出现编译错误，可在执行中增加-j{线程数}来减少线程数量。如`bash build.sh -e gpu -M on -z -j4`。 + > - `build.sh`中默认的编译线程数为8，如果编译机性能较差可能会出现编译错误，可在执行中增加-j{线程数}来减少线程数量。如`bash build.sh -e gpu -j4`。 3. 执行如下命令安装MindSpore。 @@ -124,7 +123,7 @@ | 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | | ---- | :--- | :--- | :--- | -| MindInsight master | - Ubuntu 16.04（及以上） x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindinsight/blob/master/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**安装依赖：**
与可执行文件安装依赖相同 | +| MindInsight 0.5.0-beta | - Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindinsight/blob/r0.5/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**安装依赖：**
与可执行文件安装依赖相同 | - 在联网状态下，安装whl包时会自动下载`requirements.txt`中的依赖项，其余情况需自行安装。 @@ -149,7 +148,7 @@ 1. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindinsight.git + git clone https://gitee.com/mindspore/mindinsight.git -b r0.5 ``` > **不能**直接在仓库主页下载zip包获取源码。 @@ -189,7 +188,7 @@ | 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | | ---------------------- | :------------------ | :----------------------------------------------------------- | :----------------------- | -| MindArmour master | Ubuntu 16.04（及以上） x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- 其他依赖项参见[setup.py](https://gitee.com/mindspore/mindarmour/blob/master/setup.py) | 与可执行文件安装依赖相同 | +| MindArmour 0.5.0-beta | Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- 其他依赖项参见[setup.py](https://gitee.com/mindspore/mindarmour/blob/r0.5/setup.py) | 与可执行文件安装依赖相同 | - 在联网状态下，安装whl包时会自动下载`setup.py`中的依赖项，其余情况需自行安装。 @@ -214,7 +213,7 @@ 1. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindarmour.git + git clone https://gitee.com/mindspore/mindarmour.git -b r0.5 ``` 2. 在源码根目录下，执行如下命令编译并安装MindArmour。 diff --git a/install/mindspore_gpu_install_en.md b/install/mindspore_gpu_install_en.md index d8ed2663716a0aa00562a282559c4a898050f171..626c1e1aa14d12c73bfeb6ec1d70fbec54102189 100644 --- a/install/mindspore_gpu_install_en.md +++ b/install/mindspore_gpu_install_en.md @@ -28,9 +28,8 @@ This document describes how to quickly install MindSpore on a NVIDIA GPU environ | Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | | ---- | :--- | :--- | :--- | -| MindSpore master | Ubuntu 16.04 or later x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- [CUDA 9.2](https://developer.nvidia.com/cuda-92-download-archive) / [CUDA 10.1](https://developer.nvidia.com/cuda-10.1-download-archive-base)
- [CuDNN](https://developer.nvidia.com/rdp/cudnn-archive) >= 7.6
- [OpenMPI](https://www.open-mpi.org/faq/?category=building#easy-build) 3.1.5 (optional, required for single-node/multi-GPU and multi-node/multi-GPU training)
- [NCCL](https://docs.nvidia.com/deeplearning/sdk/nccl-install-guide/index.html#debian) 2.4.8-1 (optional, required for single-node/multi-GPU and multi-node/multi-GPU training)
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindspore/blob/master/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
- [Autoconf](https://www.gnu.org/software/autoconf) >= 2.69
- [Libtool](https://www.gnu.org/software/libtool) >= 2.4.6-29.fc30
- [Automake](https://www.gnu.org/software/automake) >= 1.15.1
- [CUDA 9.2](https://developer.nvidia.com/cuda-92-download-archive) / [CUDA 10.1](https://developer.nvidia.com/cuda-10.1-download-archive-base)
- [CuDNN](https://developer.nvidia.com/rdp/cudnn-archive) >= 7.6
**Installation dependencies:**
same as the executable file installation dependencies. | +| MindSpore 0.5.0-beta | Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- [CUDA 10.1](https://developer.nvidia.com/cuda-10.1-download-archive-base)
- [CuDNN](https://developer.nvidia.com/rdp/cudnn-archive) >= 7.6
- [OpenMPI](https://www.open-mpi.org/faq/?category=building#easy-build) 3.1.5 (optional, required for single-node/multi-GPU and multi-node/multi-GPU training)
- [NCCL](https://docs.nvidia.com/deeplearning/sdk/nccl-install-guide/index.html#debian) 2.4.8-1 (optional, required for single-node/multi-GPU and multi-node/multi-GPU training)
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindspore/blob/r0.5/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5
- [Autoconf](https://www.gnu.org/software/autoconf) >= 2.69
- [Libtool](https://www.gnu.org/software/libtool) >= 2.4.6-29.fc30
- [Automake](https://www.gnu.org/software/automake) >= 1.15.1
- [CUDA 9.2](https://developer.nvidia.com/cuda-92-download-archive) / [CUDA 10.1](https://developer.nvidia.com/cuda-10.1-download-archive-base)
- [CuDNN](https://developer.nvidia.com/rdp/cudnn-archive) >= 7.6
**Installation dependencies:**
same as the executable file installation dependencies. | -- When Ubuntu version is 18.04, GCC 7.3.0 can be installed by using apt command. - When the network is connected, dependency items in the `requirements.txt` file are automatically downloaded during `.whl` package installation. In other cases, you need to manually install dependency items. - MindSpore reduces dependency on Autoconf, Libtool, Automake versions for the convenience of users, default versions of these tools built in their systems are now supported. @@ -64,16 +63,16 @@ This document describes how to quickly install MindSpore on a NVIDIA GPU environ 1. Download the source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 2. Run the following command in the root directory of the source code to compile MindSpore: ```bash - bash build.sh -e gpu -M on -z + bash build.sh -e gpu ``` > - Before running the preceding command, ensure that the paths where the executable files `cmake` and `patch` store have been added to the environment variable PATH. > - In the `build.sh` script, the `git clone` command will be executed to obtain the code in the third-party dependency database. Ensure that the network settings of Git are correct. - > - In the `build.sh` script, the default number of compilation threads is 8. If the compiler performance is poor, compilation errors may occur. You can add -j{Number of threads} in to script to reduce the number of threads. For example, `bash build.sh -e gpu -M on -z -j4`. + > - In the `build.sh` script, the default number of compilation threads is 8. If the compiler performance is poor, compilation errors may occur. You can add -j{Number of threads} in to script to reduce the number of threads. For example, `bash build.sh -e gpu -j4`. 3. Run the following command to install MindSpore: @@ -124,7 +123,7 @@ If you need to analyze information such as model scalars, graphs, and model trac | Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | | ---- | :--- | :--- | :--- | -| MindInsight master | - Ubuntu 16.04 or later x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindinsight/blob/master/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**Installation dependencies:**
same as the executable file installation dependencies. | +| MindInsight 0.5.0-beta | - Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindinsight/blob/r0.5/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**Installation dependencies:**
same as the executable file installation dependencies. | - When the network is connected, dependency items in the `requirements.txt` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items. @@ -149,7 +148,7 @@ If you need to analyze information such as model scalars, graphs, and model trac 1. Download the source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindinsight.git + git clone https://gitee.com/mindspore/mindinsight.git -b r0.5 ``` > You are **not** supposed to obtain the source code from the zip package downloaded from the repository homepage. @@ -191,7 +190,7 @@ If you need to conduct AI model security research or enhance the security of the | Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | | ---- | :--- | :--- | :--- | -| MindArmour master | Ubuntu 16.04 or later x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- For details about other dependency items, see [setup.py](https://gitee.com/mindspore/mindarmour/blob/master/setup.py). | Same as the executable file installation dependencies. | +| MindArmour 0.5.0-beta | Ubuntu 18.04 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore 0.5.0-beta
- For details about other dependency items, see [setup.py](https://gitee.com/mindspore/mindarmour/blob/r0.5/setup.py). | Same as the executable file installation dependencies. | - When the network is connected, dependency items in the `setup.py` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items. @@ -216,7 +215,7 @@ If you need to conduct AI model security research or enhance the security of the 1. Download the source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindarmour.git + git clone https://gitee.com/mindspore/mindarmour.git -b r0.5 ``` 2. Run the following command in the root directory of the source code to compile and install MindArmour: diff --git a/resource/api_mapping.md b/resource/api_mapping.md index bb3471b378ec24b365eb784563d60598c72dc3d0..4743f88c02b7b56530ad9ba7ecfc48eb0f5f5db3 100644 --- a/resource/api_mapping.md +++ b/resource/api_mapping.md @@ -9,12 +9,18 @@ Mapping between PyTorch APIs and MindSpore APIs, which is provided by the commun | torch.add | mindspore.ops.operations.TensorAdd | | torch.argmax | mindspore.ops.operations.Argmax | | torch.argmin | mindspore.ops.operations.Argmin | +| torch.asin | mindspore.ops.operations.Asin | +| torch.atan | mindspore.ops.operations.Atan | | torch.atan2 | mindspore.ops.operations.Atan2 | +| torch.bitwise_and | mindspore.ops.operations.BitwiseAnd | +| torch.bitwise_or | mindspore.ops.operations.BitwiseOr | | torch.bmm | mindspore.ops.operations.BatchMatMul | | torch.cat | mindspore.ops.operations.Concat | +| torch.ceil | mindspore.ops.operations.Ceil | | torch.chunk | mindspore.ops.operations.Split | | torch.clamp | mindspore.ops.composite.clip_by_value | | torch.cos | mindspore.ops.operations.Cos | +| torch.cosh | mindspore.ops.operations.Cosh | | torch.cuda.device_count | mindspore.communication.get_group_size | | torch.cuda.set_device | mindspore.context.set_context | | torch.cumprod | mindspore.ops.operations.CumProd | @@ -27,9 +33,11 @@ Mapping between PyTorch APIs and MindSpore APIs, which is provided by the commun | torch.eq | mindspore.ops.operations.Equal | | torch.erfc | mindspore.ops.operations.Erfc | | torch.exp | mindspore.ops.operations.Exp | +| torch.expm1 | mindspore.ops.operations.Expm1 | | torch.eye | mindspore.ops.operations.Eye | | torch.flatten | mindspore.ops.operations.Flatten | | torch.floor | mindspore.ops.operations.Floor | +| torch.linspace | mindspore.nn.LinSpace | | torch.load | mindspore.train.serialization.load_checkpoint | | torch.log | mindspore.ops.operations.Log | | torch.log1p | mindspore.ops.operations.Log1p | @@ -96,7 +104,10 @@ Mapping between PyTorch APIs and MindSpore APIs, which is provided by the commun | torch.numel | mindspore.ops.operations.Size | | torch.ones | mindspore.ops.operations.OnesLike | | torch.ones_like | mindspore.ops.operations.OnesLike | +| torch.optim.Adadelta | mindspore.ops.operations.ApplyAdadelta | +| torch.optim.Adagrad | mindspore.ops.operations.ApplyAdagrad | | torch.optim.Adam | mindspore.nn.Adam | +| torch.optim.Adamax | mindspore.ops.operations.ApplyAdaMax | | torch.optim.AdamW | mindspore.nn.AdamWeightDecay | | torch.optim.lr_scheduler.CosineAnnealingWarmRestarts | mindspore.nn.dynamic_lr.cosine_decay_lr | | torch.optim.lr_scheduler.StepLR | mindspore.nn.dynamic_lr.piecewise_constant_lr | @@ -106,10 +117,12 @@ Mapping between PyTorch APIs and MindSpore APIs, which is provided by the commun | torch.pow | mindspore.ops.operations.Pow | | torch.prod | mindspore.ops.operations.ReduceProd | | torch.randn | mindspore.ops.operations.TruncatedNormal | +| torch.range | mindspore.nn.Range | | torch.round | mindspore.ops.operations.Round | | torch.save | mindspore.train.serialization.save_checkpoint | | torch.sigmoid | mindspore.ops.operations.Sigmoid | | torch.sin | mindspore.ops.operations.Sin | +| torch.sinh | mindspore.ops.operations.Sinh | | torch.sparse.FloatTensor | mindspore.Tensor | | torch.split | mindspore.ops.operations.Split | | torch.sqrt | mindspore.ops.operations.Sqrt | @@ -121,8 +134,10 @@ Mapping between PyTorch APIs and MindSpore APIs, which is provided by the commun | torch.tensor | mindspore.Tensor | | torch.Tensor | mindspore.Tensor | | torch.Tensor.chunk | mindspore.ops.operations.Split | +| torch.Tensor.expand | mindspore.ops.operations.BroadcastTo | | torch.Tensor.fill_ | mindspore.ops.operations.Fill | | torch.Tensor.float | mindspore.ops.operations.Cast | +| torch.Tensor.index_add | mindspore.ops.operations.InplaceAdd | | torch.Tensor.mm | mindspore.ops.operations.MatMul | | torch.Tensor.mul | mindspore.ops.operations.Mul | | torch.Tensor.pow | mindspore.ops.operations.Pow | @@ -157,4 +172,4 @@ Mapping between PyTorch APIs and MindSpore APIs, which is provided by the commun | torchvision.transforms.Normalize | mindspore.dataset.transforms.vision.py_transforms.Normalize | | torchvision.transforms.RandomHorizontalFlip | mindspore.dataset.transforms.vision.py_transforms.RandomHorizontalFlip | | torchvision.transforms.Resize | mindspore.dataset.transforms.vision.py_transforms.Resize | -| torchvision.transforms.ToTensor | mindspore.dataset.transforms.vision.py_transforms.ToTensor | \ No newline at end of file +| torchvision.transforms.ToTensor | mindspore.dataset.transforms.vision.py_transforms.ToTensor | diff --git a/resource/faq/FAQ_en.md b/resource/faq/FAQ_en.md index 7fa3b3034787ff8ea08b355864c47e199bc1cfb6..d2c66f32ef9569d9497c050d75f78ab63b80ad76 100644 --- a/resource/faq/FAQ_en.md +++ b/resource/faq/FAQ_en.md @@ -68,13 +68,13 @@ A: Please install the software manually if there is any suggestion of certain `s Q: What types of model is currently supported by MindSpore for training ? -A: MindSpore has basic support for common training scenarios, please refer to [Release note](https://gitee.com/mindspore/mindspore/blob/master/RELEASE.md) for detailed information. +A: MindSpore has basic support for common training scenarios, please refer to [Release note](https://gitee.com/mindspore/mindspore/blob/r0.5/RELEASE.md) for detailed information.
Q: What are the available recommendation or text generation networks or models provided by MindSpore? -A: Currently, recommendation models such as Wide & Deep, DeepFM, and NCF are under development. In the natural language processing (NLP) field, Bert\_NEZHA is available and models such as MASS are under development. You can rebuild the network into a text generation network based on the scenario requirements. Please stay tuned for updates on the [MindSpore Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/mindspore/model_zoo). +A: Currently, recommendation models such as Wide & Deep, DeepFM, and NCF are under development. In the natural language processing (NLP) field, Bert\_NEZHA is available and models such as MASS are under development. You can rebuild the network into a text generation network based on the scenario requirements. Please stay tuned for updates on the [MindSpore Model Zoo](https://gitee.com/mindspore/mindspore/tree/r0.5/model_zoo). ### Backend Support @@ -92,13 +92,13 @@ A: MindSpore provides pluggable device management interface so that developer co Q: What hardware does MindSpore require? -A: Currently, you can try out MindSpore through Docker images on laptops or in environments with GPUs. Some models in MindSpore Model Zoo support GPU-based training and inference, and other models are being improved. For distributed parallel training, MindSpore supports multi-GPU training. You can obtain the latest information from [RoadMap](https://www.mindspore.cn/docs/en/master/roadmap.html) and project [Release Notes](https://gitee.com/mindspore/mindspore/blob/master/RELEASE.md). +A: Currently, you can try out MindSpore through Docker images on laptops or in environments with GPUs. Some models in MindSpore Model Zoo support GPU-based training and inference, and other models are being improved. For distributed parallel training, MindSpore supports multi-GPU training. You can obtain the latest information from [RoadMap](https://www.mindspore.cn/docs/en/r0.5/roadmap.html) and project [Release Notes](https://gitee.com/mindspore/mindspore/blob/r0.5/RELEASE.md). ### System Support Q: Does MindSpore support Windows 10? -A: The MindSpore CPU version can be installed on Windows 10. For details about the installation procedure, see tutorials on the [MindSpore official website](https://www.mindspore.cn/tutorial/en/master/advanced_use/mindspore_cpu_win_install.html). +A: The MindSpore CPU version can be installed on Windows 10. For details about the installation procedure, see tutorials on the [MindSpore official website](https://www.mindspore.cn/tutorial/en/r0.5/advanced_use/mindspore_cpu_win_install.html). ### Programming Language @@ -122,7 +122,7 @@ A: The MindSpore framework does not support FCA. For semantic models, you can ca Q: Where can I view the sample code or tutorial of MindSpore training and inference? -A: Please visit the [MindSpore official website](https://www.mindspore.cn/tutorial/en/master/index.html). +A: Please visit the [MindSpore official website](https://www.mindspore.cn/tutorial/en/r0.5/index.html). ## Features @@ -140,7 +140,7 @@ A: Automatic parallelism on CPUs and GPUs are being improved. You are advised to Q: What is the relationship between MindSpore and ModelArts? Can MindSpore be used on ModelArts? -A: ModelArts is an online training and inference platform on HUAWEI CLOUD. MindSpore is a Huawei deep learning framework. You can view the tutorials on the [MindSpore official website](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/use_on_the_cloud.html) to learn how to train MindSpore models on ModelArts. +A: ModelArts is an online training and inference platform on HUAWEI CLOUD. MindSpore is a Huawei deep learning framework. You can view the tutorials on the [MindSpore official website](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/use_on_the_cloud.html) to learn how to train MindSpore models on ModelArts. ## Capabilities @@ -152,7 +152,7 @@ A: The TensorFlow's object detection pipeline API belongs to the TensorFlow's Mo Q: How do I migrate scripts or models of other frameworks to MindSpore? -A: For details about script or model migration, please visit the [MindSpore official website](https://www.mindspore.cn/tutorial/en/master/advanced_use/network_migration.html). +A: For details about script or model migration, please visit the [MindSpore official website](https://www.mindspore.cn/tutorial/en/r0.5/advanced_use/network_migration.html).
diff --git a/resource/faq/FAQ_zh_cn.md b/resource/faq/FAQ_zh_cn.md index db448da184e2b5370da5245b4992fa995be8aa38..ae58ffd522c94adde5ad8e29ea3d28afe6fdbdad 100644 --- a/resource/faq/FAQ_zh_cn.md +++ b/resource/faq/FAQ_zh_cn.md @@ -67,13 +67,13 @@ A：当有此提示时说明要用户安装`tclsh`；如果仍提示缺少其他 Q：MindSpore支持哪些模型的训练？ -A：MindSpore针对典型场景均有模型训练支持，支持情况详见[Release note](https://gitee.com/mindspore/mindspore/blob/master/RELEASE.md)。 +A：MindSpore针对典型场景均有模型训练支持，支持情况详见[Release note](https://gitee.com/mindspore/mindspore/blob/r0.5/RELEASE.md)。
Q：MindSpore有哪些现成的推荐类或生成类网络或模型可用？ -A：目前正在开发Wide & Deep、DeepFM、NCF等推荐类模型，NLP领域已经支持Bert_NEZHA，正在开发MASS等模型，用户可根据场景需要改造为生成类网络，可以关注[MindSpore Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/mindspore/model_zoo)。 +A：目前正在开发Wide & Deep、DeepFM、NCF等推荐类模型，NLP领域已经支持Bert_NEZHA，正在开发MASS等模型，用户可根据场景需要改造为生成类网络，可以关注[MindSpore Model Zoo](https://gitee.com/mindspore/mindspore/tree/r0.5/model_zoo)。 ### 后端支持 @@ -91,13 +91,13 @@ A：MindSpore提供了可插拔式的设备管理接口，其他计算单元（ Q：MindSpore需要什么硬件支持？ -A：目前笔记本电脑或者有GPU的环境，都可以通过Docker镜像来试用。当前MindSpore Model Zoo中有部分模型已经支持GPU的训练和推理，其他模型也在不断地进行完善。在分布式并行训练方面，MindSpore当前支持GPU多卡训练。你可以通过[RoadMap](https://www.mindspore.cn/docs/zh-CN/master/roadmap.html)和项目[Release note](https://gitee.com/mindspore/mindspore/blob/master/RELEASE.md)获取最新信息。 +A：目前笔记本电脑或者有GPU的环境，都可以通过Docker镜像来试用。当前MindSpore Model Zoo中有部分模型已经支持GPU的训练和推理，其他模型也在不断地进行完善。在分布式并行训练方面，MindSpore当前支持GPU多卡训练。你可以通过[RoadMap](https://www.mindspore.cn/docs/zh-CN/r0.5/roadmap.html)和项目[Release note](https://gitee.com/mindspore/mindspore/blob/r0.5/RELEASE.md)获取最新信息。 ### 系统支持 Q：MindSpore是否支持Windows 10？ -A：MindSpore CPU版本已经支持在Windows 10系统中安装，具体安装步骤可以查阅[MindSpore官网教程](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/mindspore_cpu_win_install.html)。 +A：MindSpore CPU版本已经支持在Windows 10系统中安装，具体安装步骤可以查阅[MindSpore官网教程](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/mindspore_cpu_win_install.html)。 ### 编程语言扩展 @@ -121,7 +121,7 @@ A：MindSpore框架本身并不需要支持FCA。对于语义类模型，用户 Q：从哪里可以查看MindSpore训练及推理的样例代码或者教程？ -A：可以访问[MindSpore官网教程](https://www.mindspore.cn/tutorial/zh-CN/master/index.html)。 +A：可以访问[MindSpore官网教程](https://www.mindspore.cn/tutorial/zh-CN/r0.5/index.html)。 ## 特性 @@ -139,7 +139,7 @@ A：自动并行特性对CPU GPU的支持还在完善中。推荐用户在Ascend Q：MindSpore与ModelArts是什么关系，在ModelArts中能使用MindSpore吗？ -A：ModelArts是华为公有云线上训练及推理平台，MindSpore是华为深度学习框架，可以查阅[MindSpore官网教程](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/use_on_the_cloud.html)，教程中详细展示了用户如何使用ModelArts来做MindSpore的模型训练。 +A：ModelArts是华为公有云线上训练及推理平台，MindSpore是华为深度学习框架，可以查阅[MindSpore官网教程](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/use_on_the_cloud.html)，教程中详细展示了用户如何使用ModelArts来做MindSpore的模型训练。 ## 能力 @@ -151,7 +151,7 @@ A：TensorFlow的对象检测Pipeline接口属于TensorFlow Model模块。待Min Q：其他框架的脚本或者模型怎么迁移到MindSpore？ -A：关于脚本或者模型迁移，可以查询MindSpore官网中关于[网络迁移](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/network_migration.html)的介绍。 +A：关于脚本或者模型迁移，可以查询MindSpore官网中关于[网络迁移](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/network_migration.html)的介绍。
diff --git a/resource/release/release_list_en.md b/resource/release/release_list_en.md index f4634a873d5b6bee40823a3b30a8b391ca42845a..5c43336038ddc2fac9f5d84005a924ecfaeb6c51 100644 --- a/resource/release/release_list_en.md +++ b/resource/release/release_list_en.md @@ -3,6 +3,12 @@ - [Release List](#release-list) + - [0.5.0-beta](#050-beta) + - [Releasenotes](#releasenotes) + - [Downloads](#downloads) + - [Tutorials](#tutorials) + - [API](#api) + - [Docs](#docs) - [0.3.0-alpha](#030-alpha) - [Releasenotes](#releasenotes) - [Downloads](#downloads) @@ -25,6 +31,44 @@ +## 0.5.0-beta + +### Releasenotes + + + +### Downloads + +| Module Name | Hardware Platform | Operating System | Download Links | SHA-256 | +| --- | --- | --- | --- | --- | +| MindSpore | Ascend910 | Ubuntu-x86 | | 0c5afb5cef15065424cfa60beb6bb3a6073c977e815fae1004299f8de4bd0fac | +| | | Ubuntu-aarch64 | | eda47fc6e4646f0b3bcee3e37af5eb8426208f162fcee2d53b2c8310f13509c3 | +| | | EulerOS-x86 | | a108b9f238a91dee75c3005f81454a4a4e82972c54d062ebd8d62951704c0a56 | +| | | EulerOS-aarch64 | | c7aba79315c6fabdc8587e8f62f26b0069e0057d308eb4d81257f05f27b4c154 | +| | GPU CUDA 10.1 | Ubuntu-x86 | | 8532e060f31e96fc0bef6c196959ede665a9d049d60ce9e2e533ddc1d6b6222d | +| | CPU | Ubuntu-x86 | | 72e0755120060ee450e74a8ef953133b6c22a203e19de25dcba8b861fae08d52 | +| | | Windows-x64 | | ecd9144406ec7415cdfce8b55a9fd1616b528c84d6fde5c53cf329420dfb6409 | +| MindInsight | Ascend910 | Ubuntu-x86 | | 34b3c1a5ffbf9fa5e46dc6f295abde0308b65d76fd18d4551103ca0e222e3651 | +| | | Ubuntu-aarch64 | | 97f92b556f8e97e250f311f5d11caace4ac5686015b099b98462d9603e2c5724 | +| | | EulerOS-x86 | | 5fab87c3dfda57851a9981c7567200f0f0d856462b8dd521402b085830e6554f | +| | | EulerOS-aarch64 | | 7a157fb849f078fef6792353414737a8eccd98ba7a6fdd3c4ba3b497bc3f019f | +| | GPU CUDA 10.1 | Ubuntu-x86 | | 34b3c1a5ffbf9fa5e46dc6f295abde0308b65d76fd18d4551103ca0e222e3651 | +| MindArmour | Ascend910 | Ubuntu-x86/EulerOS-x86 | | 1c80113575e27d8330f6f951fd3a68b7a01b2b642a3d2b2d8c070325d71161e5 | +| | | Ubuntu-aarch64/EulerOS-aarch64 | | f5d9bf5941d5f3273deb72cf77dc63767ee5ab09f9e329b4020899195f67d951 | +| | GPU CUDA 10.1/CPU | Ubuntu-x86 | | 1c80113575e27d8330f6f951fd3a68b7a01b2b642a3d2b2d8c070325d71161e5 | + +### Tutorials + + + +### API + + + +### Docs + + + ## 0.3.0-alpha ### Releasenotes diff --git a/resource/release/release_list_zh_cn.md b/resource/release/release_list_zh_cn.md index e1d1672fc3e89d922af2350455f2742492d3c0d1..20cd0b4fa496923fd992734740a749514cc02475 100644 --- a/resource/release/release_list_zh_cn.md +++ b/resource/release/release_list_zh_cn.md @@ -3,6 +3,12 @@ - [发布版本列表](#发布版本列表) + - [0.5.0-beta](#050-beta) + - [版本说明](#版本说明) + - [下载地址](#下载地址) + - [教程](#教程) + - [API](#api) + - [文档](#文档) - [0.3.0-alpha](#030-alpha) - [版本说明](#版本说明) - [下载地址](#下载地址) @@ -25,6 +31,39 @@ +## 0.5.0-beta +### 版本说明 + + + +### 下载地址 +| 组件 | 硬件平台 | 操作系统 | 链接 | SHA-256 | +| --- | --- | --- | --- | --- | +| MindSpore | Ascend910 | Ubuntu-x86 | | 0c5afb5cef15065424cfa60beb6bb3a6073c977e815fae1004299f8de4bd0fac | +| | | Ubuntu-aarch64 | | eda47fc6e4646f0b3bcee3e37af5eb8426208f162fcee2d53b2c8310f13509c3 | +| | | EulerOS-x86 | | a108b9f238a91dee75c3005f81454a4a4e82972c54d062ebd8d62951704c0a56 | +| | | EulerOS-aarch64 | | c7aba79315c6fabdc8587e8f62f26b0069e0057d308eb4d81257f05f27b4c154 | +| | GPU CUDA 10.1 | Ubuntu-x86 | | 8532e060f31e96fc0bef6c196959ede665a9d049d60ce9e2e533ddc1d6b6222d | +| | CPU | Ubuntu-x86 | | 72e0755120060ee450e74a8ef953133b6c22a203e19de25dcba8b861fae08d52 | +| | | Windows-x64 | | ecd9144406ec7415cdfce8b55a9fd1616b528c84d6fde5c53cf329420dfb6409 | +| MindInsight | Ascend910 | Ubuntu-x86 | | 34b3c1a5ffbf9fa5e46dc6f295abde0308b65d76fd18d4551103ca0e222e3651 | +| | | Ubuntu-aarch64 | | 97f92b556f8e97e250f311f5d11caace4ac5686015b099b98462d9603e2c5724 | +| | | EulerOS-x86 | | 5fab87c3dfda57851a9981c7567200f0f0d856462b8dd521402b085830e6554f | +| | | EulerOS-aarch64 | | 7a157fb849f078fef6792353414737a8eccd98ba7a6fdd3c4ba3b497bc3f019f | +| | GPU CUDA 10.1 | Ubuntu-x86 | | 34b3c1a5ffbf9fa5e46dc6f295abde0308b65d76fd18d4551103ca0e222e3651 | +| MindArmour | Ascend910 | Ubuntu-x86/EulerOS-x86 | | 1c80113575e27d8330f6f951fd3a68b7a01b2b642a3d2b2d8c070325d71161e5 | +| | | Ubuntu-aarch64/EulerOS-aarch64 | | f5d9bf5941d5f3273deb72cf77dc63767ee5ab09f9e329b4020899195f67d951 | +| | GPU CUDA 10.1/CPU | Ubuntu-x86 | | 1c80113575e27d8330f6f951fd3a68b7a01b2b642a3d2b2d8c070325d71161e5 | + +### 教程 + + +### API + + +### 文档 + + ## 0.3.0-alpha ### 版本说明 diff --git a/tutorials/notebook/mindinsight/images/data_lineage.png b/tutorials/notebook/mindinsight/images/data_lineage.png new file mode 100644 index 0000000000000000000000000000000000000000..583707d9d796d63b04a66ce48dced48b162aad8f Binary files /dev/null and b/tutorials/notebook/mindinsight/images/data_lineage.png differ diff --git a/tutorials/notebook/mindinsight/images/histogram.png b/tutorials/notebook/mindinsight/images/histogram.png new file mode 100644 index 0000000000000000000000000000000000000000..a0bea73d058cbd971e56801a0c92d035b20b35c3 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/histogram.png differ diff --git a/tutorials/notebook/mindinsight/images/histogram_func.png b/tutorials/notebook/mindinsight/images/histogram_func.png new file mode 100644 index 0000000000000000000000000000000000000000..15437442b436b8d1783ad36a816b05df4f156c06 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/histogram_func.png differ diff --git a/tutorials/notebook/mindinsight/images/histogram_only.png b/tutorials/notebook/mindinsight/images/histogram_only.png new file mode 100644 index 0000000000000000000000000000000000000000..27febd13cefef260a93842f2c5012898dc2bf6ef Binary files /dev/null and b/tutorials/notebook/mindinsight/images/histogram_only.png differ diff --git a/tutorials/notebook/mindinsight/images/histogram_only_all.png b/tutorials/notebook/mindinsight/images/histogram_only_all.png new file mode 100644 index 0000000000000000000000000000000000000000..c3f9bdadeb51f1e3f14d8c8135b73bb03d9b6df2 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/histogram_only_all.png differ diff --git a/tutorials/notebook/mindinsight/images/histogram_panel.png b/tutorials/notebook/mindinsight/images/histogram_panel.png new file mode 100644 index 0000000000000000000000000000000000000000..fbf9de6e1fcdd025c47ac97812242c13d292cffc Binary files /dev/null and b/tutorials/notebook/mindinsight/images/histogram_panel.png differ diff --git a/tutorials/notebook/mindinsight/images/image_function.png b/tutorials/notebook/mindinsight/images/image_function.png new file mode 100644 index 0000000000000000000000000000000000000000..8e9e0bfe1deec2fca17e0c5e653ed4634b33c0c5 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/image_function.png differ diff --git a/tutorials/notebook/mindinsight/images/image_only.png b/tutorials/notebook/mindinsight/images/image_only.png new file mode 100644 index 0000000000000000000000000000000000000000..e08bd9f0c7ef2d26b4303a314f1a46d513cad65c Binary files /dev/null and b/tutorials/notebook/mindinsight/images/image_only.png differ diff --git a/tutorials/notebook/mindinsight/images/image_panel.png b/tutorials/notebook/mindinsight/images/image_panel.png new file mode 100644 index 0000000000000000000000000000000000000000..19221e047e5f36b63193aa54f6911f4c9b98f0ec Binary files /dev/null and b/tutorials/notebook/mindinsight/images/image_panel.png differ diff --git a/tutorials/notebook/mindinsight/images/image_vi.png b/tutorials/notebook/mindinsight/images/image_vi.png new file mode 100644 index 0000000000000000000000000000000000000000..a941238ee2fd945bdef5619410ba32a9157e2ff7 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/image_vi.png differ diff --git a/tutorials/notebook/mindinsight/images/loss_scalar_only.png b/tutorials/notebook/mindinsight/images/loss_scalar_only.png new file mode 100644 index 0000000000000000000000000000000000000000..71ef414dd9dd7f1e0c25017e4c14624ad8f644e9 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/loss_scalar_only.png differ diff --git a/tutorials/notebook/mindinsight/images/mindinsight_panel.png b/tutorials/notebook/mindinsight/images/mindinsight_panel.png new file mode 100644 index 0000000000000000000000000000000000000000..8eb80073b47556ea1759bc44b3b02b0e8f5ed022 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/mindinsight_panel.png differ diff --git a/tutorials/notebook/mindinsight/images/mindinsight_panel2.png b/tutorials/notebook/mindinsight/images/mindinsight_panel2.png new file mode 100644 index 0000000000000000000000000000000000000000..1122a225c18174c4c968232a9a888f834e0bef58 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/mindinsight_panel2.png differ diff --git a/tutorials/notebook/mindinsight/images/mnist_dataset.png b/tutorials/notebook/mindinsight/images/mnist_dataset.png new file mode 100644 index 0000000000000000000000000000000000000000..9cd3787d4ddb932f1a79177955066602d439a317 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/mnist_dataset.png differ diff --git a/tutorials/notebook/mindinsight/images/model_lineage_all.png b/tutorials/notebook/mindinsight/images/model_lineage_all.png new file mode 100644 index 0000000000000000000000000000000000000000..a8761f3b33fa8fbc90113a2c8202f504ccf049b3 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/model_lineage_all.png differ diff --git a/tutorials/notebook/mindinsight/images/model_lineage_cp.png b/tutorials/notebook/mindinsight/images/model_lineage_cp.png new file mode 100644 index 0000000000000000000000000000000000000000..b8a3226990209c2289a7d56ffb49d2e8937d2961 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/model_lineage_cp.png differ diff --git a/tutorials/notebook/mindinsight/images/multi_scalars.png b/tutorials/notebook/mindinsight/images/multi_scalars.png new file mode 100644 index 0000000000000000000000000000000000000000..4e43be097b6fbf7108a8cfffc6beaebf0a0e6d73 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/multi_scalars.png differ diff --git a/tutorials/notebook/mindinsight/images/multi_scalars_select.png b/tutorials/notebook/mindinsight/images/multi_scalars_select.png new file mode 100644 index 0000000000000000000000000000000000000000..182beb2c5ae782294fa7f5619eb3542308861989 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/multi_scalars_select.png differ diff --git a/tutorials/notebook/mindinsight/images/scalar.png b/tutorials/notebook/mindinsight/images/scalar.png new file mode 100644 index 0000000000000000000000000000000000000000..91d687c3d3c2cbd0f61c9486ec57c61b39f7f3b6 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/scalar.png differ diff --git a/tutorials/notebook/mindinsight/images/scalar_panel.png b/tutorials/notebook/mindinsight/images/scalar_panel.png new file mode 100644 index 0000000000000000000000000000000000000000..7e0794a2f4270a1dff868f29f41554c353ce2dc9 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/scalar_panel.png differ diff --git a/tutorials/notebook/mindinsight/images/scalar_select.png b/tutorials/notebook/mindinsight/images/scalar_select.png new file mode 100644 index 0000000000000000000000000000000000000000..a5a75e646fd236b4350b6f5dc57f4fa08b2096c6 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/scalar_select.png differ diff --git a/tutorials/notebook/mindinsight/images/summary_list.png b/tutorials/notebook/mindinsight/images/summary_list.png new file mode 100644 index 0000000000000000000000000000000000000000..5b3f170433d0fee73d4d462efe6cfd6dfeb5a166 Binary files /dev/null and b/tutorials/notebook/mindinsight/images/summary_list.png differ diff --git a/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar.ipynb b/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..c4a69bcf7e47a34c8df8ae305fc255d117c5a0bc --- /dev/null +++ b/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar.ipynb @@ -0,0 +1,600 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# MindInsight之标量、直方图和图像\n", + "\n", + "MindInsight可以将神经网络训练过程中的损失值标量、直方图、图像信息记录到日志文件中，通过可视化界面解析以供用户查看。\n", + "\n", + "整体流程:\n", + "\n", + "1. 下载MNIST数据集。\n", + "\n", + "2. 原始数据预处理。\n", + "\n", + "3. 初始化`lenet`网络。\n", + "\n", + "4. 执行主程序，使用`SummaryCollector`记录图像信息、损失值标量、权重梯度等参数，启动MindInsight服务。\n", + "\n", + "5. 在MindInsight可视化面板中查看结果。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 数据集操作\n", + "\n", + "本次流程用到MNIST数据集，MNIST数据集是一个手写的数据文件，数据库里的图像都是28x28的灰度图像，每个像素都是一个八位字节，包含了60000张训练图像和10000张测试图像，常被用作神经网络训练和测试任务的数据集。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/mnist_dataset.png)\n", + "\n", + "## 下载MNIST数据集\n", + "\n", + "下面一段代码分为两部分：\n", + "\n", + "1. 判断是否存在MNIST数据集目录，不存在则创建目录，存在则跳至[**数据预处理**](#数据预处理)。\n", + "\n", + "2. 判断是否存在MNIST数据集，不存在则下载MNIST数据集，存在则跳至[**数据预处理**](#数据预处理)。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import urllib.request\n", + "from urllib.parse import urlparse\n", + "import gzip\n", + "\n", + "def unzipfile(gzip_path):\n", + " \"\"\"unzip dataset file\n", + " Args:\n", + " gzip_path: dataset file path\n", + " \"\"\"\n", + " open_file = open(gzip_path.replace('.gz',''), 'wb')\n", + " gz_file = gzip.GzipFile(gzip_path)\n", + " open_file.write(gz_file.read())\n", + " gz_file.close()\n", + "\n", + "\n", + "def download_dataset():\n", + " \"\"\"Download the dataset from http://yann.lecun.com/exdb/mnist/.\"\"\"\n", + " print(\"******Downloading the MNIST dataset******\")\n", + " train_path = \"./MNIST_Data/train/\"\n", + " test_path = \"./MNIST_Data/test/\"\n", + " train_path_check = os.path.exists(train_path)\n", + " test_path_check = os.path.exists(test_path)\n", + " if train_path_check == False and test_path_check ==False:\n", + " os.makedirs(train_path)\n", + " os.makedirs(test_path)\n", + " train_url = {\"http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\"}\n", + " test_url = {\"http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\"}\n", + " for url in train_url:\n", + " url_parse = urlparse(url)\n", + " # split the file name from url\n", + " file_name = os.path.join(train_path,url_parse.path.split('/')[-1])\n", + " if not os.path.exists(file_name.replace('.gz','')):\n", + " file = urllib.request.urlretrieve(url, file_name)\n", + " unzipfile(file_name)\n", + " os.remove(file_name)\n", + " for url in test_url:\n", + " url_parse = urlparse(url)\n", + " # split the file name from url\n", + " file_name = os.path.join(test_path,url_parse.path.split('/')[-1])\n", + " if not os.path.exists(file_name.replace('.gz','')):\n", + " file = urllib.request.urlretrieve(url, file_name)\n", + " unzipfile(file_name)\n", + " os.remove(file_name)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 数据预处理\n", + "\n", + "好的数据集可以有效提高训练精度和效率，在加载数据集前，会进行一些处理，增加数据的可用性和随机性。下面一段代码定义`create_dataset`函数进行数据处理操作。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import mindspore.dataset as ds\n", + "import mindspore.dataset.transforms.vision.c_transforms as CV\n", + "import mindspore.dataset.transforms.c_transforms as C\n", + "from mindspore.dataset.transforms.vision import Inter\n", + "from mindspore.common import dtype as mstype\n", + "\n", + "\n", + "def create_dataset(data_path, batch_size=32, repeat_size=1,\n", + " num_parallel_workers=1):\n", + " \"\"\"create dataset for train or test.\"\"\"\n", + " # define dataset\n", + " mnist_ds = ds.MnistDataset(data_path)\n", + "\n", + " resize_height, resize_width = 32, 32\n", + " rescale = 1.0 / 255.0\n", + " shift = 0.0\n", + " rescale_nml = 1 / 0.3081\n", + " shift_nml = -1 * 0.1307 / 0.3081\n", + "\n", + " # define map operations\n", + " resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR) # Bilinear mode\n", + " rescale_op = CV.Rescale(rescale, shift)\n", + " hwc2chw_op = CV.HWC2CHW()\n", + " type_cast_op = C.TypeCast(mstype.int32)\n", + "\n", + " # apply map operations on images\n", + " mnist_ds = mnist_ds.map(input_columns=\"label\", operations=type_cast_op, num_parallel_workers=num_parallel_workers)\n", + " mnist_ds = mnist_ds.map(input_columns=\"image\", operations=resize_op, num_parallel_workers=num_parallel_workers)\n", + " mnist_ds = mnist_ds.map(input_columns=\"image\", operations=rescale_op, num_parallel_workers=num_parallel_workers)\n", + " mnist_ds = mnist_ds.map(input_columns=\"image\", operations=hwc2chw_op, num_parallel_workers=num_parallel_workers)\n", + "\n", + " # apply DatasetOps\n", + " buffer_size = 10000\n", + " mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size) # 10000 as in LeNet train script\n", + " mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True)\n", + " mnist_ds = mnist_ds.repeat(repeat_size)\n", + "\n", + " return mnist_ds" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 网络初始化\n", + "\n", + "在进行训练之前，需定义神经网络模型，本流程采用最简单的[LeNet卷积神经网络](http://yann.lecun.com/exdb/lenet/)。\n", + "\n", + "LeNet网络不包括输入层的情况下，共有7层：2个卷积层、2个下采样层（池化层）、3个全连接层。每层都包含不同数量的训练参数。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import mindspore.nn as nn\n", + "from mindspore.common.initializer import TruncatedNormal\n", + "\n", + "\n", + "def conv(in_channels, out_channels, kernel_size, stride=1, padding=0):\n", + " \"\"\"weight initial for conv layer\"\"\"\n", + " weight = weight_variable()\n", + " return nn.Conv2d(in_channels, out_channels,\n", + " kernel_size=kernel_size, stride=stride, padding=padding,\n", + " weight_init=weight, has_bias=False, pad_mode=\"valid\")\n", + "\n", + "\n", + "def fc_with_initialize(input_channels, out_channels):\n", + " \"\"\"weight initial for fc layer\"\"\"\n", + " weight = weight_variable()\n", + " bias = weight_variable()\n", + " return nn.Dense(input_channels, out_channels, weight, bias)\n", + "\n", + "\n", + "def weight_variable():\n", + " \"\"\"weight initial\"\"\"\n", + " return TruncatedNormal(0.02)\n", + "\n", + "\n", + "class LeNet5(nn.Cell):\n", + " \"\"\"\n", + " Lenet network\n", + "\n", + " Args:\n", + " num_class (int): Num classes. Default: 10.\n", + "\n", + " Returns:\n", + " Tensor, output tensor\n", + "\n", + " \"\"\"\n", + " def __init__(self, num_class=10, channel=1):\n", + " super(LeNet5, self).__init__()\n", + " self.num_class = num_class\n", + " self.conv1 = conv(channel, 6, 5)\n", + " self.conv2 = conv(6, 16, 5)\n", + " self.fc1 = fc_with_initialize(16 * 5 * 5, 120)\n", + " self.fc2 = fc_with_initialize(120, 84)\n", + " self.fc3 = fc_with_initialize(84, self.num_class)\n", + " self.relu = nn.ReLU()\n", + " self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)\n", + " self.flatten = nn.Flatten()\n", + "\n", + " def construct(self, x):\n", + " x = self.conv1(x)\n", + " x = self.relu(x)\n", + " x = self.max_pool2d(x)\n", + " x = self.conv2(x)\n", + " x = self.relu(x)\n", + " x = self.max_pool2d(x)\n", + " x = self.flatten(x)\n", + " x = self.fc1(x)\n", + " x = self.relu(x)\n", + " x = self.fc2(x)\n", + " x = self.relu(x)\n", + " x = self.fc3(x)\n", + " return x" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 记录标量、直方图、图像\n", + "\n", + "在主程序中应用`SummaryCollector`来记录标量、直方图、图像信息。\n", + "\n", + "## 运行主程序\n", + "\n", + "在MindSpore中通过`Callback`机制提供支持快速简易地收集损失值、参数权重、梯度等信息的`Callback`, 叫做`SummaryCollector`。详细的用法可以参考API文档中`mindspore.train.callback.SummaryCollector`。 \n", + "\n", + "1. 为了记录损失值标量、直方图、图像信息，在主程序代码中需要在`specified`参数中指定需要记录的信息。\n", + "\n", + " ```python\n", + " specified={\"collect_metric\": True, \"histogram_regular\": \"^conv1.*|^conv2.*\", \"collect_input_data\": True}\n", + " ```\n", + "\n", + " 其中：\n", + " - `\"collect_metric\"`为记录损失值标量信息。\n", + " - `\"histogram_regular\"`为记录`conv1`层和`conv2`层直方图信息。\n", + " - `\"collect_input_data\"`为记录图像信息。\n", + "\n", + "2. 实例化`SummaryCollector`，并将其应用到`model.train`或者`model.eval`中。\n", + "\n", + "程序运行过程中将启动MindInsight服务并自动遍历读取当前notebook目录下`summary_dir`子目录下所有日志文件、解析进行可视化展示。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import mindspore.nn as nn\n", + "from mindspore import context\n", + "from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor\n", + "from mindspore.train import Model\n", + "from mindspore.nn.metrics import Accuracy\n", + "from mindspore.train.callback import SummaryCollector\n", + "from mindspore.train.serialization import load_checkpoint, load_param_into_net\n", + "\n", + "\n", + "if __name__ == \"__main__\":\n", + " device_target = \"GPU\"\n", + " summary_base_dir = \"./summary_dir\"\n", + " context.set_context(mode=context.GRAPH_MODE, device_target=device_target)\n", + " download_dataset()\n", + " ds_train = create_dataset(data_path=\"./MNIST_Data/train/\")\n", + " network = LeNet5()\n", + " net_loss = nn.SoftmaxCrossEntropyWithLogits(is_grad=False, sparse=True, reduction=\"mean\")\n", + " net_opt = nn.Momentum(network.trainable_params(), learning_rate=0.01, momentum=0.9)\n", + " time_cb = TimeMonitor(data_size=ds_train.get_dataset_size())\n", + " config_ck = CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10)\n", + " ckpoint_cb = ModelCheckpoint(prefix=\"checkpoint_lenet\", config=config_ck)\n", + " model = Model(network, net_loss, net_opt, metrics={\"Accuracy\": Accuracy()})\n", + " os.system(f\"mindinsight start --summary-base-dir {summary_base_dir} --port=8080\")\n", + " # Init a SummaryCollector callback instance, and use it in model.train or model.eval\n", + " specified = {\"collect_metric\": True, \"histogram_regular\": \"^conv1.*|^conv2.*\", \"collect_input_data\": True}\n", + " summary_collector = SummaryCollector(summary_dir=\"./summary_dir/summary_01\", collect_specified_data=specified, collect_freq=1, keep_default_action=False)\n", + " print(\"============== Starting Training ==============\")\n", + " # Note: dataset_sink_mode should be set to False, else you should modify collect freq in SummaryCollector\n", + " model.train(epoch=3, train_dataset=ds_train, callbacks=[time_cb, ckpoint_cb, LossMonitor(), summary_collector], dataset_sink_mode=False)\n", + " print(\"============== Starting Testing ==============\")\n", + " param_dict = load_checkpoint(\"checkpoint_lenet-3_1875.ckpt\")\n", + " load_param_into_net(network, param_dict)\n", + " ds_eval = create_dataset(\"./MNIST_Data/test/\")\n", + " acc = model.eval(ds_eval, callbacks=summary_collector, dataset_sink_mode=False)\n", + " print(\"============== {} ==============\".format(acc))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# MindInsight看板\n", + "\n", + "在本地浏览器中打开地址：`127.0.0.1:8080`，进入到可视化面板。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/mindinsight_panel.png)\n", + "\n", + "在上图所示面板中可以看到`summary_01`日志文件目录，点击训练看板进入到下图所示的训练数据展示面板，该面板展示了标量数据、直方图和图像信息，并随着训练、测试的进行实时刷新数据，实时显示训练过程参数的变化情况。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/mindinsight_panel2.png)\n", + "\n", + "## 标量可视化\n", + "\n", + "标量可视化用于展示训练过程中标量的变化趋势情况，点击打开标量信息展示面板，该面板记录了迭代计算过程中的学习率（下图左侧所示）和损失值（下图右侧所示）标量信息。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/scalar_panel.png)\n", + "\n", + "如下图的loss值标量可视化信息——标量趋势图。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/scalar.png)\n", + "\n", + "上图展示了神经网络在训练过程中loss值的变化过程。横坐标是训练步骤，纵坐标是loss值。\n", + "\n", + "图中右上角有几个按钮功能，从左到右功能分别是全屏展示，切换Y轴比例，开启/关闭框选，分步回退和还原图形。\n", + "\n", + "- 全屏展示即全屏展示该标量曲线，再点击一次即可恢复。\n", + "\n", + "- 切换Y轴比例是指可以将Y轴坐标进行对数转换。\n", + "\n", + "- 开启/关闭框选是指可以框选图中部分区域，并放大查看该区域， 可以在已放大的图形上叠加框选。\n", + "\n", + "- 分步回退是指对同一个区域连续框选并放大查看时，可以逐步撤销操作。\n", + "\n", + "- 还原图形是指进行了多次框选后，点击此按钮可以将图还原回原始状态。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/scalar_select.png)\n", + "\n", + "上图展示的标量可视化的功能区，提供了根据选择不同标签，水平轴的不同维度和平滑度来查看标量信息的功能。\n", + "\n", + "- 标签：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的标量信息。\n", + "\n", + "- 水平轴：可以选择“步骤”、“相对时间”、“绝对时间”中的任意一项，来作为标量曲线的水平轴。\n", + "\n", + "- 平滑度：可以通过调整平滑度，对标量曲线进行平滑处理。\n", + "\n", + "- 标量合成：可以选中两条标量曲线进行合成并展示在一个图中，以方便对两条曲线进行对比或者查看合成后的图。\n", + " 标量合成的功能区与标量可视化的功能区相似。其中与标量可视化功能区不一样的地方，在于标签选择时，标量合成功能最多只能同时选择两个标签，将其曲线合成并展示。\n", + "\n", + "## 直方图可视化\n", + "\n", + "\n", + "直方图用于将用户所指定的张量以直方图的形式展示。点击打开直方图展示面板，以直方图的形式记录了在迭代过程中所有层参数分布信息。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_panel.png)\n", + "\n", + "如下图为`conv1`层参数分布信息，点击图中右上角，可以将图放大。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram.png)\n", + "\n", + "上图展示直方图的功能区，包含以下内容：\n", + "\n", + "- 标签选择：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的直方图。\n", + "\n", + "- 纵轴：可以选择步骤、相对时间、绝对时间中的任意一项，来作为直方图纵轴显示的数据。\n", + "\n", + "- 视角：可以选择正视和俯视中的一种。正视是指从正面的角度查看直方图，此时不同步骤之间的数据会覆盖在一起。俯视是指偏移以45度角俯视直方图区域，这时可以呈现不同步骤之间数据的差异。\n", + "\n", + "## 图像可视化\n", + "\n", + "图像可视化用于展示用户所指定的图片。点击图像展示面板，展示了每个step进行处理的图像信息。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_panel.png)\n", + "\n", + "下图为展示`summary_01`记录的图像信息。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_vi.png)\n", + "\n", + "通过滑动上图中的\"步骤\"滑条，查看不同步骤的图片。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_function.png)\n", + "\n", + "上图展示图像可视化的功能区，提供了选择查看不同标签，不同亮度和不同对比度来查看图片信息。\n", + "\n", + "- 标签：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的图片信息。\n", + "\n", + "- 亮度调整：可以调整所展示的所有图片亮度。\n", + "\n", + "- 对比度调整：可以调整所展示的所有图片对比度。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 对比看板\n", + "\n", + "对比看板可视用于多个训练之间的标量数据对比，为了展示对比看板，执行以下代码，在可视化面板中可以得到`summary_02`日志记录信息。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import mindspore.nn as nn\n", + "from mindspore import context\n", + "from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor\n", + "from mindspore.train import Model\n", + "from mindspore.nn.metrics import Accuracy\n", + "from mindspore.train.callback import SummaryCollector\n", + "from mindspore.train.serialization import load_checkpoint, load_param_into_net\n", + "\n", + "\n", + "if __name__ == \"__main__\":\n", + " device_target = \"GPU\"\n", + " context.set_context(mode=context.GRAPH_MODE, device_target=device_target)\n", + " download_dataset()\n", + " ds_train = create_dataset(data_path=\"./MNIST_Data/train/\")\n", + " network = LeNet5()\n", + " net_loss = nn.SoftmaxCrossEntropyWithLogits(is_grad=False, sparse=True, reduction=\"mean\")\n", + " net_opt = nn.Momentum(network.trainable_params(), learning_rate=0.01, momentum=0.9)\n", + " time_cb = TimeMonitor(data_size=ds_train.get_dataset_size())\n", + " config_ck = CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10)\n", + " ckpoint_cb = ModelCheckpoint(prefix=\"checkpoint_lenet\", config=config_ck)\n", + " model = Model(network, net_loss, net_opt, metrics={\"Accuracy\": Accuracy()})\n", + " # Init a SummaryCollector callback instance, and use it in model.train or model.eval\n", + " specified = {\"collect_metric\": True, \"histogram_regular\": \"^conv1.*|^conv2.*\", \"collect_input_data\": True}\n", + " summary_collector = SummaryCollector(summary_dir=\"./summary_dir/summary_02\", collect_specified_data=specified, collect_freq=1, keep_default_action=False)\n", + " print(\"============== Starting Training ==============\")\n", + " # Note: dataset_sink_mode should be set to False, else you should modify collect freq in SummaryCollector\n", + " model.train(epoch=3, train_dataset=ds_train, callbacks=[time_cb, ckpoint_cb, LossMonitor(), summary_collector], dataset_sink_mode=False)\n", + " print(\"============== Starting Testing ==============\")\n", + " param_dict = load_checkpoint(\"checkpoint_lenet_1-3_1875.ckpt\")\n", + " load_param_into_net(network, param_dict)\n", + " ds_eval = create_dataset(\"./MNIST_Data/test/\")\n", + " acc = model.eval(ds_eval, callbacks=summary_collector, dataset_sink_mode=False)\n", + " print(\"============== {} ==============\".format(acc))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "打开对比看板，可以得到`summary_01`和`summary_02`标量对比信息。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/multi_scalars.png)\n", + "\n", + "上图展示了多个训练之间的标量曲线对比效果，横坐标是训练步骤，纵坐标是标量值。\n", + "\n", + "图中右上角有几个按钮功能，从左到右功能分别是全屏展示，切换Y轴比例，开启/关闭框选，分步回退和还原图形。\n", + "\n", + "- 全屏展示即全屏展示该标量曲线，再点击一次即可恢复。\n", + "\n", + "- 切换Y轴比例是指可以将Y轴坐标进行对数转换。\n", + "\n", + "- 开启/关闭框选是指可以框选图中部分区域，并放大查看该区域， 可以在已放大的图形上叠加框选。\n", + "\n", + "- 分步回退是指对同一个区域连续框选并放大查看时，可以逐步撤销操作。\n", + "\n", + "- 还原图形是指进行了多次框选后，点击此按钮可以将图还原回原始状态。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/multi_scalars_select.png)\n", + "\n", + "上图展示的对比看板可视的功能区，提供了根据选择不同训练或标签，水平轴的不同维度和平滑度来进行标量对比的功能。\n", + "\n", + "- 训练: 提供了对所有训练进行多项选择的功能，用户可以通过勾选或关键字筛选所需的训练。\n", + "\n", + "- 标签：提供了对所有标签进行多项选择的功能，用户可以通过勾选所需的标签，查看对应的标量信息。\n", + "\n", + "- 水平轴：可以选择“步骤”、“相对时间”、“绝对时间”中的任意一项，来作为标量曲线的水平轴。\n", + "\n", + "- 平滑度：可以通过调整平滑度，对标量曲线进行平滑处理。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 单独记录数据\n", + "\n", + "以上流程为整体展示Summary算子能记录到的所有数据，也可以单独记录关心的数据，以降低性能开销和日志文件大小。\n", + "\n", + "> 为了展示运行的效果，进行以下每个步骤之前先删除当前notebook根目录下的`summary_dir/summary_02`目录，配置完`specified`参数后执行[**对比看板**](#对比看板)中的代码。\n", + "\n", + "## 单独记录损失值标量\n", + "\n", + "在主程序中配置`specified`参数为：\n", + "\n", + "```python\n", + "specified={\"collect_metric\": True}\n", + "```\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/loss_scalar_only.png)\n", + "\n", + "在MindInsight面板中，如上图所示，只展示了损失值标量信息。\n", + "\n", + "## 单独记录参数分布直方图\n", + "\n", + "在主程序中配置`specified`参数为：\n", + "\n", + "```python\n", + "specified = {\"histogram_regular\": \"^conv1.*|^conv2.*|fc1.*|fc2.*|fc3.*\"}\n", + "```\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_only.png)\n", + "\n", + "在MindInsight面板中，如上图所示，只展示了参数直方图信息。\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_only_all.png)\n", + "\n", + "点击进入直方图面板，如上图所示，展示了`conv1`、`conv2`、`fc1`、`fc2`、`fc3`等各层的权重、参数信息。\n", + "\n", + "## 单独记录图像\n", + "\n", + "在主程序中配置`specified`参数为：\n", + "\n", + "```python\n", + "specified = {\"collect_input_data\": True}\n", + "```\n", + "\n", + "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_only.png)\n", + "\n", + "在MindInsight面板中，如上图所示，只展示了输入图像信息。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 关闭MindInsight服务\n", + "\n", + "执行以下代码关闭MindInsight服务。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "\n", + "os.system(\"mindinsight stop --port 8080\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 注意事项和规格\n", + "\n", + "- 为了控制列出summary列表的用时，MindInsight最多支持发现999个summary列表条目。\n", + "\n", + "- 为了控制内存占用，MindInsight对标签（tag）数目和步骤（step）数目进行了限制：\n", + "\n", + " - 每个训练看板的最大标签数量为300个标签。标量标签、图片标签、计算图标签、参数分布图（直方图）标签的数量总和不得超过300个。特别地，每个训练看板最多有10个计算图标签。当实际标签数量超过这一限制时，将依照MindInsight的处理顺序，保留最近处理的300个标签。\n", + "\n", + " - 每个训练看板的每个标量标签最多有1000个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。\n", + "\n", + " - 每个训练看板的每个图片标签最多有10个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。\n", + " \n", + " - 每个训练看板的每个参数分布图（直方图）标签最多有50个步骤的数据。当实际步骤的数目超过这一限制时，将对数据进行随机采样，以满足这一限制。\n", + " \n", + "- 出于性能上的考虑，MindInsight对比看板使用缓存机制加载训练的标量曲线数据，并进行以下限制：\n", + " \n", + " - 对比看板只支持在缓存中的训练进行比较标量曲线对比。\n", + " \n", + " - 缓存最多保留最新（按修改时间排列）的15个训练。\n", + " \n", + " - 用户最多同时对比5个训练的标量曲线。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.5" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb b/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..2d98c64623b94002380bfa9bce94aa646d43704f --- /dev/null +++ b/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb @@ -0,0 +1,713 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#
MindInsight的模型溯源和数据溯源体验" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 概述\n", + "在AI训练的过程中，面对陌生的神经网络训练，经常需要事先优化神经网络训练中的参数，毕竟在训练一个十分复杂的神经网络时，有时候需要花费少则几天多则几周甚至更多的时间，为了更好的管理、调试和优化神经网络的训练过程，我们需要一个工具来对训练过程中的计算图、各种指标随着时间的变化趋势以及训练中使用到的图像信息进行分析和记录工作，而MindSpore就提供了一个对用户十分易用友好的可视化工具MindInsight，赋能给用户进行数据溯源和模型溯源的可视化分析，能明显提升用户对网络搭建过程和数据增强过程的纠错调优能力。而本次体验会从MindInsight的数据记录，可视化效果，如何方便用户在模型调优，数据调优上做一次整体流程的体验。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "下面按照MindSpore的训练数据模型的正常步骤进行，当使用到MindInsight或者`SummaryCollector`算子进行数据保存操作时，会增加相应的说明，本次体验的整体流程如下：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1、数据集的准备，这里使用的是MNIST数据集。\n", + "\n", + "2、构建一个网络，这里使用LeNet网络。(此处将使用第二种记录方式`ImageSummary`)。\n", + "\n", + "3、训练网络和测试网络的搭建及运行。（此处将操作`SummaryCollector`初始化，并记录模型训练和模型测试相关信息）。\n", + "\n", + "4、启动MindInsight服务。\n", + "\n", + "5、模型溯源的使用。调整模型参数多次存储数据，并使用MindInsight的模型溯源功能对不同优化参数下训练产生的模型作对比，了解MindSpore中的各类优化对训练过程的影响及如何调优训练过程。\n", + "\n", + "6、数据溯源的使用。调整数据参数多次存储数据，并使用MindInsight的数据溯源功能对不同数据集下训练产生的模型进行对比分析，了解如何调优。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "本次体验将使用快速入门案例作为基础用例，将MindInsight的模型溯源和数据溯源的数据记录功能加入到案例中，快速入门案例的源码请参考：https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/lenet.py 。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 一、训练的数据集下载" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 1、数据集准备" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 方法一：\n", + "从以下网址下载，并将数据包解压缩后放至Jupyter的工作目录下：
训练数据集：{\"http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\"}\n", + "
测试数据集：{\"http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\"}
我们用下面代码查询jupyter的工作目录。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "os.getcwd()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "训练数据集放在----`Jupyter工作目录+\\MNIST_Data\\train\\`，此时`train`文件夹内应该包含两个文件，`train-images-idx3-ubyte`和`train-labels-idx1-ubyte`
测试数据集放在----`Jupyter工作目录+\\MNIST_Data\\test\\`，此时`test`文件夹内应该包含两个文件，`t10k-images-idx3-ubyte`和`t10k-labels-idx1-ubyte`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 方法二：\n", + "直接执行下面代码，会自动进行训练集的下载与解压，但是整个过程根据网络好坏情况会需要花费几分钟时间。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Network request module, data download module, decompression module\n", + "import urllib.request \n", + "from urllib.parse import urlparse\n", + "import gzip \n", + "\n", + "def unzipfile(gzip_path):\n", + " \"\"\"unzip dataset file\n", + " Args:\n", + " gzip_path: dataset file path\n", + " \"\"\"\n", + " open_file = open(gzip_path.replace('.gz',''), 'wb')\n", + " gz_file = gzip.GzipFile(gzip_path)\n", + " open_file.write(gz_file.read())\n", + " gz_file.close()\n", + " \n", + "def download_dataset():\n", + " \"\"\"Download the dataset from http://yann.lecun.com/exdb/mnist/.\"\"\"\n", + " print(\"******Downloading the MNIST dataset******\")\n", + " train_path = \"./MNIST_Data/train/\" \n", + " test_path = \"./MNIST_Data/test/\"\n", + " train_path_check = os.path.exists(train_path)\n", + " test_path_check = os.path.exists(test_path)\n", + " if train_path_check == False and test_path_check == False:\n", + " os.makedirs(train_path)\n", + " os.makedirs(test_path)\n", + " train_url = {\"http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\"}\n", + " test_url = {\"http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\"}\n", + " \n", + " for url in train_url:\n", + " url_parse = urlparse(url)\n", + " # split the file name from url\n", + " file_name = os.path.join(train_path,url_parse.path.split('/')[-1])\n", + " if not os.path.exists(file_name.replace('.gz', '')):\n", + " file = urllib.request.urlretrieve(url, file_name)\n", + " unzipfile(file_name)\n", + " os.remove(file_name)\n", + " \n", + " for url in test_url:\n", + " url_parse = urlparse(url)\n", + " # split the file name from url\n", + " file_name = os.path.join(test_path,url_parse.path.split('/')[-1])\n", + " if not os.path.exists(file_name.replace('.gz', '')):\n", + " file = urllib.request.urlretrieve(url, file_name)\n", + " unzipfile(file_name)\n", + " os.remove(file_name)\n", + "\n", + "download_dataset()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "这样就完成了数据集的下载解压缩工作。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 2、数据集处理" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "数据集处理对于训练非常重要，好的数据集可以有效提高训练精度和效率。在加载数据集前，我们通常会对数据集进行一些处理。\n", + "
我们定义一个函数`create_dataset`来创建数据集。在这个函数中，我们定义好需要进行的数据增强和处理操作：\n", + "
1、定义数据集。\n", + "
2、定义进行数据增强和处理所需要的一些参数。\n", + "
3、根据参数，生成对应的数据增强操作。\n", + "
4、使用`map`映射函数，将数据操作应用到数据集。\n", + "
5、对生成的数据集进行处理。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "> 具体的数据集操作可以在MindInsight的数据溯源中进行可视化分析。另外提取图像需要将`normalize`算子的数据处理(`CV.Rescale`)操作取消，否则取出来的图像为全黑图像。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import mindspore.dataset.transforms.vision.c_transforms as CV\n", + "import mindspore.dataset.transforms.c_transforms as C\n", + "from mindspore.dataset.transforms.vision import Inter\n", + "from mindspore.common import dtype as mstype\n", + "import mindspore.dataset as ds\n", + "\n", + "def create_dataset(data_path, batch_size=32, repeat_size=1,\n", + " num_parallel_workers=1):\n", + " \"\"\" create dataset for train or test\n", + " Args:\n", + " data_path: Data path\n", + " batch_size: The number of data records in each group\n", + " repeat_size: The number of replicated data records\n", + " num_parallel_workers: The number of parallel workers\n", + " \"\"\"\n", + " # define dataset\n", + " mnist_ds = ds.MnistDataset(data_path)\n", + "\n", + " # Define some parameters needed for data enhancement and rough justification\n", + " resize_height, resize_width = 32, 32\n", + " rescale = 1.0 / 255.0\n", + " shift = 0.0\n", + " rescale_nml = 1 / 0.3081\n", + " shift_nml = -1 * 0.1307 / 0.3081\n", + "\n", + " # According to the parameters, generate the corresponding data enhancement method\n", + " resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR) # Resize images to (32, 32) by bilinear interpolation\n", + " rescale_nml_op = CV.Rescale(rescale_nml, shift_nml) # normalize images\n", + " rescale_op = CV.Rescale(rescale, shift) # rescale images\n", + " hwc2chw_op = CV.HWC2CHW() # change shape from (height, width, channel) to (channel, height, width) to fit network.\n", + " type_cast_op = C.TypeCast(mstype.int32) # change data type of label to int32 to fit network\n", + "\n", + " # Using map() to apply operations to a dataset\n", + " mnist_ds = mnist_ds.map(input_columns=\"label\", operations=type_cast_op, num_parallel_workers=num_parallel_workers)\n", + " mnist_ds = mnist_ds.map(input_columns=\"image\", operations=resize_op, num_parallel_workers=num_parallel_workers)\n", + " mnist_ds = mnist_ds.map(input_columns=\"image\", operations=rescale_op, num_parallel_workers=num_parallel_workers)\n", + " mnist_ds = mnist_ds.map(input_columns=\"image\", operations=rescale_nml_op, num_parallel_workers=num_parallel_workers)\n", + " mnist_ds = mnist_ds.map(input_columns=\"image\", operations=hwc2chw_op, num_parallel_workers=num_parallel_workers)\n", + " \n", + " # Process the generated dataset\n", + " buffer_size = 10000\n", + " mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size) # 10000 as in LeNet train script\n", + " mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True)\n", + " mnist_ds = mnist_ds.repeat(repeat_size)\n", + "\n", + " return mnist_ds" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 二、构建LeNet5网络" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 使用ImageSummary记录图像数据" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在构建LeNet5网络的`__init__`中，初始化`ImageSummary`算子，同时在`construct`中将`ImageSummary`放在第一步，其第一个参数`image`为抽取出来的图片的自定义命名，第二个参数`x`是图像数据。此方法与`SummaryCollector`抽取图像的方法不冲突，可以同时使用。" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from mindspore.ops import operations as P\n", + "import mindspore.nn as nn\n", + "from mindspore.common.initializer import TruncatedNormal\n", + "\n", + "# Initialize 2D convolution function\n", + "def conv(in_channels, out_channels, kernel_size, stride=1, padding=0):\n", + " \"\"\"Conv layer weight initial.\"\"\"\n", + " weight = weight_variable()\n", + " return nn.Conv2d(in_channels, out_channels,\n", + " kernel_size=kernel_size, stride=stride, padding=padding,\n", + " weight_init=weight, has_bias=False, pad_mode=\"valid\")\n", + "\n", + "# Initialize full connection layer\n", + "def fc_with_initialize(input_channels, out_channels):\n", + " \"\"\"Fc layer weight initial.\"\"\"\n", + " weight = weight_variable()\n", + " bias = weight_variable()\n", + " return nn.Dense(input_channels, out_channels, weight, bias)\n", + "\n", + "# Set truncated normal distribution\n", + "def weight_variable():\n", + " \"\"\"Weight initial.\"\"\"\n", + " return TruncatedNormal(0.02)\n", + "\n", + "class LeNet5(nn.Cell):\n", + " \"\"\"Lenet network structure.\"\"\"\n", + " # define the operator required\n", + " def __init__(self):\n", + " super(LeNet5, self).__init__()\n", + " self.batch_size = 32 # 32 pictures in each group\n", + " self.conv1 = conv(1, 6, 5) # Convolution layer 1, 1 channel input (1 Figure), 6 channel output (6 figures), convolution core 5 * 5\n", + " self.conv2 = conv(6, 16, 5) # Convolution layer 2,6-channel input, 16 channel output, convolution kernel 5 * 5\n", + " self.fc1 = fc_with_initialize(16 * 5 * 5, 120)\n", + " self.fc2 = fc_with_initialize(120, 84)\n", + " self.fc3 = fc_with_initialize(84, 10)\n", + " self.relu = nn.ReLU()\n", + " self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)\n", + " self.flatten = nn.Flatten()\n", + " #Init ImageSummary\n", + " self.sm_image = P.ImageSummary()\n", + "\n", + " # use the preceding operators to construct networks\n", + " def construct(self, x):\n", + " self.sm_image(\"image\",x)\n", + " x = self.conv1(x) # 1*32*32-->6*28*28\n", + " x = self.relu(x) # 6*28*28-->6*14*14\n", + " x = self.max_pool2d(x) # Pool layer\n", + " x = self.conv2(x) # Convolution layer\n", + " x = self.relu(x) # Function excitation layer\n", + " x = self.max_pool2d(x) # Pool layer\n", + " x = self.flatten(x) # Dimensionality reduction\n", + " x = self.fc1(x) # Full connection\n", + " x = self.relu(x) # Function excitation layer\n", + " x = self.fc2(x) # Full connection\n", + " x = self.relu(x) # Function excitation layer\n", + " x = self.fc3(x) # Full connection\n", + " return x" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 三、训练网络和测试网络构建" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 1、使用SummaryCollector放入到训练网络中记录训练数据" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`summary_callback`，即是`SummaryCollector`，在`model.train`的回调函数中使用，可以记录训练数据溯源和模型溯源信息。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Training and testing related modules\n", + "from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, SummaryCollector, Callback\n", + "from mindspore.train import Model\n", + "import os\n", + "\n", + "\n", + "def train_net(model, epoch_size, mnist_path, repeat_size, ckpoint_cb, summary_collector):\n", + " \"\"\"Define the training method.\"\"\"\n", + " print(\"============== Starting Training ==============\")\n", + " # load training dataset\n", + " ds_train = create_dataset(os.path.join(mnist_path, \"train\"), 32, repeat_size)\n", + " model.train(epoch_size, ds_train, callbacks=[ckpoint_cb, LossMonitor(), summary_collector], dataset_sink_mode=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 2、使用SummaryCollector放入到测试网络中记录测试数据" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`summary_callback`，即是`SummaryCollector`，在`model.eval`的回调函数中使用，可以记录训练精度信息和测试样本数量信息。" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from mindspore.train.serialization import load_checkpoint, load_param_into_net\n", + "\n", + "def test_net(network, model, mnist_path, summary_collector):\n", + " \"\"\"Define the evaluation method.\"\"\"\n", + " print(\"============== Starting Testing ==============\")\n", + " # load the saved model for evaluation\n", + " param_dict = load_checkpoint(\"checkpoint_lenet-3_1875.ckpt\")\n", + " # load parameter to the network\n", + " load_param_into_net(network, param_dict)\n", + " # load testing dataset\n", + " ds_eval = create_dataset(os.path.join(mnist_path, \"test\"))\n", + " acc = model.eval(ds_eval, callbacks=[summary_collector], dataset_sink_mode=True)\n", + " print(\"============== Accuracy:{} ==============\".format(acc))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 3、主程序运行入口" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "初始化`SummaryCollector`，使用`collect_specified_data`控制需要记录的数据，我们这里只需要记录模型溯源和数据溯源，所以将`collect_train_lineage`和`collect_eval_lineage`参数设置成`True`,其他的参数使用`keep_default_action`设置成`False`，SummaryCollector能够记录哪些数据，请参考官网：https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.train.html?highlight=collector#mindspore.train.callback.SummaryCollector 。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from mindspore.train.callback import SummaryCollector\n", + "from mindspore.train.summary.summary_record import SummaryRecord\n", + "from mindspore.nn.metrics import Accuracy\n", + "from mindspore import context\n", + "from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits\n", + "\n", + "if __name__==\"__main__\":\n", + " context.set_context(mode=context.GRAPH_MODE, device_target = \"GPU\")\n", + " lr = 0.01 # learning rate\n", + " momentum = 0.9 \n", + " epoch_size = 3\n", + " mnist_path = \"./MNIST_Data\"\n", + " \n", + " net_loss = SoftmaxCrossEntropyWithLogits(is_grad=False, sparse=True, reduction='mean')\n", + " repeat_size = epoch_size\n", + " # create the network\n", + " network = LeNet5()\n", + "\n", + " # define the optimizer\n", + " net_opt = nn.Momentum(network.trainable_params(), lr, momentum)\n", + " config_ck = CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10)\n", + " ckpoint_cb = ModelCheckpoint(prefix=\"checkpoint_lenet\", config=config_ck)\n", + " model = Model(network, net_loss, net_opt, metrics={\"Accuracy\": Accuracy()})\n", + " \n", + " collect_specified_data = {\"collect_eval_lineage\":True,\"collect_train_lineage\":True}\n", + " summary_collector = SummaryCollector(summary_dir=\"./summary_base/quick_start_summary01\", collect_specified_data=collect_specified_data, keep_default_action=False) \n", + " train_net(model, epoch_size, mnist_path, repeat_size, ckpoint_cb, summary_collector)\n", + " test_net(network, model, mnist_path, summary_collector)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 四、启动及关闭MindInsight服务" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "这里主要展示如何启用及关闭MindInsight，更多的命令集信息，请参考MindSpore官方网站：https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/visualization_tutorials.html 。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "- 启动MindInsight服务\n", + "\n", + " 在安装过MindInsight的环境中启动MindInsight服务：\n", + " - `--summary-base-dir`：MindInsight指定启动工作路径的命令。\n", + " - `./summary_base`：SummaryRecord保存文件夹的目录。\n", + " - `--port`：MindInsight指定启动的端口，数值可以任意为1~65535的范围内。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "os.system(\"mindinsight start --summary-base-dir=./summary_base --port=8090\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "查询是否启动成功，在网址输入:`127.0.0.1:8090`，如果看到如下界面说明启动成功。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/summary_list.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "- 关闭MindInsight服务\n", + "\n", + " 在安装过MindInsight的环境中输入命令：`mindinsight stop --port=8090`\n", + " - `mindinsight stop`：MindInsight关闭服务命令。\n", + " - `--port=8090`：即MindInsight服务开启在`8090`端口，所以这里写成`--port=8090`。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 五、模型溯源" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 1、连接到模型溯源地址" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "浏览器中输入:`127.0.0.1:8090`，点击模型溯源，如下模型溯源界面：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/model_lineage_all.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "我们可以勾选展示列，由于训练过程涉及的参数很多，在调整训练参数时，一般只会调整少量参数，所以对大部分相同参数可以去掉勾选，不显示出来，使得用户更方便的观察不同参数对模型训练的影响，上图中的不同参数的竖直线段代表的各个参数，数根连接各个参数的折线图代表不同的模型训练过程，其中各参数从左到右如下：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "- `summary路径`：表示存储记录数据的文件夹路径，即`summary_dir`。\n", + "- `Accuracy`：模型的精度值。\n", + "- `loss`：模型的loss值。\n", + "- 网络：表示神经网络名称（用户可自行命名）。\n", + "- 优化器：表示训练过程中采用的优化器。\n", + "- 训练样本数量：训练样本数量。\n", + "- 测试样本数量：测试样本数量。\n", + "- 学习率：learning_rate的值。\n", + "- `epoch`：训练圈数。\n", + "- `steps`：训练步数。\n", + "- device数目：启用的训练卡数目。\n", + "- 模型大小：生成的模型文件`.ckpt`的大小。\n", + "- 损失函数：表示训练过程中采用的损失函数（用户可自行命名）。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "根据上述记录的信息，我们可以调整模型训练过程中的参数，训练生成模型，然后选择要对比的训练，进行比对观察分析。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 2、观察分析记录下来的溯源参数" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "下图选择了数条不同参数下训练生成的模型进行对比：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/model_lineage_cp.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在这几次训练的参数中，优化器，epoch和学习率都不一致，可以看到不同的训练生成的模型精度`Accuracy`和loss值是不一致的，当然最好是调整单个参数来观察对模型生成的影响，避免多重因素干扰，难以分辨哪个参数是正影响，哪个参数是负影响。这需要我们调整不同的参数，多训练几遍生成模型，分析各参数对训练产生的影响，这对前期学习AI训练时很有帮助。在以后应对复杂训练时，可以节省不少时间。\n", + "> 在多次训练时，需要将`summary_dir`的保存路径的文件夹进行重命名操作，否则训练记录的数据会生成在同一个文件夹下，而在同一文件夹下MindInsight只会读取最后一位数字比较大的文件即最后生成的文件。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 六、数据溯源" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 1、连接到数据溯源地址" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "浏览器中输入：127.0.0.1:8090连接上MindInsight的服务，点击模型溯源，如下图数据溯源界面：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/data_lineage.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "数据溯源的根本是重现数据集从左到右进行数据增强的整个过程，方便自己发现增强过程中是否有遗漏的步骤或者不合理的操作，方便自己查找错误，也方便自己找到最优的数据增强方式，毕竟一个好的数据集对模型训练是有事半功倍的效果的。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "- `summary路径`：表示存储记录数据的文件夹名称，即为`SummaryRecord`的路径下的文件夹名称。\n", + "- `MnistDataset`：表示数据集信息，包含数据集路径。\n", + "- `Map_TypeCast`：定义数据集的类型。\n", + "- `Map_Resize`：图像缩放后的尺寸。\n", + "- `Map_Rescale`：图像的缩放比例。\n", + "- `Map_HWC2CHW`：数据集的张量由：高×宽×通道-->通道×高×宽。\n", + "- `Shuffle`：数据集混洗的缓存空间。\n", + "- `Batch`：每组训练样本数量。\n", + "- `Repeat`：数据图片复制次数，用于增强数据的数量。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 2、观察分析数据溯源参数" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "可以从上图看到数据增强过程由原数据集MnistDataset开始，按照先后顺序经过了下面的操作：label的数据类型转换（`Map_Typecast`），图像的高宽缩放（`Map_Resize`），图像的比例缩放（`Map_Rescale`），图像数据的张量变换（`Map_HWC2CHW`），图像混洗（`Shuffle`），图像成组（`Batch`），图像数量增强（`Repeat`）然后输出训练需要的数据。显然这样的可视化的数据溯源功能，在你检查数据增强操作是否有误的时候，比起一行行的去检查代码效率多了。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 最后关闭MindInsight服务" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "os.system(\"mindinsight stop --port=8090\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "以上就是这次对MindInsight的使用方法，模型溯源和数据溯源的全部过程。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/tutorials/notebook/quick_start.ipynb b/tutorials/notebook/quick_start.ipynb index da57dbd4aa139f5e2d60a607f98a96e8e2034dfd..31fec669e4d778ced5c9901f307101763a6a3016 100644 --- a/tutorials/notebook/quick_start.ipynb +++ b/tutorials/notebook/quick_start.ipynb @@ -34,7 +34,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "说明：
你可以在这里找到完整可运行的样例代码：https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/lenet.py" + "说明：
你可以在这里找到完整可运行的样例代码：https://gitee.com/mindspore/docs/blob/r0.5/tutorials/tutorial_code/lenet.py" ] }, { diff --git a/tutorials/source_en/advanced_use/checkpoint_for_hybrid_parallel.md b/tutorials/source_en/advanced_use/checkpoint_for_hybrid_parallel.md index b30c0fe5a900e0021e8a21b40930e8f6eaa4ca90..0c0dea1397fa137a31521df6c5c999fc120b1d91 100644 --- a/tutorials/source_en/advanced_use/checkpoint_for_hybrid_parallel.md +++ b/tutorials/source_en/advanced_use/checkpoint_for_hybrid_parallel.md @@ -313,7 +313,7 @@ User process: 3. Execute stage 2 training: There are two devices in stage 2 training environment. The weight shape of the MatMul operator on each device is \[4, 8]. Load the initialized model parameter data from the integrated checkpoint file and then perform training. -> For details about the distributed environment configuration and training code, see [Distributed Training](https://www.mindspore.cn/tutorial/en/master/advanced_use/distributed_training.html). +> For details about the distributed environment configuration and training code, see [Distributed Training](https://www.mindspore.cn/tutorial/en/r0.5/advanced_use/distributed_training.html). > > This document provides the example code for integrating checkpoint files and loading checkpoint files before distributed training. The code is for reference only. diff --git a/tutorials/source_en/advanced_use/computer_vision_application.md b/tutorials/source_en/advanced_use/computer_vision_application.md index be0522cd85e7f938dce1dc0ec88fda95a0bd8b55..70fbc929a513e1f85da7b5676080637479361827 100644 --- a/tutorials/source_en/advanced_use/computer_vision_application.md +++ b/tutorials/source_en/advanced_use/computer_vision_application.md @@ -16,7 +16,7 @@ - + ## Overview @@ -64,7 +64,7 @@ Next, let's use MindSpore to solve the image classification task. The overall pr 5. Call the high-level `Model` API to train and save the model file. 6. Load the saved model for inference. -> This example is for the hardware platform of the Ascend 910 AI processor. You can find the complete executable sample code at： . +> This example is for the hardware platform of the Ascend 910 AI processor. You can find the complete executable sample code at： . The key parts of the task process code are explained below. diff --git a/tutorials/source_en/advanced_use/customized_debugging_information.md b/tutorials/source_en/advanced_use/customized_debugging_information.md index d7721e19b3956762b5d679dff41282949ce51f36..3e18520988367531452729ee69abfd7a74744cce 100644 --- a/tutorials/source_en/advanced_use/customized_debugging_information.md +++ b/tutorials/source_en/advanced_use/customized_debugging_information.md @@ -14,7 +14,7 @@ - + ## Overview diff --git a/tutorials/source_en/advanced_use/dashboard_and_lineage.md b/tutorials/source_en/advanced_use/dashboard_and_lineage.md index 4ef01904e3a473e9a41bed4149ebb5b1ee7ac78b..8fc357c4b43e562ec08bcb65450096abbe99ea28 100644 --- a/tutorials/source_en/advanced_use/dashboard_and_lineage.md +++ b/tutorials/source_en/advanced_use/dashboard_and_lineage.md @@ -21,7 +21,7 @@ - + ## Overview Scalars, images, computational graphs, and model hyperparameters during training are recorded in files and can be viewed on the web page. diff --git a/tutorials/source_en/advanced_use/debugging_in_pynative_mode.md b/tutorials/source_en/advanced_use/debugging_in_pynative_mode.md index 1142684a4ef8d9f5b052995d07280ac6c9c7d00d..569b3a8efd3f4461b0a330fc2a47235965070689 100644 --- a/tutorials/source_en/advanced_use/debugging_in_pynative_mode.md +++ b/tutorials/source_en/advanced_use/debugging_in_pynative_mode.md @@ -11,7 +11,7 @@ - + ## Overview diff --git a/tutorials/source_en/advanced_use/differential_privacy.md b/tutorials/source_en/advanced_use/differential_privacy.md index 50717af4aa584d0f93e7b5a8d2f0dd41ccbaf8de..20c1383d744781679816fbbfbf9ec56eea81d4cc 100644 --- a/tutorials/source_en/advanced_use/differential_privacy.md +++ b/tutorials/source_en/advanced_use/differential_privacy.md @@ -1,4 +1,4 @@ -# Differential Privacy in Machine Learning +# Differential Privacy in Machine Learning @@ -14,7 +14,7 @@ - + ## Overview @@ -43,7 +43,7 @@ MindArmour differential privacy module Differential-Privacy implements the diffe The LeNet model and MNIST dataset are used as an example to describe how to use the differential privacy optimizer to train a neural network model on MindSpore. -> This example is for the Ascend 910 AI processor and supports PYNATIVE_MODE. You can download the complete sample code from . +> This example is for the Ascend 910 AI processor and supports PYNATIVE_MODE. You can download the complete sample code from . ## Implementation diff --git a/tutorials/source_en/advanced_use/distributed_training.md b/tutorials/source_en/advanced_use/distributed_training.md index 20cd61d6253bb7392aa9e8c422aeedadfb102ee4..eb249ee93c3cad904e5a316c36903ddc9d506bd6 100644 --- a/tutorials/source_en/advanced_use/distributed_training.md +++ b/tutorials/source_en/advanced_use/distributed_training.md @@ -18,7 +18,7 @@ - + ## Overview In deep learning, the increasing number of datasets and parameters prolongs the training time and requires more hardware resources, becoming a training bottleneck. Parallel distributed training is an important optimization method for training, which can reduce requirements on hardware, such as memory and computing performance. Based on different parallel principles and modes, parallelism is generally classified into the following types: @@ -34,7 +34,7 @@ MindSpore also provides the parallel distributed training function. It supports This tutorial describes how to train the ResNet-50 network in data parallel and automatic parallel modes on MindSpore. > The example in this tutorial applies to hardware platforms based on the Ascend 910 AI processor, whereas does not support CPU and GPU scenarios. -> Download address of the complete sample code: +> Download address of the complete sample code: ## Preparations @@ -177,7 +177,7 @@ Different from the single-node system, the multi-node system needs to transfer t ## Defining the Network -In data parallel and automatic parallel modes, the network definition method is the same as that in a single-node system. The reference code is as follows: +In data parallel and automatic parallel modes, the network definition method is the same as that in a single-node system. The reference code is as follows: ## Defining the Loss Function and Optimizer diff --git a/tutorials/source_en/advanced_use/images/data_op_profile.png b/tutorials/source_en/advanced_use/images/data_op_profile.png index 6a03815bac3797b1333050e6eae1c89950e01a1c..b83408e92777181f6447ec20239fc92e28084a6a 100644 Binary files a/tutorials/source_en/advanced_use/images/data_op_profile.png and b/tutorials/source_en/advanced_use/images/data_op_profile.png differ diff --git a/tutorials/source_en/advanced_use/images/minddata_profile.png b/tutorials/source_en/advanced_use/images/minddata_profile.png index 035939f5e3d548f39e2f5c6c16b2bc7d0c7469ce..79dfad25e6828769a2efc697bb7b02a171dbbdd0 100644 Binary files a/tutorials/source_en/advanced_use/images/minddata_profile.png and b/tutorials/source_en/advanced_use/images/minddata_profile.png differ diff --git a/tutorials/source_en/advanced_use/images/performance_overall.png b/tutorials/source_en/advanced_use/images/performance_overall.png index 2d627f972cac0b7848eff1114b0fd2fa4f030e74..3aa536d5f24fc348ad013fa07084fddb1b4f01af 100644 Binary files a/tutorials/source_en/advanced_use/images/performance_overall.png and b/tutorials/source_en/advanced_use/images/performance_overall.png differ diff --git a/tutorials/source_en/advanced_use/images/step_trace.png b/tutorials/source_en/advanced_use/images/step_trace.png index 6c54e790e34f52780e4c16f487f81a39906512bf..49c8bb72741173cd3285bfdbacfb206dbc33e3a9 100644 Binary files a/tutorials/source_en/advanced_use/images/step_trace.png and b/tutorials/source_en/advanced_use/images/step_trace.png differ diff --git a/tutorials/source_en/advanced_use/images/timeline.png b/tutorials/source_en/advanced_use/images/timeline.png index 21453967d9799b73795fae05529cda6fcb82f6ee..19c60e104169d86f1022758eda15bbc9c8a0dcf6 100644 Binary files a/tutorials/source_en/advanced_use/images/timeline.png and b/tutorials/source_en/advanced_use/images/timeline.png differ diff --git a/tutorials/source_en/advanced_use/mindinsight_commands.md b/tutorials/source_en/advanced_use/mindinsight_commands.md new file mode 100644 index 0000000000000000000000000000000000000000..cc3ab201ac4a70a09d083b4aa8c559d0fc31ad61 --- /dev/null +++ b/tutorials/source_en/advanced_use/mindinsight_commands.md @@ -0,0 +1,64 @@ +# MindInsight Commands + +1. View the command help information. + + ```shell + mindinsight --help + ``` + +2. View the version information. + + ```shell + mindinsight --version + ``` + +3. Start the service. + + ```shell + mindinsight start [-h] [--config ] [--workspace ] + [--port ] [--reload-interval ] + [--summary-base-dir ] + ``` + + Optional parameters as follows: + + - `-h, --help` : Displays the help information about the startup command. + - `--config ` : Specifies the configuration file or module. CONFIG indicates the physical file path (file:/path/to/config.py), or a module path (python:path.to.config.module) that can be identified by Python. + - `--workspace ` : Specifies the working directory. The default value of WORKSPACE is $HOME/mindinsight. + - `--port ` : Specifies the port number of the web visualization service. The value ranges from 1 to 65535. The default value of PORT is 8080. + - `--url-path-prefix ` : Specifies the path prefix of the web visualization service. The default value of URL_PATH_PREFIX is empty string. + - `--reload-interval ` : Specifies the interval (unit: second) for loading data. The value 0 indicates that data is loaded only once. The default value of RELOAD_INTERVAL is 3 seconds. + - `--summary-base-dir ` : Specifies the root directory for loading training log data. MindInsight traverses the direct subdirectories in this directory and searches for log files. If a direct subdirectory contains log files, it is identified as the log file directory. If a root directory contains log files, it is identified as the log file directory. SUMMARY_BASE_DIR is the current directory path by default. + + > When the service is started, the parameter values of the command line are saved as the environment variables of the process and start with `MINDINSIGHT_`, for example, `MINDINSIGHT_CONFIG`, `MINDINSIGHT_WORKSPACE`, and `MINDINSIGHT_PORT`. + +4. View the service process information. + + MindInsight provides user with web services. Run the following command to view the running web service process: + + ```shell + ps -ef | grep mindinsight + ``` + + Run the following command to access the working directory `WORKSPACE` corresponding to the service process based on the service process ID: + + ```shell + lsof -p | grep access + ``` + + Output with the working directory `WORKSPACE` as follows: + + ```shell + gunicorn /log/gunicorn/access.log + ``` + +5. Stop the service. + + ```shell + mindinsight stop [-h] [--port PORT] + ``` + + Optional parameters as follows: + + - `-h, --help` : Displays the help information about the stop command. + - `--port ` : Specifies the port number of the web visualization service. The value ranges from 1 to 65535. The default value of PORT is 8080. diff --git a/tutorials/source_en/advanced_use/mixed_precision.md b/tutorials/source_en/advanced_use/mixed_precision.md index b88cbd8ec2b4dc17f8dd86e58432df7ab1fd96d7..1f09f5bca7fdf8d25a7a4a74d6132b54db2e2c88 100644 --- a/tutorials/source_en/advanced_use/mixed_precision.md +++ b/tutorials/source_en/advanced_use/mixed_precision.md @@ -10,7 +10,7 @@ - + ## Overview diff --git a/tutorials/source_en/advanced_use/model_security.md b/tutorials/source_en/advanced_use/model_security.md index 0b4990aace93c35300d1726b516c767480dec688..55a2bdb6f7b0d7ff5b29f547c400bc58ff4b9a9c 100644 --- a/tutorials/source_en/advanced_use/model_security.md +++ b/tutorials/source_en/advanced_use/model_security.md @@ -15,7 +15,7 @@ - + ## Overview @@ -29,7 +29,7 @@ At the beginning of AI algorithm design, related security threats are sometimes This section describes how to use MindArmour in adversarial attack and defense by taking the Fast Gradient Sign Method (FGSM) attack algorithm and Natural Adversarial Defense (NAD) algorithm as examples. -> The current sample is for CPU, GPU and Ascend 910 AI processor. You can find the complete executable sample code at： +> The current sample is for CPU, GPU and Ascend 910 AI processor. You can find the complete executable sample code at： > - `mnist_attack_fgsm.py`: contains attack code. > - `mnist_defense_nad.py`: contains defense code. diff --git a/tutorials/source_en/advanced_use/network_migration.md b/tutorials/source_en/advanced_use/network_migration.md index 70fab25c04fa994d30aa02c7ccf45a1a72669f21..1be6d4953e169715af12f07844298e316904aae8 100644 --- a/tutorials/source_en/advanced_use/network_migration.md +++ b/tutorials/source_en/advanced_use/network_migration.md @@ -17,7 +17,7 @@ - + ## Overview @@ -29,9 +29,9 @@ Before you start working on your scripts, prepare your operator assessment and h ### Operator Assessment -Analyze the operators contained in the network to be migrated and figure out how does MindSpore support these operators based on the [Operator List](https://www.mindspore.cn/docs/en/master/operator_list.html). +Analyze the operators contained in the network to be migrated and figure out how does MindSpore support these operators based on the [Operator List](https://www.mindspore.cn/docs/en/r0.5/operator_list.html). -Take ResNet-50 as an example. The two major operators [Conv](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2d) and [BatchNorm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d) exist in the MindSpore Operator List. +Take ResNet-50 as an example. The two major operators [Conv](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2d) and [BatchNorm](https://www.mindspore.cn/api/en/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d) exist in the MindSpore Operator List. If any operator does not exist, you are advised to perform the following operations: @@ -57,17 +57,17 @@ Prepare the hardware environment, find a platform corresponding to your environm MindSpore differs from TensorFlow and PyTorch in the network structure. Before migration, you need to clearly understand the original script and information of each layer, such as shape. -> You can also use [MindConverter Tool](https://gitee.com/mindspore/mindinsight/tree/master/mindinsight/mindconverter) to automatically convert the PyTorch network definition script to MindSpore network definition script. +> You can also use [MindConverter Tool](https://gitee.com/mindspore/mindinsight/tree/r0.5/mindinsight/mindconverter) to automatically convert the PyTorch network definition script to MindSpore network definition script. The ResNet-50 network migration and training on the Ascend 910 is used as an example. 1. Import MindSpore modules. - Import the corresponding MindSpore modules based on the required APIs. For details about the module list, see . + Import the corresponding MindSpore modules based on the required APIs. For details about the module list, see . 2. Load and preprocess a dataset. - Use MindSpore to build the required dataset. Currently, MindSpore supports common datasets. You can call APIs in the original format, `MindRecord`, and `TFRecord`. In addition, MindSpore supports data processing and data augmentation. For details, see the [Data Preparation](https://www.mindspore.cn/tutorial/en/master/use/data_preparation/data_preparation.html). + Use MindSpore to build the required dataset. Currently, MindSpore supports common datasets. You can call APIs in the original format, `MindRecord`, and `TFRecord`. In addition, MindSpore supports data processing and data augmentation. For details, see the [Data Preparation](https://www.mindspore.cn/tutorial/en/r0.5/use/data_preparation/data_preparation.html). In this example, the CIFAR-10 dataset is loaded, which supports both single-GPU and multi-GPU scenarios. @@ -79,7 +79,7 @@ The ResNet-50 network migration and training on the Ascend 910 is used as an exa num_shards=device_num, shard_id=rank_id) ``` - Then, perform data augmentation, data cleaning, and batch processing. For details about the code, see . + Then, perform data augmentation, data cleaning, and batch processing. For details about the code, see . 3. Build a network. @@ -214,7 +214,7 @@ The ResNet-50 network migration and training on the Ascend 910 is used as an exa 6. Build the entire network. - The [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/resnet.py) network structure is formed by connecting multiple defined subnets. Follow the rule of defining subnets before using them and define all the subnets used in the `__init__` and connect subnets in the `construct`. + The [ResNet-50](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/resnet/src/resnet.py) network structure is formed by connecting multiple defined subnets. Follow the rule of defining subnets before using them and define all the subnets used in the `__init__` and connect subnets in the `construct`. 7. Define a loss function and an optimizer. @@ -235,7 +235,7 @@ The ResNet-50 network migration and training on the Ascend 910 is used as an exa loss_scale = FixedLossScaleManager(config.loss_scale, drop_overflow_update=False) ``` - You can use a built-in assessment method of `Model` by setting the [metrics](https://www.mindspore.cn/tutorial/en/master/advanced_use/customized_debugging_information.html#mindspore-metrics) attribute. + You can use a built-in assessment method of `Model` by setting the [metrics](https://www.mindspore.cn/tutorial/en/r0.5/advanced_use/customized_debugging_information.html#mindspore-metrics) attribute. ```python model = Model(net, loss_fn=loss, optimizer=opt, loss_scale_manager=loss_scale, metrics={'acc'}) @@ -264,17 +264,15 @@ The accuracy optimization process is as follows: #### On-Cloud Integration -Run your scripts on ModelArts. For details, see [Using MindSpore on Cloud](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/use_on_the_cloud.html). +Run your scripts on ModelArts. For details, see [Using MindSpore on Cloud](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/use_on_the_cloud.html). ### Inference Phase -Models trained on the Ascend 910 AI processor can be used for inference on different hardware platforms. Refer to the [Multi-platform Inference Tutorial](https://www.mindspore.cn/tutorial/en/master/use/multi_platform_inference.html) for detailed steps. +Models trained on the Ascend 910 AI processor can be used for inference on different hardware platforms. Refer to the [Multi-platform Inference Tutorial](https://www.mindspore.cn/tutorial/en/r0.5/use/multi_platform_inference.html) for detailed steps. ## Examples -1. [Common network script examples](https://gitee.com/mindspore/mindspore/tree/master/example) +1. [Common dataset examples](https://www.mindspore.cn/tutorial/en/r0.5/use/data_preparation/loading_the_datasets.html) -2. [Common dataset examples](https://www.mindspore.cn/tutorial/en/master/use/data_preparation/loading_the_datasets.html) - -3. [Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/mindspore/model_zoo) +2. [Model Zoo](https://gitee.com/mindspore/mindspore/tree/r0.5/model_zoo) diff --git a/tutorials/source_en/advanced_use/nlp_application.md b/tutorials/source_en/advanced_use/nlp_application.md index 1d2b11258bcb4644dd55b84a49ebac3af1c35431..41e59319a18079a8caa3d2b4ecbe6dabf3beb6ab 100644 --- a/tutorials/source_en/advanced_use/nlp_application.md +++ b/tutorials/source_en/advanced_use/nlp_application.md @@ -20,7 +20,7 @@ - + ## Overview @@ -85,7 +85,7 @@ Currently, MindSpore GPU and CPU supports SentimentNet network based on the long Long short-term memory (LSTM) is an artificial recurrent neural network (RNN) architecture used for processing and predicting an important event with a long interval and delay in a time sequence. For details, refer to online documentation. 3. After the model is obtained, use the validation dataset to check the accuracy of model. -> The current sample is for the Ascend 910 AI processor. You can find the complete executable sample code at： +> The current sample is for the Ascend 910 AI processor. You can find the complete executable sample code at： > - `main.py`: code file, including code for data preprocessing, network definition, and model training. > - `config.py`: some configurations on the network, including the `batch size` and number of training epochs. diff --git a/tutorials/source_en/advanced_use/on_device_inference.md b/tutorials/source_en/advanced_use/on_device_inference.md index 41d0999bff841ecd2befcb6bb3bbc8d112437d8f..f8c130951834fec999103a264e067d81bc2c06dd 100644 --- a/tutorials/source_en/advanced_use/on_device_inference.md +++ b/tutorials/source_en/advanced_use/on_device_inference.md @@ -11,7 +11,7 @@ - + ## Overview @@ -28,8 +28,8 @@ The environment requirements are as follows: - Hard disk space: 10 GB or above - System requirements - - System: Ubuntu = 16.04.02LTS (availability is checked) - - Kernel: 4.4.0-62-generic (availability is checked) + - System: Ubuntu = 18.04.02LTS (availability is checked) + - Kernel: 4.15.0-45-generic (availability is checked) - Software dependencies - [cmake](https://cmake.org/download/) >= 3.14.1 @@ -60,7 +60,7 @@ The compilation procedure is as follows: 2. Download source code from the code repository. ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 3. Run the following command in the root directory of the source code to compile MindSpore Predict: -I indicates options for compiling MindSpore Predict and the parameter is the target platform architecture. Currently, only the Android arm64 platform is supported. @@ -87,7 +87,7 @@ To perform on-device model inference using MindSpore, perform the following step ### Generating an On-Device Model File 1. After training is complete, load the generated checkpoint file to the defined network. ```python - param_dict = load_checkpoint(ckpoint_file_name=ckpt_file_path) + param_dict = load_checkpoint(ckpt_file_name=ckpt_file_path) load_param_into_net(net, param_dict) ``` 2. Call the `export` API to export the `.ms` model file on the device. @@ -145,7 +145,7 @@ if __name__ == '__main__': is_ckpt_exist = os.path.exists(ckpt_file_path) if is_ckpt_exist: - param_dict = load_checkpoint(ckpoint_file_name=ckpt_file_path) + param_dict = load_checkpoint(ckpt_file_name=ckpt_file_path) load_param_into_net(net, param_dict) export(net, input_data, file_name="./lenet.ms", file_format='LITE') print("export model success.") diff --git a/tutorials/source_en/advanced_use/performance_profiling.md b/tutorials/source_en/advanced_use/performance_profiling.md index 3152b8150655b5256cae13a0366e06caa3ccc408..ce7fcf410fc35367ee63dec44a37a1b6789b3c58 100644 --- a/tutorials/source_en/advanced_use/performance_profiling.md +++ b/tutorials/source_en/advanced_use/performance_profiling.md @@ -16,14 +16,14 @@ - + ## Overview -Performance data like operators' execution time are recorded in files and can be viewed on the web page, this can help the user optimize the performance of neural networks. MindInsight Profiler can only support the Ascend chip now. +Performance data like operators' execution time is recorded in files and can be viewed on the web page, this can help the user optimize the performance of neural networks. MindInsight Profiler can only support the Ascend chip now. ## Operation Process -- Prepare a training script, add profiler apis in the training script, and run the training script. +- Prepare a training script, add profiler APIs in the training script, and run the training script. - Start MindInsight and specify the profiler data directory using startup parameters. After MindInsight is started, access the visualization page based on the IP address and port number. The default access IP address is `http://127.0.0.1:8080`. - Find the training in the list, click the performance profiling link, and view the data on the web page. @@ -64,7 +64,7 @@ def test_profiler(): ## Launch MindInsight -The MindInsight launch command can refer to the **MindInsight Command** part in [Training Process Visualization](https://www.mindspore.cn/tutorial/en/master/advanced_use/visualization_tutorials.html). +The MindInsight launch command can refer to [MindInsight Commands](https://www.mindspore.cn/tutorial/en/r0.5/advanced_use/mindinsight_commands.html). ### Performance Analysis @@ -164,6 +164,10 @@ Users can get the most detailed information from the Timeline: - From high level, users can analyse whether the stream split strategy can be optimized and whether is step tail is too long. - From low level, users can analyse the execution time for all the operators, etc. +Users can click the download button on the overall performance page to view Timeline details. The Timeline data file (json format) will be stored on local machine, and can be displayed by tools. We suggest to use `chrome://tracing` or [Perfetto](https://ui.perfetto.dev/#!viewer) to visualize the Timeline. +- Chrome tracing: Click "load" on the upper left to load the file. +- Perfetto: Click "Open trace file" on the left to load the file. + ![timeline.png](./images/timeline.png) Figure 7: Timeline Analysis diff --git a/tutorials/source_en/advanced_use/visualization_tutorials.rst b/tutorials/source_en/advanced_use/visualization_tutorials.rst index 9c6c8a82031dc7cbc511f20e0517bd66426e75c1..5092b38d50b911d991e2ac351eecd3eba9938207 100644 --- a/tutorials/source_en/advanced_use/visualization_tutorials.rst +++ b/tutorials/source_en/advanced_use/visualization_tutorials.rst @@ -6,71 +6,4 @@ Training Process Visualization dashboard_and_lineage performance_profiling - -MindInsight Commands --------------------- - -1. View the command help information. - - .. code-block:: - - mindinsight --help - -2. View the version information. - - .. code-block:: - - mindinsight --version - -3. Start the service. - - .. code-block:: - - mindinsight start [-h] [--config ] [--workspace ] - [--port ] [--reload-interval ] - [--summary-base-dir ] - - Optional parameters as follows: - - - `-h, --help` : Displays the help information about the startup command. - - `--config ` : Specifies the configuration file or module. CONFIG indicates the physical file path (file:/path/to/config.py), or a module path (python:path.to.config.module) that can be identified by Python. - - `--workspace ` : Specifies the working directory. The default value of WORKSPACE is $HOME/mindinsight. - - `--port ` : Specifies the port number of the web visualization service. The value ranges from 1 to 65535. The default value of PORT is 8080. - - `--url-path-prefix ` : Specifies the path prefix of the web visualization service. The default value of URL_PATH_PREFIX is empty string. - - `--reload-interval ` : Specifies the interval (unit: second) for loading data. The value 0 indicates that data is loaded only once. The default value of RELOAD_INTERVAL is 3 seconds. - - `--summary-base-dir ` : Specifies the root directory for loading training log data. MindInsight traverses the direct subdirectories in this directory and searches for log files. If a direct subdirectory contains log files, it is identified as the log file directory. If a root directory contains log files, it is identified as the log file directory. SUMMARY_BASE_DIR is the current directory path by default. - - .. note:: - - When the service is started, the parameter values of the command line are saved as the environment variables of the process and start with `MINDINSIGHT_`, for example, `MINDINSIGHT_CONFIG`, `MINDINSIGHT_WORKSPACE`, and `MINDINSIGHT_PORT`. - -4. View the service process information. - - MindInsight provides user with web services. Run the following command to view the running web service process: - - .. code-block:: - - ps -ef | grep mindinsight - - Run the following command to access the working directory `WORKSPACE` corresponding to the service process based on the service process ID: - - .. code-block:: - - lsof -p | grep access - - Output with the working directory `WORKSPACE` as follows: - - .. code-block:: - - gunicorn /log/gunicorn/access.log - -5. Stop the service. - - .. code-block:: - - mindinsight stop [-h] [--port PORT] - - Optional parameters as follows: - - - `-h, --help` : Displays the help information about the stop command. - - `--port ` : Specifies the port number of the web visualization service. The value ranges from 1 to 65535. The default value of PORT is 8080. + mindinsight_commands diff --git a/tutorials/source_en/conf.py b/tutorials/source_en/conf.py index 0a00ad8da18607c9f0ac88017972211d04c763c0..39e33baad482db9ee42d0a065d0689abb1d068ce 100644 --- a/tutorials/source_en/conf.py +++ b/tutorials/source_en/conf.py @@ -21,7 +21,7 @@ copyright = '2020, MindSpore' author = 'MindSpore' # The full version, including alpha/beta/rc tags -release = 'master' +release = 'r0.5' # -- General configuration --------------------------------------------------- diff --git a/tutorials/source_en/quick_start/quick_start.md b/tutorials/source_en/quick_start/quick_start.md index 500c9614e3fa7e445a33a0822c73895495ead5d6..b2c7373db962d815e0e1db8f5fb8aa5b1f0a3a4d 100644 --- a/tutorials/source_en/quick_start/quick_start.md +++ b/tutorials/source_en/quick_start/quick_start.md @@ -24,7 +24,7 @@ - + ## Overview @@ -38,7 +38,7 @@ During the practice, a simple image classification function is implemented. The 5. Load the saved model for inference. 6. Validate the model, load the test dataset and trained model, and validate the result accuracy. -> You can find the complete executable sample code at . +> You can find the complete executable sample code at . This is a simple and basic application process. For other advanced and complex applications, extend this basic process as needed. @@ -83,7 +83,7 @@ Currently, the `os` libraries are required. For ease of understanding, other req import os ``` -For details about MindSpore modules, search on the [MindSpore API Page](https://www.mindspore.cn/api/en/master/index.html). +For details about MindSpore modules, search on the [MindSpore API Page](https://www.mindspore.cn/api/en/r0.5/index.html). ### Configuring the Running Information @@ -179,7 +179,7 @@ In the preceding information: Perform the shuffle and batch operations, and then perform the repeat operation to ensure that data during an epoch is unique. -> MindSpore supports multiple data processing and augmentation operations, which are usually combined. For details, see section "Data Processing and Augmentation" in the MindSpore Tutorials (https://www.mindspore.cn/tutorial/en/master/use/data_preparation/data_processing_and_augmentation.html). +> MindSpore supports multiple data processing and augmentation operations, which are usually combined. For details, see section "Data Processing and Augmentation" in the MindSpore Tutorials (https://www.mindspore.cn/tutorial/en/r0.5/use/data_preparation/data_processing_and_augmentation.html). ## Defining the Network diff --git a/tutorials/source_en/use/custom_operator.md b/tutorials/source_en/use/custom_operator.md index c6f4296a3df4ed91a1834627fa4e5e79b5b90425..1e497934bb231d7a75268a2309e9322ebf179276 100644 --- a/tutorials/source_en/use/custom_operator.md +++ b/tutorials/source_en/use/custom_operator.md @@ -14,7 +14,7 @@ - + ## Overview @@ -27,14 +27,14 @@ The related concepts are as follows: - Operator implementation: describes the implementation of the internal computation logic for an operator through the DSL API provided by the Tensor Boost Engine (TBE). The TBE supports the development of custom operators based on the Ascend AI chip. You can apply for Open Beta Tests (OBTs) by visiting . - Operator information: describes basic information about a TBE operator, such as the operator name and supported input and output types. It is the basis for the backend to select and map operators. -This section takes a Square operator as an example to describe how to customize an operator. For details, see cases in [tests/st/ops/custom_ops_tbe](https://gitee.com/mindspore/mindspore/tree/master/tests/st/ops/custom_ops_tbe) in the MindSpore source code. +This section takes a Square operator as an example to describe how to customize an operator. For details, see cases in [tests/st/ops/custom_ops_tbe](https://gitee.com/mindspore/mindspore/tree/r0.5/tests/st/ops/custom_ops_tbe) in the MindSpore source code. ## Registering the Operator Primitive The primitive of an operator is a subclass inherited from `PrimitiveWithInfer`. The type name of the subclass is the operator name. The definition of the custom operator primitive is the same as that of the built-in operator primitive. -- The attribute is defined by the input parameter of the constructor function `__init__`. The operator in this test case has no attribute. Therefore, `__init__` has only one input parameter. For details about test cases in which operators have attributes, see [custom add3](https://gitee.com/mindspore/mindspore/tree/master/tests/st/ops/custom_ops_tbe/cus_add3.py) in the MindSpore source code. +- The attribute is defined by the input parameter of the constructor function `__init__`. The operator in this test case has no attribute. Therefore, `__init__` has only one input parameter. For details about test cases in which operators have attributes, see [custom add3](https://gitee.com/mindspore/mindspore/tree/r0.5/tests/st/ops/custom_ops_tbe/cus_add3.py) in the MindSpore source code. - The input and output names are defined by the `init_prim_io_names` function. - The shape inference method of the output tensor is defined in the `infer_shape` function, and the dtype inference method of the output tensor is defined in the `infer_dtype` function. diff --git a/tutorials/source_en/use/data_preparation/converting_datasets.md b/tutorials/source_en/use/data_preparation/converting_datasets.md index 50fbc8687ef6a22486af9847c3a66f1505be3afa..966794ca3030d28915bd726d2504cf4f9cc99047 100644 --- a/tutorials/source_en/use/data_preparation/converting_datasets.md +++ b/tutorials/source_en/use/data_preparation/converting_datasets.md @@ -14,7 +14,7 @@ - + ## Overview @@ -178,12 +178,14 @@ You can use the `ImageNetToMR` class to convert the raw ImageNet data (images an Store the downloaded ImageNet dataset in a folder. The folder contains all images and a mapping file that records labels of the images. - In the mapping file, there are three columns, which are separated by spaces. They indicate image classes, label IDs, and label names. The following is an example of the mapping file: + In the mapping file, there are two columns, which are separated by spaces. They indicate image classes and label IDs. The following is an example of the mapping file: ``` - n02119789 1 pen - n02100735 2 notbook - n02110185 3 mouse - n02096294 4 orange + n01440760 0 + n01443537 1 + n01484850 2 + n01491361 3 + n01494475 4 + n01496331 5 ``` 2. Import the `ImageNetToMR` class for dataset converting. diff --git a/tutorials/source_en/use/data_preparation/data_processing_and_augmentation.md b/tutorials/source_en/use/data_preparation/data_processing_and_augmentation.md index 94973c079f3c50f3409349aae8b7775afff89826..6fa0aec89dd5ae9478ee90f86d3ef352b1ebbca6 100644 --- a/tutorials/source_en/use/data_preparation/data_processing_and_augmentation.md +++ b/tutorials/source_en/use/data_preparation/data_processing_and_augmentation.md @@ -16,7 +16,7 @@ - + ## Overview @@ -279,7 +279,7 @@ Data augmentation requires the `map` function. For details about how to use the ``` 2. Define data augmentation operators. The following uses `Resize` as an example: ```python - dataset = ds.ImageFolderDatasetV2(DATA_DIR, decode=True) # Deocde images. + dataset = ds.ImageFolderDatasetV2(DATA_DIR, decode=True) # Decode images. resize_op = transforms.Resize(size=(500,500), interpolation=Inter.LINEAR) dataset.map(input_columns="image", operations=resize_op) diff --git a/tutorials/source_en/use/data_preparation/loading_the_datasets.md b/tutorials/source_en/use/data_preparation/loading_the_datasets.md index 6a47a1a22b3b896b632aaf386421184f93306909..6b04978c61d6a69a0ff1664b4b7c240a20a20bae 100644 --- a/tutorials/source_en/use/data_preparation/loading_the_datasets.md +++ b/tutorials/source_en/use/data_preparation/loading_the_datasets.md @@ -13,7 +13,7 @@ - + ## Overview @@ -65,7 +65,7 @@ To read a dataset using the `MindDataset` object, perform the following steps: data_set = ds.MindDataset(dataset_file=CV_FILE_NAME) ``` In the preceding information: - `dataset_file`: specifies the MindRecord file, including the path and file name. + `dataset_file`: specifies the MindRecord file or list of MindRecord files. 2. Create a dictionary iterator and read data records through the iterator. ```python @@ -148,32 +148,112 @@ MindSpore can also read datasets in the `TFRecord` data format through the `TFRe ``` ## Loading a Custom Dataset -You can load a custom dataset using the `GeneratorDataset` object. +In real scenarios, there are virous datasets. For a custom dataset or a dataset that can't be loaded by APIs directly, there are tow ways. +One is converting the dataset to MindSpore data format (for details, see [Converting Datasets to the Mindspore Data Format](https://www.mindspore.cn/tutorial/en/r0.5/use/data_preparation/converting_datasets.html)). The other one is using the `GeneratorDataset` object. +The following shows how to use `GeneratorDataset`. -1. Define a function (for example, `Generator1D`) to generate a dataset. - > The custom generation function returns the objects that can be called. Each time, tuples of `numpy array` are returned as a row of data. +1. Define an iterable object to generate a dataset. There are two examples following. One is a customized function which contains `yield`. The other one is a customized class which contains `__getitem__`. + Both of them will generator a dataset with numbers from 0 to 9. + > The custom iterable object returns a tuple of `numpy arrays` as a row of data each time. An example of a custom function is as follows: ```python import numpy as np # Import numpy lib. - def Generator1D(): - for i in range(64): + def generator_func(num): + for i in range(num): yield (np.array([i]),) # Notice, tuple of only one element needs following a comma at the end. ``` -2. Transfer `Generator1D` to `GeneratorDataset` to create a dataset and set `column` to data. + An example of a custom class is as follows: ```python - dataset = ds.GeneratorDataset(Generator1D, ["data"]) + import numpy as np # Import numpy lib. + class Generator(): + + def __init__(self, num): + self.num = num + + def __getitem__(self, item): + return (np.array([item]),) # Notice, tuple of only one element needs following a comma at the end. + + def __len__(self): + return self.num + ``` + +2. Create a dataset with `GeneratorDataset`. Transfer `generator_func` to `GeneratorDataset` to create a dataset and set `column` to `data`. +Define a `Generator` and transfer it to `GeneratorDataset` to create a dataset and set `column` to `data`. + ```python + dataset1 = ds.GeneratorDataset(source=generator_func(10), column_names=["data"], shuffle=False) + dataset2 = ds.GeneratorDataset(source=Generator(10), column_names=["data"], shuffle=False) ``` 3. After creating a dataset, create an iterator for the dataset to obtain the corresponding data. Iterator creation methods are as follows: - - Create an iterator whose return value is of the sequence type. + - Create an iterator whose return value is of the sequence type. As is shown in the following, create the iterators for `dataset1` and `dataset2`, and print the output. ```python - for data in dataset.create_tuple_iterator(): # each data is a sequence + print("dataset1:") + for data in dataset1.create_tuple_iterator(): # each data is a sequence + print(data[0]) + + print("dataset2:") + for data in dataset2.create_tuple_iterator(): # each data is a sequence print(data[0]) ``` + The output is as follows: + ``` + dataset1: + [array([0], dtype=int64)] + [array([1], dtype=int64)] + [array([2], dtype=int64)] + [array([3], dtype=int64)] + [array([4], dtype=int64)] + [array([5], dtype=int64)] + [array([6], dtype=int64)] + [array([7], dtype=int64)] + [array([8], dtype=int64)] + [array([9], dtype=int64)] + dataset2: + [array([0], dtype=int64)] + [array([1], dtype=int64)] + [array([2], dtype=int64)] + [array([3], dtype=int64)] + [array([4], dtype=int64)] + [array([5], dtype=int64)] + [array([6], dtype=int64)] + [array([7], dtype=int64)] + [array([8], dtype=int64)] + [array([9], dtype=int64)] + ``` - - Create an iterator whose return value is of the dictionary type. - ```python - for data in dataset.create_dict_iterator(): # each data is a dictionary + - Create an iterator whose return value is of the dictionary type. As is shown in the following, create the iterators for `dataset1` and `dataset2`, and print the output. + ```python + print("dataset1:") + for data in dataset1.create_dict_iterator(): # each data is a dictionary + print(data["data"]) + + print("dataset2:") + for data in dataset2.create_dict_iterator(): # each data is a dictionary print(data["data"]) ``` + The output is as follows: + ``` + dataset1: + {'data': array([0], dtype=int64)} + {'data': array([1], dtype=int64)} + {'data': array([2], dtype=int64)} + {'data': array([3], dtype=int64)} + {'data': array([4], dtype=int64)} + {'data': array([5], dtype=int64)} + {'data': array([6], dtype=int64)} + {'data': array([7], dtype=int64)} + {'data': array([8], dtype=int64)} + {'data': array([9], dtype=int64)} + dataset2: + {'data': array([0], dtype=int64)} + {'data': array([1], dtype=int64)} + {'data': array([2], dtype=int64)} + {'data': array([3], dtype=int64)} + {'data': array([4], dtype=int64)} + {'data': array([5], dtype=int64)} + {'data': array([6], dtype=int64)} + {'data': array([7], dtype=int64)} + {'data': array([8], dtype=int64)} + {'data': array([9], dtype=int64)} + ``` \ No newline at end of file diff --git a/tutorials/source_en/use/defining_the_network.rst b/tutorials/source_en/use/defining_the_network.rst index a530f34dbdf92434d8727839433770e23124cfbd..05a9a81f7c3ff190858bb4953c65622e30aa751b 100644 --- a/tutorials/source_en/use/defining_the_network.rst +++ b/tutorials/source_en/use/defining_the_network.rst @@ -4,5 +4,5 @@ Defining the Network .. toctree:: :maxdepth: 1 - Network List + Network List custom_operator \ No newline at end of file diff --git a/tutorials/source_en/use/multi_platform_inference.md b/tutorials/source_en/use/multi_platform_inference.md index 230af0e3e0cdd929127e1f1bab2b8f7b9b080701..2b3cff1f8fdd3e8e6d5df6a6599f9ec751223f62 100644 --- a/tutorials/source_en/use/multi_platform_inference.md +++ b/tutorials/source_en/use/multi_platform_inference.md @@ -8,7 +8,7 @@ - + ## Overview @@ -16,7 +16,7 @@ Models based on MindSpore training can be used for inference on different hardwa 1. Inference on the Ascend 910 AI processor - MindSpore provides the `model.eval` API for model validation. You only need to import the validation dataset. The processing method of the validation dataset is the same as that of the training dataset. For details about the complete code, see . + MindSpore provides the `model.eval` API for model validation. You only need to import the validation dataset. The processing method of the validation dataset is the same as that of the training dataset. For details about the complete code, see . ```python res = model.eval(dataset) @@ -26,16 +26,16 @@ Models based on MindSpore training can be used for inference on different hardwa 2. Inference on the Ascend 310 AI processor - 1. Export the ONNX or GEIR model by referring to the [Export GEIR Model and ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#geironnx). + 1. Export the ONNX or GEIR model by referring to the [Export GEIR Model and ONNX Model](https://www.mindspore.cn/tutorial/en/r0.5/use/saving_and_loading_model_parameters.html#geironnx). 2. For performing inference in the cloud environment, see the [Ascend 910 training and Ascend 310 inference samples](https://support.huaweicloud.com/bestpractice-modelarts/modelarts_10_0026.html). For details about the bare-metal environment (compared with the cloud environment where the Ascend 310 AI processor is deployed locally), see the description document of the Ascend 310 AI processor software package. 3. Inference on a GPU - 1. Export the ONNX model by referring to the [Export GEIR Model and ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#geironnx). + 1. Export the ONNX model by referring to the [Export GEIR Model and ONNX Model](https://www.mindspore.cn/tutorial/en/r0.5/use/saving_and_loading_model_parameters.html#geironnx). 2. Perform inference on the NVIDIA GPU by referring to [TensorRT backend for ONNX](https://github.com/onnx/onnx-tensorrt). ## On-Device Inference -The On-Device Inference is based on the MindSpore Predict. Please refer to [On-Device Inference Tutorial](https://www.mindspore.cn/tutorial/en/master/advanced_use/on_device_inference.html) for details. +The On-Device Inference is based on the MindSpore Predict. Please refer to [On-Device Inference Tutorial](https://www.mindspore.cn/tutorial/en/r0.5/advanced_use/on_device_inference.html) for details. diff --git a/tutorials/source_en/use/saving_and_loading_model_parameters.md b/tutorials/source_en/use/saving_and_loading_model_parameters.md index 78fc04726f1a0977a87396c7c0ef02082efb852e..2bd4bb7806eb1520e76cf1d93723a21224d19728 100644 --- a/tutorials/source_en/use/saving_and_loading_model_parameters.md +++ b/tutorials/source_en/use/saving_and_loading_model_parameters.md @@ -13,7 +13,7 @@ - + ## Overview diff --git a/tutorials/source_zh_cn/advanced_use/aware_quantization.md b/tutorials/source_zh_cn/advanced_use/aware_quantization.md index fcb5a520ae3ec77bbeb5c01c00a873643cc89805..5798ca12c5bcdac877275fafcb17b97c3f6aafee 100644 --- a/tutorials/source_zh_cn/advanced_use/aware_quantization.md +++ b/tutorials/source_zh_cn/advanced_use/aware_quantization.md @@ -3,147 +3,192 @@ - [量化](#量化) - - [概述](#概述) - - [感知量化训练](#感知量化训练) + - [背景](#背景) + - [概念](#概念) + - [量化](#量化-1) - [伪量化节点](#伪量化节点) - - [感知量化示例](#感知量化示例) - - [导入模型重训与推理](#导入模型重训与推理) + - [感知量化训练](#感知量化训练) + - [感知量化训练示例](#感知量化训练示例) + - [定义融合网络](#定义融合网络) + - [转化为量化网络](#转化为量化网络) + - [重训和推理](#重训和推理) + - [导入模型重新训练](#导入模型重新训练) + - [进行推理](#进行推理) - [参考文献](#参考文献) - - -## 概述 - -与FP32类型相比，FP16、INT8、INT4的低精度数据表达类型所占用空间更小，因此对应的存储空间和传输时间都可以大幅下降。以手机为例，为了提供更人性化和智能的服务，现在越来越多的OS和APP集成了深度学习的功能，自然需要包含大量的模型及权重文件。经典的AlexNet，原始权重文件的大小已经超过了200MB，而最近出现的新模型正在往结构更复杂、参数更多的方向发展。显然，低精度类型的空间受益还是很明显的。低比特的计算性能也更高，INT8相对比FP32的加速比可达到3倍甚至更高，功耗上也对应有所减少。 + -量化即以较低的推理精度损失将连续取值（或者大量可能的离散取值）的浮点型模型权重或流经模型的张量数据定点近似（通常为int8）为有限多个（或较少的）离散值的过程，它是以更少位数的数据类型用于近似表示32位有限范围浮点型数据的过程，而模型的输入输出依然是浮点型，从而达到减少模型尺寸大小、减少模型内存消耗及加快模型推理速度等目标。 +## 背景 -量化方案主要分为两种：感知量化训练（aware quantization training）和训练后量化（post-training quantization）。 +越来越多的应用选择在移动设备或者边缘设备上使用深度学习技术。以手机为例，为了提供人性化和智能的服务，现在操作系统和应用都开始集成深度学习功能。而使用该功能，涉及训练或者推理，自然包含大量的模型及权重文件。经典的AlexNet，原始权重文件已经超过了200MB，而最近出现的新模型正往结构更复杂、参数更多的方向发展。由于移动设备、边缘设备的硬件资源有限，需要对模型进行精简，而量化（Quantization）技术就是应对该类问题衍生出的技术之一。 -## 感知量化训练 +## 概念 -感知量化训练为在网络模型训练的过程中，插入伪量化节点进行伪量化训练的过程。 +### 量化 -MindSpore的感知量化训练是一种伪量化的过程，它是在可识别的某些操作内嵌入伪量化节点，用以统计训练时流经该节点数据的最大最小值。其目的是减少精度损失，其参与模型训练的前向推理过程令模型获得量化损失，但梯度更新需要在浮点下进行，因而其并不参与反向传播过程。 +量化即以较低的推理精度损失将连续取值（或者大量可能的离散取值）的浮点型模型权重或流经模型的张量数据定点近似（通常为INT8）为有限多个（或较少的）离散值的过程，它是以更少位数的数据类型用于近似表示32位有限范围浮点型数据的过程，而模型的输入输出依然是浮点型。这样的好处是可以减小模型尺寸大小，减少模型内存占用，加快模型推理速度，降低功耗等。 -在MindSpore的伪量化训练中，支持非对称和对称的量化算法，支持4、7和8bit的量化方案。 +如上所述，与FP32类型相比，FP16、INT8、INT4等低精度数据表达类型所占用空间更小。使用低精度数据表达类型替换高精度数据表达类型，可以大幅降低存储空间和传输时间。而低比特的计算性能也更高，INT8相对比FP32的加速比可达到3倍甚至更高，对于相同的计算，功耗上也有明显优势。 -目前MindSpore感知量化训练支持的后端有GPU和Ascend。 +当前业界量化方案主要分为两种：感知量化训练（Aware Quantization Training）和训练后量化（Post-training Quantization）。 ### 伪量化节点 -伪量化节点的作用：（1）找到网络数据的分布，即找到待量化参数的最大值和最小值；（2）模拟量化到低比特操作的时候的精度损失，把该损失作用到网络模型中，传递给损失函数，让优化器去在训练过程中对该损失值进行优化。 - -伪量化节点的意义在于统计流经数据的min和max值，并参与前向传播，让损失函数的值增大，优化器感知到损失值的增加，并进行持续性地反向传播学习，进一步减少因为伪量化操作而引起的精度下降，从而提升精确度。 - -对于权值和数据的量化，MindSpore都采用参考文献[1]中的方案进行量化。 - -### 感知量化示例 - -使用感知量化训练特性，主要的步骤为： - -1. 定义网络模型 -2. 量化自动构图 - -代码样例如下： - - -1. 定义网络模型 - - 以LeNet5网络模型为例子，原网络模型的定义如下所示。 - - ```python - class LeNet5(nn.Cell): - def __init__(self, num_class=10): - super(LeNet5, self).__init__() - self.num_class = num_class - - self.conv1 = nn.Conv2d(1, 6, kernel_size=5) - self.bn1 = nn.batchnorm(6) - self.act1 = nn.relu() - - self.conv2 = nn.Conv2d(6, 16, kernel_size=5) - self.bn2 = nn.batchnorm(16) - self.act2 = nn.relu() - - self.fc1 = nn.Dense(16 * 5 * 5, 120) - self.fc2 = nn.Dense(120, 84) - self.act3 = nn.relu() - self.fc3 = nn.Dense(84, self.num_class) - self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) - - def construct(self, x): - x = self.conv1(x) - x = self.bn1(x) - x = self.act1(x) - x = self.max_pool2d(x) - x = self.conv2(x) - x = self.bn2(x) - x = self.act2(x) - x = self.max_pool2d(x) - x = self.flattern(x) - x = self.fc1(x) - x = self.act3(x) - x = self.fc2(x) - x = self.act3(x) - x = self.fc3(x) - return x - ``` - - 融合网络模型定义： - - 使用`nn.Conv2dBnAct`算子替换原网络模型中的三个算子`nn.Conv2d`、`nn.batchnorm`和`nn.relu`； - - 同理，使用`nn.DenseBnAct`算子替换原网络模型中的对应的算子`nn.Dense`、`nn.batchnorm`和`nn.relu`。 - - 即使`nn.Dense`和`nn.Conv2d`算子后面没有`nn.batchnorm`和`nn.relu`，都要按规定使用上述两个算子进行融合替换。 - - ```python - class LeNet5(nn.Cell): - def __init__(self, num_class=10): - super(LeNet5, self).__init__() - self.num_class = num_class - - self.conv1 = nn.Conv2dBnAct(1, 6, kernel_size=5, batchnorm=True, activation='relu') - self.conv2 = nn.Conv2dBnAct(6, 16, kernel_size=5, batchnorm=True, activation='relu') - - self.fc1 = nn.DenseBnAct(16 * 5 * 5, 120, activation='relu') - self.fc2 = nn.DenseBnAct(120, 84, activation='relu') - self.fc3 = nn.DenseBnAct(84, self.num_class) - self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) - - def construct(self, x): - x = self.conv1(x) - x = self.max_pool2d(x) - x = self.conv2(x) - x = self.max_pool2d(x) - x = self.flattern(x) - x = self.fc1(x) - x = self.fc2(x) - x = self.fc3(x) - return x - ``` -2. 量化自动构图 - - 使用`create_training_network`接口封装网络模型，该步骤将会自动在融合网络模型中插入伪量化算子。 - - ```python - from mindspore.train.quant import quant as qat - net = qat.convert_quant_network(net, quant_delay=0, bn_fold=False, freeze_bn=10000, weight_bits=8, act_bits=8) - ``` - - 其余步骤（如定义损失函数、优化器、超参数和训练网络等）与普通网络训练相同。 - -### 导入模型重训与推理 - -经过`create_training_network`函数之后，融合网络模型的图自动转换为感知量化的图。在训练和推理的时候分为下面三种情况： - -- 使用融合网络模型训练得到的checkpoint文件导入，进行感知量化训练，步骤为：a)定义融合网络模型，b)加载checkpoint文件，c)转换量化自动构图，d)训练。 -- 使用感知量化训练得到的checkpoint文件导入，进行感知量化推理，步骤为：a)定义融合网络模型，b)转换量化自动构图，c)加载checkpoint文件，d)推理。 -- 使用正常网络模型训练得到的checkpoint文件导入，进行感知量化训练，步骤为：a)定义融合网络模型，b)加载checkpoint文件，c)训练并保存为融合网络模型对应的checkpoint文件，d)使用融合网络模型训练得到的checkpoint文件导入，进行感知量化训练。 +伪量化节点，是指感知量化训练中插入的节点，用以寻找网络数据分布，并反馈损失精度，具体作用如下： +- 找到网络数据的分布，即找到待量化参数的最大值和最小值； +- 模拟量化为低比特时的精度损失，把该损失作用到网络模型中，传递给损失函数，让优化器在训练过程中对该损失值进行优化。 + +## 感知量化训练 +MindSpore的感知量化训练是在训练基础上，使用低精度数据替换高精度数据来简化训练模型的过程。这个过程不可避免引入精度的损失，这时使用伪量化节点来模拟引入的精度损失，并通过反向传播学习，来减少精度损失。对于权值和数据的量化，MindSpore采用了参考文献[1]中的方案。 + +感知量化训练规格 +| 规格 | 规格说明 | +| --- | --- | +| 硬件支持 | GPU、Ascend AI 910处理器的硬件平台 | +| 网络支持 | 已实现的网络包括LeNet、ResNet50等网络，具体请参见。 | +| 算法支持 | 在MindSpore的伪量化训练中，支持非对称和对称的量化算法。 | +| 方案支持 | 支持4、7和8比特的量化方案。 | + +## 感知量化训练示例 + +感知量化训练模型与一般训练步骤一致，在定义网络和最后生成模型阶段后，需要进行额外的操作，完整流程如下： + +1. 数据处理加载数据集。 +2. 定义网络。 +3. 定义融合网络。在完成定义网络后，替换指定的算子，完成融合网络的定义。 +4. 定义优化器和损失函数。 +5. 进行模型训练。基于融合网络训练生成融合模型。 +6. 转化量化网络。基于融合网络训练后得到的融合模型，使用转化接口在融合模型中插入伪量化节点，生成的量化网络。 +7. 进行量化训练。基于量化网络训练，生成量化模型。 + +在上面流程中，第3、6、7步是感知量化训练区别普通训练需要额外进行的步骤。 + +> - 融合网络：使用指定算子（`nn.Conv2dBnAct`、`nn.DenseBnAct`）替换后的网络。 +> - 融合模型：使用融合网络训练生成的checkpoint格式的模型。 +> - 量化网络：融合模型使用转换接口（`convert_quant_network`）插入伪量化节点后得到的网络。 +> - 量化模型：量化网络训练后得到的checkpoint格式的模型。 + +接下来，以LeNet网络为例，展开叙述3、6两个步骤。 + +> 你可以在这里找到完整可运行的样例代码： 。 + +### 定义融合网络 + +定义融合网络，在定义网络后，替换指定的算子。 + +1. 使用`nn.Conv2dBnAct`算子替换原网络模型中的3个算子`nn.Conv2d`、`nn.batchnorm`和`nn.relu`。 +2. 使用`nn.DenseBnAct`算子替换原网络模型中的3个算子`nn.Dense`、`nn.batchnorm`和`nn.relu`。 + +> 即使`nn.Dense`和`nn.Conv2d`算子后面没有`nn.batchnorm`和`nn.relu`，都要按规定使用上述两个算子进行融合替换。 + +原网络模型的定义如下所示： + +```python +class LeNet5(nn.Cell): + def __init__(self, num_class=10): + super(LeNet5, self).__init__() + self.num_class = num_class + + self.conv1 = nn.Conv2d(1, 6, kernel_size=5) + self.bn1 = nn.batchnorm(6) + self.act1 = nn.relu() + + self.conv2 = nn.Conv2d(6, 16, kernel_size=5) + self.bn2 = nn.batchnorm(16) + self.act2 = nn.relu() + + self.fc1 = nn.Dense(16 * 5 * 5, 120) + self.fc2 = nn.Dense(120, 84) + self.act3 = nn.relu() + self.fc3 = nn.Dense(84, self.num_class) + self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) + + def construct(self, x): + x = self.conv1(x) + x = self.bn1(x) + x = self.act1(x) + x = self.max_pool2d(x) + x = self.conv2(x) + x = self.bn2(x) + x = self.act2(x) + x = self.max_pool2d(x) + x = self.flattern(x) + x = self.fc1(x) + x = self.act3(x) + x = self.fc2(x) + x = self.act3(x) + x = self.fc3(x) + return x +``` + +替换算子后的融合网络如下： + +```python +class LeNet5(nn.Cell): + def __init__(self, num_class=10): + super(LeNet5, self).__init__() + self.num_class = num_class + + self.conv1 = nn.Conv2dBnAct(1, 6, kernel_size=5, batchnorm=True, activation='relu') + self.conv2 = nn.Conv2dBnAct(6, 16, kernel_size=5, batchnorm=True, activation='relu') + + self.fc1 = nn.DenseBnAct(16 * 5 * 5, 120, activation='relu') + self.fc2 = nn.DenseBnAct(120, 84, activation='relu') + self.fc3 = nn.DenseBnAct(84, self.num_class) + self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) + + def construct(self, x): + x = self.conv1(x) + x = self.max_pool2d(x) + x = self.conv2(x) + x = self.max_pool2d(x) + x = self.flattern(x) + x = self.fc1(x) + x = self.fc2(x) + x = self.fc3(x) + return x +``` + +### 转化为量化网络 + +使用`convert_quant_network`接口自动在融合模型中插入伪量化节点，将融合模型转化为量化网络。 + +```python +from mindspore.train.quant import quant as qat + +net = qat.convert_quant_network(net, quant_delay=0, bn_fold=False, freeze_bn=10000, weight_bits=8, act_bits=8) +``` + +## 重训和推理 + +### 导入模型重新训练 + +上面介绍了从零开始进行感知量化训练。更常见情况是已有一个模型文件，希望生成量化模型，这时已有正常网络模型训练得到的模型文件及训练脚本，进行感知量化训练。这里使用checkpoint文件重新训练的功能，详细步骤为： + + 1. 数据处理加载数据集。 + 2. 定义网络。 + 3. 定义融合网络。 + 4. 定义优化器和损失函数。 + 5. 加载模型文件模型重训。加载已有模型文件，基于融合网络重新训练生成融合模型。详细模型重载训练，请参见 + 6. 转化量化网络。 + 7. 进行量化训练。 + +### 进行推理 + +使用量化模型进行推理，与普通模型推理一致，分为直接checkpoint文件推理及转化为通用模型格式（ONNX、GEIR等）进行推理。 + +> 推理详细说明请参见。 + +- 使用感知量化训练后得到的checkpoint文件进行推理： + + 1. 加载量化模型。 + 2. 推理。 + +- 转化为ONNX等通用格式进行推理（暂不支持，开发完善后补充）。 + ## 参考文献 [1] Jacob B, Kligys S, Chen B, et al. Quantization and training of neural networks for efficient integer-arithmetic-only inference[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 2704-2713. @@ -151,4 +196,3 @@ MindSpore的感知量化训练是一种伪量化的过程，它是在可识别 [2] Krishnamoorthi R. Quantizing deep convolutional networks for efficient inference: A whitepaper[J]. arXiv preprint arXiv:1806.08342, 2018. [3] Jacob B, Kligys S, Chen B, et al. Quantization and training of neural networks for efficient integer-arithmetic-only inference[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 2704-2713. - diff --git a/tutorials/source_zh_cn/advanced_use/checkpoint_for_hybrid_parallel.md b/tutorials/source_zh_cn/advanced_use/checkpoint_for_hybrid_parallel.md index 6aaf7eb9e7b1628b60282eee3ce068133b8950bf..12a21ea3a1170f9fb2d583e8710656ed57a95fab 100644 --- a/tutorials/source_zh_cn/advanced_use/checkpoint_for_hybrid_parallel.md +++ b/tutorials/source_zh_cn/advanced_use/checkpoint_for_hybrid_parallel.md @@ -26,7 +26,7 @@ - + ## 概述 @@ -316,7 +316,7 @@ load_param_into_net(opt, param_dict) 3. 执行阶段2训练：阶段2为2卡训练环境，每卡上MatMul算子weight的shape为[4, 8]，从合并后的CheckPoint文件加载初始化模型参数数据，之后执行训练。 -> 具体分布式环境配置和训练部分代码，此处不做详细说明，可以参考[分布式并行训练](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/distributed_training.html) +> 具体分布式环境配置和训练部分代码，此处不做详细说明，可以参考[分布式并行训练](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/distributed_training.html) 章节。 > > 本文档附上对CheckPoint文件做合并处理以及分布式训练前加载CheckPoint文件的示例代码，仅作为参考，实际请参考具体情况实现。 diff --git a/tutorials/source_zh_cn/advanced_use/computer_vision_application.md b/tutorials/source_zh_cn/advanced_use/computer_vision_application.md index d8839c1b946e1f51615f5aa23e9d652531e6b8bc..6497d34f07a5851053844b12f1b0b5c4ae6e944a 100644 --- a/tutorials/source_zh_cn/advanced_use/computer_vision_application.md +++ b/tutorials/source_zh_cn/advanced_use/computer_vision_application.md @@ -16,7 +16,7 @@ - + ## 概述 @@ -65,7 +65,7 @@ MindSpore当前支持的图像分类网络包括：典型网络LeNet、AlexNet 6. 加载保存的模型进行推理 -> 本例面向Ascend 910 AI处理器硬件平台，你可以在这里下载完整的样例代码： +> 本例面向Ascend 910 AI处理器硬件平台，你可以在这里下载完整的样例代码： 下面对任务流程中各个环节及代码关键片段进行解释说明。 diff --git a/tutorials/source_zh_cn/advanced_use/customized_debugging_information.md b/tutorials/source_zh_cn/advanced_use/customized_debugging_information.md index 9febbe7c2eb790fc25fd2eca73334faf30515f07..2e25b94c22824c023c10dc570fa141a0e0c80124 100644 --- a/tutorials/source_zh_cn/advanced_use/customized_debugging_information.md +++ b/tutorials/source_zh_cn/advanced_use/customized_debugging_information.md @@ -13,7 +13,7 @@ - + ## 概述 diff --git a/tutorials/source_zh_cn/advanced_use/dashboard_and_lineage.md b/tutorials/source_zh_cn/advanced_use/dashboard_and_lineage.md index d7ac86f0c089880abb8d737c185cb5f05bf1ba64..22c8011d107f5d009736ff0e8a7750e6b0d8f46d 100644 --- a/tutorials/source_zh_cn/advanced_use/dashboard_and_lineage.md +++ b/tutorials/source_zh_cn/advanced_use/dashboard_and_lineage.md @@ -21,7 +21,8 @@ - +   + ## 概述 训练过程中的标量、图像、计算图以及模型超参等信息记录到文件中，通过可视化界面供用户查看。 diff --git a/tutorials/source_zh_cn/advanced_use/debugging_in_pynative_mode.md b/tutorials/source_zh_cn/advanced_use/debugging_in_pynative_mode.md index 08ac3f3f4d8f89b5fe56791ece4f90250c49d125..c5a3e29d10b51bb821b2dc39a9679426a8eaaa9d 100644 --- a/tutorials/source_zh_cn/advanced_use/debugging_in_pynative_mode.md +++ b/tutorials/source_zh_cn/advanced_use/debugging_in_pynative_mode.md @@ -11,7 +11,7 @@ - + ## 概述 diff --git a/tutorials/source_zh_cn/advanced_use/differential_privacy.md b/tutorials/source_zh_cn/advanced_use/differential_privacy.md index 84c381f7cfc2ade4fe58fe837697b83fb2f81b64..0b5e00fc0b67f49e67021aa24d0be69052504135 100644 --- a/tutorials/source_zh_cn/advanced_use/differential_privacy.md +++ b/tutorials/source_zh_cn/advanced_use/differential_privacy.md @@ -1,6 +1,6 @@ # 机器学习中的差分隐私 - + ## 概述 @@ -29,7 +29,7 @@ MindArmour的差分隐私模块Differential-Privacy，实现了差分隐私优 这里以LeNet模型，MNIST 数据集为例，说明如何在MindSpore上使用差分隐私优化器训练神经网络模型。 -> 本例面向Ascend 910 AI处理器，支持PYNATIVE_MODE，你可以在这里下载完整的样例代码： +> 本例面向Ascend 910 AI处理器，支持PYNATIVE_MODE，你可以在这里下载完整的样例代码： ## 实现阶段 diff --git a/tutorials/source_zh_cn/advanced_use/distributed_training.md b/tutorials/source_zh_cn/advanced_use/distributed_training.md index 52627db1c5abfa1614582dcab03a7571d2cbff70..4044307d695d7d6818af0b76f3d3bd84de3881a6 100644 --- a/tutorials/source_zh_cn/advanced_use/distributed_training.md +++ b/tutorials/source_zh_cn/advanced_use/distributed_training.md @@ -18,7 +18,7 @@ - + ## 概述 在深度学习中，当数据集和参数量的规模越来越大，训练所需的时间和硬件资源会随之增加，最后会变成制约训练的瓶颈。分布式并行训练，可以降低对内存、计算性能等硬件的需求，是进行训练的重要优化手段。根据并行的原理及模式不同，业界主流的并行类型有以下几种： @@ -34,7 +34,7 @@ 本篇教程我们主要讲解如何在MindSpore上通过数据并行及自动并行模式训练ResNet-50网络。 > 本例面向Ascend 910 AI处理器硬件平台，暂不支持CPU和GPU场景。 -> 你可以在这里下载完整的样例代码： +> 你可以在这里下载完整的样例代码： ## 准备环节 @@ -175,7 +175,7 @@ def create_dataset(data_path, repeat_num=1, batch_size=32, rank_id=0, rank_size= ## 定义网络 -数据并行及自动并行模式下，网络定义方式与单机一致。代码请参考： +数据并行及自动并行模式下，网络定义方式与单机一致。代码请参考： ## 定义损失函数及优化器 diff --git a/tutorials/source_zh_cn/advanced_use/graph_kernel_fusion.md b/tutorials/source_zh_cn/advanced_use/graph_kernel_fusion.md index 961085f13e9ffc8fab4be64cab69796a6fc22108..26771c1ffe84368d718aa265a11ab5128a76e8bc 100644 --- a/tutorials/source_zh_cn/advanced_use/graph_kernel_fusion.md +++ b/tutorials/source_zh_cn/advanced_use/graph_kernel_fusion.md @@ -12,7 +12,7 @@ - + ## 概述 @@ -99,7 +99,7 @@ context.set_context(enable_graph_kernel=True) 2. `BERT-large`训练网络 以`BERT-large`网络的训练模型为例，数据集和训练脚本可参照 - ，同样我们只需修改`context`参数即可。 + ，同样我们只需修改`context`参数即可。 ## 效果评估 @@ -111,29 +111,42 @@ context.set_context(enable_graph_kernel=True) ```bash pytest -s test_graph_kernel_fusion::test_basic_fuse - ``` - 脚本执行结束后，我们在脚本运行目录可以得到一些`.dot`文件，使用`dot`工具可以将`.dot`文件转换为`.png`文件进行查看。我们以`6_validate.dot`和`hwopt_d_fuse_basic_opt_end_graph_0.dot`生成初始计算图和基础算子融合后计算图。 - 如下图所示，我们构造的网络的初始计算中有两个基础算子计算，打开图算融合的开关之后会自动将两个基础算子(`Mul`、`TensorAdd`)融合为一个算子(组合算子)。第二张图中，右上角部分即为融合之后的组合算子，现在网络只需要执行一个组合算子就可以完成原有的`Mul`、`TensorAdd`两次计算。 + ``` + + 脚本执行结束后，我们在脚本运行目录可以得到一些`.dot`文件，使用`dot`工具可以将`.dot`文件转换为`.png`文件进行查看。我们以`6_validate.dot`和`hwopt_d_fuse_basic_opt_end_graph_0.dot`生成初始计算图和基础算子融合后计算图。 + + 如图1所示，我们构造的网络的初始计算中有两个基础算子计算，打开图算融合的开关之后会自动将两个基础算子(`Mul`、`TensorAdd`)融合为一个算子(组合算子)。图2中，右上角部分即为融合之后的组合算子，现在网络只需要执行一个组合算子就可以完成原有的`Mul`、`TensorAdd`两次计算。 + + ![初始计算图](./images/graph_kernel_fusion_example_fuse_basic_before.png) - | 类别 | 计算图 | - | ------ | ------ | - | 初始计算图 | ![初始计算图](./images/graph_kernel_fusion_example_fuse_basic_before.png) | - | 基础算子融合后计算图 | ![基础算子融合](./images/graph_kernel_fusion_example_fuse_basic_after.png) | + 图1：初始计算图 + + ![基础算子融合](./images/graph_kernel_fusion_example_fuse_basic_after.png) + + 图2：基础算子融合后计算图 2. 组合算子融合场景：组合算子融合是指将原有的组合算子和与其相关的基础算子进行分析，在可以得到性能收益的条件下，将原有的组合算子和基础算子融合成为一个更大的组合算子，以简单样例`NetCompositeFuse`说明。 ```bash pytest -s test_graph_kernel_fusion::test_composite_fuse - ``` - 同样，我们以`6_validate.dot`、`hwopt_d_fuse_basic_opt_end_graph_0.dot`和`hwopt_d_composite_opt_end_graph_0.dot`生成初始计算图、基础算子融合后计算图和组合算子融合后计算图。 - 如下图所示，我们构造的网络的初始计算中有三个基础算子计算，打开图算融合的开关之后，在基础算子融合阶段，会自动将前两个基础算子(`Mul`、`TensorAdd`)融合为一个算子(组合算子)，第二张图中可以看到，右上角部分即为融合之后的组合算子，左下角的主图中还有一个基础算子`Pow`。在接下来的组合算子融合阶段分析后，会进一步将剩余的基础算子(`Pow`)和已有的一个组合算子进行融合，形成一个新的组合算子。第三张图中，右上角部分即为融合三个基础算子之后的组合算子，现在网络只需要执行一个组合算子就可以完成原有的`Mul`、`TensorAdd`、`Pow`三次计算。 + ``` + + 同样，我们以`6_validate.dot`、`hwopt_d_fuse_basic_opt_end_graph_0.dot`和`hwopt_d_composite_opt_end_graph_0.dot`生成初始计算图、基础算子融合后计算图和组合算子融合后计算图。 + + 如图3所示，我们构造的网络的初始计算中有三个基础算子计算，打开图算融合的开关之后，在基础算子融合阶段，会自动将前两个基础算子(`Mul`、`TensorAdd`)融合为一个算子(组合算子)。从图4中可以看到，右上角部分即为融合之后的组合算子，左下角的主图中还有一个基础算子`Pow`。在接下来的组合算子融合阶段分析后，会进一步将剩余的基础算子(`Pow`)和已有的一个组合算子进行融合，形成一个新的组合算子。图5中，右上角部分即为融合三个基础算子之后的组合算子，现在网络只需要执行一个组合算子就可以完成原有的`Mul`、`TensorAdd`、`Pow`三次计算。 + + ![初始计算图](./images/graph_kernel_fusion_example_fuse_composite_before.png) + + 图3：初始计算图 + + ![基础算子融合](./images/graph_kernel_fusion_example_fuse_composite_middle.png) + + 图4：基础算子融合后计算图 - | 类别 | 计算图 | - | ------ | ------ | - | 初始计算图 | ![初始计算图](./images/graph_kernel_fusion_example_fuse_composite_before.png) | - | 基础算子融合后计算图 | ![基础算子融合](./images/graph_kernel_fusion_example_fuse_composite_middle.png) | - | 组合算子融合后计算图 | ![组合算子融合](./images/graph_kernel_fusion_example_fuse_composite_after.png) | + ![组合算子融合](./images/graph_kernel_fusion_example_fuse_composite_after.png) + 图5：组合算子融合后计算图 + ### 训练单step时间 BERT-large场景：BERT-large网络启用图算融合后，在保持与启用前精度一致的前提下，训练的单step时间可提升5%左右。 diff --git a/tutorials/source_zh_cn/advanced_use/images/data_op_profile.png b/tutorials/source_zh_cn/advanced_use/images/data_op_profile.png index 6a03815bac3797b1333050e6eae1c89950e01a1c..b83408e92777181f6447ec20239fc92e28084a6a 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/data_op_profile.png and b/tutorials/source_zh_cn/advanced_use/images/data_op_profile.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png index fe12a44349f8edad6945312acb21403f5115695f..bcc85cbde11d73e8dd24e1744269cbf1543c87a0 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png and b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_after.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png index a31ec3cff7e49704151518a718e2157dce2c1d23..7e400d641f7fd1eaf0654c0802fd2ba9b77b78fb 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png and b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_basic_before.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png index 1ee038af8f83b3877b605c8fa5399b45c9115e64..74b761f33fb710de15acc0bcdcbbe8a6af0d05d4 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png and b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_after.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png index 5a18b93dc616936bef6723c47772a405af02130c..85faa754d9c18738260e1dfddf791bbd714f0d78 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png and b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_before.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png index 36081dd9b1bf4e6199912af63b5cc86abb1cf748..638f7b73bdfc5be6248073e14c7a26588792233c 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png and b/tutorials/source_zh_cn/advanced_use/images/graph_kernel_fusion_example_fuse_composite_middle.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/minddata_profile.png b/tutorials/source_zh_cn/advanced_use/images/minddata_profile.png index 035939f5e3d548f39e2f5c6c16b2bc7d0c7469ce..79dfad25e6828769a2efc697bb7b02a171dbbdd0 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/minddata_profile.png and b/tutorials/source_zh_cn/advanced_use/images/minddata_profile.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/performance_overall.png b/tutorials/source_zh_cn/advanced_use/images/performance_overall.png index 2d627f972cac0b7848eff1114b0fd2fa4f030e74..3aa536d5f24fc348ad013fa07084fddb1b4f01af 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/performance_overall.png and b/tutorials/source_zh_cn/advanced_use/images/performance_overall.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/step_trace.png b/tutorials/source_zh_cn/advanced_use/images/step_trace.png index 6c54e790e34f52780e4c16f487f81a39906512bf..49c8bb72741173cd3285bfdbacfb206dbc33e3a9 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/step_trace.png and b/tutorials/source_zh_cn/advanced_use/images/step_trace.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/timeline.png b/tutorials/source_zh_cn/advanced_use/images/timeline.png index 21453967d9799b73795fae05529cda6fcb82f6ee..19c60e104169d86f1022758eda15bbc9c8a0dcf6 100644 Binary files a/tutorials/source_zh_cn/advanced_use/images/timeline.png and b/tutorials/source_zh_cn/advanced_use/images/timeline.png differ diff --git a/tutorials/source_zh_cn/advanced_use/mindinsight_commands.md b/tutorials/source_zh_cn/advanced_use/mindinsight_commands.md new file mode 100644 index 0000000000000000000000000000000000000000..b611bab4f433fe71ce4e80efba03c126a04f7d37 --- /dev/null +++ b/tutorials/source_zh_cn/advanced_use/mindinsight_commands.md @@ -0,0 +1,64 @@ +# MindInsight相关命令 + +1. 查看命令帮助信息 + + ```shell + mindinsight --help + ``` + +2. 查看版本信息 + + ```shell + mindinsight --version + ``` + +3. 启动服务 + + ```shell + mindinsight start [-h] [--config ] [--workspace ] + [--port ] [--reload-interval ] + [--summary-base-dir ] + ``` + + 参数含义如下: + + - `-h, --help` : 显示启动命令的帮助信息。 + - `--config ` : 指定配置文件或配置模块，CONFIG为物理文件路径（file:/path/to/config.py）或Python可识别的模块路径（python:path.to.config.module）。 + - `--workspace ` : 指定工作目录路径，WORKSPACE默认为 $HOME/mindinsight。 + - `--port ` : 指定Web可视化服务端口，取值范围是1~65535，PORT默认为8080。 + - `--url-path-prefix ` : 指定Web服务地址前缀，URL_PATH_PREFIX默认为空。 + - `--reload-interval ` : 指定加载数据的时间间隔（单位：秒），设置为0时表示只加载一次数据，RELOAD_INTERVAL默认为3秒。 + - `--summary-base-dir ` : 指定加载训练日志数据的根目录路径，MindInsight将遍历此路径下的直属子目录。若某个直属子目录包含日志文件，则该子目录被识别为日志文件目录，若根目录包含日志文件，则根目录被识别为日志文件目录。SUMMARY_BASE_DIR默认为当前目录路径。 + + > 服务启动时，命令行参数值将被保存为进程的环境变量，并以 `MINDINSIGHT_` 开头作为标识，如 `MINDINSIGHT_CONFIG`，`MINDINSIGHT_WORKSPACE`，`MINDINSIGHT_PORT` 等。 + +4. 查看服务进程信息 + + MindInsight向用户提供Web服务，可通过以下命令，查看当前运行的Web服务进程。 + + ```shell + ps -ef | grep mindinsight + ``` + + 根据服务进程PID，可通过以下命令，查看当前服务进程对应的工作目录`WORKSPACE`。 + + ```shell + lsof -p | grep access + ``` + + 输出如下，可查看`WORKSPACE`。 + + ```shell + gunicorn /log/gunicorn/access.log + ``` + +5. 停止服务 + + ```shell + mindinsight stop [-h] [--port PORT] + ``` + + 参数含义如下: + + - `-h, --help` : 显示停止命令的帮助信息。 + - `--port ` : 指定Web可视化服务端口，取值范围是1~65535，PORT默认为8080。 diff --git a/tutorials/source_zh_cn/advanced_use/mixed_precision.md b/tutorials/source_zh_cn/advanced_use/mixed_precision.md index 4cacb8c7a40d5cd55ea31bbc7a078f6dab3ab10e..4df6ab97dfe93bc43bbe9a81894480691e4d3670 100644 --- a/tutorials/source_zh_cn/advanced_use/mixed_precision.md +++ b/tutorials/source_zh_cn/advanced_use/mixed_precision.md @@ -10,7 +10,7 @@ - + ## 概述 diff --git a/tutorials/source_zh_cn/advanced_use/model_security.md b/tutorials/source_zh_cn/advanced_use/model_security.md index 9daf517e31f8ea9c295ecc3fc9b993eab417148a..d958bbc494a29f7d2027e78690ab52e629541365 100644 --- a/tutorials/source_zh_cn/advanced_use/model_security.md +++ b/tutorials/source_zh_cn/advanced_use/model_security.md @@ -15,7 +15,7 @@ - + ## 概述 @@ -28,7 +28,7 @@ AI算法设计之初普遍未考虑相关的安全威胁，使得AI算法的判 这里通过图像分类任务上的对抗性攻防，以攻击算法FGSM和防御算法NAD为例，介绍MindArmour在对抗攻防上的使用方法。 -> 本例面向CPU、GPU、Ascend 910 AI处理器，你可以在这里下载完整的样例代码： +> 本例面向CPU、GPU、Ascend 910 AI处理器，你可以在这里下载完整的样例代码： > - `mnist_attack_fgsm.py`：包含攻击代码。 > - `mnist_defense_nad.py`：包含防御代码。 diff --git a/tutorials/source_zh_cn/advanced_use/network_migration.md b/tutorials/source_zh_cn/advanced_use/network_migration.md index 3e9cf2e776d0ffc163e2cb8dd83da375bfd4d752..8e26ee5e145e0c92c4f68bf3ab3b3b448df58ab2 100644 --- a/tutorials/source_zh_cn/advanced_use/network_migration.md +++ b/tutorials/source_zh_cn/advanced_use/network_migration.md @@ -17,7 +17,7 @@ - + ## 概述 @@ -29,9 +29,9 @@ ### 算子评估 -分析待迁移的网络中所包含的算子，结合[MindSpore算子支持列表](https://www.mindspore.cn/docs/zh-CN/master/operator_list.html)，梳理出MindSpore对这些算子的支持程度。 +分析待迁移的网络中所包含的算子，结合[MindSpore算子支持列表](https://www.mindspore.cn/docs/zh-CN/r0.5/operator_list.html)，梳理出MindSpore对这些算子的支持程度。 -以ResNet-50为例，[Conv](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2d)和[BatchNorm](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d)是其中最主要的两个算子，它们已在MindSpore支持的算子列表中。 +以ResNet-50为例，[Conv](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2d)和[BatchNorm](https://www.mindspore.cn/api/zh-CN/r0.5/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d)是其中最主要的两个算子，它们已在MindSpore支持的算子列表中。 如果发现没有对应算子，建议： - 使用其他算子替换：分析算子实现公式，审视是否可以采用MindSpore现有算子叠加达到预期目标。 @@ -55,17 +55,17 @@ MindSpore与TensorFlow、PyTorch在网络结构组织方式上，存在一定差别，迁移前需要对原脚本有较为清晰的了解，明确地知道每一层的shape等信息。 -> 你也可以使用[MindConverter工具](https://gitee.com/mindspore/mindinsight/tree/master/mindinsight/mindconverter)实现PyTorch网络定义脚本到MindSpore网络定义脚本的自动转换。 +> 你也可以使用[MindConverter工具](https://gitee.com/mindspore/mindinsight/tree/r0.5/mindinsight/mindconverter)实现PyTorch网络定义脚本到MindSpore网络定义脚本的自动转换。 下面，我们以ResNet-50的迁移，并在Ascend 910上训练为例： 1. 导入MindSpore模块。 - 根据所需使用的接口，导入相应的MindSpore模块，模块列表详见。 + 根据所需使用的接口，导入相应的MindSpore模块，模块列表详见。 2. 加载数据集和预处理。 - 使用MindSpore构造你需要使用的数据集。目前MindSpore已支持常见数据集，你可以通过原始格式、`MindRecord`、`TFRecord`等多种接口调用，同时还支持数据处理以及数据增强等相关功能，具体用法可参考[准备数据教程](https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/data_preparation.html)。 + 使用MindSpore构造你需要使用的数据集。目前MindSpore已支持常见数据集，你可以通过原始格式、`MindRecord`、`TFRecord`等多种接口调用，同时还支持数据处理以及数据增强等相关功能，具体用法可参考[准备数据教程](https://www.mindspore.cn/tutorial/zh-CN/r0.5/use/data_preparation/data_preparation.html)。 本例中加载了Cifar-10数据集，可同时支持单卡和多卡的场景。 @@ -77,7 +77,7 @@ MindSpore与TensorFlow、PyTorch在网络结构组织方式上，存在一定差 num_shards=device_num, shard_id=rank_id) ``` - 然后对数据进行了数据增强、数据清洗和批处理等操作。代码详见。 + 然后对数据进行了数据增强、数据清洗和批处理等操作。代码详见。 3. 构建网络。 @@ -210,7 +210,7 @@ MindSpore与TensorFlow、PyTorch在网络结构组织方式上，存在一定差 6. 构造整网。 - 将定义好的多个子网连接起来就是整个[ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/resnet.py)网络的结构了。同样遵循先定义后使用的原则，在`__init__`中定义所有用到的子网，在`construct`中连接子网。 + 将定义好的多个子网连接起来就是整个[ResNet-50](https://gitee.com/mindspore/mindspore/blob/r0.5/model_zoo/resnet/src/resnet.py)网络的结构了。同样遵循先定义后使用的原则，在`__init__`中定义所有用到的子网，在`construct`中连接子网。 7. 定义损失函数和优化器。 @@ -231,7 +231,7 @@ MindSpore与TensorFlow、PyTorch在网络结构组织方式上，存在一定差 loss_scale = FixedLossScaleManager(config.loss_scale, drop_overflow_update=False) ``` - 如果希望使用`Model`内置的评估方法，则可以使用[metrics](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/customized_debugging_information.html#mindspore-metrics)属性设置希望使用的评估方法。 + 如果希望使用`Model`内置的评估方法，则可以使用[metrics](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/customized_debugging_information.html#mindspore-metrics)属性设置希望使用的评估方法。 ```python model = Model(net, loss_fn=loss, optimizer=opt, loss_scale_manager=loss_scale, metrics={'acc'}) @@ -259,16 +259,14 @@ MindSpore与TensorFlow、PyTorch在网络结构组织方式上，存在一定差 #### 云上集成 -请参考[在云上使用MindSpore](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/use_on_the_cloud.html)，将你的脚本运行在ModelArts。 +请参考[在云上使用MindSpore](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/use_on_the_cloud.html)，将你的脚本运行在ModelArts。 ### 推理阶段 -在Ascend 910 AI处理器上训练后的模型，支持在不同的硬件平台上执行推理。详细步骤可参考[多平台推理教程](https://www.mindspore.cn/tutorial/zh-CN/master/use/multi_platform_inference.html)。 +在Ascend 910 AI处理器上训练后的模型，支持在不同的硬件平台上执行推理。详细步骤可参考[多平台推理教程](https://www.mindspore.cn/tutorial/zh-CN/r0.5/use/multi_platform_inference.html)。 ## 样例参考 -1. [常用网络脚本样例](https://gitee.com/mindspore/mindspore/tree/master/example) +1. [常用数据集读取样例](https://www.mindspore.cn/tutorial/zh-CN/r0.5/use/data_preparation/loading_the_datasets.html) -2. [常用数据集读取样例](https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/loading_the_datasets.html) - -3. [Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/mindspore/model_zoo) \ No newline at end of file +2. [Model Zoo](https://gitee.com/mindspore/mindspore/tree/r0.5/model_zoo) \ No newline at end of file diff --git a/tutorials/source_zh_cn/advanced_use/nlp_application.md b/tutorials/source_zh_cn/advanced_use/nlp_application.md index ace94f759da81b922d1654991efa567987cfb8cb..8f6ba8d445c20709b4e23e93a16dd0870f94e557 100644 --- a/tutorials/source_zh_cn/advanced_use/nlp_application.md +++ b/tutorials/source_zh_cn/advanced_use/nlp_application.md @@ -20,7 +20,7 @@ - + ## 概述 @@ -85,7 +85,7 @@ $F1分数 = (2 * Precision * Recall) / (Precision + Recall)$ > LSTM（Long short-term memory，长短期记忆）网络是一种时间循环神经网络，适合于处理和预测时间序列中间隔和延迟非常长的重要事件。具体介绍可参考网上资料，在此不再赘述。 3. 得到模型之后，使用验证数据集，查看模型精度情况。 -> 本例面向GPU或CPU硬件平台，你可以在这里下载完整的样例代码： +> 本例面向GPU或CPU硬件平台，你可以在这里下载完整的样例代码： > - `main.py`：代码文件，包括数据预处理、网络定义、模型训练等代码。 > - `config.py`：网络中的一些配置，包括`batch size`、进行几次epoch训练等。 diff --git a/tutorials/source_zh_cn/advanced_use/on_device_inference.md b/tutorials/source_zh_cn/advanced_use/on_device_inference.md index 446a65e5de3a143a08864b228f3b2bd79d42a256..7de3f8f563cd4c2eee1e6048217590ae866d9de6 100644 --- a/tutorials/source_zh_cn/advanced_use/on_device_inference.md +++ b/tutorials/source_zh_cn/advanced_use/on_device_inference.md @@ -11,7 +11,7 @@ - + ## 概述 @@ -28,8 +28,8 @@ MindSpore Predict是一个轻量级的深度神经网络推理引擎，提供了 - 硬盘空间10GB以上 - 系统要求 - - 系统：Ubuntu = 16.04.02LTS（验证可用） - - 内核：4.4.0-62-generic（验证可用） + - 系统：Ubuntu = 18.04.02LTS（验证可用） + - 内核：4.15.0-45-generic（验证可用） - 软件依赖 - [cmake](https://cmake.org/download/) >= 3.14.1 @@ -59,7 +59,7 @@ MindSpore Predict是一个轻量级的深度神经网络推理引擎，提供了 2. 从代码仓下载源码。 ```bash - git clone https://gitee.com/mindspore/mindspore.git + git clone https://gitee.com/mindspore/mindspore.git -b r0.5 ``` 3. 在源码根目录下，执行如下命令编译MindSpore Predict。-I为编译MindSpore Predict的编译参数，-I的参数为目标端侧平台，目前仅支持安卓arm64平台。 @@ -86,7 +86,7 @@ MindSpore进行端侧模型推理的步骤如下。 ### 生成端侧模型文件 1. 加载训练完毕所生成的CheckPoint文件至定义好的网络中。 ```python - param_dict = load_checkpoint(ckpoint_file_name=ckpt_file_path) + param_dict = load_checkpoint(ckpt_file_name=ckpt_file_path) load_param_into_net(net, param_dict) ``` 2. 调用`export`接口，导出端侧模型文件(`.ms`)。 @@ -144,7 +144,7 @@ if __name__ == '__main__': is_ckpt_exist = os.path.exists(ckpt_file_path) if is_ckpt_exist: - param_dict = load_checkpoint(ckpoint_file_name=ckpt_file_path) + param_dict = load_checkpoint(ckpt_file_name=ckpt_file_path) load_param_into_net(net, param_dict) export(net, input_data, file_name="./lenet.ms", file_format='LITE') print("export model success.") diff --git a/tutorials/source_zh_cn/advanced_use/performance_profiling.md b/tutorials/source_zh_cn/advanced_use/performance_profiling.md index 94c9fc8ae7838ce3261dda151609b976529f3b94..47b3a1cc28a312e073ef7155f169cd3c0155514a 100644 --- a/tutorials/source_zh_cn/advanced_use/performance_profiling.md +++ b/tutorials/source_zh_cn/advanced_use/performance_profiling.md @@ -17,7 +17,7 @@ - + ## 概述 将训练过程中的算子耗时等信息记录到文件中，通过可视化界面供用户查看分析，帮助用户更高效地调试神经网络性能。目前仅支持在Ascend芯片上的性能调试。 @@ -66,7 +66,7 @@ def test_profiler(): ## 启动MindInsight -启动命令请参考[训练过程可视](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/visualization_tutorials.html)中**MindInsight相关命令**小节。 +启动命令请参考[MindInsight相关命令](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/mindinsight_commands.html)。 ### 性能分析 @@ -164,6 +164,10 @@ Timeline组件可以展示： 通过分析Timeline，用户可以对训练过程进行细粒度分析：从High Level层面，可以分析流切分方法是否合理、迭代间隙和拖尾时间是否过长等；从Low Level层面，可以分析算子执行时间等。 +用户可以点击总览页面Timeline部分的下载按钮，将Timeline数据文件 (json格式) 保存至本地，再通过工具查看Timeline的详细信息。推荐使用 `chrome://tracing` 或者 [Perfetto](https://ui.perfetto.dev/#!viewer) 做Timeline展示。 +- Chrome tracing：点击左上角"load"加载文件。 +- Perfetto：点击左侧"Open trace file"加载文件。 + ![timeline.png](./images/timeline.png) 图7：Timeline分析 diff --git a/tutorials/source_zh_cn/advanced_use/use_on_the_cloud.md b/tutorials/source_zh_cn/advanced_use/use_on_the_cloud.md index 8b6a147e3140b500abfd9bee18bdc7c70d390e5a..d648ced35b10c99d3226a1768a1b571a06fdd4bc 100644 --- a/tutorials/source_zh_cn/advanced_use/use_on_the_cloud.md +++ b/tutorials/source_zh_cn/advanced_use/use_on_the_cloud.md @@ -24,7 +24,7 @@ - + ## 概述 @@ -69,7 +69,7 @@ ModelArts使用对象存储服务（Object Storage Service，简称OBS）进行 ### 执行脚本准备 新建一个自己的OBS桶（例如：`resnet50-train`），在桶中创建代码目录（例如：`resnet50_cifar10_train`），并将以下目录中的所有脚本上传至代码目录： -> 脚本使用ResNet-50网络在CIFAR-10数据集上进行训练，并在训练结束后验证精度。脚本可以在ModelArts采用`1*Ascend`或`8*Ascend`两种不同规格进行训练任务。 +> 脚本使用ResNet-50网络在CIFAR-10数据集上进行训练，并在训练结束后验证精度。脚本可以在ModelArts采用`1*Ascend`或`8*Ascend`两种不同规格进行训练任务。 为了方便后续创建训练作业，先创建训练输出目录和日志输出目录，本示例创建的目录结构如下： diff --git a/tutorials/source_zh_cn/advanced_use/visualization_tutorials.rst b/tutorials/source_zh_cn/advanced_use/visualization_tutorials.rst index 2b6b4aa018d3d849bc0963a2a134ac16a0383d8f..814a815ae3f5ee65736374f2f474cad13fec9887 100644 --- a/tutorials/source_zh_cn/advanced_use/visualization_tutorials.rst +++ b/tutorials/source_zh_cn/advanced_use/visualization_tutorials.rst @@ -6,71 +6,4 @@ dashboard_and_lineage performance_profiling - -MindInsight相关命令 --------------------- - -1. 查看命令帮助信息 - - .. code-block:: - - mindinsight --help - -2. 查看版本信息 - - .. code-block:: - - mindinsight --version - -3. 启动服务 - - .. code-block:: - - mindinsight start [-h] [--config ] [--workspace ] - [--port ] [--reload-interval ] - [--summary-base-dir ] - - 参数含义如下: - - - `-h, --help` : 显示启动命令的帮助信息。 - - `--config ` : 指定配置文件或配置模块，CONFIG为物理文件路径（file:/path/to/config.py）或Python可识别的模块路径（python:path.to.config.module）。 - - `--workspace ` : 指定工作目录路径，WORKSPACE默认为 $HOME/mindinsight。 - - `--port ` : 指定Web可视化服务端口，取值范围是1~65535，PORT默认为8080。 - - `--url-path-prefix ` : 指定Web服务地址前缀，URL_PATH_PREFIX默认为空。 - - `--reload-interval ` : 指定加载数据的时间间隔（单位：秒），设置为0时表示只加载一次数据，RELOAD_INTERVAL默认为3秒。 - - `--summary-base-dir ` : 指定加载训练日志数据的根目录路径，MindInsight将遍历此路径下的直属子目录。若某个直属子目录包含日志文件，则该子目录被识别为日志文件目录，若根目录包含日志文件，则根目录被识别为日志文件目录。SUMMARY_BASE_DIR默认为当前目录路径。 - - .. note:: - - 服务启动时，命令行参数值将被保存为进程的环境变量，并以 `MINDINSIGHT_` 开头作为标识，如 `MINDINSIGHT_CONFIG`，`MINDINSIGHT_WORKSPACE`，`MINDINSIGHT_PORT` 等。 - -4. 查看服务进程信息 - - MindInsight向用户提供Web服务，可通过以下命令，查看当前运行的Web服务进程。 - - .. code-block:: - - ps -ef | grep mindinsight - - 根据服务进程PID，可通过以下命令，查看当前服务进程对应的工作目录`WORKSPACE`。 - - .. code-block:: - - lsof -p | grep access - - 输出如下，可查看`WORKSPACE`。 - - .. code-block:: - - gunicorn /log/gunicorn/access.log - -5. 停止服务 - - .. code-block:: - - mindinsight stop [-h] [--port PORT] - - 参数含义如下: - - - `-h, --help` : 显示停止命令的帮助信息。 - - `--port ` : 指定Web可视化服务端口，取值范围是1~65535，PORT默认为8080。 + mindinsight_commands diff --git a/tutorials/source_zh_cn/conf.py b/tutorials/source_zh_cn/conf.py index c2d1fb828249e37f44afaa4cda7e45d641784785..b47a4a74ecbd7554d930889e9c41b51e552ec0b7 100644 --- a/tutorials/source_zh_cn/conf.py +++ b/tutorials/source_zh_cn/conf.py @@ -21,7 +21,7 @@ copyright = '2020, MindSpore' author = 'MindSpore' # The full version, including alpha/beta/rc tags -release = 'master' +release = 'r0.5' # -- General configuration --------------------------------------------------- diff --git a/tutorials/source_zh_cn/index.rst b/tutorials/source_zh_cn/index.rst index 8a241a818c611eabfeecac74d6987e2368cb0eb7..7a0252dd78f775ec5212ae1dd479e027454ed3cc 100644 --- a/tutorials/source_zh_cn/index.rst +++ b/tutorials/source_zh_cn/index.rst @@ -48,6 +48,7 @@ MindSpore教程 advanced_use/distributed_training_tutorials advanced_use/mixed_precision + advanced_use/graph_kernel_fusion advanced_use/aware_quantization .. toctree:: diff --git a/tutorials/source_zh_cn/quick_start/quick_start.md b/tutorials/source_zh_cn/quick_start/quick_start.md index 19a7c8e7a442b7760079e5297f2f1eb8788c8c8c..7212b63fda8596d98f4892011b62efde09a16574 100644 --- a/tutorials/source_zh_cn/quick_start/quick_start.md +++ b/tutorials/source_zh_cn/quick_start/quick_start.md @@ -24,9 +24,9 @@ - +    - + ## 概述 @@ -40,7 +40,7 @@ 5. 加载保存的模型，进行推理。 6. 验证模型，加载测试数据集和训练后的模型，验证结果精度。 -> 你可以在这里找到完整可运行的样例代码： 。 +> 你可以在这里找到完整可运行的样例代码： 。 这是简单、基础的应用流程，其他高级、复杂的应用可以基于这个基本流程进行扩展。 @@ -86,7 +86,7 @@ import os ``` -详细的MindSpore的模块说明，可以在[MindSpore API页面](https://www.mindspore.cn/api/zh-CN/master/index.html)中搜索查询。 +详细的MindSpore的模块说明，可以在[MindSpore API页面](https://www.mindspore.cn/api/zh-CN/r0.5/index.html)中搜索查询。 ### 配置运行信息 @@ -182,7 +182,7 @@ def create_dataset(data_path, batch_size=32, repeat_size=1, 先进行shuffle、batch操作，再进行repeat操作，这样能保证1个epoch内数据不重复。 -> MindSpore支持进行多种数据处理和增强的操作，各种操作往往组合使用，具体可以参考[数据处理与数据增强](https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/data_processing_and_augmentation.html)章节。 +> MindSpore支持进行多种数据处理和增强的操作，各种操作往往组合使用，具体可以参考[数据处理与数据增强](https://www.mindspore.cn/tutorial/zh-CN/r0.5/use/data_preparation/data_processing_and_augmentation.html)章节。 ## 定义网络 diff --git a/tutorials/source_zh_cn/quick_start/quick_video.md b/tutorials/source_zh_cn/quick_start/quick_video.md index ccc368390d3236920e52914bf6640ea35df81120..5f0e8be9f6c195e9398cef2331036baea5e83e8a 100644 --- a/tutorials/source_zh_cn/quick_start/quick_video.md +++ b/tutorials/source_zh_cn/quick_start/quick_video.md @@ -43,7 +43,7 @@ -**查看代码**： +**查看代码**： ### 模型参数保存与加载 diff --git a/tutorials/source_zh_cn/use/custom_operator.md b/tutorials/source_zh_cn/use/custom_operator.md index 064b271fd8daedef06cebffb2b552f8322daebd1..640d9f8d93645d8a12472a04a7e0b51888b8a043 100644 --- a/tutorials/source_zh_cn/use/custom_operator.md +++ b/tutorials/source_zh_cn/use/custom_operator.md @@ -14,7 +14,7 @@ - + ## 概述 @@ -27,14 +27,14 @@ - 算子实现：通过TBE（Tensor Boost Engine）提供的特性语言接口，描述算子内部计算逻辑的实现。TBE提供了开发昇腾AI芯片自定义算子的能力。你可以在页面申请公测。 - 算子信息：描述TBE算子的基本信息，如算子名称、支持的输入输出类型等。它是后端做算子选择和映射时的依据。 -本文将以自定义Square算子为例，介绍自定义算子的步骤。更多详细内容可参考MindSpore源码中[tests/st/ops/custom_ops_tbe](https://gitee.com/mindspore/mindspore/tree/master/tests/st/ops/custom_ops_tbe)下的用例。 +本文将以自定义Square算子为例，介绍自定义算子的步骤。更多详细内容可参考MindSpore源码中[tests/st/ops/custom_ops_tbe](https://gitee.com/mindspore/mindspore/tree/r0.5/tests/st/ops/custom_ops_tbe)下的用例。 ## 注册算子原语 每个算子的原语是一个继承于`PrimitiveWithInfer`的子类，其类型名称即是算子名称。 自定义算子原语与内置算子原语的接口定义完全一致： -- 属性由构造函数`__init__`的入参定义。本用例的算子没有属性，因此`__init__`没有额外的入参。带属性的用例可参考MindSpore源码中的[custom add3](https://gitee.com/mindspore/mindspore/tree/master/tests/st/ops/custom_ops_tbe/cus_add3.py)用例。 +- 属性由构造函数`__init__`的入参定义。本用例的算子没有属性，因此`__init__`没有额外的入参。带属性的用例可参考MindSpore源码中的[custom add3](https://gitee.com/mindspore/mindspore/tree/r0.5/tests/st/ops/custom_ops_tbe/cus_add3.py)用例。 - 输入输出的名称通过`init_prim_io_names`函数定义。 - 输出Tensor的shape推理方法在`infer_shape`函数中定义，输出Tensor的dtype推理方法在`infer_dtype`函数中定义。 diff --git a/tutorials/source_zh_cn/use/data_preparation/converting_datasets.md b/tutorials/source_zh_cn/use/data_preparation/converting_datasets.md index 7bd585166ad30dc26656b09a6686fd7cacc8b398..84f91840551297c65bb37ac44ddbcc55c17045f1 100644 --- a/tutorials/source_zh_cn/use/data_preparation/converting_datasets.md +++ b/tutorials/source_zh_cn/use/data_preparation/converting_datasets.md @@ -14,7 +14,7 @@ - + ## 概述 @@ -178,12 +178,14 @@ MindSpore提供转换常见数据集的工具类，将常见数据集转换为Mi 对下载后的ImageNet数据集，整理数据集组织形式为一个包含所有图片的文件夹，以及一个记录图片对应标签的映射文件。 - 标签映射文件包含3列，分别为各类别图片目录、标签ID、标签名，用空格隔开，映射文件示例如下： + 标签映射文件包含2列，分别为各类别图片目录、标签ID，用空格隔开，映射文件示例如下： ``` - n02119789 1 pen - n02100735 2 notbook - n02110185 3 mouse - n02096294 4 orange + n01440760 0 + n01443537 1 + n01484850 2 + n01491361 3 + n01494475 4 + n01496331 5 ``` 2. 导入转换数据集的工具类`ImageNetToMR`。 diff --git a/tutorials/source_zh_cn/use/data_preparation/data_processing_and_augmentation.md b/tutorials/source_zh_cn/use/data_preparation/data_processing_and_augmentation.md index 13fc97b52e907834b2d88751ae82adc472a1803e..a5537f1e96a110c002998e721867e33da0ea6967 100644 --- a/tutorials/source_zh_cn/use/data_preparation/data_processing_and_augmentation.md +++ b/tutorials/source_zh_cn/use/data_preparation/data_processing_and_augmentation.md @@ -16,7 +16,7 @@ - + ## 概述 @@ -278,7 +278,7 @@ MindSpore提供`c_transforms`模块以及`py_transforms`模块函数供用户进 ``` 2. 定义数据增强算子，以`Resize`为例： ```python - dataset = ds.ImageFolderDatasetV2(DATA_DIR, decode=True) # Deocde images. + dataset = ds.ImageFolderDatasetV2(DATA_DIR, decode=True) # Decode images. resize_op = transforms.Resize(size=(500,500), interpolation=Inter.LINEAR) dataset.map(input_columns="image", operations=resize_op) diff --git a/tutorials/source_zh_cn/use/data_preparation/loading_the_datasets.md b/tutorials/source_zh_cn/use/data_preparation/loading_the_datasets.md index 3186acaa2bd0c2779064fc2ae411bd324d8898f2..31ae059449b925e9f2be56d72ec189cf05ce286f 100644 --- a/tutorials/source_zh_cn/use/data_preparation/loading_the_datasets.md +++ b/tutorials/source_zh_cn/use/data_preparation/loading_the_datasets.md @@ -13,7 +13,7 @@ - + ## 概述 @@ -65,7 +65,7 @@ MindSpore天然支持读取MindSpore数据格式——`MindRecord`存储的数 data_set = ds.MindDataset(dataset_file=CV_FILE_NAME) ``` 其中， - `dataset_file`：指定MindRecord的文件，含路径及文件名。 + `dataset_file`：指定MindRecord的文件或文件列表。 2. 创建字典迭代器，通过迭代器读取数据记录。 ```python @@ -148,32 +148,110 @@ MindSpore也支持读取`TFRecord`数据格式的数据集，可以通过`TFReco ``` ## 加载自定义数据集 -对于自定义数据集，可以通过`GeneratorDataset`对象加载。 +现实场景中，数据集的种类多种多样，对于自定义数据集或者目前不支持直接加载的数据集，有两种方法可以处理。 +一种方法是将数据集转成MindRecord格式（请参考[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/r0.5/use/data_preparation/converting_datasets.html)章节），另一种方法是通过`GeneratorDataset`对象加载，以下将展示如何使用`GeneratorDataset`。 -1. 定义一个函数（示例函数名为`Generator1D`）用于生成数据集的函数。 - > 自定义的生成函数返回的是可调用的对象，每次返回`numpy array`的元组，作为一行数据。 +1. 定义一个可迭代的对象，用于生成数据集。以下展示了两种示例，一种是含有`yield`返回值的自定义函数，另一种是含有`__getitem__`的自定义类。两种示例都将产生一个含有从0到9数字的数据集。 + > 自定义的可迭代对象，每次返回`numpy array`的元组，作为一行数据。 自定义函数示例如下： ```python import numpy as np # Import numpy lib. - def Generator1D(): - for i in range(64): + def generator_func(num): + for i in range(num): yield (np.array([i]),) # Notice, tuple of only one element needs following a comma at the end. ``` -2. 将`Generator1D`传入`GeneratorDataset`创建数据集，并设定`column`名为“data”。 + + 自定义类示例如下： ```python - dataset = ds.GeneratorDataset(Generator1D, ["data"]) + import numpy as np # Import numpy lib. + class Generator(): + + def __init__(self, num): + self.num = num + + def __getitem__(self, item): + return (np.array([item]),) # Notice, tuple of only one element needs following a comma at the end. + + def __len__(self): + return self.num + ``` +2. 使用`GeneratorDataset`创建数据集。将`generator_func`传入`GeneratorDataset`创建数据集`dataset1`，并设定`column`名为“data” 。 + 将定义的`Generator`对象传入`GeneratorDataset`创建数据集`dataset2`，并设定`column`名为“data” 。 + ```python + dataset1 = ds.GeneratorDataset(source=generator_func(10), column_names=["data"], shuffle=False) + dataset2 = ds.GeneratorDataset(source=Generator(10), column_names=["data"], shuffle=False) ``` 3. 在创建数据集后，可以通过给数据创建迭代器的方式，获取相应的数据。有两种创建迭代器的方法。 - - 创建返回值为序列类型的迭代器。 + - 创建返回值为序列类型的迭代器。以下分别对`dataset1`和`dataset2`创建迭代器，并打印输出数据观察结果。 ```python - for data in dataset.create_tuple_iterator(): # each data is a sequence - print(data[0]) + print("dataset1:") + for data in dataset1.create_tuple_iterator(): # each data is a sequence + print(data) + + print("dataset2:") + for data in dataset2.create_tuple_iterator(): # each data is a sequence + print(data) ``` - - - 创建返回值为字典类型的迭代器。 - ```python - for data in dataset.create_dict_iterator(): # each data is a dictionary + 输出如下所示： + ``` + dataset1: + [array([0], dtype=int64)] + [array([1], dtype=int64)] + [array([2], dtype=int64)] + [array([3], dtype=int64)] + [array([4], dtype=int64)] + [array([5], dtype=int64)] + [array([6], dtype=int64)] + [array([7], dtype=int64)] + [array([8], dtype=int64)] + [array([9], dtype=int64)] + dataset2: + [array([0], dtype=int64)] + [array([1], dtype=int64)] + [array([2], dtype=int64)] + [array([3], dtype=int64)] + [array([4], dtype=int64)] + [array([5], dtype=int64)] + [array([6], dtype=int64)] + [array([7], dtype=int64)] + [array([8], dtype=int64)] + [array([9], dtype=int64)] + ``` + + - 创建返回值为字典类型的迭代器。以下分别对`dataset1`和`dataset2`创建迭代器，并打印输出数据观察结果。 + ```python + print("dataset1:") + for data in dataset1.create_dict_iterator(): # each data is a dictionary + print(data["data"]) + + print("dataset2:") + for data in dataset2.create_dict_iterator(): # each data is a dictionary print(data["data"]) ``` + 输出如下所示： + ``` + dataset1: + {'data': array([0], dtype=int64)} + {'data': array([1], dtype=int64)} + {'data': array([2], dtype=int64)} + {'data': array([3], dtype=int64)} + {'data': array([4], dtype=int64)} + {'data': array([5], dtype=int64)} + {'data': array([6], dtype=int64)} + {'data': array([7], dtype=int64)} + {'data': array([8], dtype=int64)} + {'data': array([9], dtype=int64)} + dataset2: + {'data': array([0], dtype=int64)} + {'data': array([1], dtype=int64)} + {'data': array([2], dtype=int64)} + {'data': array([3], dtype=int64)} + {'data': array([4], dtype=int64)} + {'data': array([5], dtype=int64)} + {'data': array([6], dtype=int64)} + {'data': array([7], dtype=int64)} + {'data': array([8], dtype=int64)} + {'data': array([9], dtype=int64)} + ``` diff --git a/tutorials/source_zh_cn/use/defining_the_network.rst b/tutorials/source_zh_cn/use/defining_the_network.rst index d6d2bfba310f930c14a4d450256705391e596012..89ba1da65475ee82c506af1c4f20cf8ce024155e 100644 --- a/tutorials/source_zh_cn/use/defining_the_network.rst +++ b/tutorials/source_zh_cn/use/defining_the_network.rst @@ -4,5 +4,5 @@ .. toctree:: :maxdepth: 1 - 网络支持 + 网络支持 custom_operator \ No newline at end of file diff --git a/tutorials/source_zh_cn/use/multi_platform_inference.md b/tutorials/source_zh_cn/use/multi_platform_inference.md index 3d2bb3fb10cb68f63dc43af2084ed4d5940870cc..2e965eacfcd5b807c84856becf2f2ce1a6850ed8 100644 --- a/tutorials/source_zh_cn/use/multi_platform_inference.md +++ b/tutorials/source_zh_cn/use/multi_platform_inference.md @@ -19,7 +19,7 @@ - + ## 概述 @@ -59,15 +59,15 @@ CPU | ONNX格式 | 支持ONNX推理的runtime/SDK，如TensorRT。 res = model.eval(dataset) ``` 其中， - `model.eval`为模型验证接口，对应接口说明：。 - > 推理样例代码：。 + `model.eval`为模型验证接口，对应接口说明：。 + > 推理样例代码：。 2. 使用`model.predict`接口来进行推理操作。 ```python model.predict(input_data) ``` 其中， - `model.eval`为推理接口，对应接口说明：。 + `model.eval`为推理接口，对应接口说明：。 ## Ascend 310 AI处理器上推理 @@ -79,7 +79,7 @@ CPU | ONNX格式 | 支持ONNX推理的runtime/SDK，如TensorRT。 Ascend 310 AI处理器上搭载了ACL框架，他支持OM格式，而OM格式需要从ONNX或者GEIR模型进行转换。所以在Ascend 310 AI处理器上推理，需要下述两个步骤： -1. 在训练平台上生成ONNX或GEIR格式模型，具体步骤请参考[模型导出-导出GEIR模型和ONNX模型](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#geironnx)。 +1. 在训练平台上生成ONNX或GEIR格式模型，具体步骤请参考[模型导出-导出GEIR模型和ONNX模型](https://www.mindspore.cn/tutorial/zh-CN/r0.5/use/saving_and_loading_model_parameters.html#geironnx)。 2. 将ONNX/GEIR格式模型文件，转化为OM格式模型，并进行推理。 - 云上（ModelArt环境），请参考[Ascend910训练和Ascend310推理的样例](https://support.huaweicloud.com/bestpractice-modelarts/modelarts_10_0026.html)完成推理操作。 @@ -93,7 +93,7 @@ Ascend 310 AI处理器上搭载了ACL框架，他支持OM格式，而OM格式需 ### 使用ONNX格式文件推理 -1. 在训练平台上生成ONNX格式模型，具体步骤请参考[模型导出-导出GEIR模型和ONNX模型](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#geironnx)。 +1. 在训练平台上生成ONNX格式模型，具体步骤请参考[模型导出-导出GEIR模型和ONNX模型](https://www.mindspore.cn/tutorial/zh-CN/r0.5/use/saving_and_loading_model_parameters.html#geironnx)。 2. 在GPU上进行推理，具体可以参考推理使用runtime/SDK的文档。如在Nvidia GPU上进行推理，使用常用的TensorRT，可参考[TensorRT backend for ONNX](https://github.com/onnx/onnx-tensorrt)。 @@ -105,10 +105,10 @@ Ascend 310 AI处理器上搭载了ACL框架，他支持OM格式，而OM格式需 ### 使用ONNX格式文件推理 与在GPU上进行推理类似，需要以下几个步骤： -1. 在训练平台上生成ONNX格式模型，具体步骤请参考[模型导出-导出GEIR模型和ONNX模型](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#geironnx)。 +1. 在训练平台上生成ONNX格式模型，具体步骤请参考[模型导出-导出GEIR模型和ONNX模型](https://www.mindspore.cn/tutorial/zh-CN/r0.5/use/saving_and_loading_model_parameters.html#geironnx)。 2. 在CPU上进行推理，具体可以参考推理使用runtime/SDK的文档。如使用ONNX Runtime，可以参考[ONNX Runtime说明文档](https://github.com/microsoft/onnxruntime)。 ## 端侧推理 -端侧推理需使用MindSpore Predict推理引擎，详细操作请参考[端侧推理教程](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/on_device_inference.html)。 \ No newline at end of file +端侧推理需使用MindSpore Predict推理引擎，详细操作请参考[端侧推理教程](https://www.mindspore.cn/tutorial/zh-CN/r0.5/advanced_use/on_device_inference.html)。 \ No newline at end of file diff --git a/tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md b/tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md index dc4848e1872805f99426f8cd7ec15baa30b076c2..111c493105d535e7107add358b8dfb2f6cf0b317 100644 --- a/tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md +++ b/tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md @@ -13,7 +13,7 @@ - + ## 概述