diff --git a/CONTRIBUTING_DOC.md b/CONTRIBUTING_DOC.md index 93f03a5a9af4677495a63b1286a83f5c596992a6..9453ed63d158f21db9acbcda591a6b61f107e77e 100644 --- a/CONTRIBUTING_DOC.md +++ b/CONTRIBUTING_DOC.md @@ -1,5 +1,7 @@ # Contributing Documents +[查看中文](./CONTRIBUTING_DOC_CN.md) + You are welcome to contribute MindSpore documents. Documents that meet requirements will be displayed on the [MindSpore official website](https://www.mindspore.cn). diff --git a/CONTRIBUTING_DOC_CN.md b/CONTRIBUTING_DOC_CN.md new file mode 100644 index 0000000000000000000000000000000000000000..914ba5b6e5820fe8fd1441df90507ca55fd3d1c2 --- /dev/null +++ b/CONTRIBUTING_DOC_CN.md @@ -0,0 +1,92 @@ +# 贡献文档 + +[View English](./CONTRIBUTING_DOC.md) + +欢迎参与MindSpore文档贡献，符合要求的文档将会在[MindSpore官网](http://www.mindspore.cn)中呈现。 + + + +- [贡献文档](#贡献文档) + - [新建或更新文档](#新建或更新文档) + - [提交修改](#提交修改) + - [文档写作规范](#文档写作规范) + + + +## 新建或更新文档 + +本项目支持MarkDown和reStructuredText格式的内容贡献，对应地可创建```.md```和```.rst```为后缀的文档或修改已存在的文档。 + +## 提交修改 + +提交修改的步骤与代码相同，请参考[代码贡献指南](https://gitee.com/mindspore/mindspore/blob/master/CONTRIBUTING.md)。 + +## 文档写作规范 + +- 标题仅支持Atx风格，标题与上下文需用空行隔开。 + + ``` + # 一级标题 + + ## 二级标题 + + ### 三级标题 + ``` + +- 列表标题和内容如需换行显示，标题和内容间需增加一个空行，否则无法实现换行。 + + ``` + - 标题 + + 内容。 + ``` + +- 目录中的锚点（超链接）只能使用中文、小写英文字母或“-”，不能带有空格或其他特殊字符，否则会导致链接无效。 + +- 注意事项使用“>”标识。 + + ``` + > 注意事项内容。 + ``` + +- 参考文献需列举在文末，并在文中标注。 + + ``` + 引用文字或图片说明后，增加标注[编号]。 + + ## 参考文献 + + [1] 作者. [有链接的文献名](http://xxx). + + [2] 作者. 没有链接的文献名. + ``` + +- 示例代码注释需遵循如下要求： + + - 注释用英文写作； + - Python函数、方法、类的注释使用```"""```； + - Python其他代码注释使用```#```； + - C++代码注释使用```//```。 + + ``` + """ + Python函数、方法、类的注释 + """ + + # Python代码注释 + + // C++代码注释 + + ``` + +- 图和图标题前后需增加一个空行，否则会导致排版异常。 + + ``` + 如下图所示： + + ![](./xxx.png) + + 图1：xxx + + 下文内容。 + ``` \ No newline at end of file diff --git a/README.md b/README.md index 9957788d3a2e7bf8bac9febb56dfc55eea2574a8..fe8f90ffeaad28d4c66d0c4672bae86222f5f3db 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,9 @@ -# MindSpore Documents +# MindSpore Documents ![MindSpore Logo](resource/MindSpore-logo.png) +[查看中文](./README_CN.md) + ## Overview This project provides the source files of the installation guide, tutorials, and other documents, as well as API configurations on the MindSpore official website . @@ -50,4 +52,4 @@ MindSpore tutorials and API documents can be generated by [Sphinx](https://www.s ## License - [Apache License 2.0](LICENSE) -- [Creative Commons License version 4.0](LICENSE-CC-BY-4.0) +- [Creative Commons License version 4.0](LICENSE-CC-BY-4.0) \ No newline at end of file diff --git a/README_CN.md b/README_CN.md new file mode 100644 index 0000000000000000000000000000000000000000..b65ca489d3fd8d94724839fa5fb4f277968106c1 --- /dev/null +++ b/README_CN.md @@ -0,0 +1,55 @@ +# MindSpore文档 + +![MindSpore Logo](resource/MindSpore-logo.png) + +[View English](./README.md) + +## 简介 + +此工程提供MindSpore官方网站所呈现安装指南、教程、文档的源文件以及API的相关配置。 + +## 贡献 + +我们非常欢迎您贡献文档！如果想要参与，请阅读[CONTRIBUTING_DOC_CN.md](./CONTRIBUTING_DOC_CN.md)，务必遵守文档写作规范，并按照流程规则提交，审核通过后，改动会在文档工程和官网中呈现。 + +同时，如果您对文档有任何意见或建议，请在Issues中提交。 + +## 目录结构说明 + +``` +docs +├───api //用于生成API的相关配置文件 +│ +├───docs //介绍文档 +│ +├───install //安装指南 +│ +├───resource //资源相关文档 +│ +├───tutorials //教程相关文档 +│ +└───README.md //Docs仓说明 +``` + +## 文档构建 + +MindSpore的教程和API文档均可由[Sphinx](https://www.sphinx-doc.org/en/master/)工具生成。下面以API文档为例介绍具体步骤。 + +1. 下载MindSpore Docs仓代码。 + ```shell + git clone https://gitee.com/mindspore/docs.git + ``` +2. 进入docs目录，安装该目录下```requirements.txt```文件中的依赖项。 + ```shell + cd docs/api + pip install -r requirements.txt + ``` +3. 在docs目录下执行如下命令，完成后会新建build_zh_cn/html目录，该目录中存放了生成后的文档网页，打开```build_zh_cn/html/index.html```即可查看API文档内容。 + ``` + make html + ``` + +## 版权 + +- [Apache License 2.0](LICENSE) +- [Creative Commons License version 4.0](LICENSE-CC-BY-4.0) diff --git a/resource/faq/FAQ_en.md b/resource/faq/FAQ_en.md index efe6b4da2667539fa6e2284e2b0e310f719c4abd..7fa3b3034787ff8ea08b355864c47e199bc1cfb6 100644 --- a/resource/faq/FAQ_en.md +++ b/resource/faq/FAQ_en.md @@ -9,6 +9,7 @@ - [Support](#support) - [Model Support](#model-support) - [Backend Support](#backend-support) + - [System Support](#system-support) - [Programming Language](#programming-language) - [Others](#others) - [Features](#features) @@ -69,6 +70,12 @@ Q: What types of model is currently supported by MindSpore for training ? A: MindSpore has basic support for common training scenarios, please refer to [Release note](https://gitee.com/mindspore/mindspore/blob/master/RELEASE.md) for detailed information. +
+ +Q: What are the available recommendation or text generation networks or models provided by MindSpore? + +A: Currently, recommendation models such as Wide & Deep, DeepFM, and NCF are under development. In the natural language processing (NLP) field, Bert\_NEZHA is available and models such as MASS are under development. You can rebuild the network into a text generation network based on the scenario requirements. Please stay tuned for updates on the [MindSpore Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/mindspore/model_zoo). + ### Backend Support Q: When install or run MindSpore, are there any requirements for hardwares like GPU, NPU and so forth ? @@ -81,6 +88,18 @@ Q: Does MindSpore have any plan on supporting other types of heterogeneous compu A: MindSpore provides pluggable device management interface so that developer could easily integrate other types of heterogeneous computing hardwares like FPGA to MindSpore. We welcome more backend support in MindSpore from the community. +
+ +Q: What hardware does MindSpore require? + +A: Currently, you can try out MindSpore through Docker images on laptops or in environments with GPUs. Some models in MindSpore Model Zoo support GPU-based training and inference, and other models are being improved. For distributed parallel training, MindSpore supports multi-GPU training. You can obtain the latest information from [RoadMap](https://www.mindspore.cn/docs/en/master/roadmap.html) and project [Release Notes](https://gitee.com/mindspore/mindspore/blob/master/RELEASE.md). + +### System Support + +Q: Does MindSpore support Windows 10? + +A: The MindSpore CPU version can be installed on Windows 10. For details about the installation procedure, see tutorials on the [MindSpore official website](https://www.mindspore.cn/tutorial/en/master/advanced_use/mindspore_cpu_win_install.html). + ### Programming Language Q: The recent announced programming language such as taichi got Python extensions that could be directly used as `import taichi as ti`. Does MindSpore have similar support ? @@ -99,14 +118,50 @@ Q: How does MindSpore implement semantic collaboration and processing? Is the po A: The MindSpore framework does not support FCA. For semantic models, you can call third-party tools to perform FCA in the data preprocessing phase. MindSpore supports Python therefore `import FCA` could do the trick. +
+ +Q: Where can I view the sample code or tutorial of MindSpore training and inference? + +A: Please visit the [MindSpore official website](https://www.mindspore.cn/tutorial/en/master/index.html). + ## Features Q: Does MindSpore have any plan or consideration on the edge and device when the training and inference functions on the cloud are relatively mature? A: MindSpore is a unified cloud-edge-device training and inference framework. Edge has been considered in its design, so MindSpore can perform inference at the edge. The open-source version will support Ascend 310-based inference. Currently, inference supports optimization operations, including quantization, operator fusion, and memory overcommitment. +
+ +Q: How does MindSpore support automatic parallelism? + +A: Automatic parallelism on CPUs and GPUs are being improved. You are advised to use the automatic parallelism feature on the Ascend 910 AI processor. Follow our open source community and apply for a MindSpore developer experience environment for trial use. + +
+ +Q: What is the relationship between MindSpore and ModelArts? Can MindSpore be used on ModelArts? + +A: ModelArts is an online training and inference platform on HUAWEI CLOUD. MindSpore is a Huawei deep learning framework. You can view the tutorials on the [MindSpore official website](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/use_on_the_cloud.html) to learn how to train MindSpore models on ModelArts. + ## Capabilities Q: Does MindSpore have a module that can implement object detection algorithms as TensorFlow does? A: The TensorFlow's object detection pipeline API belongs to the TensorFlow's Model module. After MindSpore's detection models are complete, similar pipeline APIs will be provided. + +
+ +Q: How do I migrate scripts or models of other frameworks to MindSpore? + +A: For details about script or model migration, please visit the [MindSpore official website](https://www.mindspore.cn/tutorial/en/master/advanced_use/network_migration.html). + +
+ +Q: Does MindSpore provide open-source e-commerce datasets? + +A: No. Please stay tuned for updates on the [MindSpore official website](https://www.mindspore.cn/en). + +
+ +Q: How simple can the MindSpore model training code be? + +A: MindSpore provides Model APIs except for network definitions. In most scenarios, model training can be completed using only a few lines of code. diff --git a/tutorials/source_en/advanced_use/customized_debugging_information.md b/tutorials/source_en/advanced_use/customized_debugging_information.md index feba4f504dbfce995a23b9d6c08c66aea69eb431..0e4fdd17dc346225ce4d59736305ca8b56033546 100644 --- a/tutorials/source_en/advanced_use/customized_debugging_information.md +++ b/tutorials/source_en/advanced_use/customized_debugging_information.md @@ -224,4 +224,31 @@ MindSpore uses glog to output logs. The following environment variables are comm 1. GLOG_v specifies the log level. The default value is 2, indicating the WARNING level. The values are as follows: 0: DEBUG; 1: INFO; 2: WARNING; 3: ERROR. 2. When GLOG_logtostderr is set to 1, logs are output to the screen. If the value is set to 0, logs are output to a file. Default value: 1 3. GLOG_log_dir=YourPath specifies the log output path. If GLOG_logtostderr is set to 0, value of this variable must be specified. If GLOG_log_dir is specified and the value of GLOG_logtostderr is 1, logs are output to the screen but not to a file. Logs of C++ and Python will be output to different files. The file name of C++ log complies with the naming rule of GLOG log file. Here, the name is `mindspore.MachineName.UserName.log.LogLevel.Timestamp`. The file name of Python log is `mindspore.log`. - +4. MS_SUBMODULE_LOG_v={SubModule1:LogLevel1,SubModule2:LogLevel2,...} specifies log levels of sub modules of MindSpore. The specified sub module log level will overwrite the global log level. The meaning of submodule log level is same as GLOG_v, the sub modules of MindSpore grouped by source directory is as the bellow table. E.g. when set `GLOG_v=1 MS_SUBMODULE_LOG_v={PARSER:2,ANALYZER:2}` then log levels of `PARSER` and `ANALYZER` are WARNING, other modules' log levels are INFO. + + Sub moudles of MindSpore grouped by source directory: + | Source Files | Sub Module Name | + | ------------ | --------------- | + | mindspore/ccsrc/common | COMMON | + | mindspore/ccsrc/dataset | MD | + | mindspore/ccsrc/debug | DEBUG | + | mindspore/ccsrc/device | DEVICE | + | mindspore/ccsrc/gvar | COMMON | + | mindspore/ccsrc/ir | IR | + | mindspore/ccsrc/kernel | KERNEL | + | mindspore/ccsrc/mindrecord | MD | + | mindspore/ccsrc | ME | + | mindspore/ccsrc/onnx | ONNX | + | mindspore/ccsrc/operator | ANALYZER | + | mindspore/ccsrc/optimizer | OPTIMIZER | + | mindspore/ccsrc/parallel | PARALLEL | + | mindspore/ccsrc/pipeline/*.cc | PIPELINE | + | mindspore/ccsrc/pipeline/parse | PARSER | + | mindspore/ccsrc/pipeline/static_analysis | ANALYZER | + | mindspore/ccsrc/pre_activate | PRE_ACT | + | mindspore/ccsrc/pybind_api | COMMON | + | mindspore/ccsrc/pynative | PYNATIVE | + | mindspore/ccsrc/session | SESSION | + | mindspore/ccsrc/transform | GE_ADPT | + | mindspore/ccsrc/utils | UTILS | + | mindspore/ccsrc/vm | VM | diff --git a/tutorials/source_en/advanced_use/network_migration.md b/tutorials/source_en/advanced_use/network_migration.md new file mode 100644 index 0000000000000000000000000000000000000000..78a6aa7460e6697438b9e7bb101bf3fa3ab7d397 --- /dev/null +++ b/tutorials/source_en/advanced_use/network_migration.md @@ -0,0 +1,298 @@ +# Network Migration + + + +- [Network Migration](#network-migration) + - [Overview](#overview) + - [Preparations](#preparations) + - [Operator Assessment](#operator-assessment) + - [Software and Hardware Environments](#software-and-hardware-environments) + - [E2E Network Migration](#e2e-network-migration) + - [Training Phase](#training-phase) + - [Script Migration](#script-migration) + - [Accuracy Debugging](#accuracy-debugging) + - [On-Cloud Integration](#on-cloud-integration) + - [Inference Phase](#inference-phase) + - [Examples](#examples) + + + + + +## Overview + +You've probably written scripts for frameworks such as TensorFlow and PyTorch. This tutorial describes how to migrate existing TensorFlow and PyTorch networks to MindSpore, including key steps and operation recommendations which help you quickly migrate your network. + +## Preparations + +Before you start working on your scripts, prepare your operator assessment and hardware and software environments to make sure that MindSpore can support the network you want to migrate. + +### Operator Assessment + +Analyze the operators contained in the network to be migrated and figure out how does MindSpore support these operators based on the [Operator List](https://www.mindspore.cn/docs/en/master/operator_list.html). + +Take ResNet-50 as an example. The two major operators [Conv](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Conv2d) and [BatchNorm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.BatchNorm2d) exist in the MindSpore Operator List. + +If any operator does not exist, you are advised to perform the following operations: + +- Operator replacement: Analyze the operator implementation formula and check whether a combination of existing operators of MindSpore can be used to achieve the expected objective. +- Substitution solution: For example, if a loss operator is not supported, check whether it can be replaced with a loss operator of the same type supported by MindSpore; alternatively, check whether the current network structure can be replaced by another mainstream network of the same type. + +If the operators used for replacement are not able to fulfill complete function, you are advised to perform the following operations: + +- Delete unnecessary functions. +- Find a substitution solution for necessary functions. + +If the preceding requirements cannot be met, you can raise requirements in the [MindSpore community](https://gitee.com/mindspore/mindspore). + +### Software and Hardware Environments + +Prepare the hardware environment, find a platform corresponding to your environment by referring to the [installation guide](https://www.mindspore.cn/install/en), and install MindSpore. + +## E2E Network Migration + +### Training Phase + +#### Script Migration + +MindSpore differs from TensorFlow and PyTorch in the network structure. Before migration, you need to clearly understand the original script and information of each layer, such as shape. + +The ResNet-50 network migration and training on the Ascend 910 is used as an example. + +1. Import MindSpore modules. + + Import the corresponding MindSpore modules based on the required APIs. For details about the module list, see . + +2. Load and preprocess a dataset. + + Use MindSpore to build the required dataset. Currently, MindSpore supports common datasets. You can call APIs in the original format, `MindRecord`, and `TFRecord`. In addition, MindSpore supports data processing and data augmentation. For details, see the [Data Preparation](https://www.mindspore.cn/tutorial/en/master/use/data_preparation/data_preparation.html). + + In this example, the CIFAR-10 dataset is loaded, which supports both single-GPU and multi-GPU scenarios. + + ```python + if device_num == 1: + ds = de.Cifar10Dataset(dataset_path, num_parallel_workers=4, shuffle=True) + else: + ds = de.Cifar10Dataset(dataset_path, num_parallel_workers=4, shuffle=True, + num_shards=device_num, shard_id=rank_id) + ``` + + Then, perform data augmentation, data cleaning, and batch processing. For details about the code, see . + +3. Build a network. + + The biggest difference between MindSpore and TensorFlow in convolution is the data format. `NCHW` is used in MindSpore by default, while `NHWC` is used in TensorFlow. + + The following uses the first convolutional layer on the ResNet-50 network whose batch\_size is set to 32 as an example: + + - In TensorFlow, the format of the input feature is \[32, 224, 224, 3], and the size of the convolution kernel is \[7, 7, 3, 64]. + + - In MindSpore, the format of the input feature is \[32, 3, 224, 224], and the size of the convolution kernel is \[64, 3, 7, 7]. + + ```python + def _conv7x7(in_channel, out_channel, stride=1): + weight_shape = (out_channel, in_channel, 7, 7) + weight = _weight_variable(weight_shape) + return nn.Conv2d(in_channel, out_channel, + kernel_size=7, stride=stride, padding=0, pad_mode='same', weight_init=weight) + + + def _bn(channel): + return nn.BatchNorm2d(channel, eps=1e-4, momentum=0.9, + gamma_init=1, beta_init=0, moving_mean_init=0, moving_var_init=1) + ``` + +4. Build a subnet. + + In MindSpore, `nn.Cell` is used to build a subnet structure. The network structure must be defined before being used in a subnet. Define each operator to be used in the `__init__` function of the Cell, connect the defined operators in the `construct` function, and then return the output of the subnet through `return`. + + ```python + class ResidualBlock(nn.Cell): + """ + ResNet V1 residual block definition. + + Args: + in_channel (int): Input channel. + out_channel (int): Output channel. + stride (int): Stride size for the first convolutional layer. Default: 1. + + Returns: + Tensor, output tensor. + + Examples: + >>> ResidualBlock(3, 256, stride=2) + """ + expansion = 4 + + def __init__(self, + in_channel, + out_channel, + stride=1): + super(ResidualBlock, self).__init__() + + channel = out_channel + self.conv1 = _conv1x1(in_channel, channel, stride=1) + self.bn1 = _bn(channel) + + self.conv2 = _conv3x3(channel, channel, stride=stride) + self.bn2 = _bn(channel) + + self.conv3 = _conv1x1(channel, out_channel, stride=1) + self.bn3 = _bn_last(out_channel) + + self.relu = nn.ReLU() + + self.down_sample = False + + if stride != 1 or in_channel != out_channel: + self.down_sample = True + self.down_sample_layer = None + + if self.down_sample: + self.down_sample_layer = nn.SequentialCell([_conv1x1(in_channel, out_channel, stride), + _bn(out_channel)]) + self.add = P.TensorAdd() + + def construct(self, x): + identity = x + + out = self.conv1(x) + out = self.bn1(out) + out = self.relu(out) + + out = self.conv2(out) + out = self.bn2(out) + out = self.relu(out) + + out = self.conv3(out) + out = self.bn3(out) + + if self.down_sample: + identity = self.down_sample_layer(identity) + + out = self.add(out, identity) + out = self.relu(out) + + return out + ``` + +5. Define a concatenated structure. + + The ResNet-50 network has a large number of repeated structures. In TensorFlow, you can use the for loop function to reduce repeated code. In MindSpore, each defined Cell object is independent. Especially for subnets with weight parameters, the defined Cell cannot be used repeatedly. If a large number of repeated concatenated structures exist, you can construct multiple Cell instances using the for loop function and concatenate them by using `SequentialCell`. + + ```python + def _make_layer(self, block, layer_num, in_channel, out_channel, stride): + """ + Make stage network of ResNet. + + Args: + block (Cell): Resnet block. + layer_num (int): Layer number. + in_channel (int): Input channel. + out_channel (int): Output channel. + stride (int): Stride size for the first convolutional layer. + + Returns: + SequentialCell, the output layer. + + Examples: + >>> _make_layer(ResidualBlock, 3, 128, 256, 2) + """ + layers = [] + + resnet_block = block(in_channel, out_channel, stride=stride) + layers.append(resnet_block) + + for _ in range(1, layer_num): + resnet_block = block(out_channel, out_channel, stride=1) + layers.append(resnet_block) + + return nn.SequentialCell(layers) + ``` + +6. Build the entire network. + + The [ResNet-50](https://gitee.com/mindspore/mindspore/blob/master/mindspore/model_zoo/resnet.py) network structure is formed by connecting multiple defined subnets. Follow the rule of defining subnets before using them and define all the subnets used in the `__init__` and connect subnets in the `construct`. + +7. Define a loss function and an optimizer. + + After the network is defined, the loss function and optimizer need to be defined accordingly. + + ```python + loss = SoftmaxCrossEntropyWithLogits(sparse=True) + opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), lr, config.momentum, config.weight_decay, config.loss_scale) + ``` + +8. Build a model. + + Similar to the `Estimator` API of TensorFlow, the defined network prototype, loss function, and optimizer are transferred to the `Model` API of MindSpore and automatically combined into a network that can be used for training. + + To use loss scale in training, define a loss\_scale\_manager and transfer it to the `Model` API. + + ```python + loss_scale = FixedLossScaleManager(config.loss_scale, drop_overflow_update=False) + ``` + + You can use a built-in assessment method of `Model` by setting the [metrics](https://www.mindspore.cn/tutorial/en/master/advanced_use/customized_debugging_information.html#mindspore-metrics) attribute. + + ```python + model = Model(net, loss_fn=loss, optimizer=opt, loss_scale_manager=loss_scale, metrics={'acc'}) + ``` + + Similar to `estimator.train()` of TensorFlow, you can call the `model.train` API to perform training. Functions such as CheckPoint and intermediate result printing can be defined on the `model.train` API in Callback mode. + + ```python + time_cb = TimeMonitor(data_size=step_size) + loss_cb = LossMonitor() + cb = [time_cb, loss_cb] + if config.save_checkpoint: + config_ck = CheckpointConfig(save_checkpoint_steps=config.save_checkpoint_steps, + keep_checkpoint_max=config.keep_checkpoint_max) + ckpt_cb = ModelCheckpoint(prefix="resnet", directory=config.save_checkpoint_path, config=config_ck) + cb += [ckpt_cb] + model.train(epoch_size, dataset, callbacks=cb) + ``` + +#### Accuracy Debugging + +The accuracy optimization process is as follows: + +1. When validating the single-GPU accuracy, you are advised to use a small dataset for training. After the validation is successful, use the full dataset for multi-GPU accuracy validation. This helps improve the debugging efficiency. +2. Delete unnecessary skills (such as augmentation configuration and dynamic loss scale in an optimizer) from the script. After the validation is successful, add functions one by one. After a new function is confirmed to be normal, add the next function. In this way, you can quickly locate the fault. + +#### On-Cloud Integration + +Run your scripts on ModelArts. For details, see [Using MindSpore on Cloud](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/use_on_the_cloud.html). + +### Inference Phase + +Models trained on the Ascend 910 AI processor can be used for inference on different hardware platforms. + +1. Inference on the Ascend 910 AI processor + + Similar to the `estimator.evaluate()` API of TensorFlow, MindSpore provides the `model.eval()` API for model validation. You only need to import the validation dataset. The processing method of the validation dataset is the same as that of the training dataset. For details about the complete code, see . + + ```python + res = model.eval(dataset) + ``` + +2. Inference on the Ascend 310 AI processor + + 1. Export the ONNX or GEIR model by referring to the [Export GEIR Model and ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#geironnx). + + 2. For performing inference in the cloud environment, see the [Ascend 910 training and Ascend 310 inference samples](https://support.huaweicloud.com/bestpractice-modelarts/modelarts_10_0026.html). For details about the bare-metal environment (compared with the cloud environment where the Ascend 310 AI processor is deployed locally), see the description document of the Ascend 310 AI processor software package. + +3. Inference on a GPU + + 1. Export the ONNX model by referring to the [Export GEIR Model and ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#geironnx). + + 2. Perform inference on the NVIDIA GPU by referring to [TensorRT backend for ONNX](https://github.com/onnx/onnx-tensorrt). + +## Examples + +1. [Common network script examples](https://gitee.com/mindspore/mindspore/tree/master/example) + +2. [Common dataset examples](https://www.mindspore.cn/tutorial/en/master/use/data_preparation/loading_the_datasets.html) + +3. [Model Zoo](https://gitee.com/mindspore/mindspore/tree/master/mindspore/model_zoo) + diff --git a/tutorials/source_en/index.rst b/tutorials/source_en/index.rst index 37008d6b60065ae2b9b246213940a0f110f90962..59753beb6e26ba59af56c6602b3dabc2c4c7755f 100644 --- a/tutorials/source_en/index.rst +++ b/tutorials/source_en/index.rst @@ -34,6 +34,7 @@ MindSpore Tutorials advanced_use/nlp_application advanced_use/customized_debugging_information advanced_use/on_device_inference + advanced_use/network_migration advanced_use/model_security advanced_use/mindspore_cpu_win_install advanced_use/community \ No newline at end of file diff --git a/tutorials/source_zh_cn/advanced_use/customized_debugging_information.md b/tutorials/source_zh_cn/advanced_use/customized_debugging_information.md index 5ea8d27a88f622fd8f39506fe9c0c6868f34f8f5..f10e2983e85daf9c53820de8ba498a439f9175d7 100644 --- a/tutorials/source_zh_cn/advanced_use/customized_debugging_information.md +++ b/tutorials/source_zh_cn/advanced_use/customized_debugging_information.md @@ -223,4 +223,32 @@ MindSpore采用glog来输出日志，常用的几个环境变量如下： 1. GLOG_v 控制日志的级别，默认值为2，即WARNING级别，对应关系如下：0-DEBUG、1-INFO、2-WARNING、3-ERROR。 2. GLOG_logtostderr 值设置为1时，日志输出到屏幕；值设置为0时，日志输出到文件。默认值为1。 3. GLOG_log_dir=YourPath 指定日志输出的路径。若GLOG_logtostderr的值为0，则必须设置此变量。若指定了GLOG_log_dir且GLOG_logtostderr的值为1时，则日志输出到屏幕，不输出到文件。C++和Python的日志会被输出到不同的文件中，C++日志的文件名遵从GLOG日志文件的命名规则，这里是`mindspore.机器名.用户名.log.日志级别.时间戳`，Python日志的文件名为`mingspore.log`。 +4. MS_SUBMODULE_LOG_v={SubModule1:LogLevel1,SubModule2:LogLevel2,...} 指定MindSpore子模块的日志级别，被指定的子模块的日志级别将覆盖GLOG_v在此模块内的设置，此处子模块的日志级别LogLevel与GLOG_v的日志级别含义相同，MindSpore子模块的划分如下表。如可以通过`GLOG_v=1 MS_SUBMODULE_LOG_v={PARSER:2,ANALYZER:2}`把`PARSER`和`ANALYZER`模块的日志级别设为WARNING，其他模块的日志级别设为INFO。 + + MindSpore子模块按照目录划分如下: + | Source Files | Sub Module Name | + | ------------ | --------------- | + | mindspore/ccsrc/common | COMMON | + | mindspore/ccsrc/dataset | MD | + | mindspore/ccsrc/debug | DEBUG | + | mindspore/ccsrc/device | DEVICE | + | mindspore/ccsrc/gvar | COMMON | + | mindspore/ccsrc/ir | IR | + | mindspore/ccsrc/kernel | KERNEL | + | mindspore/ccsrc/mindrecord | MD | + | mindspore/ccsrc | ME | + | mindspore/ccsrc/onnx | ONNX | + | mindspore/ccsrc/operator | ANALYZER | + | mindspore/ccsrc/optimizer | OPTIMIZER | + | mindspore/ccsrc/parallel | PARALLEL | + | mindspore/ccsrc/pipeline/*.cc | PIPELINE | + | mindspore/ccsrc/pipeline/parse | PARSER | + | mindspore/ccsrc/pipeline/static_analysis | ANALYZER | + | mindspore/ccsrc/pre_activate | PRE_ACT | + | mindspore/ccsrc/pybind_api | COMMON | + | mindspore/ccsrc/pynative | PYNATIVE | + | mindspore/ccsrc/session | SESSION | + | mindspore/ccsrc/transform | GE_ADPT | + | mindspore/ccsrc/utils | UTILS | + | mindspore/ccsrc/vm | VM | diff --git a/tutorials/source_zh_cn/index.rst b/tutorials/source_zh_cn/index.rst index 50fd39c66a48b28062e5b3fd8483e912655caa16..1cd6053fa98125e9830117f53ba01d933e4783e3 100644 --- a/tutorials/source_zh_cn/index.rst +++ b/tutorials/source_zh_cn/index.rst @@ -39,4 +39,4 @@ MindSpore教程 advanced_use/network_migration advanced_use/checkpoint_for_hybrid_parallel advanced_use/mindspore_cpu_win_install - advanced_use/community + advanced_use/community \ No newline at end of file