diff --git a/docs/golden_stick/docs/source_en/index.rst b/docs/golden_stick/docs/source_en/index.rst index d7b91ca7a568ec821cd09990c28c962e0b8d8f60..feec0d48b956976f795dabac0bafa6df5783d402 100644 --- a/docs/golden_stick/docs/source_en/index.rst +++ b/docs/golden_stick/docs/source_en/index.rst @@ -46,7 +46,7 @@ During network training, the MindSpore Golden Stick does not have great impact o - **Optimize the network using the MindSpore Golden Stick:** In the original training process, after the original network is defined and before the network is trained, use the MindSpore Golden Stick to optimize the network structure. Generally, this step is implemented by calling the `apply` API of MindSpore Golden Stick. For details, see `Applying the SimQAT Algorithm `_ . -- **Register the MindSpore Golden Stick callback:** Register the callback of the MindSpore Golden Stick into the model to be trained. Generally, in this step, the `callback` function of MindSpore Golden Stick is called to obtain the corresponding callback object and `register the object into the model `_ . +- **Register the MindSpore Golden Stick callback:** Register the callback of the MindSpore Golden Stick into the model to be trained. Generally, in this step, the `callback` function of MindSpore Golden Stick is called to obtain the corresponding callback object and `register the object into the model `_ . 2. Deployment @@ -54,9 +54,9 @@ During network training, the MindSpore Golden Stick does not have great impact o .. note:: - For details about how to apply the MindSpore Golden Stick, see the detailed description and sample code in each algorithm section. - - For details about the "network training or retraining" step in the process, see `MindSpore Training and Evaluation `_ . - - For details about the "ms.export" step in the process, see `Exporting MINDIR Model `_ . - - For details about the "MindSpore infer" step in the process, see `MindSpore Inference Runtime `_ . + - For details about the "network training or retraining" step in the process, see `MindSpore Training and Evaluation `_ . + - For details about the "ms.export" step in the process, see `Exporting MINDIR Model `_ . + - For details about the "MindSpore infer" step in the process, see `MindSpore Inference Runtime `_ . Roadmap --------------------------------------- diff --git a/tutorials/source_en/advanced/pynative_graph.rst b/tutorials/source_en/advanced/compute_graph.rst similarity index 37% rename from tutorials/source_en/advanced/pynative_graph.rst rename to tutorials/source_en/advanced/compute_graph.rst index 7e87850ff60398cce9a25f19ab4b60616936066f..ea9a86788ea7395daa6e737cbce111959f98e766 100644 --- a/tutorials/source_en/advanced/pynative_graph.rst +++ b/tutorials/source_en/advanced/compute_graph.rst @@ -1,9 +1,9 @@ -Dynamic and Static Graphs +Computation Graphs ========================== .. toctree:: :maxdepth: 1 - pynative_graph/mode - pynative_graph/combine - pynative_graph/pynative + compute_graph/mode + compute_graph/combine + compute_graph/pynative diff --git a/tutorials/source_en/advanced/pynative_graph/combine.md b/tutorials/source_en/advanced/compute_graph/combine.md similarity index 97% rename from tutorials/source_en/advanced/pynative_graph/combine.md rename to tutorials/source_en/advanced/compute_graph/combine.md index 117c3d0b5123eda02361a90f7dcec61255f0adfc..79118e760927819714354dcbd9339167cfd7d577 100644 --- a/tutorials/source_en/advanced/pynative_graph/combine.md +++ b/tutorials/source_en/advanced/compute_graph/combine.md @@ -1,6 +1,6 @@ # Combination of Dynamic and Static Graphs - + Currently, dynamic and static graphs are supported in the industry. Dynamic graphs are executed through explanation, with dynamic syntax affinity and flexible expression. Static graphs are executed through just in time (JIT) build, which focuses on static syntax and has many syntax constraints. The build process of the dynamic graph is different from that of the static graph. As a result, the syntax constraints are also different. @@ -21,6 +21,7 @@ from mindspore import ms_function class Add(nn.Cell): """Define a class to implement the x self addition.""" def construct(self, x): + x = x + x x = x + x return x @@ -28,6 +29,7 @@ class Mul(nn.Cell): """Define a class to implement the x self multiplication.""" @ms_function # Use ms_function to modify the function. This function is executed in static graph mode. def construct(self, x): + x = x * x x = x * x return x @@ -59,9 +61,9 @@ print("\nx:\n", x) [1. 1. 1.]] x: - [[8. 8. 8.] - [8. 8. 8.] - [8. 8. 8.]] + [[1024. 1024. 1024.] + [1024. 1024. 1024.] + [1024. 1024. 1024.]] ``` According to the preceding information, after the test operation, the final value of x is a 3\*3 matrix whose each element is 8. The following figure shows the build method of this test case according to the execution sequence. diff --git a/tutorials/source_en/advanced/pynative_graph/images/framework1.png b/tutorials/source_en/advanced/compute_graph/images/framework1.png similarity index 100% rename from tutorials/source_en/advanced/pynative_graph/images/framework1.png rename to tutorials/source_en/advanced/compute_graph/images/framework1.png diff --git a/tutorials/source_en/advanced/pynative_graph/images/framework2.png b/tutorials/source_en/advanced/compute_graph/images/framework2.png similarity index 100% rename from tutorials/source_en/advanced/pynative_graph/images/framework2.png rename to tutorials/source_en/advanced/compute_graph/images/framework2.png diff --git a/tutorials/source_en/advanced/pynative_graph/images/ms_function.png b/tutorials/source_en/advanced/compute_graph/images/ms_function.png similarity index 100% rename from tutorials/source_en/advanced/pynative_graph/images/ms_function.png rename to tutorials/source_en/advanced/compute_graph/images/ms_function.png diff --git a/tutorials/source_en/advanced/pynative_graph/mode.md b/tutorials/source_en/advanced/compute_graph/mode.md similarity index 98% rename from tutorials/source_en/advanced/pynative_graph/mode.md rename to tutorials/source_en/advanced/compute_graph/mode.md index 457a0121331ff2529c3b303a8c4840e2d19d8854..cb2a18f740f4b94ea0fba23322a095b859176bde 100644 --- a/tutorials/source_en/advanced/pynative_graph/mode.md +++ b/tutorials/source_en/advanced/compute_graph/mode.md @@ -1,6 +1,6 @@ # Dynamic and Static Graphs - + Currently, there are two execution modes of a mainstream deep learning framework: a static graph mode (Graph) and a dynamic graph mode (PyNative). diff --git a/tutorials/source_en/advanced/pynative_graph/pynative.md b/tutorials/source_en/advanced/compute_graph/pynative.md similarity index 95% rename from tutorials/source_en/advanced/pynative_graph/pynative.md rename to tutorials/source_en/advanced/compute_graph/pynative.md index 461171b729659ea8d22de69de0728dff92024c76..64f696627b80d60e8477304796a7bd7e2a75b5fd 100644 --- a/tutorials/source_en/advanced/pynative_graph/pynative.md +++ b/tutorials/source_en/advanced/compute_graph/pynative.md @@ -1,6 +1,6 @@ # Dynamic Graph Mode Application - + In dynamic graph mode, MindSpore supports single-operator execution, common function execution, network execution, and independent gradient computation. The following uses sample code to describe how to use these operations and precautions. diff --git a/tutorials/source_en/advanced/dataset.rst b/tutorials/source_en/advanced/dataset.rst index 745d3f349bac638fe835c2dd93c0948761d25148..8f7c14b3eaeb96d1974568442e4f136c931815a1 100644 --- a/tutorials/source_en/advanced/dataset.rst +++ b/tutorials/source_en/advanced/dataset.rst @@ -1,5 +1,5 @@ -Data Processing -=================== +Advanced Data Processing +========================= .. toctree:: :maxdepth: 1 diff --git a/tutorials/source_en/advanced/network/derivation.md b/tutorials/source_en/advanced/derivation.md similarity index 98% rename from tutorials/source_en/advanced/network/derivation.md rename to tutorials/source_en/advanced/derivation.md index 9f10b5a3d096da55cf8cbcba306faa3078e4b5a3..4dfc155ecf628b8226dbd9ca53beb3fc5f26fbcf 100644 --- a/tutorials/source_en/advanced/network/derivation.md +++ b/tutorials/source_en/advanced/derivation.md @@ -1,6 +1,6 @@ -# Automatic Derivation +# Advanced Automatic Differentiation - + The `GradOperation` API provided by the `mindspore.ops` module can be used to generate a gradient of a network model. The following describes how to use the `GradOperation` API to perform first-order and second-order derivations and how to stop gradient computation. diff --git a/tutorials/source_en/advanced/lenet_mnist.md b/tutorials/source_en/advanced/lenet_mnist.md new file mode 100644 index 0000000000000000000000000000000000000000..09ae192413e976d99ec45fa40f6a2fb5ac8df835 --- /dev/null +++ b/tutorials/source_en/advanced/lenet_mnist.md @@ -0,0 +1,207 @@ +# Advanced Case: Handwritten Digit Recognition + + + +This section runs through the basic process of MindSpore deep learning, using the LeNet5 network model as an example to implement common tasks in deep learning. + +## Downloading and Processing the Dataset + +Datasets are crucial for model training. A good dataset can effectively improve training accuracy and efficiency. The MNIST dataset used in the example consists of 28 x 28 grayscale images of 10 classes. The training dataset contains 60,000 images, and the test dataset contains 10,000 images. + +![mnist](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/tutorials/source_zh_cn/beginner/images/mnist.png) + +> You can download the dataset from [MNIST dataset download page](http://yann.lecun.com/exdb/mnist/), decompress it, and save it according to the following directory structure. + +The [MindSpore Vision](https://mindspore.cn/vision/docs/en/master/index.html) suite provides the Mnist module for downloading and processing MNIST datasets. The following sample code downloads and decompresses the datasets to the specified location for data processing: + +> The sample code in this chapter relies on `mindvision`, which can be installed by using the command `pip install mindvision`. If this document is run as Notebook, you need to restart the kernel after installation to execute subsequent code. + +```python +from mindvision.dataset import Mnist + +# Download and process the MNIST dataset. +download_train = Mnist(path="./mnist", split="train", batch_size=32, repeat_num=1, shuffle=True, resize=32, download=True) + +download_eval = Mnist(path="./mnist", split="test", batch_size=32, resize=32, download=True) + +dataset_train = download_train.run() +dataset_eval = download_eval.run() +``` + +Parameters description: + +- path: dataset path. +- split: dataset type. The value can be train, test, or infer. The default value is train. +- batch_size: data size set for each training batch. The default value is 32. +- repeat_num: number of times that the dataset is traversed during training. The default value is 1. +- shuffle: determines whether to randomly shuffle the dataset. This parameter is optional. +- resize: size of the output image. The default value is 32 x 32. +- download: determines whether to download the dataset. The default value is False. + +The directory structure of the downloaded dataset files is as follows: + +```text +./mnist/ +├── test +│ ├── t10k-images-idx3-ubyte +│ └── t10k-labels-idx1-ubyte +└── train + ├── train-images-idx3-ubyte + └── train-labels-idx1-ubyte +``` + +## Building the Model + +Except the input layer, LeNet contains seven layers: three convolutional layers, two subsampling layers, and two fully-connected layers. + +![](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/tutorials/source_zh_cn/advanced/images/lenet.png) + +The MindSpore Vision Suite provides the LeNet model interface `lenet`, which defines the network model as follows: + +```python +from mindvision.classification.models import lenet + +network = lenet(num_classes=10, pretrained=False) +``` + +## Defining a Loss Function and an Optimizer + +To train a neural network model, you need to define a loss function and an optimizer function. + +- The following uses the cross-entropy loss function `SoftmaxCrossEntropyWithLogits`. +- The optimizer is `Momentum`. + +```python +import mindspore.nn as nn + +# Define the loss function. +net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean') + +# Define the optimizer function. +net_opt = nn.Momentum(network.trainable_params(), learning_rate=0.01, momentum=0.9) +``` + +## Training and Saving the Model + +Before training, MindSpore needs to declare whether the intermediate process and result of the network model need to be saved during training. Therefore, the `ModelCheckpoint` API is used to save the network model and parameters for subsequent fine-tuning. + +```python +import mindspore as ms + +# Set the model saving parameters. The checkpoint steps are 1875. +config_ck = ms.CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10) + +# Apply the model saving parameters. +ckpoint = ms.ModelCheckpoint(prefix="lenet", directory="./lenet", config=config_ck) +``` + +The `model.train` API provided by MindSpore can be used to easily train the network. `LossMonitor` can monitor the changes of the `loss` value during the training process. + +```python +import mindspore as ms +from mindvision.engine.callback import LossMonitor + +# Initialize the model parameters. +model = ms.Model(network, loss_fn=net_loss, optimizer=net_opt, metrics={'accuracy'}) + +# Train the network model. The model name is lenet-1_1875.ckpt. +model.train(10, dataset_train, callbacks=[ckpoint, LossMonitor(0.01, 1875)]) +``` + +```text +Epoch:[ 0/ 10], step:[ 1875/ 1875], loss:[0.314/0.314], time:2237.313 ms, lr:0.01000 +Epoch time: 3577.754 ms, per step time: 1.908 ms, avg loss: 0.314 +Epoch:[ 1/ 10], step:[ 1875/ 1875], loss:[0.031/0.031], time:1306.982 ms, lr:0.01000 +Epoch time: 1307.792 ms, per step time: 0.697 ms, avg loss: 0.031 +Epoch:[ 2/ 10], step:[ 1875/ 1875], loss:[0.007/0.007], time:1324.625 ms, lr:0.01000 +Epoch time: 1325.340 ms, per step time: 0.707 ms, avg loss: 0.007 +Epoch:[ 3/ 10], step:[ 1875/ 1875], loss:[0.021/0.021], time:1396.733 ms, lr:0.01000 +Epoch time: 1397.495 ms, per step time: 0.745 ms, avg loss: 0.021 +Epoch:[ 4/ 10], step:[ 1875/ 1875], loss:[0.028/0.028], time:1594.762 ms, lr:0.01000 +Epoch time: 1595.549 ms, per step time: 0.851 ms, avg loss: 0.028 +Epoch:[ 5/ 10], step:[ 1875/ 1875], loss:[0.007/0.007], time:1242.175 ms, lr:0.01000 +Epoch time: 1242.928 ms, per step time: 0.663 ms, avg loss: 0.007 +Epoch:[ 6/ 10], step:[ 1875/ 1875], loss:[0.033/0.033], time:1199.938 ms, lr:0.01000 +Epoch time: 1200.627 ms, per step time: 0.640 ms, avg loss: 0.033 +Epoch:[ 7/ 10], step:[ 1875/ 1875], loss:[0.175/0.175], time:1228.845 ms, lr:0.01000 +Epoch time: 1229.548 ms, per step time: 0.656 ms, avg loss: 0.175 +Epoch:[ 8/ 10], step:[ 1875/ 1875], loss:[0.009/0.009], time:1237.200 ms, lr:0.01000 +Epoch time: 1237.969 ms, per step time: 0.660 ms, avg loss: 0.009 +Epoch:[ 9/ 10], step:[ 1875/ 1875], loss:[0.000/0.000], time:1287.693 ms, lr:0.01000 +Epoch time: 1288.413 ms, per step time: 0.687 ms, avg loss: 0.000 +``` + +During the training, the loss value is printed and fluctuates. However, the loss value gradually decreases and the precision gradually increases. Loss values displayed each time may be different because of their randomness. + +Validate the generalization capability of the model based on the result obtained by running the test dataset. + +1. Read the test dataset using the `model.eval` API. +2. Use the saved model parameters for inference. + +```python +acc = model.eval(dataset_eval) + +print("{}".format(acc)) +``` + +```text +{'accuracy': 0.9903846153846154} +``` + +The model accuracy data is displayed in the output content. In the example, the accuracy reaches 95%, indicating a good model quality. As the number of network epochs increases, the model accuracy will be further improved. + +## Loading the Model + +```python +import mindspore as ms + +# Load the saved model used for testing. +param_dict = ms.load_checkpoint("./lenet/lenet-1_1875.ckpt") +# Load parameters to the network. +ms.load_param_into_net(network, param_dict) +``` + +```text +[] +``` + +> For more information about loading a model in mindspore, see [Loading the Model](https://www.mindspore.cn/tutorials/en/master/beginner/save_load.html#loading-the-model). + +## Validating the Model + +Use the generated model to predict the classification of a single image. The procedure is as follows: + +> - The predicted image is randomly generated, and the execution result may be different each time. +> - The example uses the mindspore Tensor module, please see [Tensor](https://www.mindspore.cn/tutorials/en/master/beginner/tensor.html). + +```python +import numpy as np +import mindspore as ms +import matplotlib.pyplot as plt + +mnist = Mnist("./mnist", split="train", batch_size=6, resize=32) +dataset_infer = mnist.run() +ds_test = dataset_infer.create_dict_iterator() +data = next(ds_test) +images = data["image"].asnumpy() +labels = data["label"].asnumpy() + +plt.figure() +for i in range(1, 7): + plt.subplot(2, 3, i) + plt.imshow(images[i-1][0], interpolation="None", cmap="gray") +plt.show() + +# Use the model.predict function to predict the classification of the image. +output = model.predict(ms.Tensor(data['image'])) +predicted = np.argmax(output.asnumpy(), axis=1) + +# Output the predicted classification and the actual classification. +print(f'Predicted: "{predicted}", Actual: "{labels}"') +``` + +```text +Predicted: "[4 6 2 3 5 1]", Actual: "[4 6 2 3 5 1]" +``` + +The preceding information shows that the predicted values are the same as the target values. diff --git a/tutorials/source_en/advanced/linear_fitting.md b/tutorials/source_en/advanced/linear_fitting.md deleted file mode 100644 index ea078c41ec29d4b303362f76c371d25d4e51e8ed..0000000000000000000000000000000000000000 --- a/tutorials/source_en/advanced/linear_fitting.md +++ /dev/null @@ -1,469 +0,0 @@ -# Case: Linear Fitting - - - -MindSpore provides high-level, medium-level, and low-level APIs. For details, see [API Level Structure](https://www.mindspore.cn/tutorials/en/master/beginner/introduction.html#api-level-structure). - -To facilitate the control of the network execution process, MindSpore provides the high-level training and inference API `mindspore.Model`. By specifying the neural network model to be trained and common training settings, MindSpore calls the `train` and `eval` methods to train and infer the network. In addition, if you want to customize a specific module, you can call the corresponding medium- and low-level APIs to define the network training process. - -The following uses the medium- and low-level APIs provided by MindSpore to fit linear functions. - -$$f(x) = 2x + 3 \tag {1}$$ - -Before initializing the network, you need to configure the `context` parameter to control the program execution policy. For example, configure the static graph or dynamic graph mode and configure the hardware environment for network running. - -The following describes how to configure information and use medium- and low-level APIs provided by MindSpore to customize loss functions, optimizers, training processes, metrics, and evaluation processes. - -## Configuration Information - -Before initializing the network, you need to configure the `context` parameter to control the program execution policy. For example, configure the static graph or dynamic graph mode and configure the hardware environment for network running. Before initializing the network, you need to configure the `context` parameter to control the program execution policy. The following describes the execution mode management and hardware management. - -### Execution Mode - -MindSpore supports two running modes: Graph and PyNative. By default, MindSpore uses the Graph mode, and the PyNative mode is used for debugging. - -- Graph mode (static graph mode): The neural network model is built into an entire graph and then delivered to the hardware for execution. This mode uses graph optimization to improve the running performance and facilitates large-scale deployment and cross-platform running. - -- PyNative mode (dynamic graph mode): Operators in the neural network are delivered to the hardware one by one for execution. This mode facilitates code writing and neural network model debugging. - -MindSpore provides a unified encoding mode for static and dynamic graphs, significantly enhancing compatibility between both types of graphs. This enables you to switch between the static and dynamic graph modes by changing only one line of code, eliminating the need to develop multiple sets of code. When switching the mode, pay attention to the [constraints](https://www.mindspore.cn/docs/en/master/note/static_graph_syntax_support.html) of the target mode. - -Set the running mode to dynamic graph mode. - -```python -import mindspore as ms - -ms.set_context(mode=ms.PYNATIVE_MODE) -``` - -Similarly, when MindSpore is in dynamic image mode, you can run the `set_context(mode=GRAPH_MODE)` command to switch to the static graph mode. - -```python -import mindspore as ms - -ms.set_context(mode=ms.GRAPH_MODE) -``` - -### Hardware Management - -Hardware management involves the `device_target` and `device_id` parameters. - -- `device_target`: target device to be run. The value can be `Ascend`, `GPU`, or `CPU`. You can set this parameter based on the actual situation or use the default value. - -- `device_id`: ID of the target device. The value is in the range of [0,`device_num_per_host` - 1]. `device_num_per_host` indicates the total number of devices on the server. The value of `device_num_per_host` cannot exceed 4096. The default value of `device_id` is 0. - -> When the program is executed in non-distributed mode, you can set `device_id` to determine the ID of the device where the program is executed to avoid device usage conflicts. - -A code example is as follows: - -```Python -import mindspore as mst - -ms.set_context(device_target="Ascend", device_id=6) -``` - -## Processing Datasets - -### Generating a Dataset - -Define the dataset generation function `get_data` to generate the training dataset and test dataset. - -Since linear data is fitted, the required training datasets should be randomly distributed around the objective function. Assume that the objective function to be fitted is $f(x)=2x+3$. $f(x)=2x+3+noise$ is used to generate training datasets, and `noise` is a random value that complies with standard normal distribution rules. - -```python -import numpy as np - -def get_data(num, w=2.0, b=3.0): - for _ in range(num): - x = np.random.uniform(-10.0, 10.0) - noise = np.random.normal(0, 1) - y = x * w + b + noise - yield np.array([x]).astype(np.float32), np.array([y]).astype(np.float32) -``` - -Use get_data to generate 50 groups of evaluation data and visualize the data. - -```python -import matplotlib.pyplot as plt - -train_data = list(get_data(50)) -x_target_label = np.array([-10, 10, 0.1]) -y_target_label = x_target_label * 2 + 3 -x_eval_label, y_eval_label = zip(*train_data) - -plt.scatter(x_eval_label, y_eval_label, color="red", s=5) -plt.plot(x_target_label, y_target_label, color="green") -plt.title("Eval data") -plt.show() -``` - -![png](./images/output_8_0.png) - -In the figure shown above, the green line indicates the target function, and the red point indicates the evaluation data (`train_data`). - -### Loading a Dataset - -Loads the dataset generated by the `get_data` function to the system memory and performs basic data processing operations. - -- `ds.GeneratorDataset`: converts the generated data into a MindSpore dataset and saves the x and y values of the generated data to arrays of `data` and `label`. -- `batch`: combines `batch_size` pieces of data into a batch. -- `repeat`: Multiplies the number of data sets. - -```python -from mindspore import dataset as ds - -def create_dataset(num_data, batch_size=16, repeat_size=1): - input_data = ds.GeneratorDataset(list(get_data(num_data)), column_names=['data', 'label']) - input_data = input_data.batch(batch_size, drop_remainder=True) - input_data = input_data.repeat(repeat_size) - return input_data -``` - -Use the dataset augmentation function to generate training data. Use the defined `create_dataset` to augment the generated 1600 pieces of data into 100 datasets with the shape of 16 x 1. - -```python -data_number = 1600 -batch_number = 16 -repeat_number = 1 - -ds_train = create_dataset(data_number, batch_size=batch_number, repeat_size=repeat_number) -print("The dataset size of ds_train:", ds_train.get_dataset_size()) -step_size = ds_train.get_dataset_size() -dict_datasets = next(ds_train.create_dict_iterator()) - -print(dict_datasets.keys()) -print("The x label value shape:", dict_datasets["data"].shape) -print("The y label value shape:", dict_datasets["label"].shape) -``` - -```text - The dataset size of ds_train: 100 - dict_keys(['data', 'label']) - The x label value shape: (16, 1) - The y label value shape: (16, 1) -``` - -## Defining a Network Model - -The `mindspore.nn` class is the base class for setting up all networks and the basic unit of a network. In order to customize a network, you can inherit the `nn.Cell` class and overwrite the `__init__` and `construct` methods. - -The `mindspore.ops` module provides the implementation of basic operators. The `nn.Cell` module further encapsulates basic operators. You can flexibly use different operators as required. - -The following example uses `nn.Cell` to build a simple fully-connected network for subsequent customized content. In MindSpore, use `nn.Dense` to generate a linear function model with a single data input and a single data output. - -$$f(x)=wx+b\tag{2}$$ - -Use the Normal operator to randomly initialize the $w$ and $b$ parameters in formula (2). - -```python -from mindspore import nn -from mindspore.common.initializer import Normal - -class LinearNet(nn.Cell): - def __init__(self): - super(LinearNet, self).__init__() - self.fc = nn.Dense(1, 1, Normal(0.02), Normal(0.02)) - - def construct(self, x): - fx = self.fc(x) - return fx -``` - -After initializing the network model, visualize the initialized network function and training dataset to understand the model function before fitting. - -```python -import mindspore as ms - -net = LinearNet() # Initialize the linear regression network. - -model_params = net.trainable_params() # Obtain network parameters w and b before training. - -x_model_label = np.array([-10, 10, 0.1]) -y_model_label = (x_model_label * model_params[0].asnumpy()[0] + model_params[1].asnumpy()[0]) - -plt.axis([-10, 10, -20, 25]) -plt.scatter(x_eval_label, y_eval_label, color="red", s=5) -plt.plot(x_model_label, y_model_label, color="blue") -plt.plot(x_target_label, y_target_label, color="green") -plt.show() -``` - -![png](./images/output_16_0.png) - -## Customized Loss Functions - -A loss function is used to measure the difference between the predicted value and the actual value. In deep learning, model training is a process of reducing a loss function value through continuous iteration. Therefore, it is very important to select a loss function in a model training process. Defining a good loss function can help the loss function value converge faster and achieve better accuracy. - -[mindspore.nn](https://www.mindspore.cn/docs/en/master/api_python/mindspore.nn.html#loss-function) provides many common loss functions for users to select and allows users to customize loss functions as required. - -When customizing a loss function class, you can inherit the base class `nn.Cell` of the network or the base class `nn.LossBase` of the loss function. `nn.LossBase` is based on `nn.Cell` and provides the `get_loss` method. The `reduction` parameter is used to obtain a sum or mean loss value and output a scalar. The following describes how to define the mean absolute error (MAE) function by inheriting LossBase. The formula of the MAE algorithm is as follows: - -$$ loss= \frac{1}{m}\sum_{i=1}^m\lvert y_i-f(x_i) \rvert \tag{3}$$ - -In the preceding formula, $f(x)$ indicates the predicted value, $y$ indicates the actual value of the sample, and $loss$ indicates the mean distance between the predicted value and the actual value. - -When using the method inherited from LossBase to define the loss function, you need to rewrite the `__init__` and `construct` methods and use the `get_loss` method to compute the loss. The sample code is as follows: - -```python -from mindspore import nn, ops - -class MyMAELoss(nn.LossBase): - """Define the loss.""" - def __init__(self): - super(MyMAELoss, self).__init__() - self.abs = ops.Abs() - - def construct(self, predict, target): - x = self.abs(target - predict) - return self.get_loss(x) -``` - -## Customized Optimizer - -During model training, the optimizer is used to compute and update network parameters. A proper optimizer can effectively reduce the training time and improve model performance. - -[mindspore.nn](https://www.mindspore.cn/docs/en/master/api_python/mindspore.nn.html#optimizer) provides many common optimizers for users to select and allows users to customize optimizers as required. - -When customizing an optimizer, you can inherit the optimizer base class `nn.Optimizer` and rewrite the `__init__` and `construct` methods to update parameters. - -The following example implements the customized optimizer Momentum (SGD algorithm with momentum): - -$$ v_{t+1} = v_t × u+grad \tag{4}$$ - -$$p_{t+1} = p_t - lr × v_{t+1} \tag{5}$$ - -$grad$, $lr$, $p$, $v$, and $u$ respectively represent a gradient, a learning rate, a weight parameter, a momentum parameter, and an initial speed. - -```python -import mindspore as ms -from mindspore import nn, ops - -class MyMomentum(nn.Optimizer): - """Define the optimizer.""" - - def __init__(self, params, learning_rate, momentum=0.9): - super(MyMomentum, self).__init__(learning_rate, params) - self.moment = ms.Parameter(ms.Tensor(momentum, ms.float32), name="moment") - self.momentum = self.parameters.clone(prefix="momentum", init="zeros") - self.assign = ops.Assign() - - def construct(self, gradients): - """The input of construct is gradient. Gradients are automatically transferred during training.""" - lr = self.get_lr() - params = self.parameters # Weight parameter to be updated - for i in range(len(params)): - self.assign(self.momentum[i], self.momentum[i] * self.moment + gradients[i]) - update = params[i] - self.momentum[i] * lr # SGD algorithm with momentum - self.assign(params[i], update) - return params -``` - -## Customized Training Process - -`mindspore.Model` provides `train` and `eval` APIs for users to use during training. However, this API does not apply to all scenarios, such as multi-data and multi-label scenarios, where users need to define the training process. - -The following uses linear regression as an example to describe the customized training process. First, define the loss network and connect the forward network to the loss function. Then, define the training process. Generally, the training process inherits `nn.TrainOneStepCell`. `nn.TrainOneStepCell` encapsulates the loss network and optimizer to implement the backward propagation network to update the weight parameters. - -### Defining a Loss Network - -Define the loss network `MyWithLossCell` to connect the feedforward network to the loss function. - -```python -class MyWithLossCell(nn.Cell): - """Define the loss network.""" - - def __init__(self, backbone, loss_fn): - """Transfer the feedforward network and loss function as parameters during instantiation.""" - super(MyWithLossCell, self).__init__(auto_prefix=False) - self.backbone = backbone - self.loss_fn = loss_fn - - def construct(self, data, label): - """Connect the feedforward network and loss function.""" - out = self.backbone(data) - return self.loss_fn(out, label) - - def backbone_network(self): - """Backbone network to be encapsulated.""" - return self.backbone -``` - -### Defining the Training Process - -Define the training process `MyTrainStep`. This class inherits `nn.TrainOneStepCell`. `nn.TrainOneStepCell` encapsulates the loss network and optimizer. During training, the `ops.GradOperation` operator is used to obtain the gradient, the optimizer is used to update the weight. - -```python -class MyTrainStep(nn.TrainOneStepCell): - """Define the training process.""" - - def __init__(self, network, optimizer): - """Initialize parameters.""" - super(MyTrainStep, self).__init__(network, optimizer) - self.grad = ops.GradOperation(get_by_list=True) - - def construct(self, data, label): - """Build the training process.""" - weights = self.weights - loss = self.network(data, label) - grads = self.grad(self.network, weights)(data, label) - return loss, self.optimizer(grads) -``` - -The following defines the drawing function `plot_model_and_datasets` to draw the test data, objective function, and network model fitting function, and view the loss value. - -```python -from IPython import display -import matplotlib.pyplot as plt -import time - -def plot_model_and_datasets(net, data, loss): - weight = net.trainable_params()[0] - bias = net.trainable_params()[1] - x = np.arange(-10, 10, 0.1) - y = x * ms.Tensor(weight).asnumpy()[0][0] + ms.Tensor(bias).asnumpy()[0] - x1, y1 = zip(*data) - x_target = x - y_target = x_target * 2 + 3 - - plt.axis([-11, 11, -20, 25]) - plt.scatter(x1, y1, color="red", s=5) # Raw data - plt.plot(x, y, color="blue") # Predicted data - plt.plot(x_target, y_target, color="green") # Fitting function - plt.title(f"Loss:{loss}") # Printed loss value - - plt.show() - time.sleep(0.2) - display.clear_output(wait=True) -``` - -### Training - -Use the training data `ds_train` to train the training network `train_net` and visualize the training process. - -```python -loss_func = MyMAELoss () # Loss function -opt = MyMomentum(net.trainable_params(), 0.01) # Optimizer - -net_with_criterion = MyWithLossCell(net, loss_func) # Build a loss network. -train_net = MyTrainStep(net_with_criterion, opt) # Build a training network. - -for data in ds_train.create_dict_iterator(): - train_net(data['data'], data['label']) # Perform training and update the weight. - loss = net_with_criterion(data['data'], data['label']) # Compute the loss value. - plot_model_and_datasets(train_net, train_data, loss) # Visualize the. -``` - -![png](./images/output_28_0.png) - -## Customized Evaluation Metrics - -When a training task is complete, an evaluation function (Metric) is often required to evaluate the quality of a model. Common evaluation metrics include confusion matrix, accuracy, precision, and recall. - -The [mindspore.nn](https://www.mindspore.cn/docs/en/master/api_python/mindspore.nn.html#evaluation-metrics) module provides common evaluation functions. You can also define evaluation metrics as required. The customized Metric function needs to inherit the `nn.Metric` parent class and re-implement the `clear`, `update`, and `eval` methods in the parent class. The following formula shows the mean absolute error (MAE) algorithm. The following uses MAE as an example to describe the three functions and their usage. - -$$ MAE=\frac{1}{n}\sum_{i=1}^n\lvert y\_pred_i - y_i \rvert \tag{6}$$ - -- `clear`: initializes related internal parameters. -- `update`: receives network prediction output and labels, computes errors, and updates internal evaluation results. Generally, the calculation is performed after each step and the statistical value is updated. -- `eval`: computes the final evaluation result after each epoch ends. - -```python -class MyMAE(nn.Metric): - """Define metrics.""" - - def __init__(self): - super(MyMAE, self).__init__() - self.clear() - - def clear(self): - """Initialize variables abs_error_sum and samples_num.""" - self.abs_error_sum = 0 - self.samples_num = 0 - - def update(self, *inputs): - """Update abs_error_sum and samples_num.""" - y_pred = inputs[0].asnumpy() - y = inputs[1].asnumpy() - - # Compute the absolute error between the predicted value and the actual value. - error_abs = np.abs(y.reshape(y_pred.shape) - y_pred) - self.abs_error_sum += error_abs.sum() - self.samples_num += y.shape[0] # Total number of samples - - def eval(self): - """Compute the final evaluation result."" - return self.abs_error_sum / self.samples_num -``` - -## Customized Evaluation Process - -The mindspore.nn module provides the evaluation network packaging function [nn.WithEvalCell](https://www.mindspore.cn/docs/en/master/api_python/nn/mindspore.nn.WithEvalCell.html#mindspore.nn.WithEvalCell). Because `nn.WithEvalCell` has only two inputs `data` and `label`, it is not applicable to the scenario with multiple data or labels. Therefore, you need to customize the evaluation network. For details about how to customize the evaluation network in the multi-label scenario, see [Customized Training and Evaluation Networks](https://www.mindspore.cn/tutorials/en/master/advanced/train/train_eval.html). - -The following example implements a simple customized evaluation network `MyWithEvalCell`. Enter the input `data` and `label`. - -```python -class MyWithEvalCell(nn.Cell): - """Define the evaluation process.""" - - def __init__(self, network): - super(MyWithEvalCell, self).__init__(auto_prefix=False) - self.network = network - - def construct(self, data, label): - outputs = self.network(data) - return outputs, label -``` - -Perform inference and evaluation: - -```python -data_number = 160 -batch_number = 16 -repeat_number = 1 - -# Obtain evaluation data. -ds_eval = create_dataset(data_number, batch_size=batch_number, repeat_size=repeat_number) - -eval_net = MyWithEvalCell(net) # Define the evaluation network. -eval_net.set_train(False) -mae = MyMAE() - -# Execute the inference process. -for data in ds_eval.create_dict_iterator(): - output, eval_y = eval_net(data['data'], data['label']) - mae.update(output, eval_y) - -mae_result = mae.eval() -print("MAE: ", mae_result) -``` - -```text - MAE: 0.9605088472366333 -``` - -Output evaluation error. The effect of MAE is similar to that of the model in the training set. - -## Saving and Exporting a Model - -Save the trained model parameters to a checkpoint (CKPT) file, and export the checkpoint file as a MindIR file for cross-platform inference. - -```python -import numpy as np -import mindspore as ms - -ms.save_checkpoint(net, "./linear.ckpt") # Save model parameters in a CKPT file. -param_dict = ms.load_checkpoint("./linear.ckpt") # Save the model parameters to the param_dict dictionary. - -# View the model parameters. -for param in param_dict: - print(param, ":", param_dict[param].asnumpy()) - -net1 = LinearNet() -input_np = np.random.uniform(0.0, 1.0, size=[1, 1]).astype(np.float32) -ms.export(net1, ms.Tensor(input_np), file_name='linear', file_format='MINDIR') -``` - -```text - fc.weight : [[1.894384]] - fc.bias : [3.0015702] -``` diff --git a/tutorials/source_en/advanced/model.rst b/tutorials/source_en/advanced/model.rst new file mode 100644 index 0000000000000000000000000000000000000000..19e2cf094f54f2f7870303277cb8d432f304c681 --- /dev/null +++ b/tutorials/source_en/advanced/model.rst @@ -0,0 +1,11 @@ +Advanced Encapsulation: Model +============================== + +.. toctree:: + :maxdepth: 1 + + model/metric + model/train_eval + model/model + model/callback + model/save \ No newline at end of file diff --git a/tutorials/source_en/advanced/train/callback.md b/tutorials/source_en/advanced/model/callback.md similarity index 99% rename from tutorials/source_en/advanced/train/callback.md rename to tutorials/source_en/advanced/model/callback.md index 284b72a8257609ce5f8d9e4fda376fdf9d78b220..d6e379a10ea45b1b40d5b064fe7d2353e0bedbfa 100644 --- a/tutorials/source_en/advanced/train/callback.md +++ b/tutorials/source_en/advanced/model/callback.md @@ -1,6 +1,6 @@ # Callback Mechanism - + During deep learning training, MindSpore provides the callback mechanism to promptly learn about the training status of the network model, observe the changes of network model parameters in real time, and implement customized operations during training. diff --git a/tutorials/source_en/advanced/train/images/model.png b/tutorials/source_en/advanced/model/images/model.png similarity index 100% rename from tutorials/source_en/advanced/train/images/model.png rename to tutorials/source_en/advanced/model/images/model.png diff --git a/tutorials/source_en/advanced/train/metric.md b/tutorials/source_en/advanced/model/metric.md similarity index 99% rename from tutorials/source_en/advanced/train/metric.md rename to tutorials/source_en/advanced/model/metric.md index a0bb31624ee84f89b5e3ac0051eeb9266fe98ef4..e07b63227291bd7bee551a6331319b5ae3b76ee5 100644 --- a/tutorials/source_en/advanced/train/metric.md +++ b/tutorials/source_en/advanced/model/metric.md @@ -1,6 +1,6 @@ # Evaluation Metrics - + When a training task is complete, an evaluation function (Metric) is often required to evaluate the quality of a model. Different training tasks usually require different metric functions. For example, for a binary classification problem, common evaluation metrics include precision, recall, and the like. For a multiclass classification task, macro and micro may be used for evaluation. diff --git a/tutorials/source_en/advanced/train/model.md b/tutorials/source_en/advanced/model/model.md similarity index 99% rename from tutorials/source_en/advanced/train/model.md rename to tutorials/source_en/advanced/model/model.md index 811c68dd06906843b072d93d0e52a9b652eb96be..1a5866ac646ef5c99749cc123699a33932384b2b 100644 --- a/tutorials/source_en/advanced/train/model.md +++ b/tutorials/source_en/advanced/model/model.md @@ -1,6 +1,6 @@ # Basic Usage of Models - + Generally, defining a training and evaluation network and running it directly can meet basic requirements. diff --git a/tutorials/source_en/advanced/train/save.md b/tutorials/source_en/advanced/model/save.md similarity index 99% rename from tutorials/source_en/advanced/train/save.md rename to tutorials/source_en/advanced/model/save.md index 676106d3c30b3ecf151c01a799f0be683df9b7ae..167bcc760020dbe8d8f93a2a9e6a862dc40c8737 100644 --- a/tutorials/source_en/advanced/train/save.md +++ b/tutorials/source_en/advanced/model/save.md @@ -1,6 +1,6 @@ # Saving and Exporting Models - + ## Overview diff --git a/tutorials/source_en/advanced/train/train_eval.md b/tutorials/source_en/advanced/model/train_eval.md similarity index 99% rename from tutorials/source_en/advanced/train/train_eval.md rename to tutorials/source_en/advanced/model/train_eval.md index 9345c1e4107fe8a6cf4bbfab55401344d34efdba..7295419090dd3ab4fac929c3c31a1e21ea62973a 100644 --- a/tutorials/source_en/advanced/train/train_eval.md +++ b/tutorials/source_en/advanced/model/train_eval.md @@ -1,6 +1,6 @@ # Training and Evaluation - + The preceding section describes the basic elements used by MindSpore to build a network, such as the basic network unit, loss function, optimizer, and evaluation function of MindSpore. diff --git a/tutorials/source_en/advanced/modules.rst b/tutorials/source_en/advanced/modules.rst new file mode 100644 index 0000000000000000000000000000000000000000..069a897407f2c44e275977b97d27c6d9d7d2e340 --- /dev/null +++ b/tutorials/source_en/advanced/modules.rst @@ -0,0 +1,12 @@ +Module Customization +===================== + +.. toctree:: + :maxdepth: 1 + + modules/parameter + modules/loss + modules/optim + modules/forward + modules/derivation + modules/control_flow diff --git a/tutorials/source_en/advanced/network/control_flow.md b/tutorials/source_en/advanced/modules/control_flow.md similarity index 99% rename from tutorials/source_en/advanced/network/control_flow.md rename to tutorials/source_en/advanced/modules/control_flow.md index a32b8a4fd8ead9c7c59e12e54c96b3b796acd611..14981714eb9b3ad47d7d1fce3e4929e95105b745 100644 --- a/tutorials/source_en/advanced/network/control_flow.md +++ b/tutorials/source_en/advanced/modules/control_flow.md @@ -1,6 +1,6 @@ # Process Control Statements - + Currently, there are two execution modes of a mainstream deep learning framework: a static graph mode (`GRAPH_MODE`) and a dynamic graph mode (`PYNATIVE_MODE`). diff --git a/tutorials/source_en/advanced/network/forward.md b/tutorials/source_en/advanced/modules/forward.md similarity index 99% rename from tutorials/source_en/advanced/network/forward.md rename to tutorials/source_en/advanced/modules/forward.md index 1e88d28de6ae77a1a792ff5ab231f819e755eeac..4fae41dc68af426f80229eb3fb61741ce1453338 100644 --- a/tutorials/source_en/advanced/network/forward.md +++ b/tutorials/source_en/advanced/modules/forward.md @@ -1,6 +1,6 @@ # Building a Network - + The `Cell` class of MindSpore is the base class for setting up all networks and the basic unit of a network. When customizing a network, you need to inherit the `Cell` class. The following describes the basic network unit `Cell` and customized feedforward network. diff --git a/tutorials/source_en/advanced/network/loss.md b/tutorials/source_en/advanced/modules/loss.md similarity index 99% rename from tutorials/source_en/advanced/network/loss.md rename to tutorials/source_en/advanced/modules/loss.md index 0dde9e90c30dab7ad742799d307ade7f3479f975..c2d6c0a418149a99f79ae2b21c057aec69826862 100644 --- a/tutorials/source_en/advanced/network/loss.md +++ b/tutorials/source_en/advanced/modules/loss.md @@ -1,6 +1,6 @@ # Loss Function - + A loss function is also called objective function and is used to measure the difference between a predicted value and an actual value. diff --git a/tutorials/source_en/advanced/network/optim.md b/tutorials/source_en/advanced/modules/optim.md similarity index 99% rename from tutorials/source_en/advanced/network/optim.md rename to tutorials/source_en/advanced/modules/optim.md index 187b7c97adb999f441e6e462e645003c2bc49ef6..d3cda461a3e10531ebfa59ae27d3f89fa2560767 100644 --- a/tutorials/source_en/advanced/network/optim.md +++ b/tutorials/source_en/advanced/modules/optim.md @@ -1,6 +1,6 @@ # Optimizer - + During model training, the optimizer is used to compute gradients and update network parameters. A proper optimizer can effectively reduce the training time and improve model performance. diff --git a/tutorials/source_en/advanced/network/parameter.md b/tutorials/source_en/advanced/modules/parameter.md similarity index 99% rename from tutorials/source_en/advanced/network/parameter.md rename to tutorials/source_en/advanced/modules/parameter.md index b128ec41fad5ae75eff37483833823974752bd55..4afaf2a5e929d477b5dea6586d0e80679ee10a6e 100644 --- a/tutorials/source_en/advanced/network/parameter.md +++ b/tutorials/source_en/advanced/modules/parameter.md @@ -1,6 +1,6 @@ # Network Arguments - + MindSpore provides initialization modules for parameters and network arguments. You can initialize network arguments by encapsulating operators to call character strings, Initializer subclasses, or customized tensors. diff --git a/tutorials/source_en/advanced/network.rst b/tutorials/source_en/advanced/network.rst deleted file mode 100644 index e94141feb8d45acfecefdd817af9c335e31e1866..0000000000000000000000000000000000000000 --- a/tutorials/source_en/advanced/network.rst +++ /dev/null @@ -1,12 +0,0 @@ -Network Building -=================== - -.. toctree:: - :maxdepth: 1 - - network/parameter - network/loss - network/optim - network/forward - network/derivation - network/control_flow diff --git a/tutorials/source_en/advanced/train.rst b/tutorials/source_en/advanced/train.rst deleted file mode 100644 index a741cb5bec57593ae51f8c3dba67c2df093d9f7b..0000000000000000000000000000000000000000 --- a/tutorials/source_en/advanced/train.rst +++ /dev/null @@ -1,11 +0,0 @@ -Training and Evaluation -======================= - -.. toctree:: - :maxdepth: 1 - - train/metric - train/train_eval - train/model - train/callback - train/save \ No newline at end of file diff --git a/tutorials/source_en/beginner/quick_start.md b/tutorials/source_en/beginner/quick_start.md index 097e5b0409081a97791954c8352c7220fee087fd..5dec8c2e77f2afbb27a6977485413d723bd99e58 100644 --- a/tutorials/source_en/beginner/quick_start.md +++ b/tutorials/source_en/beginner/quick_start.md @@ -1,207 +1,469 @@ -# Quickstart: Handwritten Digit Recognition +# Quick Start: Linear Fitting -This section runs through the basic process of MindSpore deep learning, using the LeNet5 network model as an example to implement common tasks in deep learning. +MindSpore provides high-level, medium-level, and low-level APIs. For details, see [API Level Structure](https://www.mindspore.cn/tutorials/en/master/beginner/introduction.html#api-level-structure). -## Downloading and Processing the Dataset +To facilitate the control of the network execution process, MindSpore provides the high-level training and inference API `mindspore.Model`. By specifying the neural network model to be trained and common training settings, MindSpore calls the `train` and `eval` methods to train and infer the network. In addition, if you want to customize a specific module, you can call the corresponding medium- and low-level APIs to define the network training process. -Datasets are crucial for model training. A good dataset can effectively improve training accuracy and efficiency. The MNIST dataset used in the example consists of 28 x 28 grayscale images of 10 classes. The training dataset contains 60,000 images, and the test dataset contains 10,000 images. +The following uses the medium- and low-level APIs provided by MindSpore to fit linear functions. -![mnist](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/tutorials/source_zh_cn/beginner/images/mnist.png) +$$f(x) = 2x + 3 \tag {1}$$ -> You can download the dataset from [MNIST dataset download page](http://yann.lecun.com/exdb/mnist/), decompress it, and save it according to the following directory structure. +Before initializing the network, you need to configure the `context` parameter to control the program execution policy. For example, configure the static graph or dynamic graph mode and configure the hardware environment for network running. -The [MindSpore Vision](https://mindspore.cn/vision/docs/en/master/index.html) suite provides the Mnist module for downloading and processing MNIST datasets. The following sample code downloads and decompresses the datasets to the specified location for data processing: +The following describes how to configure information and use medium- and low-level APIs provided by MindSpore to customize loss functions, optimizers, training processes, metrics, and evaluation processes. -> The sample code in this chapter relies on `mindvision`, which can be installed by using the command `pip install mindvision`. If this document is run as Notebook, you need to restart the kernel after installation to execute subsequent code. +## Configuration Information + +Before initializing the network, you need to configure the `context` parameter to control the program execution policy. For example, configure the static graph or dynamic graph mode and configure the hardware environment for network running. Before initializing the network, you need to configure the `context` parameter to control the program execution policy. The following describes the execution mode management and hardware management. + +### Execution Mode + +MindSpore supports two running modes: Graph and PyNative. By default, MindSpore uses the Graph mode, and the PyNative mode is used for debugging. + +- Graph mode (static graph mode): The neural network model is built into an entire graph and then delivered to the hardware for execution. This mode uses graph optimization to improve the running performance and facilitates large-scale deployment and cross-platform running. + +- PyNative mode (dynamic graph mode): Operators in the neural network are delivered to the hardware one by one for execution. This mode facilitates code writing and neural network model debugging. + +MindSpore provides a unified encoding mode for static and dynamic graphs, significantly enhancing compatibility between both types of graphs. This enables you to switch between the static and dynamic graph modes by changing only one line of code, eliminating the need to develop multiple sets of code. When switching the mode, pay attention to the [constraints](https://www.mindspore.cn/docs/en/master/note/static_graph_syntax_support.html) of the target mode. + +Set the running mode to dynamic graph mode. ```python -from mindvision.dataset import Mnist +import mindspore as ms -# Download and process the MNIST dataset. -download_train = Mnist(path="./mnist", split="train", batch_size=32, repeat_num=1, shuffle=True, resize=32, download=True) +ms.set_context(mode=ms.PYNATIVE_MODE) +``` -download_eval = Mnist(path="./mnist", split="test", batch_size=32, resize=32, download=True) +Similarly, when MindSpore is in dynamic image mode, you can run the `set_context(mode=GRAPH_MODE)` command to switch to the static graph mode. -dataset_train = download_train.run() -dataset_eval = download_eval.run() +```python +import mindspore as ms + +ms.set_context(mode=ms.GRAPH_MODE) ``` -Parameters description: +### Hardware Management -- path: dataset path. -- split: dataset type. The value can be train, test, or infer. The default value is train. -- batch_size: data size set for each training batch. The default value is 32. -- repeat_num: number of times that the dataset is traversed during training. The default value is 1. -- shuffle: determines whether to randomly shuffle the dataset. This parameter is optional. -- resize: size of the output image. The default value is 32 x 32. -- download: determines whether to download the dataset. The default value is False. +Hardware management involves the `device_target` and `device_id` parameters. -The directory structure of the downloaded dataset files is as follows: +- `device_target`: target device to be run. The value can be `Ascend`, `GPU`, or `CPU`. You can set this parameter based on the actual situation or use the default value. -```text -./mnist/ -├── test -│ ├── t10k-images-idx3-ubyte -│ └── t10k-labels-idx1-ubyte -└── train - ├── train-images-idx3-ubyte - └── train-labels-idx1-ubyte +- `device_id`: ID of the target device. The value is in the range of [0,`device_num_per_host` - 1]. `device_num_per_host` indicates the total number of devices on the server. The value of `device_num_per_host` cannot exceed 4096. The default value of `device_id` is 0. + +> When the program is executed in non-distributed mode, you can set `device_id` to determine the ID of the device where the program is executed to avoid device usage conflicts. + +A code example is as follows: + +```Python +import mindspore as mst + +ms.set_context(device_target="Ascend", device_id=6) ``` -## Building the Model +## Processing Datasets -Except the input layer, LeNet contains seven layers: three convolutional layers, two subsampling layers, and two fully-connected layers. +### Generating a Dataset -![](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/tutorials/source_zh_cn/beginner/images/lenet.png) +Define the dataset generation function `get_data` to generate the training dataset and test dataset. -The MindSpore Vision Suite provides the LeNet model interface `lenet`, which defines the network model as follows: +Since linear data is fitted, the required training datasets should be randomly distributed around the objective function. Assume that the objective function to be fitted is $f(x)=2x+3$. $f(x)=2x+3+noise$ is used to generate training datasets, and `noise` is a random value that complies with standard normal distribution rules. ```python -from mindvision.classification.models import lenet +import numpy as np -network = lenet(num_classes=10, pretrained=False) +def get_data(num, w=2.0, b=3.0): + for _ in range(num): + x = np.random.uniform(-10.0, 10.0) + noise = np.random.normal(0, 1) + y = x * w + b + noise + yield np.array([x]).astype(np.float32), np.array([y]).astype(np.float32) ``` -## Defining a Loss Function and an Optimizer +Use get_data to generate 50 groups of evaluation data and visualize the data. + +```python +import matplotlib.pyplot as plt + +train_data = list(get_data(50)) +x_target_label = np.array([-10, 10, 0.1]) +y_target_label = x_target_label * 2 + 3 +x_eval_label, y_eval_label = zip(*train_data) + +plt.scatter(x_eval_label, y_eval_label, color="red", s=5) +plt.plot(x_target_label, y_target_label, color="green") +plt.title("Eval data") +plt.show() +``` + +![png](./images/output_8_0.png) + +In the figure shown above, the green line indicates the target function, and the red point indicates the evaluation data (`train_data`). -To train a neural network model, you need to define a loss function and an optimizer function. +### Loading a Dataset -- The following uses the cross-entropy loss function `SoftmaxCrossEntropyWithLogits`. -- The optimizer is `Momentum`. +Loads the dataset generated by the `get_data` function to the system memory and performs basic data processing operations. + +- `ds.GeneratorDataset`: converts the generated data into a MindSpore dataset and saves the x and y values of the generated data to arrays of `data` and `label`. +- `batch`: combines `batch_size` pieces of data into a batch. +- `repeat`: Multiplies the number of data sets. ```python -import mindspore.nn as nn +from mindspore import dataset as ds + +def create_dataset(num_data, batch_size=16, repeat_size=1): + input_data = ds.GeneratorDataset(list(get_data(num_data)), column_names=['data', 'label']) + input_data = input_data.batch(batch_size, drop_remainder=True) + input_data = input_data.repeat(repeat_size) + return input_data +``` -# Define the loss function. -net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean') +Use the dataset augmentation function to generate training data. Use the defined `create_dataset` to augment the generated 1600 pieces of data into 100 datasets with the shape of 16 x 1. -# Define the optimizer function. -net_opt = nn.Momentum(network.trainable_params(), learning_rate=0.01, momentum=0.9) +```python +data_number = 1600 +batch_number = 16 +repeat_number = 1 + +ds_train = create_dataset(data_number, batch_size=batch_number, repeat_size=repeat_number) +print("The dataset size of ds_train:", ds_train.get_dataset_size()) +step_size = ds_train.get_dataset_size() +dict_datasets = next(ds_train.create_dict_iterator()) + +print(dict_datasets.keys()) +print("The x label value shape:", dict_datasets["data"].shape) +print("The y label value shape:", dict_datasets["label"].shape) +``` + +```text + The dataset size of ds_train: 100 + dict_keys(['data', 'label']) + The x label value shape: (16, 1) + The y label value shape: (16, 1) ``` -## Training and Saving the Model +## Defining a Network Model + +The `mindspore.nn` class is the base class for setting up all networks and the basic unit of a network. In order to customize a network, you can inherit the `nn.Cell` class and overwrite the `__init__` and `construct` methods. + +The `mindspore.ops` module provides the implementation of basic operators. The `nn.Cell` module further encapsulates basic operators. You can flexibly use different operators as required. + +The following example uses `nn.Cell` to build a simple fully-connected network for subsequent customized content. In MindSpore, use `nn.Dense` to generate a linear function model with a single data input and a single data output. + +$$f(x)=wx+b\tag{2}$$ + +Use the Normal operator to randomly initialize the $w$ and $b$ parameters in formula (2). + +```python +from mindspore import nn +from mindspore.common.initializer import Normal + +class LinearNet(nn.Cell): + def __init__(self): + super(LinearNet, self).__init__() + self.fc = nn.Dense(1, 1, Normal(0.02), Normal(0.02)) + + def construct(self, x): + fx = self.fc(x) + return fx +``` -Before training, MindSpore needs to declare whether the intermediate process and result of the network model need to be saved during training. Therefore, the `ModelCheckpoint` API is used to save the network model and parameters for subsequent fine-tuning. +After initializing the network model, visualize the initialized network function and training dataset to understand the model function before fitting. ```python import mindspore as ms -# Set the model saving parameters. The checkpoint steps are 1875. -config_ck = ms.CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10) +net = LinearNet() # Initialize the linear regression network. -# Apply the model saving parameters. -ckpoint = ms.ModelCheckpoint(prefix="lenet", directory="./lenet", config=config_ck) +model_params = net.trainable_params() # Obtain network parameters w and b before training. + +x_model_label = np.array([-10, 10, 0.1]) +y_model_label = (x_model_label * model_params[0].asnumpy()[0] + model_params[1].asnumpy()[0]) + +plt.axis([-10, 10, -20, 25]) +plt.scatter(x_eval_label, y_eval_label, color="red", s=5) +plt.plot(x_model_label, y_model_label, color="blue") +plt.plot(x_target_label, y_target_label, color="green") +plt.show() ``` -The `model.train` API provided by MindSpore can be used to easily train the network. `LossMonitor` can monitor the changes of the `loss` value during the training process. +![png](./images/output_16_0.png) + +## Customized Loss Functions + +A loss function is used to measure the difference between the predicted value and the actual value. In deep learning, model training is a process of reducing a loss function value through continuous iteration. Therefore, it is very important to select a loss function in a model training process. Defining a good loss function can help the loss function value converge faster and achieve better accuracy. + +[mindspore.nn](https://www.mindspore.cn/docs/en/master/api_python/mindspore.nn.html#loss-function) provides many common loss functions for users to select and allows users to customize loss functions as required. + +When customizing a loss function class, you can inherit the base class `nn.Cell` of the network or the base class `nn.LossBase` of the loss function. `nn.LossBase` is based on `nn.Cell` and provides the `get_loss` method. The `reduction` parameter is used to obtain a sum or mean loss value and output a scalar. The following describes how to define the mean absolute error (MAE) function by inheriting LossBase. The formula of the MAE algorithm is as follows: + +$$ loss= \frac{1}{m}\sum_{i=1}^m\lvert y_i-f(x_i) \rvert \tag{3}$$ + +In the preceding formula, $f(x)$ indicates the predicted value, $y$ indicates the actual value of the sample, and $loss$ indicates the mean distance between the predicted value and the actual value. + +When using the method inherited from LossBase to define the loss function, you need to rewrite the `__init__` and `construct` methods and use the `get_loss` method to compute the loss. The sample code is as follows: + +```python +from mindspore import nn, ops + +class MyMAELoss(nn.LossBase): + """Define the loss.""" + def __init__(self): + super(MyMAELoss, self).__init__() + self.abs = ops.Abs() + + def construct(self, predict, target): + x = self.abs(target - predict) + return self.get_loss(x) +``` + +## Customized Optimizer + +During model training, the optimizer is used to compute and update network parameters. A proper optimizer can effectively reduce the training time and improve model performance. + +[mindspore.nn](https://www.mindspore.cn/docs/en/master/api_python/mindspore.nn.html#optimizer) provides many common optimizers for users to select and allows users to customize optimizers as required. + +When customizing an optimizer, you can inherit the optimizer base class `nn.Optimizer` and rewrite the `__init__` and `construct` methods to update parameters. + +The following example implements the customized optimizer Momentum (SGD algorithm with momentum): + +$$ v_{t+1} = v_t × u+grad \tag{4}$$ + +$$p_{t+1} = p_t - lr × v_{t+1} \tag{5}$$ + +$grad$, $lr$, $p$, $v$, and $u$ respectively represent a gradient, a learning rate, a weight parameter, a momentum parameter, and an initial speed. ```python import mindspore as ms -from mindvision.engine.callback import LossMonitor +from mindspore import nn, ops + +class MyMomentum(nn.Optimizer): + """Define the optimizer.""" + + def __init__(self, params, learning_rate, momentum=0.9): + super(MyMomentum, self).__init__(learning_rate, params) + self.moment = ms.Parameter(ms.Tensor(momentum, ms.float32), name="moment") + self.momentum = self.parameters.clone(prefix="momentum", init="zeros") + self.assign = ops.Assign() + + def construct(self, gradients): + """The input of construct is gradient. Gradients are automatically transferred during training.""" + lr = self.get_lr() + params = self.parameters # Weight parameter to be updated + for i in range(len(params)): + self.assign(self.momentum[i], self.momentum[i] * self.moment + gradients[i]) + update = params[i] - self.momentum[i] * lr # SGD algorithm with momentum + self.assign(params[i], update) + return params +``` + +## Customized Training Process + +`mindspore.Model` provides `train` and `eval` APIs for users to use during training. However, this API does not apply to all scenarios, such as multi-data and multi-label scenarios, where users need to define the training process. -# Initialize the model parameters. -model = ms.Model(network, loss_fn=net_loss, optimizer=net_opt, metrics={'accuracy'}) +The following uses linear regression as an example to describe the customized training process. First, define the loss network and connect the forward network to the loss function. Then, define the training process. Generally, the training process inherits `nn.TrainOneStepCell`. `nn.TrainOneStepCell` encapsulates the loss network and optimizer to implement the backward propagation network to update the weight parameters. -# Train the network model. The model name is lenet-1_1875.ckpt. -model.train(10, dataset_train, callbacks=[ckpoint, LossMonitor(0.01, 1875)]) +### Defining a Loss Network + +Define the loss network `MyWithLossCell` to connect the feedforward network to the loss function. + +```python +class MyWithLossCell(nn.Cell): + """Define the loss network.""" + + def __init__(self, backbone, loss_fn): + """Transfer the feedforward network and loss function as parameters during instantiation.""" + super(MyWithLossCell, self).__init__(auto_prefix=False) + self.backbone = backbone + self.loss_fn = loss_fn + + def construct(self, data, label): + """Connect the feedforward network and loss function.""" + out = self.backbone(data) + return self.loss_fn(out, label) + + def backbone_network(self): + """Backbone network to be encapsulated.""" + return self.backbone ``` -```text -Epoch:[ 0/ 10], step:[ 1875/ 1875], loss:[0.314/0.314], time:2237.313 ms, lr:0.01000 -Epoch time: 3577.754 ms, per step time: 1.908 ms, avg loss: 0.314 -Epoch:[ 1/ 10], step:[ 1875/ 1875], loss:[0.031/0.031], time:1306.982 ms, lr:0.01000 -Epoch time: 1307.792 ms, per step time: 0.697 ms, avg loss: 0.031 -Epoch:[ 2/ 10], step:[ 1875/ 1875], loss:[0.007/0.007], time:1324.625 ms, lr:0.01000 -Epoch time: 1325.340 ms, per step time: 0.707 ms, avg loss: 0.007 -Epoch:[ 3/ 10], step:[ 1875/ 1875], loss:[0.021/0.021], time:1396.733 ms, lr:0.01000 -Epoch time: 1397.495 ms, per step time: 0.745 ms, avg loss: 0.021 -Epoch:[ 4/ 10], step:[ 1875/ 1875], loss:[0.028/0.028], time:1594.762 ms, lr:0.01000 -Epoch time: 1595.549 ms, per step time: 0.851 ms, avg loss: 0.028 -Epoch:[ 5/ 10], step:[ 1875/ 1875], loss:[0.007/0.007], time:1242.175 ms, lr:0.01000 -Epoch time: 1242.928 ms, per step time: 0.663 ms, avg loss: 0.007 -Epoch:[ 6/ 10], step:[ 1875/ 1875], loss:[0.033/0.033], time:1199.938 ms, lr:0.01000 -Epoch time: 1200.627 ms, per step time: 0.640 ms, avg loss: 0.033 -Epoch:[ 7/ 10], step:[ 1875/ 1875], loss:[0.175/0.175], time:1228.845 ms, lr:0.01000 -Epoch time: 1229.548 ms, per step time: 0.656 ms, avg loss: 0.175 -Epoch:[ 8/ 10], step:[ 1875/ 1875], loss:[0.009/0.009], time:1237.200 ms, lr:0.01000 -Epoch time: 1237.969 ms, per step time: 0.660 ms, avg loss: 0.009 -Epoch:[ 9/ 10], step:[ 1875/ 1875], loss:[0.000/0.000], time:1287.693 ms, lr:0.01000 -Epoch time: 1288.413 ms, per step time: 0.687 ms, avg loss: 0.000 +### Defining the Training Process + +Define the training process `MyTrainStep`. This class inherits `nn.TrainOneStepCell`. `nn.TrainOneStepCell` encapsulates the loss network and optimizer. During training, the `ops.GradOperation` operator is used to obtain the gradient, the optimizer is used to update the weight. + +```python +class MyTrainStep(nn.TrainOneStepCell): + """Define the training process.""" + + def __init__(self, network, optimizer): + """Initialize parameters.""" + super(MyTrainStep, self).__init__(network, optimizer) + self.grad = ops.GradOperation(get_by_list=True) + + def construct(self, data, label): + """Build the training process.""" + weights = self.weights + loss = self.network(data, label) + grads = self.grad(self.network, weights)(data, label) + return loss, self.optimizer(grads) ``` -During the training, the loss value is printed and fluctuates. However, the loss value gradually decreases and the precision gradually increases. Loss values displayed each time may be different because of their randomness. +The following defines the drawing function `plot_model_and_datasets` to draw the test data, objective function, and network model fitting function, and view the loss value. + +```python +from IPython import display +import matplotlib.pyplot as plt +import time + +def plot_model_and_datasets(net, data, loss): + weight = net.trainable_params()[0] + bias = net.trainable_params()[1] + x = np.arange(-10, 10, 0.1) + y = x * ms.Tensor(weight).asnumpy()[0][0] + ms.Tensor(bias).asnumpy()[0] + x1, y1 = zip(*data) + x_target = x + y_target = x_target * 2 + 3 + + plt.axis([-11, 11, -20, 25]) + plt.scatter(x1, y1, color="red", s=5) # Raw data + plt.plot(x, y, color="blue") # Predicted data + plt.plot(x_target, y_target, color="green") # Fitting function + plt.title(f"Loss:{loss}") # Printed loss value + + plt.show() + time.sleep(0.2) + display.clear_output(wait=True) +``` -Validate the generalization capability of the model based on the result obtained by running the test dataset. +### Training -1. Read the test dataset using the `model.eval` API. -2. Use the saved model parameters for inference. +Use the training data `ds_train` to train the training network `train_net` and visualize the training process. ```python -acc = model.eval(dataset_eval) +loss_func = MyMAELoss () # Loss function +opt = MyMomentum(net.trainable_params(), 0.01) # Optimizer -print("{}".format(acc)) +net_with_criterion = MyWithLossCell(net, loss_func) # Build a loss network. +train_net = MyTrainStep(net_with_criterion, opt) # Build a training network. + +for data in ds_train.create_dict_iterator(): + train_net(data['data'], data['label']) # Perform training and update the weight. + loss = net_with_criterion(data['data'], data['label']) # Compute the loss value. + plot_model_and_datasets(train_net, train_data, loss) # Visualize the. ``` -```text -{'accuracy': 0.9903846153846154} +![png](./images/output_28_0.png) + +## Customized Evaluation Metrics + +When a training task is complete, an evaluation function (Metric) is often required to evaluate the quality of a model. Common evaluation metrics include confusion matrix, accuracy, precision, and recall. + +The [mindspore.nn](https://www.mindspore.cn/docs/en/master/api_python/mindspore.nn.html#evaluation-metrics) module provides common evaluation functions. You can also define evaluation metrics as required. The customized Metric function needs to inherit the `nn.Metric` parent class and re-implement the `clear`, `update`, and `eval` methods in the parent class. The following formula shows the mean absolute error (MAE) algorithm. The following uses MAE as an example to describe the three functions and their usage. + +$$ MAE=\frac{1}{n}\sum_{i=1}^n\lvert y\_pred_i - y_i \rvert \tag{6}$$ + +- `clear`: initializes related internal parameters. +- `update`: receives network prediction output and labels, computes errors, and updates internal evaluation results. Generally, the calculation is performed after each step and the statistical value is updated. +- `eval`: computes the final evaluation result after each epoch ends. + +```python +class MyMAE(nn.Metric): + """Define metrics.""" + + def __init__(self): + super(MyMAE, self).__init__() + self.clear() + + def clear(self): + """Initialize variables abs_error_sum and samples_num.""" + self.abs_error_sum = 0 + self.samples_num = 0 + + def update(self, *inputs): + """Update abs_error_sum and samples_num.""" + y_pred = inputs[0].asnumpy() + y = inputs[1].asnumpy() + + # Compute the absolute error between the predicted value and the actual value. + error_abs = np.abs(y.reshape(y_pred.shape) - y_pred) + self.abs_error_sum += error_abs.sum() + self.samples_num += y.shape[0] # Total number of samples + + def eval(self): + """Compute the final evaluation result."" + return self.abs_error_sum / self.samples_num ``` -The model accuracy data is displayed in the output content. In the example, the accuracy reaches 95%, indicating a good model quality. As the number of network epochs increases, the model accuracy will be further improved. +## Customized Evaluation Process + +The mindspore.nn module provides the evaluation network packaging function [nn.WithEvalCell](https://www.mindspore.cn/docs/en/master/api_python/nn/mindspore.nn.WithEvalCell.html#mindspore.nn.WithEvalCell). Because `nn.WithEvalCell` has only two inputs `data` and `label`, it is not applicable to the scenario with multiple data or labels. Therefore, you need to customize the evaluation network. For details about how to customize the evaluation network in the multi-label scenario, see [Customized Training and Evaluation Networks](https://www.mindspore.cn/tutorials/en/master/advanced/train/train_eval.html). -## Loading the Model +The following example implements a simple customized evaluation network `MyWithEvalCell`. Enter the input `data` and `label`. ```python -import mindspore as ms +class MyWithEvalCell(nn.Cell): + """Define the evaluation process.""" -# Load the saved model used for testing. -param_dict = ms.load_checkpoint("./lenet/lenet-1_1875.ckpt") -# Load parameters to the network. -ms.load_param_into_net(network, param_dict) + def __init__(self, network): + super(MyWithEvalCell, self).__init__(auto_prefix=False) + self.network = network + + def construct(self, data, label): + outputs = self.network(data) + return outputs, label ``` -```text -[] +Perform inference and evaluation: + +```python +data_number = 160 +batch_number = 16 +repeat_number = 1 + +# Obtain evaluation data. +ds_eval = create_dataset(data_number, batch_size=batch_number, repeat_size=repeat_number) + +eval_net = MyWithEvalCell(net) # Define the evaluation network. +eval_net.set_train(False) +mae = MyMAE() + +# Execute the inference process. +for data in ds_eval.create_dict_iterator(): + output, eval_y = eval_net(data['data'], data['label']) + mae.update(output, eval_y) + +mae_result = mae.eval() +print("MAE: ", mae_result) ``` -> For more information about loading a model in mindspore, see [Loading the Model](https://www.mindspore.cn/tutorials/en/master/beginner/save_load.html#loading-the-model). +```text + MAE: 0.9605088472366333 +``` -## Validating the Model +Output evaluation error. The effect of MAE is similar to that of the model in the training set. -Use the generated model to predict the classification of a single image. The procedure is as follows: +## Saving and Exporting a Model -> - The predicted image is randomly generated, and the execution result may be different each time. -> - The example uses the mindspore Tensor module, please see [Tensor](https://www.mindspore.cn/tutorials/en/master/beginner/tensor.html). +Save the trained model parameters to a checkpoint (CKPT) file, and export the checkpoint file as a MindIR file for cross-platform inference. ```python import numpy as np import mindspore as ms -import matplotlib.pyplot as plt -mnist = Mnist("./mnist", split="train", batch_size=6, resize=32) -dataset_infer = mnist.run() -ds_test = dataset_infer.create_dict_iterator() -data = next(ds_test) -images = data["image"].asnumpy() -labels = data["label"].asnumpy() - -plt.figure() -for i in range(1, 7): - plt.subplot(2, 3, i) - plt.imshow(images[i-1][0], interpolation="None", cmap="gray") -plt.show() +ms.save_checkpoint(net, "./linear.ckpt") # Save model parameters in a CKPT file. +param_dict = ms.load_checkpoint("./linear.ckpt") # Save the model parameters to the param_dict dictionary. -# Use the model.predict function to predict the classification of the image. -output = model.predict(ms.Tensor(data['image'])) -predicted = np.argmax(output.asnumpy(), axis=1) +# View the model parameters. +for param in param_dict: + print(param, ":", param_dict[param].asnumpy()) -# Output the predicted classification and the actual classification. -print(f'Predicted: "{predicted}", Actual: "{labels}"') +net1 = LinearNet() +input_np = np.random.uniform(0.0, 1.0, size=[1, 1]).astype(np.float32) +ms.export(net1, ms.Tensor(input_np), file_name='linear', file_format='MINDIR') ``` ```text -Predicted: "[4 6 2 3 5 1]", Actual: "[4 6 2 3 5 1]" + fc.weight : [[1.894384]] + fc.bias : [3.0015702] ``` - -The preceding information shows that the predicted values are the same as the target values. diff --git a/tutorials/source_en/index.rst b/tutorials/source_en/index.rst index fdc531f9fc1c2fb34feb72c4ac27fd2338fa445b..0b5462dbb895bf949230420e1270d47c399580bd 100644 --- a/tutorials/source_en/index.rst +++ b/tutorials/source_en/index.rst @@ -26,8 +26,9 @@ MindSpore Tutorial :maxdepth: 1 :caption: Advanced - advanced/linear_fitting + advanced/lenet_mnist + advanced/model + advanced/modules advanced/dataset - advanced/network - advanced/train - advanced/pynative_graph \ No newline at end of file + advanced/derivation + advanced/compute_graph diff --git a/tutorials/source_zh_cn/advanced/derivation.ipynb b/tutorials/source_zh_cn/advanced/derivation.ipynb index b8e3fd1b588564808fe28b0322953b7f4d6ae697..e07f72f789b0ce2244c99936d1dee50cb6998a21 100644 --- a/tutorials/source_zh_cn/advanced/derivation.ipynb +++ b/tutorials/source_zh_cn/advanced/derivation.ipynb @@ -8,7 +8,7 @@ "\n", "[![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_notebook.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/master/tutorials/zh_cn/advanced/mindspore_derivation.ipynb) \n", "[![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_download_code.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/master/tutorials/zh_cn/advanced/indspore_derivation.py) \n", - "[![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/master/tutorials/source_zh_cn/advanced/erivation.ipynb)\n", + "[![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/master/tutorials/source_zh_cn/advanced/derivation.ipynb)\n", "\n", "`mindspore.ops`模块提供的`GradOperation`接口可以生成网络模型的梯度。本文主要介绍如何使用`GradOperation`接口进行一阶、二阶求导，以及如何停止计算梯度。\n", "\n",