diff --git a/tutorials/source_en/beginner/autograd.md b/tutorials/source_en/beginner/autograd.md
index 31c200b3eba5fee274af1509cfb082cfd17b1372..b784fc75b00021b22cc85d97362e95f5d371ca41 100644
--- a/tutorials/source_en/beginner/autograd.md
+++ b/tutorials/source_en/beginner/autograd.md
@@ -1,195 +1,205 @@
# Automatic Differentiation
-`Ascend` `GPU` `CPU` `Beginner` `Model Development`
+
-
+Automatic differentiation is able to calculate the derivative value of a derivative function at a certain point, which is a generalization of backpropagation algorithms. The main problem solved by automatic differentiation is to decompose a complex mathematical operation into a series of simple basic operations, which shields the user from a large number of details and processes of differentiation, which greatly reduces the threshold for the use of the framework.
-Automatic differentiation is commonly used when implementing machine learning algorithms such as backpropagation for training neural networks. By using automatic differentiation, multi-layer composite functions could be divided into several simple computational steps, thereby helping users avoid implementing complex derivation codes. As a result, automatic differentiation enables ease of use of MindSpore.
+MindSpore uses `ops.GradOperation` to calculate the first derivative, and the attribute of the `ops.GradOperation`:
-The first-order derivative method of MindSpore is `mindspore.ops.GradOperation (get_all=False, get_by_list=False, sens_param=False)`. When `get_all` is set to `False`, the first input derivative is computed. When `get_all` is set to `True`, all input derivatives are computed. When `get_by_list` is set to `False`, weight derivatives are not computed. When `get_by_list` is set to `True`, the weight derivative is computed. `sens_param` scales the output value of the network to change the final gradient. The following uses the MatMul operator derivative for in-depth analysis.
+- `get_all`：Whether to derive the input parameters, the default value is False.
+- `get_by_list`：Whether to derive the weight parameter, the default value is False.
+- `sens_param`：Whether to scale the output value of the network to change the final gradient, the default value is False.
-Import the required modules and APIs:
+This chapter uses `ops.GradOperation` in the MindSpore to find a first derivative for the function y=wx+b.
-```python
-import numpy as np
-import mindspore.nn as nn
-import mindspore.ops as ops
-from mindspore import Tensor
-from mindspore import ParameterTuple, Parameter
-from mindspore import dtype as mstype
-```
+## First Order Dderivative for Input
-## First-order Derivative of the Input
+Before the required modules is imported, the formula needs to be defined first:
-To compute the input derivative, you need to define a network requiring a derivative. The following uses a network $f(x,y)=z *x* y$ formed by the MatMul operator as an example.
+ y=wx+b (1)
-The network structure is as follows:
+The following code is the expression of the formula (1). Since the function programming is applied in the MindSpore, the functions are applied in all the calculation formulas to express.
```python
+import numpy as np
+import mindspore.nn as nn
+from mindspore import Tensor, Parameter
+
class Net(nn.Cell):
def __init__(self):
super(Net, self).__init__()
- self.matmul = ops.MatMul()
- self.z = Parameter(Tensor(np.array([1.0], np.float32)), name='z')
+ self.w = Parameter(Tensor(np.array([6], np.float32)), name='w')
+ self.b = Parameter(Tensor(np.array([1.0], np.float32)), name='b')
- def construct(self, x, y):
- x = x * self.z
- out = self.matmul(x, y)
+ def construct(self, x):
+ out = x * self.w + self.b
return out
```
-Define the network requiring the derivative. In the `__init__` function, define the `self.net` and `ops.GradOperation` networks. In the `construct` function, compute the derivative of `self.net`.
-
-The network structure is as follows:
+Then define derivation type `GradNet`. In the type of `__init__`, the definition needs internet `self.net` and `ops.GradOperation` of the derivation, and the type of `construct` derivates to the `self.net`. The inside of the MindSpore may generate the following formula (2):
+ 𝑦′=𝑤 (2)
```python
-class GradNetWrtX(nn.Cell):
+from mindspore import dtype as mstype
+import mindspore.ops as ops
+
+class GradNet(nn.Cell):
def __init__(self, net):
- super(GradNetWrtX, self).__init__()
+ super(GradNet, self).__init__()
self.net = net
self.grad_op = ops.GradOperation()
- def construct(self, x, y):
+ def construct(self, x):
gradient_function = self.grad_op(self.net)
- return gradient_function(x, y)
+ return gradient_function(x)
```
-Define the input and display the output:
+At last define the input, and the function is found for the input.
```python
-x = Tensor([[0.8, 0.6, 0.2], [1.8, 1.3, 1.1]], dtype=mstype.float32)
-y = Tensor([[0.11, 3.3, 1.1], [1.1, 0.2, 1.4], [1.1, 2.2, 0.3]], dtype=mstype.float32)
-output = GradNetWrtX(Net())(x, y)
-print(output)
-```
+x = Tensor([6], dtype=mstype.float32)
+output = GradNet(Net())(x)
-```text
- [[4.5099998 2.7 3.6000001]
- [4.5099998 2.7 3.6000001]]
+print(output)
```
-If the derivatives of the `x` and `y` inputs are considered, you only need to set `self.grad_op = GradOperation(get_all=True)` in `GradNetWrtX`.
-
-## First-order Derivative of the Weight
-
-To compute weight derivatives, you need to set `get_by_list` in `ops.GradOperation` to `True`.
+## First Derivative for Weight
-The `GradNetWrtX` structure is as follows:
+Finding the function for the weight parameters, `get_by_list` of `ops.GradOperation` needs to be set as `True`.
```python
-class GradNetWrtX(nn.Cell):
+from mindspore import ParameterTuple
+
+class GradNet(nn.Cell):
def __init__(self, net):
- super(GradNetWrtX, self).__init__()
+ super(GradNet, self).__init__()
self.net = net
self.params = ParameterTuple(net.trainable_params())
- self.grad_op = ops.GradOperation(get_by_list=True)
+ self.grad_op = ops.GradOperation(get_by_list=True) # set the first order derivative for the weight parameter
- def construct(self, x, y):
+ def construct(self, x):
gradient_function = self.grad_op(self.net, self.params)
- return gradient_function(x, y)
+ return gradient_function(x)
```
-Run and display the output:
+Derivate the function:
```python
-output = GradNetWrtX(Net())(x, y)
+# Perform a derivative calculation on the function
+x = Tensor([6], dtype=mstype.float32)
+output = GradNet(Net())(x)
+
+# print results
print(output)
+print(f"wgrad: {output[0]}\nbgrad: {output[1]}")
```
```text
-(Tensor(shape=[1], dtype=Float32, value= [ 2.15359993e+01]),)
+(Tensor(shape=[1], dtype=Float32, value= [ 6.00000000e+00]), Tensor(shape=[1], dtype=Float32, value= [ 1.00000000e+00]))
+wgrad: [6.]
+bgrad: [1.]
```
-If computation of certain weight derivatives is not required, set `requirements_grad` to `False` when defining the network requiring derivatives.
+If some weights do not need to be derived, when the derivative network is defined, during the corresponding weight parameter declaration definition, the `requires_grad` needs to be set to `False`.
-```Python
-self.z = Parameter(Tensor(np.array([1.0], np.float32)), name='z', requires_grad=False)
-```
+```python
+class Net(nn.Cell):
+ def __init__(self):
+ super(Net, self).__init__()
+ self.w = Parameter(Tensor(np.array([6], np.float32)), name='w')
+ self.b = Parameter(Tensor(np.array([1.0], np.float32)), name='b', requires_grad=False)
-## Gradient Value Scaling
+ def construct(self, x):
+ out = x * self.w + self.b
+ return out
-You can use the `sens_param` parameter to scale the output value of the network to change the final gradient. Set `sens_param` in `ops.GradOperation` to `True` and determine the scaling index. The dimension must be the same as the output dimension.
+class GradNet(nn.Cell):
+ def __init__(self, net):
+ super(GradNet, self).__init__()
+ self.net = net
+ self.params = ParameterTuple(net.trainable_params())
+ self.grad_op = ops.GradOperation(get_by_list=True)
-The scaling index `self.grad_wrt_output` may be in the following format:
+ def construct(self, x):
+ gradient_function = self.grad_op(self.net, self.params)
+ return gradient_function(x)
-```python
-self.grad_wrt_output = Tensor([[s1, s2, s3], [s4, s5, s6]])
+# Construct the derivative network
+x = Tensor([6], dtype=mstype.float32)
+output = GradNet(Net())(x)
+
+print(output)
```
-The `GradNetWrtX` structure is as follows:
+```text
+(Tensor(shape=[1], dtype=Float32, value= [ 6.00000000e+00]),)
+```
+
+## Gradient Value Scaling
+
+The output value of the network is scaled by `sens_param` parameter to change the final gradient. First the `sens_param` in `ops. GradOperation` is set to True and the zoom index is determined, whose dimension is consistent with the output dimension.
```python
-class GradNetWrtX(nn.Cell):
+class GradNet(nn.Cell):
def __init__(self, net):
- super(GradNetWrtX, self).__init__()
+ super(GradNet, self).__init__()
self.net = net
+ # derivation operation
self.grad_op = ops.GradOperation(sens_param=True)
- self.grad_wrt_output = Tensor([[0.1, 0.6, 0.2], [0.8, 1.3, 1.1]], dtype=mstype.float32)
+ # zoom index
+ self.grad_wrt_output = Tensor([0.1], dtype=mstype.float32)
- def construct(self, x, y):
+ def construct(self, x):
gradient_function = self.grad_op(self.net)
- return gradient_function(x, y, self.grad_wrt_output)
+ return gradient_function(x, self.grad_wrt_output)
+
+x = Tensor([6], dtype=mstype.float32)
+output = GradNet(Net())(x)
-output = GradNetWrtX(Net())(x, y)
print(output)
```
```text
-[[2.211 0.51 1.49 ]
- [5.588 2.68 4.07 ]]
+[0.6]
```
## Stop Gradient
-We can use `stop_gradient` to disable calculation of gradient for certain operators. For example:
+We can use `ops.stop_gradient` to disable calculation of gradient. For example:
```python
-import numpy as np
-import mindspore.nn as nn
-import mindspore.ops as ops
-from mindspore import Tensor
-from mindspore import ParameterTuple, Parameter
-from mindspore import dtype as mstype
from mindspore.ops import stop_gradient
class Net(nn.Cell):
def __init__(self):
super(Net, self).__init__()
- self.matmul = ops.MatMul()
+ self.w = Parameter(Tensor(np.array([6], np.float32)), name='w')
+ self.b = Parameter(Tensor(np.array([1.0], np.float32)), name='b')
- def construct(self, x, y):
- out1 = self.matmul(x, y)
- out2 = self.matmul(x, y)
- out2 = stop_gradient(out2)
- out = out1 + out2
+ def construct(self, x):
+ out = x * self.w + self.b
+ # Stop upgrading the gradient, and the putput has no contribution to the gradient circulation
+ out = stop_gradient(out)
return out
-class GradNetWrtX(nn.Cell):
+class GradNet(nn.Cell):
def __init__(self, net):
- super(GradNetWrtX, self).__init__()
+ super(GradNet, self).__init__()
self.net = net
- self.grad_op = ops.GradOperation()
+ self.params = ParameterTuple(net.trainable_params())
+ self.grad_op = ops.GradOperation(get_by_list=True)
- def construct(self, x, y):
- gradient_function = self.grad_op(self.net)
- return gradient_function(x, y)
+ def construct(self, x):
+ gradient_function = self.grad_op(self.net, self.params)
+ return gradient_function(x)
-x = Tensor([[0.8, 0.6, 0.2], [1.8, 1.3, 1.1]], dtype=mstype.float32)
-y = Tensor([[0.11, 3.3, 1.1], [1.1, 0.2, 1.4], [1.1, 2.2, 0.3]], dtype=mstype.float32)
-output = GradNetWrtX(Net())(x, y)
-print(output)
-```
+x = Tensor([6], dtype=mstype.float32)
+output = GradNet(Net())(x)
-```text
- [[4.5, 2.7, 3.6],
- [4.5, 2.7, 3.6]]
+print(f"wgrad: {output[0]}\nbgrad: {output[1]}")
```
-Here, we set `stop_gradient` to `out2`, so this operator does not have any contribution to gradient. If we delete `out2 = stop_gradient(out2)`, the result is:
-
```text
- [[9.0, 5.4, 7.2],
- [9.0, 5.4, 7.2]]
+wgrad: [0.]
+bgrad: [0.]
```
-
-After we do not set `stop_gradient` to `out2`, it will make the same contribution to gradient as `out1`. So we can see that each result has doubled.
\ No newline at end of file
diff --git a/tutorials/source_en/beginner/dataset.md b/tutorials/source_en/beginner/dataset.md
index 35de7d86320b8ba6418225c3072942f2b601dc73..cf2d6e88ea186774051a440cf56abd17fc9f3fab 100644
--- a/tutorials/source_en/beginner/dataset.md
+++ b/tutorials/source_en/beginner/dataset.md
@@ -1,252 +1,152 @@
-# Loading and Processing Data
+# Processing Data
-`Ascend` `GPU` `CPU` `Beginner` `Data Preparation`
+
-
+Data is the foundation of deep learning, and high-quality data input plays an active role in the entire deep neural network. [mindspore.dataset](https://www.mindspore.cn/docs/api/zh-CN/master/api_python/mindspore.dataset.html) provides loading interfaces for some commonly used datasets and datasets with the standard format, enabling users to quickly perform data processing operations. For the figure dataset, the users can load and process data by using `mindvision.dataset`.
-MindSpore provides APIs for loading common datasets and datasets in standard formats. You can directly use the corresponding dataset loading class in mindspore.dataset to load data. The dataset class provides common data processing APIs for users to quickly process data.
+> `mindvision.dataset` is a dataset interface developed on the basis of `mindspore.dataset`. In addition to providing dataset loading, `mindvision.dataset` further provides dataset download, data processing, and data enhancements. This chapter first describes how to load and process a CIFAR-10 dataset using the `mindvision.dataset.Cifar10` interface, and then describes how to use `mindspore.dataset.GeneratorDataset` to implement custom dataset loading.
-## Data Preparation
+## Data Process
-Execute the following command to download and decompress the CIFAR-10 and MNIST dataset to the specified location.
+In the network training and inference process, the raw data is generally stored in a disk or database, and it needs to be read into the memory through the data loading step, is converted into a framework-common tensor (Tensor) format, and then mapped to a more easy-to-learn feature space through the data processing and enhancement steps, while the number of samples and generalization are increased , and is finally input the network for calculation.
-```python
-import os
-import requests
-import tarfile
-import zipfile
-
-def download_dataset(url, target_path):
- """download and decompress dataset"""
- if not os.path.exists(target_path):
- os.makedirs(target_path)
- download_file = url.split("/")[-1]
- if not os.path.exists(download_file):
- res = requests.get(url, stream=True, verify=False)
- if download_file.split(".")[-1] not in ["tgz","zip","tar","gz"]:
- download_file = os.path.join(target_path, download_file)
- with open(download_file, "wb") as f:
- for chunk in res.iter_content(chunk_size=512):
- if chunk:
- f.write(chunk)
- if download_file.endswith("zip"):
- z = zipfile.ZipFile(download_file, "r")
- z.extractall(path=target_path)
- z.close()
- if download_file.endswith(".tar.gz") or download_file.endswith(".tar") or download_file.endswith(".tgz"):
- t = tarfile.open(download_file)
- names = t.getnames()
- for name in names:
- t.extract(name, target_path)
- t.close()
-
-download_dataset("https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/cifar-10-binary.tar.gz", "./datasets")
-download_dataset("https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/MNIST_Data.zip", "./datasets")
-```
-
-The directory structure of the CIFAR-10 dataset file is as follows:
+The whole process is shown in the following figure:
-```text
-./datasets/cifar-10-batches-bin
-├── batches.meta.txt
-├── data_batch_1.bin
-├── data_batch_2.bin
-├── data_batch_3.bin
-├── data_batch_4.bin
-├── data_batch_5.bin
-├── readme.html
-└── test_batch.bin
-```
+
-Refer to [Quick Start](https://www.mindspore.cn/tutorials/en/master/quick_start.html#downloading-the-dataset) for the directory structure of MINIST dataset files.
+### Dataset
-## Loading the Dataset
+A dataset is a collection of samples, and a row of the dataset is a sample that contains one or more features, and possibly contains a label. The dataset needs to meet certain specification requirements to make it easier to evaluate the effectiveness of the model.
-In the following example, the CIFAR-10 dataset is loaded through the `Cifar10Dataset` API, and the first five samples are obtained using the sequential sampler.
+Dataset supports multiple format datasets, including MindRecord, MindSpore's self-developed data format, a commonly used public image dataset and a text datasets, a user-defined dataset, etc.
-```python
-import mindspore.dataset as ds
-
-DATA_DIR = "./datasets/cifar-10-batches-bin"
-sampler = ds.SequentialSampler(num_samples=5)
-dataset = ds.Cifar10Dataset(DATA_DIR, sampler=sampler)
-```
-
-## Iterating Dataset
+### Data Loading
-You can use `create_dict_iterator` to create a data iterator to iteratively access data. The following shows the image shapes and labels.
-
-```python
-for data in dataset.create_dict_iterator():
- print("Image shape: {}".format(data['image'].shape), ", Label: {}".format(data['label']))
-```
-
-```text
- Image shape: (32, 32, 3) , Label: 6
- Image shape: (32, 32, 3) , Label: 9
- Image shape: (32, 32, 3) , Label: 9
- Image shape: (32, 32, 3) , Label: 4
- Image shape: (32, 32, 3) , Label: 1
-```
-
-## Customizing Datasets
-
-For datasets that cannot be directly loaded by MindSpore, you can build a custom dataset class and use the `GeneratorDataset` API to customize data loading.
-
-```python
-import numpy as np
+Dataset loading allows the model to continuously obtain data for training during training. Dataset provides corresponding classes for multiple commonly used datasets to load datasets, and for data files in different storage formats, Dataset also has corresponding classes for data loading.
-np.random.seed(58)
+Dataset provides multiple uses of the sampler (Sampler), and the sampler is responsible for generating the read index sequence. The Dataset is responsible for reading the corresponding data according to the index, helping users to sample the dataset in different forms to meet the training needs and solve problems, such as the data set is too large or the distribution of the sample class is uneven. It is noted that the sampler is responsible for performing filtering and reordering operations on the sample, and will not perform the Batch operation.
-class DatasetGenerator:
- def __init__(self):
- self.data = np.random.sample((5, 2))
- self.label = np.random.sample((5, 1))
+### Data Processing
- def __getitem__(self, index):
- return self.data[index], self.label[index]
+After the Dataset loads the data into memory, the data is organized in tensor form. Tensor is also a basic data structure in the data augmentation operation.
- def __len__(self):
- return len(self.data)
-```
+## Loading the Dataset
-You need to customize the following class functions:
+In the following example, the CIFAR-10 dataset is loaded through the ` mindvision.dataset.Cifar10 ` interface. The CIFAR-10 dataset has a total of 60,000 32*32 color images, which are divided into 10 categories, each with 6,000 maps. The dataset has a total of 50,000 training pictures and 10,000 test pictures. The `Cifar10` interface provides the ability to download and load CIFAR-10 datasets.
-- **\_\_init\_\_**
+
- When a dataset object is instantiated, the `__init__` function is called. You can perform operations such as data initialization.
+- `path`：The location of the dataset root.
- ```python
- def __init__(self):
- self.data = np.random.sample((5, 2))
- self.label = np.random.sample((5, 1))
- ```
+- `split`：Train, test or inference the dataset, optionally`train`，`test` or `infer`, by default as `train`.
-- **\_\_getitem\_\_**
+- `download`：whether to download the dataset, when setting `True`, if the dataset has no dataset that can be downloaded and unzipped, `True` is set by default.
- Define the `__getitem__` function of the dataset class to support random access and obtain and return data in the dataset based on the specified `index` value.
+ ```python
+ from mindvision.dataset import Cifar10
- The return value of the `__getitem__` function needs to be a tuple of numpy arrays. When returning a single numpy array, it can be written as `return (np_array_1,)`.
+ # dataset root
+ data_dir = "./datasets"
- ```python
- def __getitem__(self, index):
- return self.data[index], self.label[index]
- ```
+ # download, unzip and load the CIFAR-10 trainning dataset
+ dataset = Cifar10(path=data_dir, split='train', download=True)
+ dataset = dataset.run()
+ ```
-- **\_\_len\_\_**
+The directory structure of the dataset files CIFAR-10 is as follows:
- Define the `__len__` function of the dataset class and return the number of samples in the dataset.
+```text
+datasets/
+├── cifar-10-batches-py
+│ ├── batches.meta
+│ ├── data_batch_1
+│ ├── data_batch_2
+│ ├── data_batch_3
+│ ├── data_batch_4
+│ ├── data_batch_5
+│ ├── readme.html
+│ └── test_batch
+└── cifar-10-python.tar.gz
+```
- ```python
- def __len__(self):
- return len(self.data)
- ```
+## Iterating Dataset
-After the dataset class is defined, the `GeneratorDataset` API can be used to load and access dataset samples in the user-defined mode.
+You can use `create_dict_iterator` to create a data iterator to iteratively access data. The accessed data is `Tensor` by default. If `output_numpy=True` is set, the type of accessed data is `Numpy` . The following shows the image shapes and labels.
```python
-dataset_generator = DatasetGenerator()
-dataset = ds.GeneratorDataset(dataset_generator, ["data", "label"], shuffle=False)
+data = next(dataset.create_dict_iterator())
+print(f"Data type:{type(data['image'])}\nImage shape: {data['image'].shape}, Label: {data['label']}")
-for data in dataset.create_dict_iterator():
- print('{}'.format(data["data"]), '{}'.format(data["label"]))
+data = next(dataset.create_dict_iterator(output_numpy=True))
+print(f"Data type:{type(data['image'])}\nImage shape: {data['image'].shape}, Label: {data['label']}")
```
```text
- [0.36510558 0.45120592] [0.78888122]
- [0.49606035 0.07562207] [0.38068183]
- [0.57176158 0.28963401] [0.16271622]
- [0.30880446 0.37487617] [0.54738768]
- [0.81585667 0.96883469] [0.77994068]
+Data type:
+Image shape: (32, 3, 224, 224), Label: [6 8 5 5 3 2 6 6 1 5 1 7 1 8 6 1 6 1 0 6 7 3 2 2 8 2 4 5 7 3 3 9]
+Data type:
+Image shape: (32, 3, 224, 224), Label: [3 8 7 3 9 8 3 0 1 0 5 2 1 5 7 9 3 3 1 1 2 0 1 0 9 1 8 2 3 2 6 7]
```
## Data Processing and Augmentation
-### Processing Data
+### Data Processing
-The dataset APIs provided by MindSpore support common data processing methods. You only need to call the corresponding function APIs to quickly process data.
+`mindvision.dataset.Cifar10` interface provides data processing. The data can be processed by simply setting the appropriate attributes.
-In the following example, the datasets are shuffled, and then two samples form a batch.
+- `shuffle`：Whether to disrupt the order of the datasets. When `True` is set, the order of the datasets is disturbed by default to `False`.
+- `batch_size`：the number of data contained in each group. `batch_size=2` means two data, and the default value of `batch_size` is 32.
+- `repeat_num`：the number of the repeated datasets. `repeat_num=1` is a dataset, and the default value of `repeat_num` is 1.
+
+The following sample implements the function to randomly scramble the dataset and form two samples into one batch.
```python
-ds.config.set_seed(58)
+dataset = Cifar10(data_dir, batch_size=2, shuffle=True, repeat_num=1)
-# Shuffle the data sequence.
-dataset = dataset.shuffle(buffer_size=10)
-# Perform batch operations on datasets.
-dataset = dataset.batch(batch_size=2)
+data = dataset.run()
+data = next(data.create_dict_iterator())
-for data in dataset.create_dict_iterator():
- print("data: {}".format(data["data"]))
- print("label: {}".format(data["label"]))
+print(f"Image shape: {data['image'].shape}, Label: {data['label']}")
```
```text
- data: [[0.36510558 0.45120592]
- [0.57176158 0.28963401]]
- label: [[0.78888122]
- [0.16271622]]
- data: [[0.30880446 0.37487617]
- [0.49606035 0.07562207]]
- label: [[0.54738768]
- [0.38068183]]
- data: [[0.81585667 0.96883469]]
- label: [[0.77994068]]
+Image shape: (2, 3, 224, 224) , Label: [5 0]
```
-Where,
-
-`buffer_size`: size of the buffer for shuffle operations in the dataset.
-
-`batch_size`: number of data records in each group. Currently, each group contains 2 data records.
-
### Data Augmentation
-If the data volume is too small or the sample scenario is simple, the model training effect is affected. You can perform the data augmentation operation to expand the sample diversity and improve the generalization capability of the model.
+Problems such as too small amount of data or a single sample scene will affect the training effect of the model, and the users can expand the diversity of samples through the data augmentation operation to improve the generalization ability of the model. `mindvision.dataset.Cifar10` interface uses the default data augmentation operation, and the users perform the data augmentation operation by setting attribute `transform` and `target_transform`.
-The following example uses the operators in the `mindspore.dataset.vision.c_transforms` module to perform data argumentation on the MNIST dataset.
+- `transform`: augment the figure data of the dataset.
+- `target_transform`: processing the label data of the dataset.
-Import the `c_transforms` module and load the MNIST dataset.
+This chapter describes that data augmentation is performed on the CIFAR-10 dataset by using operators in the `mindspore.dataset.vision .c_transforms` module.
```python
+import numpy as np
import matplotlib.pyplot as plt
-from mindspore.dataset.vision import Inter
-import mindspore.dataset.vision.c_transforms as c_vision
+import mindspore.dataset.vision.c_transforms as transforms
-DATA_DIR = './datasets/MNIST_Data/train'
+# Image augmentation
+trans = [
+ transforms.RandomHorizontalFlip(prob=0.5), # Flip the image randomly and horizontally
+ transforms.HWC2CHW(), # (h, w, c) swithes to (c, h, w)
+]
-mnist_dataset = ds.MnistDataset(DATA_DIR, num_samples=6, shuffle=False)
+dataset = Cifar10(data_dir, transform=trans)
+data_trans = dataset.run()
+data_trans = next(data_trans.create_dict_iterator(output_numpy=True))
+image_trans = data_trans["image"][0]
-# View the original image data.
-mnist_it = mnist_dataset.create_dict_iterator()
-data = next(mnist_it)
-plt.imshow(data['image'].asnumpy().squeeze(), cmap=plt.cm.gray)
-plt.title(data['label'].asnumpy(), fontsize=20)
-plt.show()
-```
-
-
-
-Define the data augmentation operator, perform the `Resize` and `RandomCrop` operations on the dataset, and insert the dataset into the data processing pipeline through `map` mapping.
-
-```python
-resize_op = c_vision.Resize(size=(200,200), interpolation=Inter.LINEAR)
-crop_op = c_vision.RandomCrop(150)
-transforms_list = [resize_op, crop_op]
-mnist_dataset = mnist_dataset.map(operations=transforms_list, input_columns=["image"])
-```
-
-View the data augmentation effect.
+# Draw an image enhancement diagram
+image_trans = np.transpose(image_trans, (1, 2, 0))
+index = data_trans['label'][0]
-```python
-mnist_dataset = mnist_dataset.create_dict_iterator()
-data = next(mnist_dataset)
-plt.imshow(data['image'].asnumpy().squeeze(), cmap=plt.cm.gray)
-plt.title(data['label'].asnumpy(), fontsize=20)
+plt.imshow(image_trans)
+plt.title(f"Transform image:{dataset.index2label[index]}")
+plt.axis("off")
plt.show()
```
-
-
-For more information, see [Data augmentation](https://www.mindspore.cn/docs/programming_guide/en/master/augmentation.html).
+
diff --git a/tutorials/source_en/beginner/introduction.ipynb b/tutorials/source_en/beginner/introduction.ipynb
index db36c0734a4613ae36ab0a2fa6d7d1f0b46cd7ff..7001bd80265c1744b6f524ce0c5500e8af3870a9 100644
--- a/tutorials/source_en/beginner/introduction.ipynb
+++ b/tutorials/source_en/beginner/introduction.ipynb
@@ -3,15 +3,17 @@
{
"cell_type": "markdown",
"source": [
- "# Overview\n",
+ "# Basic Introduction\n",
"\n",
- "`Ascend` `GPU` `CPU` `Device` `Beginner`\n",
+ "[](https://gitee.com/mindspore/docs/blob/tutorials-develop/tutorials/source_en/beginner/introduction.ipynb)\n",
"\n",
- "The following describes the Huawei AI full-stack solution and introduces the position of MindSpore in the solution. Developers who are interested in MindSpore can visit the [MindSpore community](https://gitee.com/mindspore/mindspore) and click [Watch, Star, and Fork](https://gitee.com/mindspore/mindspore).\n",
+ "This chapter describes the MindSpore and the Huawei AI full-stack solution and introduces the position of MindSpore in the solution. Developers who are interested in MindSpore can visit the [MindSpore community](https://gitee.com/mindspore/mindspore) and click [Watch, Star, and Fork](https://gitee.com/mindspore/mindspore).\n",
"\n",
"## MindSpore Introduction\n",
"\n",
- "MindSpore is a deep learning framework in all scenarios, aiming to achieve easy development, efficient execution, and all-scenario coverage. Easy development features friendly APIs and easy debugging. Efficient execution is reflected in computing, data preprocessing, and distributed training. All-scenario coverage means that the framework supports cloud, edge, and device scenarios.\n",
+ "MindSpore is a deep learning framework in all scenarios, aiming to achieve easy development, efficient execution, and all-scenario coverage.\n",
+ "\n",
+ "Easy development features friendly APIs and easy debugging. Efficient execution is reflected in computing, data preprocessing, and distributed training. All-scenario coverage means that the framework supports cloud, edge, and device scenarios.\n",
"\n",
"The following figure shows the overall MindSpore architecture:\n",
"\n",
@@ -20,12 +22,30 @@
"- **ModelZoo**: ModelZoo provides available deep learning algorithm networks, and more developers are welcome to contribute new networks ([ModelZoo](https://gitee.com/mindspore/models)).\n",
"- **MindSpore Extend**: The expansion package of MindSpore expands the support of new fields, such as GNN/deep probabilistic programming/reinforcement learning, etc. We look forward to more developers to contribute and build together.\n",
"- **MindScience**：MindScience is a scientific computing kits for various industries based on the converged MindSpore framefork. It contains the industry-leading datasets, basic network structures, high-precision pre-trained models, and pre-and post-processing tools that accelerate application development of the scientific computing ([More Information](https://mindspore.cn/mindscience/docs/en/master/index.html)).\n",
- "- **MindExpression**: Python-based frontend expression and programming interfaces. In the future, more frontends based on C/C++ will be provided. Cangjie, Huawei's self-developed programming language frontend, is now in the pre-research phase. In addition, Huawei is working on interconnection with third-party frontends to introduce more third-party ecosystems.\n",
+ "- **MindExpression**: Python-based frontend expression and programming interfaces. In the future, it is planned to provide third-party front-end docking work such as C/C++ and Cangjie, Huawei's self-developed programming language frontend (which is still in the pre-research stage) to introduce more third-party ecosystems.\n",
+ "- **MindData**: Provide functions, such as efficient data processing, common data sets loading and other functions and programming interfaces, support users to flexibly define processing registration and pipeline parallel optimization.\n",
"- **MindCompiler**: The core compiler of the layer, which implements three major functions based on the unified device-cloud MindIR, including hardware-independent optimization (type derivation, automatic differentiation, and expression simplification), hardware-related optimization (automatic parallelism, memory optimization, graph kernel fusion, and pipeline execution) and optimization related to deployment and inference (quantification and pruning).\n",
- "- **MindRT**: MindSpore runtime system, which covers the cloud-side host-side runtime system, the device-side and the lightweight runtime system of the smaller IoT.\n",
- "- **MindInsight**: Provides MindSpore's visual debugging and tuning tools, and supports users to debug and tune the training network ([More Information](https://mindspore.cn/mindinsight/docs/en/master/index.html)).\n",
+ "- **MindRT**: MindSpore's runtime system, which covers the cloud-side host-side runtime system, the device-side and the lightweight runtime system of the smaller IoT.\n",
+ "- **MindInsight**: MindSpore's visual debugging and tuning tools, which can visually view the training process, optimize model performance, debug precision issues, and interpret inference results ([More Information](https://mindspore.cn/mindinsight/docs/en/master/index.html)).\n",
"- **MindArmour**: For enterprise-level applications, security and privacy protection related enhancements, such as anti-robustness, model security testing, differential privacy training, privacy leakage risk assessment, data drift detection, etc. technology ([More Information](https://mindspore.cn/mindarmour/docs/en/master/index.html)).\n",
"\n",
+ "### Implementation Process\n",
+ "\n",
+ "After knowing the overall MindSpore architecture, we can see the overall coordination of each module. The details are shown in the figure:\n",
+ "\n",
+ "\n",
+ "As the all-scenario structure, different series of hardware supported by device (mobile phone and IoT device), edge (station and routing device) and cloud (server) scenarios, including Ascend series products, NVIDIA series products, Arm series Qualcomm Snapdragon, Huawei Kirin chips and other series of products.\n",
+ "\n",
+ "The blue box on the left is the MindSpore main framework, which mainly provides the basic API functions related to the training and verification of neural networks, and also provides automatic differentiation and automatic parallelism by default.\n",
+ "\n",
+ "Below the blue box is the MindSpore Data module, which can be used for data preprocessing, including data sampling, data iteration, data format conversion and other different data operations. In the process of training, many debugging and tuning problems will be encountered, so the MindSpore Insight module visualizes the data related to debugging and tuning such as loss curves, operator execution, and weight parameter variables, so as to facilitate users to debug and tune during training.\n",
+ "\n",
+ "The simplest way of AI security is from the perspective of attack and defense, for example, the attacker introduces malicious data during the training stage, affecting the inference ability of the AI model, so MindSpore launched the MindSpore Armour module to provide AN security mechanisms for MindSpore.\n",
+ "\n",
+ "The content above the blue box is closer to the users related to algorithm development, including storing a large number of AI algorithm model libraries ModelZoo, providing a development tool suite for different fields MindSpore DevKit, and a high-level expansion library MindSpore Extend, which is worth mentioning the scientific computing suite MindSciences in MindSpore Extend. For the first time, MindSpore explores the combination of scientific computing and deep learning, the combination of numerical computing and deep learning, and the support of electromagnetic simulation, drug molecule simulation and so on through deep learning.\n",
+ "\n",
+ "After the neural network model is trained, you can export the model or load the model that has been trained in MindSpore Hub. Then MindIR provides a unified IR format for the device and the cloud. The logical structure of the network and the attributes of the operators are defined through the unified IR, and the model file in the MindIR format is decoupled with the hardware platform to achieve multiple deployments at one time. Therefore, as shown in the figure, the model is exported to different modules through IR to perform inference.\n",
+ "\n",
"### Design Philosophy\n",
"\n",
"- Support full-scenario collaboration\n",
@@ -34,15 +54,15 @@
"\n",
"- Provide Python programming normal form to simplify AI programming\n",
"\n",
- " MindSpore provides a Python programming normal form. Users can build complex neural network models using Python's native control logic, making AI programming easy.\n",
+ " MindSpore provides a Python programming normal form. Users can build complex neural network models by using Python native control logic, making AI programming easy.\n",
"\n",
"- Provide a unified coding method for PyNative graphs and static graphs\n",
"\n",
- " At present, there are two execution modes of mainstream deep learning frameworks, they are Graph mode and PyNative mode. Graph mode has high training performance but is difficult to debug. Although the Pynative mode is easier to debug than the static graph mode, it is difficult to execute efficiently. MindSpore provides a unified coding method for PyNative graphs and static graphs, which greatly increases their compatibility. Users do not need to develop multiple sets of codes, and can switch the mode only by changing one line of code. For example, users can set `context.set_context(mode=context.PYNATIVE_MODE)` to switch to the PyNative mode and set `context.set_context(mode=context.GRAPH_MODE)` to switch to the Graph mode, users can have an easier development, debugging and performance experience.\n",
+ " At present, there are two execution modes of mainstream deep learning frameworks, Graph mode and PyNative mode. The Graph mode has high training performance but is difficult to debug. Although the Pynative mode is easier to debug than the static graph mode, it is difficult to execute efficiently. MindSpore provides a unified coding method for PyNative graphs and Graph graphs, which greatly increases their compatibility. Users do not need to develop multiple sets of codes, and can switch the mode only by changing one line of code. For example, users can set `context.set_context(mode=context.PYNATIVE_MODE)` to switch to the PyNative mode and set `context.set_context(mode=context.GRAPH_MODE)` to switch to the Graph mode, and the users can have an easier development, debugging and performance experience.\n",
"\n",
"- Use functional differentiable programming architecture, allowing users to focus on the mathematical native expression of model algorithms\n",
"\n",
- " Neural network models are usually trained based on gradient descent algorithms, but the manual derivation process is complicated and the results are prone to errors. The Automatic Differentiation mechanism based on Source Code Transformation of MindSpore adopts a functional differentiable programming architecture, and provides a Python programming interface at the interface layer, including the expression of control flow. Users can focus on the mathematically native expression of the model algorithm without manual derivation.\n",
+ " Neural network models are usually trained based on gradient descent algorithms, but the manual derivation process is complicated and the results are prone to errors. The automatic differentiation mechanism of MindSpore adopts a functional differentiable programming architecture based on source code transformation, and provides a Python programming interface at the interface layer, including the expression of the control flow. Users can focus on the mathematically native expression of the model algorithm without manual derivation.\n",
"\n",
"- Unify the coding method of single device and distributed training\n",
"\n",
@@ -50,7 +70,7 @@
"\n",
"### API Level Structure\n",
"\n",
- "To support network building, entire graph execution, subgraph execution, and single-operator execution, MindSpore provides users with three levels of APIs. In ascending order, these are Low-Level Python API, Medium-Level Python API, and High-Level Python API.\n",
+ "To support network building, entire graph execution, subgraph execution, and single-operator execution, MindSpore provides users with three levels of APIs. In an ascending order, these are Low-Level Python API, Medium-Level Python API, and High-Level Python API.\n",
"\n",
"\n",
"\n",
@@ -64,7 +84,7 @@
"\n",
"- Low-Level Python API\n",
"\n",
- " Low-level APIs are at the third layer, including tensor definition, basic operators, and automatic differential modules, enabling users to easily define tensors and perform derivative computation. For example, users can customize tensors by using the Tensor API, and use the GradOperation operator in the ops.composite module to calculate the derivative of the function at a specified position.\n",
+ " Low-level APIs are at the third layer, mainly including tensor definition, basic operators, and automatic differential modules, enabling users to easily define tensors and perform derivative computation. For example, users can customize tensors by using the Tensor API, and use the GradOperation operator in the ops.composite module to calculate the derivative of the function at a specified position.\n",
"\n",
"## Introduction to Huawei Ascend AI Full-Stack Solution\n",
"\n",
@@ -72,6 +92,11 @@
"\n",
"Huawei Atlas AI computing solution is based on Ascend series AI processors and uses various product forms such as modules, cards, edge stations, servers, and clusters to build an all-scenario AI infrastructure solution oriented to device, edge, and cloud. It covers data center and intelligent edge solutions, as well as the entire inference and training processes in the deep learning field.\n",
"\n",
+ "Ascend AI Full-Stack is shown as the following figure:\n",
+ "\n",
+ "\n",
+ "Here's a quick look at what each module does:\n",
+ "\n",
"- **Atlas series**: provides AI training, inference cards, and training servers ([learn more](https://e.huawei.com/en/products/cloud-computing-dc/atlas/)).\n",
"- **CANN at heterogeneous computing architecture**: a driver layer that enables chips ([learn more](https://www.hiascend.com/en/software/cann)).\n",
"- **MindSpore**: all-scenario AI framework ([learn more](https://www.mindspore.cn/en)).\n",
@@ -81,11 +106,12 @@
"\n",
"For details, click [Huawei Ascend official website](https://e.huawei.com/en/products/servers/ascend).\n",
"\n",
- "## Joining the Community\n",
+ "## Join the Community\n",
"\n",
"Welcome every developer to the MindSpore community and contribute to this all-scenario AI framework.\n",
"\n",
"- **MindSpore official website**: provides comprehensive MindSpore information, including installation, tutorials, documents, community, resources, and news ([learn more](https://www.mindspore.cn/en)).\n",
+ "\n",
"- **MindSpore code**:\n",
"\n",
" - [MindSpore Gitee](https://gitee.com/mindspore/mindspore): Top 1 Gitee open-source project in 2020, where you can track the latest progress of MindSpore by clicking Watch, Star, and Fork, discuss issues, and commit code.\n",
diff --git a/tutorials/source_en/beginner/model.md b/tutorials/source_en/beginner/model.md
index d5a2ba96fd7c8e6200d76985cc614f231d8297e3..422d64df3849d6c6f1e49c889b819b1e0cf05e9f 100644
--- a/tutorials/source_en/beginner/model.md
+++ b/tutorials/source_en/beginner/model.md
@@ -1,45 +1,53 @@
# Building a Neural Network
-`Ascend` `GPU` `CPU` `Beginner` `Model Development`
+
-
+A neural network model consists of multiple data operation layers. `mindspore.nn` provides various basic network modules. This chapter takes the construction of a LeNet-5 network as an example, first showing the use of `mindspore.nn` to build a neural network model, and then showing the use of `mindvision.classification.models` to quickly build a LeNet-5 network model. `mindvision.classification.models` is a network model interface developed based on `mindspore.nn`, providing some classic and commonly used network models for the convenience of users.
-A neural network model consists of multiple data operation layers. `mindspore.nn` provides various basic network modules.
+## LeNet-5 Network Model
-The following uses LeNet as an example to describe how MindSpore builds a neural network model.
+[LeNet-5](https://ieeexplore.ieee.org/document/726791) is a typical convolutional neural network proposed by Professor Yann LeCun in 1998, which achieves 99.4% accuracy on the MNIST dataset and is the first classic in the field of CNN. The model structure is shown in the following figure:
-Import the required modules and APIs:
+
-```python
-import numpy as np
-import mindspore
-import mindspore.nn as nn
-from mindspore import Tensor
-```
+According to the network structure of LeNet, LeNet has a total of 7 layers except the input layers, including 3 convolutional layers, 2 sub-sampling layers, 1 fully connected layer and 1 Gaussian connection layer.
+
+## Define Model Class
-## Defining a Model Class
+In the above figure, C is used to represent the convolutional layer, S to represent the sampling layer, and F to represent the fully connected layer.
+
+The input size of the picture is fixed at 32 *32. In order to get a good convolution effect, the number is required in the center of the picture, so inputting 32* 32 is actually a result of the 28 * 28 picture after filled. In addition, unlike the input picture of the three channels of the CNN network, the input of the LeNet picture is only a normalized binary image. The output of the network is a prediction probability of ten digits 0\~9, which can be understood as the probability that the input image belongs to 0\~9 digits.
The `Cell` class of MindSpore is the base class for building all networks and the basic unit of a network. When a neural network is required, you need to inherit the `Cell` class and overwrite the `__init__` and `construct` methods.
```python
+import mindspore.nn as nn
+
class LeNet5(nn.Cell):
"""
- Lenet network structure
+ LeNet-5 Network Structure
"""
def __init__(self, num_class=10, num_channel=1):
super(LeNet5, self).__init__()
- # Define the required operation.
+ # Convolutional layer, the number of input channel is num_channel, the number of output channel is 6, the size of the convolutional nuclei is 5*5
self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid')
+ # Convolutional layer，the number of input channel is 6, the number of output channel is 16, the size of the convolutional nuclei is 5*5
self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid')
+ # Fully connected layer, the number of input is 16*5*5, the number of output is 120
self.fc1 = nn.Dense(16 * 5 * 5, 120)
+ # Fully connected layer, the number of input is 120, the number of output is84
self.fc2 = nn.Dense(120, 84)
+ # Fully connected layer, the number of input is 84，the number of class is num_class
self.fc3 = nn.Dense(84, num_class)
+ # ReLU activation function
self.relu = nn.ReLU()
+ # Pooling layer
self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)
+ # Multidimensional arrays are flattened into one-dimensional arrays
self.flatten = nn.Flatten()
def construct(self, x):
- # Use the defined operation to build a forward network.
+ # Use defined operations to build the forward network
x = self.conv1(x)
x = self.relu(x)
x = self.max_pool2d(x)
@@ -55,17 +63,44 @@ class LeNet5(nn.Cell):
return x
```
+Next, build the neural network model defined above and look at the structure of the network model.
+
+```python
+model = LeNet5()
+
+print(model)
+```
+
+```text
+LeNet5<
+ (conv1): Conv2d
+ (conv2): Conv2d
+ (fc1): Dense
+ (fc2): Dense
+ (fc3): Dense
+ (relu): ReLU<>
+ (max_pool2d): MaxPool2d
+ (flatten): Flatten<>
+ >
+```
+
## Model Layers
-The following describes the key member functions of the `Cell` class used in LeNet, and then describes how to use the `Cell` class to access model parameters through the instantiation network.
+The following describes the key member functions of the `Cell` class used in LeNet-5 network, and then describes how to use the `Cell` class to access model parameters through the instantiation network. For more content about `cell` , refer to [mindspore.nn interface](https://www.mindspore.cn/docs/api/zh-CN/master/api_python/mindspore.nn.html).
### nn.Conv2d
Add the `nn.Conv2d` layer and add a convolution function to the network to help the neural network extract features.
```python
+import numpy as np
+
+from mindspore import Tensor
+from mindspore import dtype as mstype
+
+# The number of input channels is 1, the number of output channels is 6, the convolutional nuclei is 5*5, and the parameters are initialized by using the normal operator, and the pixels are not filled
conv2d = nn.Conv2d(1, 6, 5, has_bias=False, weight_init='normal', pad_mode='valid')
-input_x = Tensor(np.ones([1, 1, 32, 32]), mindspore.float32)
+input_x = Tensor(np.ones([1, 1, 32, 32]), mstype.float32)
print(conv2d(input_x).shape)
```
@@ -139,23 +174,45 @@ print(output.shape)
## Model Parameters
-The convolutional layer and fully-connected layer in the network will have weights and offsets after being instantiated, and these weight and offset parameters are optimized in subsequent training. In `nn.Cell`, the `parameters_and_names()` method is used to access all parameters.
+The convolutional layer and fully-connected layer in the network will have weights and offsets after being instantiated, and these weight and offset parameters are optimized in subsequent training. During the training, you can use `get_parameters()` to view information such as the name, shape, data type of the network layers, and whether the back calculation is perfored.
+
+```python
+for m in model.get_parameters():
+ print(f"layer:{m.name}, shape:{m.shape}, dtype:{m.dtype}, requeires_grad:{m.requires_grad}\n")
+```
+
+```text
+layer:conv1.weight, shape:(6, 1, 5, 5), dtype:Float32, requeires_grad:True
+layer:conv2.weight, shape:(16, 6, 5, 5), dtype:Float32, requeires_grad:True
+layer:fc1.weight, shape:(120, 400), dtype:Float32, requeires_grad:True
+layer:fc1.bias, shape:(120,), dtype:Float32, requeires_grad:True
+layer:fc2.weight, shape:(84, 120), dtype:Float32, requeires_grad:True
+layer:fc2.bias, shape:(84,), dtype:Float32, requeires_grad:True
+layer:fc3.weight, shape:(10, 84), dtype:Float32, requeires_grad:True
+layer:fc3.bias, shape:(10,), dtype:Float32, requeires_grad:True
+```
+
+## Quickly Build a LeNet-5 Network Model
-In the example, we traverse each parameter and display the name and attribute of each layer in the network.
+The above describes the use of `mindspore.nn.cell` to build a LeNet-5 network model, and the network model interface has been built in `mindvision.classification.models`, and the LeNet-5 network model can be directly built using the `lenet` interface.
```python
-model = LeNet5()
-for m in model.parameters_and_names():
- print(m)
+from mindvision.classification.models import lenet
+
+# num_classes represents the categories of the classes, and pretrained represents whether to use the class to train with a trained model
+model = lenet(num_classes=10, pretrained=False)
+
+for m in model.get_parameters():
+ print(f"layer:{m.name}, shape:{m.shape}, dtype:{m.dtype}, requeires_grad:{m.requires_grad} \n")
```
```text
-('conv1.weight', Parameter (name=conv1.weight, shape=(6, 1, 5, 5), dtype=Float32, requires_grad=True)),
-('conv2.weight', Parameter (name=conv2.weight, shape=(16, 6, 5, 5), dtype=Float32, requires_grad=True)),
-('fc1.weight', Parameter (name=fc1.weight, shape=(120, 400), dtype=Float32, requires_grad=True)),
-('fc1.bias', Parameter (name=fc1.bias, shape=(120,), dtype=Float32, requires_grad=True)),
-('fc2.weight', Parameter (name=fc2.weight, shape=(84, 120), dtype=Float32, requires_grad=True)),
-('fc2.bias', Parameter (name=fc2.bias, shape=(84,), dtype=Float32, requires_grad=True)),
-('fc3.weight', Parameter (name=fc3.weight, shape=(10, 84), dtype=Float32, requires_grad=True)),
-('fc3.bias', Parameter (name=fc3.bias, shape=(10,), dtype=Float32, requires_grad=True))
+layer:backbone.conv1.weight, shape:(6, 1, 5, 5), dtype:Float32, requeires_grad:True
+layer:backbone.conv2.weight, shape:(16, 6, 5, 5), dtype:Float32, requeires_grad:True
+layer:backbone.fc1.weight, shape:(120, 400), dtype:Float32, requeires_grad:True
+layer:backbone.fc1.bias, shape:(120,), dtype:Float32, requeires_grad:True
+layer:backbone.fc2.weight, shape:(84, 120), dtype:Float32, requeires_grad:True
+layer:backbone.fc2.bias, shape:(84,), dtype:Float32, requeires_grad:True
+layer:backbone.fc3.weight, shape:(10, 84), dtype:Float32, requeires_grad:True
+layer:backbone.fc3.bias, shape:(10,), dtype:Float32, requeires_grad:True
```