diff --git a/docs/programming_guide/source_en/initializer.md b/docs/programming_guide/source_en/initializer.md index 9d6a454772287f5ea8c376b7427c3a286857b704..6d2b16b5f7b34331cfdc715691215bfc19b94a2c 100644 --- a/docs/programming_guide/source_en/initializer.md +++ b/docs/programming_guide/source_en/initializer.md @@ -1,36 +1,32 @@ -# Initialization of Network Parameters - -Translator: [Karlos Ma](https://gitee.com/Mavendetta985) +# Network parameter initialization -- [Initialization of Network Parameters](#initialization-of-network-parameters) +- [Network parameter initialization](#Network-parameter-initialization) - [Overview](#overview) - - [Using Encapsulation Operator to Initialize Parameters](#using-encapsulation-operator-to-initialize-parameters) - - [Character String](#character-string) - - [Initializer Subclass](#initializer-subclass) - - [The Custom of the Tensor](#the-custom-of-the-tensor) - - [Using the Initializer Method to Initialize Parameters](#using-the-initializer-method-to-initialize-parameters) - - [The Parameter of Init is Tensor](#the-parameter-of-init-is-tensor) - - [The Parameter of Init is Str](#the-parameter-of-init-is-str) - - [The Parameter of Init is the Subclass of Initializer](#the-parameter-of-init-is-the-subclass-of-initializer) - - [Application in Parameter](#application-in-parameter) + - [Initialization of parameters using wrapper operators](#Initialization-of-parameters-using-wrapper-operators) + - [Str](#Str) + - [Initializer subclass](#Initializer-subclass) + - [Customized Tensor](#Customized-Tensor) + - [Initialize the parameters using the initializer method](#Initialize-the parameters-using-the-initializer-method) + - [The init parameter is Tensor](#The-init-parameter-is-Tensor) + - [The init parameter is str](#Th-init-parameter-is-str) + - [The init parameter is a subclass of Initializer](#The-init-parameter-is-a-subclass-of-Initializer) + - [Application in Parameter](#Application-in-Parameter) - - ## Overview -MindSpore provides a weight initialization module, which allows users to initialize network parameters by encapsulated operators and initializer methods to call strings, initializer subclasses, or custom Tensors. The Initializer class is the basic data structure used for initialization in MindSpore. Its subclasses contain several different types of data distribution (Zero, One, XavierUniform, Heuniform, Henormal, Constant, Uniform, Normal, TruncatedNormal). The following two parameter initialization modes, encapsulation operator and initializer method, are introduced in detail. +MindSpore provides a weight initialization module that allows the user to initialize the network parameters by wrapping operators and initializer methods to call strings, Initializer subclasses or custom Tensor, etc. The Initializer class is the basic data structure used in MindSpore to perform initialization, and its sub Its subclasses contain several different types of data distributions (Zero, One, XavierUniform, HeUniform, HeNormal, Constant, Uniform, Normal, TruncatedNormal). The following is a detailed description of the two parameter initialization modes for the wrapper operator and the initializer method. -## Using Encapsulation Operator to Initialize Parameters +## Initialization of parameters using wrapper operators -Mindspore provides a variety of methods of initializing parameters, and encapsulates parameter initialization functions in some operators. This section will introduce the method of initialization of parameters by operators with parameter initialization function. Taking `Conv2D` operator as an example, it will introduce the initialization of parameters in the network by strings, `Initializer` subclass and custom `Tensor`, etc. `Normal`, a subclass of `Initializer`, is used in the following code examples and can be replaced with any of the subclasses of Initializer in the code examples. +MindSpore provides various ways of parameter initialization and encapsulates the function of parameter initialization in some operators. In this section, we introduce how to initialize the parameters of the operator with parameter initialization function, taking the `Conv2d` operator as an example, we introduce the initialization of the parameters in the network by string, `Initializer` subclass and custom `Tensor`, etc. In the following code examples, we take `Normal`, a subclass of `Initializer`, as an example. `Normal` can be replaced by any of the `Initializer` subclasses. -### Character String +## Str -Network parameters are initialized using a string. The contents of the string need to be consistent with the name of the `Initializer` subclass. Initialization using a string will use the default parameters in the `Initializer` subclass. For example, using the string `Normal` is equivalent to using the `Initializer` subclass `Normal()`. The code sample is as follows: +Using a string to initialize the network parameters, the content of the string needs to be consistent with the name of the `Initializer` subclass, using the string approach to initialization will use the default parameters in the `Initializer` subclass, for example, using the string `Normal` is equivalent to using the `Initializer` subclass `Normal()`, code samples are as follows: ```python import numpy as np @@ -97,41 +93,9 @@ output = net(input_data) print(output) ``` -```text -[[[[ 6.2076533e-01 8.7720710e-01 8.7720710e-01 ... 8.7720710e-01 - 8.7720710e-01 2.7743810e-01] - [ 6.5210247e-01 7.0859784e-01 7.0859784e-01 ... 7.0859784e-01 - 7.0859784e-01 -1.1080378e-01] - [ 6.5210247e-01 7.0859784e-01 7.0859784e-01 ... 7.0859784e-01 - 7.0859784e-01 -1.1080378e-01] - ... - [ 6.5210247e-01 7.0859784e-01 7.0859784e-01 ... 7.0859784e-01 - 7.0859784e-01 -1.1080378e-01] - [ 6.5210247e-01 7.0859784e-01 7.0859784e-01 ... 7.0859784e-01 - 7.0859784e-01 -1.1080378e-01] - [ 1.9323981e-01 2.4820906e-01 2.4820906e-01 ... 2.4820906e-01 - 2.4820906e-01 -2.7795550e-01]] - - ... - - [[ 7.9710668e-01 -2.7093157e-02 -2.7093157e-02 ... -2.7093157e-02 - -2.7093157e-02 -2.0062150e-01] - [ 8.7680638e-02 -7.2153252e-01 -7.2153252e-01 ... -7.2153252e-01 - -7.2153252e-01 -5.9123868e-01] - [ 8.7680638e-02 -7.2153252e-01 -7.2153252e-01 ... -7.2153252e-01 - -7.2153252e-01 -5.9123868e-01] - ... - [ 8.7680638e-02 -7.2153252e-01 -7.2153252e-01 ... -7.2153252e-01 - -7.2153252e-01 -5.9123868e-01] - [ 8.7680638e-02 -7.2153252e-01 -7.2153252e-01 ... -7.2153252e-01 - -7.2153252e-01 -5.9123868e-01] - [ 2.6627803e-01 1.3488382e-03 1.3488382e-03 ... 1.3488382e-03 - 1.3488382e-03 -4.5465171e-01]]]] -``` +## Customized Tensor -### The Custom of the Tensor - -In addition to the above two initialization methods, when the network wants to use data types that are not available in MindSpore, users can customize `Tensor` to initialize the parameters. The code sample is as follows: +In addition to the above two initialization methods, when the network wants to initialize the parameters using data types not available in MindSpore, the user can initialize the parameters by means of a custom `Tensor` with the following code sample： ```python import numpy as np @@ -146,40 +110,23 @@ output = net(input_data) print(output) ``` -```text -[[[[12. 18. 18. ... 18. 18. 12.] - [18. 27. 27. ... 27. 27. 18.] - [18. 27. 27. ... 27. 27. 18.] - ... - [18. 27. 27. ... 27. 27. 18.] - [18. 27. 27. ... 27. 27. 18.] - [12. 18. 18. ... 18. 18. 12.]] +## Initialize the parameters using the initializer method - ... +In the above code sample, a method is given on how to initialize parameters in the network, such as using the nn layer in the network to encapsulate the Conv2d operator, and the parameter weight_init is passed into the Conv2d operator as the data type to be initialized, and the operator will initialize the parameter by calling the Parameter class, which in turn calls the initializer method encapsulated in the Parameter class to complete the initialization of the parameter. However, some operators do not encapsulate the function of parameter initialization internally as Conv2d does, such as the weights of the Conv3d operator are passed into the Conv3d operator as parameters, and the initialization of the weights needs to be defined manually. - [[12. 18. 18. ... 18. 18. 12.] - [18. 27. 27. ... 27. 27. 18.] - [18. 27. 27. ... 27. 27. 18.] - ... - [18. 27. 27. ... 27. 27. 18.] - [18. 27. 27. ... 27. 27. 18.] - [12. 18. 18. ... 18. 18. 12.]]]] -``` +When initializing parameters, you can use the `initializer` method to call different data types in the `Initializer` subclass to initialize the parameters and thus generate different types of data. -## Using the Initializer Method to Initialize Parameters +When using initializer for parameter initialization, the parameters passed in are `init`, `shape`, and `dtype`. -In the above code sample, the method of Parameter initialization in the network is given. For example, NN layer is used to encapsulate a `Conv2D` operator in the network, and the Parameter `weight_init` is passed into a `Conv2D` operator as the data type to be initialized. The operator will be initialized by calling `Parameter` class. Then the `initializer` method encapsulated in the `Parameter` class is called to initialize the parameters. However, some operators do not encapsulate the function of parameter initialization internally like `Conv2D`. For example, the weights of `Conv3D` operators are passed to `Conv3D` operators as parameters. In this case, it is necessary to manually define the initialization of weights. +- `init`: supports passing in subclasses of `Tensor`, `str`, and `Initializer`. -When initializing a parameter, you can use the `Initializer` method to initialize the parameter by calling different data types in the `Initializer` subclasses, resulting in different types of data. +- `shape`: supports passing `list`, `tuple`, `int`. -When initializer is used for parameter initialization, the parameters passed in are `init`, `shape`, `dtype`: - -`init`: Supported subclasses of incoming `Tensor`, `STR`, `Subclass of Initializer`. - -`shape`: Supported subclasses of incoming `list`, `tuple`, `int`. - -`dtype`: Supported subclasses of incoming `mindspore.dtype`. +- `dtype`: supports passing `mindspore.dtype`. -### The Parameter of Init is Tensor +## The init parameter is the Tensor -The code sample is shown below: +Sample codes are as follows： ```python import numpy as np @@ -239,8 +186,9 @@ output = conv3d(input_data, weight) print(output) ``` +The output is as follows： + ```text -The output is as follows: [[[[[0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0] [0 0 0 ... 0 0 0]]