diff --git a/docs/programming_guide/source_en/initializer.md b/docs/programming_guide/source_en/initializer.md index cd934f345ffd31fe2502207806817a392642f10c..fc54829515102bff56547177dee7660e725db9db 100644 --- a/docs/programming_guide/source_en/initializer.md +++ b/docs/programming_guide/source_en/initializer.md @@ -1,5 +1,148 @@ # Initialization of Network Parameters -No English version right now, welcome to contribute. - - \ No newline at end of file + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Initialization of Network Parameters\n", + "\n", + "[![](https://gitee.com/mindspore/docs/raw/master/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/master/docs/programming_guide/source_zh_cn/initializer.ipynb) [![](https://gitee.com/mindspore/docs/raw/master/resource/_static/logo_notebook.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/master/programming_guide/mindspore_initializer.ipynb) [![](https://gitee.com/mindspore/docs/raw/master/resource/_static/logo_modelarts.png)](https://authoring-modelarts-cnnorth4.huaweicloud.com/console/lab?share-url-b64=aHR0cHM6Ly9vYnMuZHVhbHN0YWNrLmNuLW5vcnRoLTQubXlodWF3ZWljbG91ZC5jb20vbWluZHNwb3JlLXdlYnNpdGUvbm90ZWJvb2svbW9kZWxhcnRzL3Byb2dyYW1taW5nX2d1aWRlL21pbmRzcG9yZV9pbml0aWFsaXplci5pcHluYg==&imagename=MindSpore1.1.1)" + ] +@@ -13,25 +13,25 @@ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Overview\n", + "\n", + "MindSpore provides modules for weight initialization, users can call string, Initializer's subclasses or customized Tensor and so on by encapsulation operator and initializer method to initialize the network parameters. The Initializer class is the basic data structure for initialization in MindSpore. Its subclasses include several different types of data distribution (Zero, One, XavierUniform, Heuniform, Henormal, Constant, Uniform, Normal, TruncatedNormal). The following is the detailed introduction for 2 parameter initialization modes of encapsulation operator and initializer method." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using encapsulation operator to initialize parameters \n", + "Mindspore provides lots of methods for parameter initialization, and encapsulates parameter initialization functions in some operators. This section introduces the method for parameters initializating by operators with parameter initialization function. Taking `Conv2d` operator as an example, it introduces the initialization of parameters in the network by strings, `Initializer`'s subclasses and customized `Tensor`, etc. The following codes all using the `Initializer`'s subclass `Normal` as examples, `Normal` can be replaced by any of the subclasses of `Initializer`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### String \n", + "When using string to initialize the network parameters, the string content needs to be the same as the name of the `Initializer` subclass. The default parameters in the `Initializer` subclass are used for string initializating. For instance, using the string `Normal` is equivalent to using the subclass `Normal()` of `Initializer`. The code sample is as follows:" + ] + }, + { +@@ -93,8 +93,8 @@ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Initializer's subclass \n", + "Using `Initializer`'s subclass to initialize the network parameter is similar to using string to do so. The difference is that using a string for parameter initialization uses the default parameters of the `Initializer` subclass. If you want to use parameters in the `Initializer` subclass, you have to use the `Initializer` subclass to initialize the parameters. Take `Normal(0.2)` as an example, code example as follows:" + ] + }, + { +@@ -157,8 +157,8 @@ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Customized Tensor \n", + "In addition to the above two initialization methods, users can initialize the parameters by customized `Tensor` when using the data type that is not in MindSpore in network. The code sample is as follows:" + ] + }, + { +@@ -207,33 +207,33 @@ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using initializer method to initialize parameters\n", + "\n", + "In the above code sample, the method for parameter initialization in the network is given. For example, nn layer is used to encapsulate `Conv2d` operator in the network, and the parameter `weight_init` is passed into a `Conv2d` operator as the initialized data type. When initializing, the operator will call the class of `Parameter`, then call the `initializer` method encapsulated in the class `Parameter` to initialize the parameters. However, some operators do not encapsulate the parameter initialization function internally like `Conv2d`. For example, the weights of `Conv3D` operators are passed to `Conv3D` operators as parameters. In this case, it is necessary to manually define the weight initialization.", + "\n", + "When initializing the parameter, the `initializer` method can be used to initialize the parameter by calling different data types in the `Initializer` subclass to produce different types of data.\n", + "\n", + "When using initializer for parameter initialization, the parameters that support passing in are `init`, `shape`, `dtype`: \n", + "\n", + "- `init`: Supports passing in `Tensor`, `str`, `Initializer`'s subclass.\n", + "\n", + "- `shape`: Supports passing in `list`, `tuple`, `int`.\n", + "\n", + "- `dtype`: Supports passing in `mindspore.dtype`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### init's parameter is Tensor" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Code example as follows: " + ] + }, + { +@@ -268,7 +268,7 @@ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Output as follows: \n", + "\n", + "```text\n", + "[[[[[108 108 108 ... 108 108 108]\n", +@@ -293,14 +293,14 @@ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### init's parameter is str" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Code example as follows: " + ] + }, + { +@@ -329,7 +329,7 @@ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Output as follows: \n", + "\n", + "```text\n", + "[[[[[0 0 0 ... 0 0 0]\n", +@@ -354,14 +354,14 @@ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### init's parameter is the subclass of Initializer" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Code example as follows: " + ] + }, + { +@@ -413,14 +413,14 @@ + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Application in Parameter" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Code example as follows: " + ] + }, + {