From 3703474d6aac2c8f40542766525d158e0368d62d Mon Sep 17 00:00:00 2001 From: liangyongxiong Date: Fri, 17 Apr 2020 16:18:24 +0800 Subject: [PATCH] add mindinsight supports for gpu --- install/mindspore_gpu_install.md | 71 +++++++++++++++++++ install/mindspore_gpu_install_en.md | 71 +++++++++++++++++++ .../advanced_use/visualization_tutorials.md | 13 ++-- .../advanced_use/visualization_tutorials.md | 13 ++-- 4 files changed, 150 insertions(+), 18 deletions(-) diff --git a/install/mindspore_gpu_install.md b/install/mindspore_gpu_install.md index 7bdf1afc63..e0996c1b3a 100644 --- a/install/mindspore_gpu_install.md +++ b/install/mindspore_gpu_install.md @@ -13,6 +13,7 @@ - [通过可执行文件安装](#通过可执行文件安装) - [从源码编译安装](#从源码编译安装) - [安装验证](#安装验证) +- [安装MindInsight](#安装mindinsight) - [安装MindArmour](#安装mindarmour) @@ -112,6 +113,76 @@ [ 2. 2. 2. 2.]]] ``` +# 安装MindInsight + +当您需要查看训练过程中的标量、图像、计算图以及模型超参等信息时，可以选装MindInsight。 + +## 环境要求 + +### 系统要求和软件依赖 + +| 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 | +| ---- | :--- | :--- | :--- | +| MindInsight master | - Ubuntu 16.04（及以上） x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindinsight/blob/master/requirements.txt) | **编译依赖：**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**安装依赖：**
与可执行文件安装依赖相同 | + +- 在联网状态下，安装whl包时会自动下载requirements.txt中的依赖项，其余情况需自行安装。 + +## 安装指南 + +### 通过可执行文件安装 + +1. 从[MindSpore网站下载地址](https://www.mindspore.cn/versions)下载whl包，建议先进行SHA-256完整性校验，执行如下命令安装MindInsight。 + + ```bash + pip install mindinsight-{version}-cp37-cp37m-linux_{arch}.whl + ``` + +2. 执行如下命令，如果提示`web address: http://127.0.0.1:8080`，则说明安装成功。 + + ```bash + mindinsight start + ``` + +### 从源码编译安装 + +1. 从代码仓下载源码。 + + ```bash + git clone https://gitee.com/mindspore/mindinsight.git + ``` + +2. 可选择以下任意一种安装方式： + + (1) 进入源码的根目录，执行安装命令。 + + ```bash + cd mindinsight + pip install -r requirements.txt + python setup.py install + ``` + + (2) 构建whl包进行安装。 + + 进入源码的build目录，执行MindInsight编译脚本。 + + ```bash + cd mindinsight/build + bash build.sh + ``` + + 进入源码的output目录，即可查看生成的MindInsight安装包，执行安装命令。 + + ```bash + cd mindinsight/output + pip install mindinsight-{version}-cp37-cp37m-linux_{arch}.whl + ``` + +3. 执行如下命令，如果提示`web address: http://127.0.0.1:8080`，则说明安装成功。 + + ```bash + mindinsight start + ``` + # 安装MindArmour 当您进行AI模型安全研究或想要增强AI应用模型的防护能力时，可以选装MindArmour。 diff --git a/install/mindspore_gpu_install_en.md b/install/mindspore_gpu_install_en.md index 8bee4177c5..f2e111607f 100644 --- a/install/mindspore_gpu_install_en.md +++ b/install/mindspore_gpu_install_en.md @@ -13,6 +13,7 @@ This document describes how to quickly install MindSpore on a NVIDIA GPU environ - [Installing Using Executable Files](#installing-using-executable-files) - [Installing Using the Source Code](#installing-using-the-source-code) - [Installation Verification](#installation-verification) +- [Installing MindInsight](#installing-mindinsight) - [Installing MindArmour](#installing-mindarmour) @@ -112,6 +113,76 @@ This document describes how to quickly install MindSpore on a NVIDIA GPU environ [ 2. 2. 2. 2.]]] ``` +# Installing MindInsight + +If you need to analyze information such as model scalars, graphs, and model traceback, you can install MindInsight. + +## Environment Requirements + +### System Requirements and Software Dependencies + +| Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies | +| ---- | :--- | :--- | :--- | +| MindInsight master | - Ubuntu 16.04 or later x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5
- MindSpore master
- For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindinsight/blob/master/requirements.txt). | **Compilation dependencies:**
- [Python](https://www.python.org/downloads/) 3.7.5
- [CMake](https://cmake.org/download/) >= 3.14.1
- [GCC](https://gcc.gnu.org/releases.html) 7.3.0
- [node.js](https://nodejs.org/en/download/) >= 10.19.0
- [wheel](https://pypi.org/project/wheel/) >= 0.32.0
- [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3
**Installation dependencies:**
same as the executable file installation dependencies. | + +- When the network is connected, dependency items in the requirements.txt file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items. + +## Installation Guide + +### Installing Using Executable Files + +1. Download the .whl package from the [MindSpore website](https://www.mindspore.cn/versions/en). It is recommended to perform SHA-256 integrity verification first and run the following command to install MindInsight: + + ```bash + pip install mindinsight-{version}-cp37-cp37m-linux_{arch}.whl + ``` + +2. Run the following command. If `web address: http://127.0.0.1:8080` is displayed, the installation is successful. + + ```bash + mindinsight start + ``` + +### Installing Using the Source Code + +1. Download the source code from the code repository. + + ```bash + git clone https://gitee.com/mindspore/mindinsight.git + ``` + +2. Install MindInsight by using either of the following installation methods: + + (1) Access the root directory of the source code and run the following installation command: + + ```bash + cd mindinsight + pip install -r requirements.txt + python setup.py install + ``` + + (2) Create a .whl package to install MindInsight. + + Access the build directory of the source code and run the MindInsight compilation script. + + ```bash + cd mindinsight/build + bash build.sh + ``` + + Access the output directory of the source code, where the generated MindInsight installation package is stored, and run the installation command. + + ```bash + cd mindinsight/output + pip install mindinsight-{version}-cp37-cp37m-linux_{arch}.whl + ``` + +3. Run the following command. If `web address: http://127.0.0.1:8080` is displayed, the installation is successful. + + ```bash + mindinsight start + ``` + # Installing MindArmour If you need to conduct AI model security research or enhance the security of the model in you applications, you can install MindArmour. diff --git a/tutorials/source_en/advanced_use/visualization_tutorials.md b/tutorials/source_en/advanced_use/visualization_tutorials.md index 8547409b5f..31048b04c3 100644 --- a/tutorials/source_en/advanced_use/visualization_tutorials.md +++ b/tutorials/source_en/advanced_use/visualization_tutorials.md @@ -6,8 +6,6 @@ - [Overview](#overview) - [Operation Process](#operation-process) - [Preparing the Training Script](#preparing-the-training-script) - - [Basic Script Editing](#basic-script-editing) - - [Recording the Computational Graph After Operator Fusion](#recording-the-computational-graph-after-operator-fusion) - [MindInsight Commands](#mindinsight-commands) - [Visualization Components](#visualization-components) - [Computational Graph Visualization](#computational-graph-visualization) @@ -35,9 +33,6 @@ Currently, MindSpore uses the `Callback` mechanism to save scalars, images, comp Scalar and image data is recorded by using the `Summary` operator. A computational graph is saved to the summary log file by using `SummaryRecord` after network compilation is complete. Model parameters are saved to the summary log file by using `TrainLineage` or `EvalLineage`. - -### Basic Script Editing - Step 1: Call the `Summary` operator in the `construct` function of the derived class that inherits `nn.Cell` to collect image or scalar data. For example, when a network is defined, image data is recorded in `construct` of the network. When the loss function is defined, the loss value is recorded in `construct` of the loss function. @@ -160,11 +155,11 @@ def test_summary(): summary_writer.close() ``` -### Recording the Computational Graph After Operator Fusion - -After completing the script by referring to "Basic Writing", use the `save_graphs` option of `context` to record the computational graph after operator fusion. +After completing the script, use the `save_graphs` option of `context` to record the computational graph after operator fusion. `ms_output_after_hwopt.pb` is the computational graph after operator fusion. +> Currently MindSpore supports recording computational graph after operator fusion for Ascend 910 AI processor only. + ## MindInsight Commands ### View the command help information. @@ -232,7 +227,7 @@ gunicorn ## Visualization Components ### Computational Graph Visualization -Computational graph visualization is used to display the graph structure, data flow direction, and control flow direction of a computational graph. +Computational graph visualization is used to display the graph structure, data flow direction, and control flow direction of a computational graph. It supports visualization of summary log files and pb files generated by `save_graphs` configuration in `context`. ![graph.png](./images/graph.png) diff --git a/tutorials/source_zh_cn/advanced_use/visualization_tutorials.md b/tutorials/source_zh_cn/advanced_use/visualization_tutorials.md index 70b54d2b84..621abd6bd6 100644 --- a/tutorials/source_zh_cn/advanced_use/visualization_tutorials.md +++ b/tutorials/source_zh_cn/advanced_use/visualization_tutorials.md @@ -6,8 +6,6 @@ - [概述](#概述) - [操作流程](#操作流程) - [准备训练脚本](#准备训练脚本) - - [基础写法](#基础写法) - - [记录算子融合后的计算图](#记录算子融合后的计算图) - [MindInsight相关命令](#mindinsight相关命令) - [查看命令帮助信息](#查看命令帮助信息) - [查看版本信息](#查看版本信息) @@ -40,9 +38,6 @@ 其中标量、图像是通过Summary算子实现记录数据，计算图是在网络编译完成后，通过 `SummaryRecord` 将其保存到summary日志文件中， 模型参数是通过 `TrainLineage` 或 `EvalLineage` 保存到summary日志文件中。 - -### 基础写法 - 步骤一：在继承 `nn.Cell` 的衍生类的 `construct` 函数中调用Summary算子来采集图像或标量数据。 比如，在定义网络时，在网络的 `construct` 中记录图像数据；在定义损失函数时，在损失函数的 `construct`中记录损失值。 @@ -165,11 +160,11 @@ def test_summary(): summary_writer.close() ``` -### 记录算子融合后的计算图 - -参照“基础写法”完成脚本后，可以通过`context`的`save_graphs`选项配置记录算子融合后的计算图。 +完成脚本后，可以通过`context`的`save_graphs`选项配置记录算子融合后的计算图。 其中`ms_output_after_hwopt.pb`为算子融合后的计算图。 +> 目前MindSpore仅支持在Ascend 910 AI处理器上导出算子融合后的计算图。 + ## MindInsight相关命令 ### 查看命令帮助信息 @@ -237,7 +232,7 @@ gunicorn ## 可视化组件 ### 计算图可视化 -计算图可视化用于展示计算图的图结构，数据流以及控制流的走向。 +计算图可视化用于展示计算图的图结构，数据流以及控制流的走向，支持展示summary日志文件与通过`context`的`save_graphs`参数导出的`pb`文件。 ![graph.png](./images/graph.png) -- Gitee