From 55562a7759cba7ce50ffb92388728e48c4618e9f Mon Sep 17 00:00:00 2001 From: yeyunpeng2020 Date: Thu, 25 Mar 2021 16:49:26 +0800 Subject: [PATCH] add quick start demo --- .../source_en/quick_start/quick_start_cpp.md | 253 +++++++++++++++++- .../source_en/quick_start/quick_start_java.md | 182 ++++++++++++- 2 files changed, 430 insertions(+), 5 deletions(-) diff --git a/tutorials/lite/source_en/quick_start/quick_start_cpp.md b/tutorials/lite/source_en/quick_start/quick_start_cpp.md index e639726806..3f4acf0bf2 100644 --- a/tutorials/lite/source_en/quick_start/quick_start_cpp.md +++ b/tutorials/lite/source_en/quick_start/quick_start_cpp.md @@ -1,5 +1,252 @@ -# Experience a MindSpore Lite C++ Simple Demo +# Simplified MindSpore Lite C++ Demo - This tutorial is being translated, please stay tuned... +`Linux` `Windows` `x86` `C++` `Whole Process``Inference Application` `Data Preparation` `Beginner` + + + +- [Simplified MindSpore Lite C++ Demo](#simplified-mindspore-lite-c-demo) + - [Overview](#overview) + - [Building and Running](#building-and-running) + - [Linux x86](#linux-x86) + - [Windows](#windows) + - [CMake Integration](#cmake-integration) + - [Model Loading](#model-loading) + - [Model Build](#model-build) + - [Model Inference](#model-inference) + - [Memory Release](#memory-release) + + + + + +## Overview + +This tutorial provides a MindSpore Lite inference demo. It demonstrates the basic on-device inference process using C++ by inputting random data, executing inference, and printing the inference result. You can quickly understand how to use inference-related APIs on MindSpore Lite. In this tutorial, the randomly generated data is used as the input data to perform the inference on the MobileNetV2 model and print the output data. The code is stored in the [mindspore/lite/examples/quick_start_cpp](https://gitee.com/mindspore/mindspore/tree/master/mindspore/lite/examples/quick_start_cpp) directory. + +The MindSpore Lite inference steps are as follows: + +1. Load the model: Read the `.ms` model converted by the [model conversion tool](https://www.mindspore.cn/tutorial/lite/en/master/use/converter_tool.html) from the file system, import the model by using [mindspore::lite::Model::Import](https://www.mindspore.cn/doc/api_cpp/en/master/lite.html#import), parse the model, and create the `Model *`. +2. Create and configure context: Create and configure [context](https://www.mindspore.cn/doc/api_cpp/en/master/lite.html#context) to save some basic configuration parameters required by a session to guide graph build and execution. +3. Create a session: Create [LiteSession](https://www.mindspore.cn/doc/api_cpp/en/master/session.html#litesession) and configure the [context](https://www.mindspore.cn/doc/api_cpp/en/master/lite.html#context) obtained in the previous step to the session. +4. Build a graph: Before performing inference, call the `CompileGraph` API of [LiteSession](https://www.mindspore.cn/doc/api_cpp/en/master/session.html#litesession) to build a graph. In the graph build phase, subgraph partition and operator selection and scheduling are performed, which takes a long time. Therefore, it is recommended that with one [LiteSession](https://www.mindspore.cn/doc/api_cpp/en/master/session.html#litesession) created, one graph be built. In this case, the inference will be performed for multiple times. +5. Input data: Before the graph is executed, data needs to be filled in the `Input Tensor`. +6. Perform inference: Use `RunGraph` of the [LiteSession](https://www.mindspore.cn/doc/api_cpp/en/master/session.html#litesession) to perform model inference. +7. Obtain the output: After the graph execution is complete, you can obtain the inference result by `outputting the tensor`. +8. Release the memory: If the MindSpore Lite inference framework is not required, release the created [LiteSession](https://www.mindspore.cn/doc/api_cpp/en/master/session.html#litesession) and [Model](https://www.mindspore.cn/doc/api_cpp/en/master/lite.html#model). + +![img](../images/lite_runtime.png) + +> To view the advanced usage of MindSpore Lite, see [Using Runtime to Perform Inference (C++)](https://www.mindspore.cn/tutorial/lite/en/master/use/runtime_cpp.html)]. + +## Building and Running + +### Linux x86 + +- Environment requirements + + - System environment: Linux x86_64 (Ubuntu 18.04.02LTS is recommended.) + - Build dependency: + - [CMake](https://cmake.org/download/) >= 3.14 + - [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0 + +- Build + + Run the build script in the `mindspore/lite/examples/quick_start_cpp` directory to automatically download related files and build the demo. + + ```bash + bash build.sh + ``` + + > If the MindSpore Lite inference framework fails to be downloaded, manually download the MindSpore Lite model inference framework [mindspore-lite-{version}-inference-linux-x64.tar.gz](https://www.mindspore.cn/tutorial/lite/en/master/use/downloads.html) whose hardware platform is CPU and operating system is Ubuntu-x64, and copy the `libmindspore-lite.a` file in the decompressed lib directory to the `mindspore/lite/examples/quick_start_cpp/lib` directory. Change the include directory to the `mindspore/lite/examples/quick_start_cpp/include` directory. + > + > If the MobileNetV2 model fails to be downloaded, manually download the model file [mobilenetv2.ms](https://download.mindspore.cn/model_zoo/official/lite/mobilenetv2_imagenet/mobilenetv2.ms) and copy it to the `mindspore/lite/examples/quick_start_cpp/model` directory. + +- Inference + + After the build, go to the `mindspore/lite/examples/quick_start_cpp/build` directory and run the following command to experience MindSpore Lite inference on the MobileNetV2 model: + + ```bash + ./mindspore_quick_start_cpp ../model/mobilenetv2.ms + ``` + + After the execution, the following information is displayed, including the tensor name, tensor size, number of output tensors, and the first 50 pieces of data. + + ```shell + tensor name is:Default/head-MobileNetV2Head/Softmax-op204 tensor size is:4000 tensor elements num is:1000 + output data is:5.26823e-05 0.00049752 0.000296722 0.000377607 0.000177048 8.02107e-05 0.000212864 0.000422286 0.000273189 0.000234105 0.00099807 0.0042331 0.00204993 0.00124968 0.00294458 0.00139795 0.00111545 0.000656357 0.000809457 0.00153731 0.000621049 0.00224637 0.00127045 0.00187557 0.000420144 0.000150638 0.000266477 0.000438628 0.000187773 0.00054668 0.000212853 0.000921661 0.000127179 0.000565873 0.00100394 0.000300159 0.000282677 0.000358067 0.00215288 0.000477845 0.00107596 0.00065134 0.000722132 0.000807501 0.000631415 0.00043247 0.00125898 0.000255094 8.2606e-05 9.91917e-05 0.000794512 + ``` + +### Windows + +- Environment requirements + + - System environment: 64-bit Windows 7 or 64-bit Windows 10 + - Build dependency: + - [CMake](https://cmake.org/download/) >= 3.14 + - [MinGW GCC](https://sourceforge.net/projects/mingw-w64/files/ToolchainstargettingWin64/PersonalBuilds/mingw-builds/7.3.0/threads-posix/seh/x86_64-7.3.0-release-posix-seh-rt_v5-rev0.7z/download) = 7.3.0 + +- Build + + - Download the library: Manually download the MindSpore Lite model inference framework [mindspore-lite-{version}-inference-win-x64.zip](https://www.mindspore.cn/tutorial/lite/en/master/use/downloads.html) whose hardware platform is CPU and operating system is Windows-x64. Copy the `libmindspore-lite.a` file in the decompressed `benchmark` directory to the `mindspore/lite/examples/quick_start_cpp/lib` directory, and change the include directory to the `mindspore/lite/examples/quick_start_cpp/include` directory. + - Download the model: Manually download the model file [mobilenetv2.ms](https://download.mindspore.cn/model_zoo/official/lite/mobilenetv2_imagenet/mobilenetv2.ms) and copy it to the `mindspore/lite/examples/quick_start_cpp/model` directory. + + - Build the demo: Run the build script in the `mindspore/lite/examples/quick_start_cpp` directory to automatically download related files and build the demo. + + ```bash + call build.bat + ``` + +- Inference + + After the build, go to the `mindspore/lite/examples/quick_start_cpp/build` directory and run the following command to experience MindSpore Lite inference on the MobileNetV2 model: + + ```bash + call ./mindspore_quick_start_cpp.exe ../model/mobilenetv2.ms + ``` + + After the execution, the following information is displayed, including the tensor name, tensor size, number of output tensors, and the first 50 pieces of data. + + ```shell + tensor name is:Default/head-MobileNetV2Head/Softmax-op204 tensor size is:4000 tensor elements num is:1000 + output data is:5.26823e-05 0.00049752 0.000296722 0.000377607 0.000177048 8.02107e-05 0.000212864 0.000422286 0.000273189 0.000234105 0.00099807 0.0042331 0.00204993 0.00124968 0.00294458 0.00139795 0.00111545 0.000656357 0.000809457 0.00153731 0.000621049 0.00224637 0.00127045 0.00187557 0.000420144 0.000150638 0.000266477 0.000438628 0.000187773 0.00054668 0.000212853 0.000921661 0.000127179 0.000565873 0.00100394 0.000300159 0.000282677 0.000358067 0.00215288 0.000477845 0.00107596 0.00065134 0.000722132 0.000807501 0.000631415 0.00043247 0.00125898 0.000255094 8.2606e-05 9.91917e-05 0.000794512 + ``` + +## CMake Integration + +When CMake integrates the `libmindspore-lite.a` static library, the `-Wl,--whole-archive` option needs to be passed to the linker. In addition, the build option for stack protection `-fstack-protector-strong` is added during the build of MindSpore Lite. Therefore, the `ssp` library in MinGW needs to be linked on the Windows platform. + +```cmake +cmake_minimum_required(VERSION 3.14) +project(QuickStartCpp) + +if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 7.3.0) + message(FATAL_ERROR "GCC version ${CMAKE_CXX_COMPILER_VERSION} must not be less than 7.3.0") +endif() + +# Add the directory to include search path +include_directories(${CMAKE_CURRENT_SOURCE_DIR}) + +# Add the directory to include search path +link_directories(${CMAKE_CURRENT_SOURCE_DIR}/lib) + +file(GLOB_RECURSE QUICK_START_CXX ${CMAKE_CURRENT_SOURCE_DIR}/*.cc) +add_executable(mindspore_quick_start_cpp ${QUICK_START_CXX}) + +target_link_libraries( + mindspore_quick_start_cpp + -Wl,--whole-archive mindspore-lite -Wl,--no-whole-archive + pthread +) + +# Due to the increased compilation options for stack protection, +# it is necessary to target link ssp library when Use the static library in Windows. +if(WIN32) + target_link_libraries( + mindspore_quick_start_cpp + ssp + ) +endif() +``` + +## Model Loading + +Read the MindSpore Lite model from the file system and use the `mindspore::lite::Model::Import` function to import the model for parsing. + +```c++ +// Read model file. +size_t size = 0; +char *model_buf = ReadFile(model_path, &size); +if (model_buf == nullptr) { + std::cerr << "Read model file failed." << std::endl; + return -1; +} +// Load the .ms model. +auto model = mindspore::lite::Model::Import(model_buf, size); +delete[](model_buf); +if (model == nullptr) { + std::cerr << "Import model file failed." << std::endl; + return -1; +} +``` + +## Model Build + +Model build includes context configuration creation, session creation, and graph build. + +```c++ +mindspore::session::LiteSession *Compile(mindspore::lite::Model *model) { + // Create and init context. + auto context = std::make_shared(); + if (context == nullptr) { + std::cerr << "New context failed while." << std::endl; + return nullptr; + } + + // Create the session. + mindspore::session::LiteSession *session = mindspore::session::LiteSession::CreateSession(context.get()); + if (session == nullptr) { + std::cerr << "CreateSession failed while running." << std::endl; + return nullptr; + } + + // Build a graph. + auto ret = session->CompileGraph(model); + if (ret != mindspore::lite::RET_OK) { + delete session; + std::cerr << "Compile failed while running." << std::endl; + return nullptr; + } + + // Note: when use model->Free(), the model can not be compiled again. + if (model != nullptr) { + model->Free(); + } + return session; +} +``` + +## Model Inference + +Model inference includes data input, inference execution, and output obtaining. In this example, the input data is randomly generated, and the output result is printed after inference. + +```c++ +int Run(mindspore::session::LiteSession *session) { + auto inputs = session->GetInputs(); + auto ret = GenerateInputDataWithRandom(inputs); + if (ret != mindspore::lite::RET_OK) { + std::cerr << "Generate Random Input Data failed." << std::endl; + return ret; + } + + ret = session->RunGraph(); + if (ret != mindspore::lite::RET_OK) { + std::cerr << "Inference error " << ret << std::endl; + return ret; + } + + auto out_tensors = session->GetOutputs(); + for (auto tensor : out_tensors) { + std::cout << "tensor name is:" << tensor.first << " tensor size is:" << tensor.second->Size() + << " tensor elements num is:" << tensor.second->ElementsNum() << std::endl; + auto out_data = reinterpret_cast(tensor.second->MutableData()); + std::cout << "output data is:"; + for (int i = 0; i < tensor.second->ElementsNum() && i <= 50; i++) { + std::cout << out_data[i] << " "; + } + std::cout << std::endl; + } + return mindspore::lite::RET_OK; +} +``` + +## Memory Release + +If the MindSpore Lite inference framework is not required, release the created `LiteSession` and `Model`. + +```c++ +// Delete model buffer. +delete model; +// Delete session buffer. +delete session; +``` - diff --git a/tutorials/lite/source_en/quick_start/quick_start_java.md b/tutorials/lite/source_en/quick_start/quick_start_java.md index f352eaf68d..ba699fd961 100644 --- a/tutorials/lite/source_en/quick_start/quick_start_java.md +++ b/tutorials/lite/source_en/quick_start/quick_start_java.md @@ -1,5 +1,183 @@ -# Experience a MindSpore Lite Java Simple Demo +# Simplified MindSpore Lite Java Demo - This tutorial is being translated, please stay tuned... +`Linux` `x86` `Java` `Whole Process` `Inference Application` `Data Preparation` `Beginner` + + + +- [Simplified MindSpore Lite Java Demo](#simplified-mindspore-lite-java-demo) + - [Overview](#overview) + - [Building and Running](#building-and-running) + - [Model Loading](#model-loading) + - [Model Build](#model-build) + - [Model Inference](#model-inference) + - [Memory Release](#memory-release) + + + +## Overview + +This tutorial provides an example program for MindSpore Lite to perform inference. It demonstrates the basic process of performing inference on the device side using [MindSpore Lite Java API](https://www.mindspore.cn/doc/api_java/en/master/index.html) by random inputting data, executing inference, and printing the inference result. You can quickly understand how to use the Java APIs related to inference on MindSpore Lite. In this tutorial, the randomly generated data is used as the input data to perform the inference on the MobileNetV2 model and print the output data. The code is stored in the [mindspore/lite/examples/quick_start_java](https://gitee.com/mindspore/mindspore/tree/master/mindspore/lite/examples/quick_start_java) directory. + +The MindSpore Lite inference steps are as follows: + +1. Load the model: Read the `.ms` model converted by the model conversion tool () from the file system and import the model using the [loadModel](https://www.mindspore.cn/doc/api_java/en/master/model.html#loadmodel). +2. Create and configure context: Create a configuration context [MSConfig](https://www.mindspore.cn/doc/api_java/en/master/msconfig.html#msconfig) to save some basic configuration parameters required by a session to guide graph build and execution. including `deviceType` (device type), `threadNum` (number of threads), `cpuBindMode` (CPU binding mode), and `enable_float16` (whether to preferentially use the float16 operator). +3. Create a session: Create [LiteSession](https://www.mindspore.cn/doc/api_java/en/master/lite_session.html#litesession) and call the [init](https://www.mindspore.cn/doc/api_java/en/master/lite_session.html#init) method to configure the [MSConfig](https://www.mindspore.cn/doc/api_java/en/master/msconfig.html#msconfig) obtained in the previous step in the session. +4. Build a graph: Before building a graph, the [compileGraph](https://www.mindspore.cn/doc/api_java/en/master/lite_session.html#compilegraph) interface of [LiteSession](https://www.mindspore.cn/doc/api_java/en/master/lite_session.html#litesession) needs to be called to build the graph, including subgraph partition and operator selection and scheduling. This takes a long time. Therefore, it is recommended that with one [LiteSession](https://www.mindspore.cn/doc/api_cpp/en/master/session.html#litesession) created, one graph be built. In this case, the inference will be performed for multiple times. +5. Input data: Before the graph is executed, data needs to be filled in the `Input Tensor`. +6. Perform inference: Use the [runGraph](https://www.mindspore.cn/doc/api_java/en/master/lite_session.html#rungraph) of the [LiteSession](https://www.mindspore.cn/doc/api_java/en/master/lite_session.html#litesession) to perform model inference. + Obtain the output: After the graph execution is complete, you can obtain the inference result by `outputting the tensor`. +8. Release the memory: If the MindSpore Lite inference framework is not required, release the created [LiteSession](https://www.mindspore.cn/doc/api_java/en/master/lite_session.html#litesession) and [Model](https://www.mindspore.cn/doc/api_java/en/master/model.html#model). + +![img](../images/lite_runtime.png) + +> To view the advanced usage of MindSpore Lite, see [Using Runtime to Perform Inference (Java)](https://www.mindspore.cn/tutorial/lite/en/master/use/runtime_java.html). + +## Building and Running + +- Environment requirements + - System environment: Linux x86_64 (Ubuntu 18.04.02LTS is recommended.) + - Build dependency: + - [Git](https://git-scm.com/downloads) >= 2.28.0 + - [Maven](https://maven.apache.org/download.cgi) >= 3.3 + - [OpenJDK](https://openjdk.java.net/install/) >= 1.8 + +- Build + + Run the build script in the `mindspore/lite/examples/quick_start_java` directory to automatically download related files and build the demo. + + ```bash + bash build.sh + ``` + + > If the MindSpore Lite inference framework fails to be downloaded, manually download the MindSpore Lite model inference framework [mindspore-lite-{version}-inference-linux-x64.tar.gz](https://www.mindspore.cn/tutorial/lite/en/master/use/downloads.html) whose hardware platform is CPU and operating system is Ubuntu-x64. Decompress the package and obtain the `libmindspore-lite.so` file in the `lib\jar` directory. Copy `libmindspore-lite-jni.so` and `libmindspore-lite-java.jar` to the `mindspore/lite/examples/quick_start_java/lib` directory. + > + > If the MobileNetV2 model fails to be downloaded, manually download the model file [mobilenetv2.ms](https://download.mindspore.cn/model_zoo/official/lite/mobilenetv2_imagenet/mobilenetv2.ms) and copy it to the `mindspore/lite/examples/quick_start_java/src/main/resources/model/` directory. + +- Inference + + After the build, go to the `mindspore/lite/examples/quick_start_java/target` directory and run the following command to experience MindSpore Lite inference on the MobileNetV2 model: + + ```bash + java -Djava.library.path=../lib/ -classpath .:./quick_start_java.jar:../lib/mindspore-lite-java.jar com.mindspore.lite.demo.Main ../src/main/resources/model/mobilenetv2.ms + ``` + + After the execution, the following information is displayed, including the tensor name, tensor size, number of output tensors, and the first 50 pieces of data. + + ```shell + out tensor shape: [1,1000,] and out data: 5.4091015E-5 4.030303E-4 3.032344E-4 4.0029243E-4 2.2730739E-4 8.366581E-5 2.629827E-4 3.512394E-4 2.879536E-4 1.9557697E-4xxxxxxxxxx MindSpore Lite 1.1.0out tensor shape: [1,1000,] and out data: 5.4091015E-5 4.030303E-4 3.032344E-4 4.0029243E-4 2.2730739E-4 8.366581E-5 2.629827E-4 3.512394E-4 2.879536E-4 1.9557697E-4tensor name is:Default/Sigmoid-op204 tensor size is:2000 tensor elements num is:500output data is:3.31223e-05 1.99382e-05 3.01624e-05 0.000108345 1.19685e-05 4.25282e-06 0.00049955 0.000340809 0.00199094 0.000997094 0.00013585 1.57605e-05 4.34131e-05 1.56114e-05 0.000550819 2.9839e-05 4.70447e-06 6.91601e-06 0.000134483 2.06795e-06 4.11612e-05 2.4667e-05 7.26248e-06 2.37974e-05 0.000134513 0.00142482 0.00011707 0.000161848 0.000395011 3.01961e-05 3.95325e-05 3.12398e-06 3.57709e-05 1.36277e-06 1.01068e-05 0.000350805 5.09019e-05 0.000805241 6.60321e-05 2.13734e-05 9.88654e-05 2.1991e-06 3.24065e-05 3.9479e-05 4.45178e-05 0.00205024 0.000780899 2.0633e-05 1.89997e-05 0.00197261 0.000259391 + ``` + +## Model Loading + +Read the MindSpore Lite model from the file system and use the `model.loadModel` function to import the model for parsing. + +```java +boolean ret = model.loadModel(modelPath); +if (!ret) { + System.err.println("Load model failed, model path is " + modelPath); + return; +} +``` + +## Model Build + +Model build includes context configuration creation, session creation, and graph build. + +```java +private static boolean compile() { + MSConfig msConfig = new MSConfig(); + boolean ret = msConfig.init(DeviceType.DT_CPU, 2); + if (!ret) { + System.err.println("Init context failed"); + return false; + } + + // Create the MindSpore lite session. + session = new LiteSession(); + ret = session.init(msConfig); + msConfig.free(); + if (!ret) { + System.err.println("Create session failed"); + model.free(); + return false; + } + + // Compile graph. + ret = session.compileGraph(model); + if (!ret) { + System.err.println("Compile graph failed"); + model.free(); + return false; + } + return true; +} +``` + +## Model Inference + +Model inference includes data input, inference execution, and output obtaining. In this example, the input data is randomly generated, and the output result is printed after inference. + +```java +private static boolean run() { + MSTensor inputTensor = session.getInputsByTensorName("2031_2030_1_construct_wrapper:x"); + if (inputTensor.getDataType() != DataType.kNumberTypeFloat32) { + System.err.println("Input tensor shape do not float, the data type is " + inputTensor.getDataType()); + return false; + } + // Generator Random Data. + int elementNums = inputTensor.elementsNum(); + float[] randomData = generateArray(elementNums); + byte[] inputData = floatArrayToByteArray(randomData); + + // Set Input Data. + inputTensor.setData(inputData); + + // Run Inference. + boolean ret = session.runGraph(); + if (!ret) { + System.err.println("MindSpore Lite run failed."); + return false; + } + + // Get Output Tensor Data. + MSTensor outTensor = session.getOutputByTensorName("Default/head-MobileNetV2Head/Softmax-op204"); + + // Print out Tensor Data. + StringBuilder msgSb = new StringBuilder(); + msgSb.append("out tensor shape: ["); + int[] shape = outTensor.getShape(); + for (int dim : shape) { + msgSb.append(dim).append(","); + } + msgSb.append("]"); + if (outTensor.getDataType() != DataType.kNumberTypeFloat32) { + System.err.println("output tensor shape do not float, the data type is " + outTensor.getDataType()); + return false; + } + float[] result = outTensor.getFloatData(); + if (result == null) { + System.err.println("decodeBytes return null"); + return false; + } + msgSb.append(" and out data:"); + for (int i = 0; i < 10 && i < outTensor.elementsNum(); i++) { + msgSb.append(" ").append(result[i]); + } + System.out.println(msgSb.toString()); + return true; +} +``` + +## Memory Release + +If the MindSpore Lite inference framework is not required, release the created `LiteSession` and `Model`. + +```java +// Delete session buffer. +session.free(); +// Delete model buffer. +model.free(); +``` -- Gitee