diff --git a/tutorials/.DS_Store b/tutorials/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..53b22e778400923cc00233fa790e0e6f3cef13df Binary files /dev/null and b/tutorials/.DS_Store differ diff --git a/tutorials/source_en/advanced_use/model_explaination.md b/tutorials/source_en/advanced_use/model_explaination.md new file mode 100644 index 0000000000000000000000000000000000000000..ab4675f963fac9b045318be38582d94affeb73ad --- /dev/null +++ b/tutorials/source_en/advanced_use/model_explaination.md @@ -0,0 +1,140 @@ +# Explain models + +`Ascend` `GPU` `Linux` `Beginner` `Intermediate` `Expert` `XAI` + + + +- [Explain Models](#explain-models) + - [Overview](#overview) + - [Operation Process](#operation-process) + - [Preparing the Script](#preparing-the-script) + - [Enabling MindInsight](#enabling-mindInsight) + - [Pages and Functions](#pages-and-functions) + - [Saliency Map Visualization](#saliency-map-visualization) + - [Explanation Method Assessment](#explanation-method-assessment) + - [Comprehensive Assessment](#comprehensive-assessment) + - [Classification Assessment](#classification-assessment) + + + +## Overview + +Currently, most deep learning models are black-box models with good performance but poor explainability. The model explanation module aims to provide users with explanation of the model decision basis, help users better understand the model, trust the model, and improve the model when an error occurs in the model. + +In some critical application scenarios, such as automatic driving, financial decision-making, etc., AI model cannot be truly applied if it is not interpretable for legal and policy supervision reasons. Therefore, the interpretability of the model is becoming more and more important. As a consequence, model explaination is an important part of improving mindspore's applicability and humanization. + +To be specific, in the task of image classification, there is one explaination method that highlights the most critical area that affects the classification decision of the model. We call it "saliency map". If people see that the part of the model that is concerned is just the key feature of the classification label, then the features learned by the model are correct, and we can trust the effect and decision of the model. If the model focuses on irrelevant parts, even if the prediction label is correct, it does not mean that the model is reliable, we still need to optimize and improve the model. This may be due to the association of some irrelevant elements in the training data. Model developers can consider targeted data enhancement to correct the bias learned by the model. + +After a variety of interpretation methods are available, we also provide a set of measurement methods to evaluate the effect of interpretation methods from various dimensions. It helps users compare and select the interpretation methods that are most suitable for a particular scene. + +## Operation Process + +### Preparing the Script + +Currently, MindSpore provides the explanation methods and explanation measurement Python API. You can use the provided explanation methods by ```mindspore.explainer.explanation``` and the provided explanation meaturement by ```mindspore.explainer.benchmark```. You need to prepare the black-box model and data to be explained, instantiate explanation methods or explanation measurement according to your need and call the explanation API in your script to collect the explanation result and explanation measurement result. + +MindSpore also provides ```mindspore.explainer.ExplainRunner``` to run all explanation methods and explanation measurement automatically. You just need to put the instantiated object into ```run``` of ```ExplainRunner``` and then all explanation methods and explanation metric are executed and explanation logs containing explanation results and explanation measurement results are automatically generated. + +The following uses ResNet-50 and 20 types of multi-tag data as an example. Add the initialization and calling code of the explanation method on the basis of the original script. The explanation method GradCAM is used to explain the model and the measurement methods are used to evaluate the explanation method. The sample code is as follows: + +``` +import mindspore.nn as nn +from mindspore.train.serialization import load_checkpoint, load_param_into_net + +from mindspore.explainer.explaination.explaination import GradCAM, GuidedBackprop +from mindspore.explainer.benchmark import Faithfulness, Localization +from mindspore.explainer import ExplainRunner + +num_classes = 20 +# please refer to model_zoo for the model architecture of resnet50 +model = resnet50(num_classes) +param_dict = load_checkpoint("resnet50.ckpt") +load_param_into_net(model, param_dict) + + +# combine the model architecture with its final activation layer, eg.Sigmoid() for multi-label models or Softmax() for single-label models +model = nn.SequentialCell([model, nn.Sigmoid()]) +model.set_grad(False) +model.set_train(False) + +# initialize explainers with the loaded black-box model +gradcam = GradCAM(model, layer='0.layer4') +guidedbackprop = GuidedBackprop(model) + + +# initialize benchmarkers to evaluate the chosen explainers +faithfulness = Faithfulness(num_labels=num_classes, metric='InsertionAUC') +localization = Localization(num_labels=num_classes, metric='PointingGame') + +# returns the dataset to be explained, when localization is chosen, the dataset is required to provide bounding box +# the columns of the dataset should be in [image], [image, labels], or [image, labels, bbox] (order matters). +# You may refer to 'mindspore.dataset.project' for columns managements. +dataset_path = "/dataset_dir" +dataset = get_dataset(dataset_path) + +# specify the class names of the dataset +classes = [ + 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', + 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', + 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor', +] + +dataset_with_classes = (dataset, classes) +explainers = [gradcam, guidedbackprop] +benchmarkers = [faithfulness, localization] + +# initialize runner with specified summary_dir +summary_dir = "./" +runner = ExplainRunner(summary_dir) + +# execute runner.run to generate explanation and evaluation results to save it to summary_dir +runner.run(dataset_with_classes, explainers, benchmarkers) +``` + +> - Only support CNN of image classification models, such as Lenet, Resnet, Alexnet. + +> - Only support PyNative mode. + +### Enabling MindInsight + +Enable MindInsight and click **Model Explanation** on the top of the page. All explanation log paths are displayed. When a log path meets the conditions, the **Saliency Map Visualization** buttons are displayed in the **Operation** column. + +![xai_index_en](./images/xai_index_en.png) + +## Pages and Functions + +### Saliency Map Visualization + +Saliency map visualization is used to display the image area that has the most significant impact on the model decision-making result. Generally, the image can be considered as a key feature of the objective classification. + +![xai_saliency_map_en](./images/xai_saliency_map_en.png) + +The following information is displayed on the **Saliency Map Visualization** page: +- Objective dataset set by a user through the Python API of the dataset. +- Ground truth tags, prediction tags, and the prediction probabilities of the model for the corresponding tags. The system adds the TP, TN, FP and FN flags(meanings are provided in the page's information) in the upper left corner of the corresponding tag based on the actual requirements. +- A saliency map given by the selected explanation method. + +Operations: +1. Select the required explanation methods. Currently, we support four explanation methods. More explanation methods will be provided in the future. +2. Click **Overlay on Original Image** in the upper right corner of the page to overlay the saliency map on the original image. +3. Click different tags to display the saliency map analysis results of the model for different tags. For different classification results, the focus of the model is usually different. +4. Use the tag filtering function on the upper part of the page to filter out images with specified tags. +5. Select an image display sequence from **Sort Images By** in the upper right corner of the page. +6. Click **View Score** on the right of an explanation method. The page for assessing all explanation methods is displayed. +7. Click image you will see the higher resolution image. + +![xai_saliency_map_detail_en](./images/xai_saliency_map_detail_en.png) + +### Explanation Method Assessment + +#### Comprehensive Assessment + +The provided explanation methods are scored from different dimensions. We provide various dimensions scores to help users compare the performance and select the most suitable one. You can configure weights for metrics in a specific scenario to obtain the comprehensive score. + +![xai_metrix_comprehensive_en](./images/xai_metrix_comprehensive_en.png) + +#### Classification Assessment + +The classification assessment page provides two types of comparison. One is to compare scores of different measurement dimensions of the same explanation method in each tag. The other is to compare scores of different explanation methods of the same measurement dimension in each tag. + +![xai_metrix_class_en](./images/xai_metrix_class_en.png) diff --git a/tutorials/source_en/use/.DS_Store b/tutorials/source_en/use/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..5008ddfcf53c02e82d7eee2e57c38e5672ef89f6 Binary files /dev/null and b/tutorials/source_en/use/.DS_Store differ diff --git a/tutorials/source_en/use/images/xai_index_en.png b/tutorials/source_en/use/images/xai_index_en.png new file mode 100644 index 0000000000000000000000000000000000000000..bb70bf262fed8e3bee441088d2a95df6f2c143a3 Binary files /dev/null and b/tutorials/source_en/use/images/xai_index_en.png differ diff --git a/tutorials/source_en/use/images/xai_metrix_class_en.png b/tutorials/source_en/use/images/xai_metrix_class_en.png new file mode 100644 index 0000000000000000000000000000000000000000..bb11e9ab12ea9c6c12b230116ad438023c7d449b Binary files /dev/null and b/tutorials/source_en/use/images/xai_metrix_class_en.png differ diff --git a/tutorials/source_en/use/images/xai_metrix_comprehensive_en.png b/tutorials/source_en/use/images/xai_metrix_comprehensive_en.png new file mode 100644 index 0000000000000000000000000000000000000000..03d49c0a12121f58f521c5478d84fdb8aaa523b4 Binary files /dev/null and b/tutorials/source_en/use/images/xai_metrix_comprehensive_en.png differ diff --git a/tutorials/source_en/use/images/xai_saliency_map_detail_en.png b/tutorials/source_en/use/images/xai_saliency_map_detail_en.png new file mode 100644 index 0000000000000000000000000000000000000000..eefe66391b395af0cfb54d20ef40f753b57011bd Binary files /dev/null and b/tutorials/source_en/use/images/xai_saliency_map_detail_en.png differ diff --git a/tutorials/source_en/use/images/xai_saliency_map_en.png b/tutorials/source_en/use/images/xai_saliency_map_en.png new file mode 100644 index 0000000000000000000000000000000000000000..d8577f04d815027ceeb401b3a936fac84ac75aa8 Binary files /dev/null and b/tutorials/source_en/use/images/xai_saliency_map_en.png differ diff --git a/tutorials/source_zh_cn/.DS_Store b/tutorials/source_zh_cn/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..6ea4700bf34e709ae3326f25e43bf687ca5acc14 Binary files /dev/null and b/tutorials/source_zh_cn/.DS_Store differ diff --git a/tutorials/source_zh_cn/advanced_use/model_explaination.md b/tutorials/source_zh_cn/advanced_use/model_explaination.md new file mode 100644 index 0000000000000000000000000000000000000000..62500c45f745ad9d4795e1b8dcf5c9a6f5a327a4 --- /dev/null +++ b/tutorials/source_zh_cn/advanced_use/model_explaination.md @@ -0,0 +1,140 @@ +# 解释模型 + +`Ascend` `GPU` `Linux` `初级` `中级` `高级` `可解释性` + + + +- [解释模型](#解释模型) + - [概述](#概述) + - [操作流程](#操作流程) + - [准备脚本](#准备脚本) + - [启动MindInsight](#启动MindInsight) + - [页面及功能介绍](#页面及功能介绍) + - [显著图可视化](#显著图可视化) + - [解释方法评估](#解释方法评估) + - [综合评估](#综合评估) + - [分类评估](#分类评估) + + + +## 概述 + +当前深度学习模型多为黑盒模型,性能表现好但可解释性较差。模型解释模块旨在为用户提供对模型决策依据的解释,帮助用户更好地理解模型、信任模型,以及当模型出现错误时有针对性地改进模型效果。 + +在一些影响至关重要的应用场景中,如自动驾驶、金融决策等,由于法律和政策监管的原因,AI模型如果不具备可解释性,是无法真正落地应用的。所以模型的可解释性的重要性越来越高,受到越来越多的关注。因此,模型解释是提升MindSpore生态应用性、人性化至关重要的一部分。 + +具体来说,在图片分类任务中,有一种解释方法会将影响模型的分类决策最关键的区域高亮出来,我们称之为“显著图”,如果人们看到模型关注的部分恰好就是分类标签的关键特征,那么说明模型学习到的特征是正确的,我们可以更加信任模型的效果和决策。如果模型关注的是不相关的部分,即使预测标签是正确的,也不代表模型是可靠的,我们还是需要优化改进模型。造成这种情况有可能是训练数据中存在某些无关元素的关联,模型开发者可以考虑有针对性的做数据增强来修正模型学习到的偏见。 + +有了多种解释方法后,我们还提供了一套对解释方法效果评分的度量方法,从多种维度评估解释方法的效果。帮助用户比较和选择最适合于特定场景的解释方法。 + +## 操作流程 + +### 准备脚本 + +当前MindSpore提供解释方法及给解释方法进行评估的度量Python API,已提供的解释方法可以通过```mindspore.explainer.explanation```包获取,度量方法可以通过```mindspore.explainer.benchmark```包获取。用户准备好待解释的黑盒模型和数据,在脚本中根据需要实例化解释方法及度量方法,调用API用于收集解释结果和解释度量结果。 + +MindSpore还提供```mindspore.explainer.ExplainRunner```运行模块,支持自动化运行所有解释方法和度量方法。用户将实例化的解释方法及度量方法传入到```ExplainRunner```的```run```方法中,即可自动运行解释方法及度量方法,并生成及保存包含解释结果及解释度量结果的解释日志。 + +下面以ResNet50及带有20类多标签数据为例,在原有脚本流程的基础上增加解释方法的初始化及调用代码,使用解释方法GradCAM对模型进行解释,并使用度量方法对解释方法进行评估。其样例代码如下: + +``` +import mindspore.nn as nn +from mindspore.train.serialization import load_checkpoint, load_param_into_net + +from mindspore.explainer.explaination.explaination import GradCAM, GuidedBackprop +from mindspore.explainer.benchmark import Faithfulness, Localization +from mindspore.explainer import ExplainRunner + +num_classes = 20 +# please refer to model_zoo for the model architecture of resnet50 +model = resnet50(num_classes) +param_dict = load_checkpoint("resnet50.ckpt") +load_param_into_net(model, param_dict) + + +# combine the model architecture with its final activation layer, eg.Sigmoid() for multi-label models or Softmax() for single-label models +model = nn.SequentialCell([model, nn.Sigmoid()]) +model.set_grad(False) +model.set_train(False) + +# initialize explainers with the loaded black-box model +gradcam = GradCAM(model, layer='0.layer4') +guidedbackprop = GuidedBackprop(model) + + +# initialize benchmarkers to evaluate the chosen explainers +faithfulness = Faithfulness(num_labels=num_classes, metric='InsertionAUC') +localization = Localization(num_labels=num_classes, metric='PointingGame') + +# returns the dataset to be explained, when localization is chosen, the dataset is required to provide bounding box +# the columns of the dataset should be in [image], [image, labels], or [image, labels, bbox] (order matters). +# You may refer to 'mindspore.dataset.project' for columns managements. +dataset_path = "/dataset_dir" +dataset = get_dataset(dataset_path) + +# specify the class names of the dataset +classes = [ + 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', + 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', + 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor', +] + +dataset_with_classes = (dataset, classes) +explainers = [gradcam, guidedbackprop] +benchmarkers = [faithfulness, localization] + +# initialize runner with specified summary_dir +summary_dir = "./" +runner = ExplainRunner(summary_dir) + +# execute runner.run to generate explanation and evaluation results to save it to summary_dir +runner.run(dataset_with_classes, explainers, benchmarkers) +``` + +>- 当前只支持图片分类下的CNN网络模型,比如:Lenet、Resnet、Alexnet。 + +>- 仅支持PyNative运行模式。 + +### **启动MindInsight** + +启动MindInsight系统,在顶部选择进入“模型解释”模块。可以看到所有的解释日志路径,当日志满足条件时,操作列会有“显著图可视化”的功能入口。 + +![xai_index](./images/xai_index.png) + +## 页面及功能介绍 + +### 显著图可视化 + +显著图可视化用于展示对模型决策结果影响最为显著的图片区域,通常可视为图片被标记为目标分类的关键特征。 + +![xai_saliency_map](./images/xai_saliency_map.png) + +进入显著图可视化界面,会展示: +- 用户通过Dataset的Python API接口设置的目标数据集。 +- 真实标签、预测标签,以及模型对对应标签的预测概率。根据具体情况,系统会在对应标签的左上角增加TP,TN,FP,FN(含义见界面提示信息)的旗标。 +- 选中的解释方法给出的显著图。 + +界面操作: +1. 通过界面上方的解释方法勾选需要的解释方法。 +2. 通过切换界面右上方的“叠加于原图”按钮可以选择让显著图叠加于原图上显示。 +3. 点击不同标签,显示模型对不同标签的显著图分析结果。对于不同的分类结果,通常模型的关注点也是不同的。 +4. 通过界面上方的标签筛选功能,筛选出指定标签图片。 +5. 通过界面右上角的图片排序改变图片显示的顺序。 +6. 点击解释方法最右边的“查看评分”,可以进入对所有解释方法评估的界面。 +7. 点击图片可查看放大图。 + +![xai_saliency_map_detail](./images/xai_saliency_map_detail.png) + +### 解释方法评估 + +#### 综合评估 + +对当前提供的解释方法,从不同的评估维度进行评分。我们提供了多种评估度量维度,帮助用户对比不同解释方法在不同维度上的表现,从而挑选最适合用户使用的解释方法。用户可以根据对特定场景下看中的指标进行权重配置,得到综合得分。 + +![xai_metrix_comprehensive](./images/xai_metrix_comprehensive.png) + +#### 分类评估 + +分类评估页提供两种形式的对比,一种是同一解释方法的不同度量维度在各个标签下的分值,一种是对于同一度量维度,不同解释方法在各个标签下的分值。 + +![xai_metrix_class](./images/xai_metrix_class.png) diff --git a/tutorials/source_zh_cn/use/images/xai_index.png b/tutorials/source_zh_cn/use/images/xai_index.png new file mode 100644 index 0000000000000000000000000000000000000000..9920e78fa3efd87c05236f2781e1a51a182f2635 Binary files /dev/null and b/tutorials/source_zh_cn/use/images/xai_index.png differ diff --git a/tutorials/source_zh_cn/use/images/xai_metrix_class.png b/tutorials/source_zh_cn/use/images/xai_metrix_class.png new file mode 100644 index 0000000000000000000000000000000000000000..667405255d59a34d2eeecb65c068c3081eacdb78 Binary files /dev/null and b/tutorials/source_zh_cn/use/images/xai_metrix_class.png differ diff --git a/tutorials/source_zh_cn/use/images/xai_metrix_comprehensive.png b/tutorials/source_zh_cn/use/images/xai_metrix_comprehensive.png new file mode 100644 index 0000000000000000000000000000000000000000..518a0c2353b9518a3b953c6273535bf00dd2ab29 Binary files /dev/null and b/tutorials/source_zh_cn/use/images/xai_metrix_comprehensive.png differ diff --git a/tutorials/source_zh_cn/use/images/xai_saliency_map.png b/tutorials/source_zh_cn/use/images/xai_saliency_map.png new file mode 100644 index 0000000000000000000000000000000000000000..27587fbe0456de72d2b2ff5be2a99f797005dd82 Binary files /dev/null and b/tutorials/source_zh_cn/use/images/xai_saliency_map.png differ diff --git a/tutorials/source_zh_cn/use/images/xai_saliency_map_detail.png b/tutorials/source_zh_cn/use/images/xai_saliency_map_detail.png new file mode 100644 index 0000000000000000000000000000000000000000..66e6b1cf714f8ee7b114a1149b3e06a941574096 Binary files /dev/null and b/tutorials/source_zh_cn/use/images/xai_saliency_map_detail.png differ