diff --git a/contrib/ActionRecognition/README.md b/contrib/ActionRecognition/README.md new file mode 100644 index 0000000000000000000000000000000000000000..d4238940e0d5e9ba467af95284a7f25e202e21ad --- /dev/null +++ b/contrib/ActionRecognition/README.md @@ -0,0 +1,444 @@ +# ActionRecgnition + +## 1 介绍 + +本开发样例演示动作识别系统 ActionRecgnition，供用户参考。 +本系统基于mxVision SDK进行开发，以昇腾Atlas300卡为主要的硬件平台，主要应用于单人独处、逗留超时、快速移动、剧烈运动、离床检测、攀高检测六种应用场景。 + +1. 单人独处：识别出单个人独处场景后报警。 +2. 逗留超时：识别出单人或多人在区域内长时间逗留的情况并发出警报。 +3. 快速移动：检测出视频中单人或多人行进速度大于阈值的情况，并发出警报。 +4. 剧烈运动：检测到视频流中有剧烈运动并进行报警。 +5. 离床检测：检测出视频中行人离开指定区域的情况并报警。 +6. 攀高检测：检测出行人中心点向上移动的情况，并发出警报。 + +## 2 环境依赖 + +* 支持的硬件形态和操作系统版本 + + | 硬件形态 | 操作系统版本 | + | ------------------------------------- | -------------- | + | x86_64+Atlas 300I 推理卡（型号3010） | Ubuntu 18.04.1 | + | x86_64+Atlas 300I 推理卡 （型号3010） | CentOS 7.6 | + | ARM+Atlas 300I 推理卡 （型号3000） | Ubuntu 18.04.1 | + | ARM+Atlas 300I 推理卡 （型号3000） | CentOS 7.6 | + +* 软件依赖 + + | 软件名称 | 版本 | + | -------- | ----- | + | cmake | 3.5.+ | + | mxVision | 2.0.2 | + | Python | 3.7.5 | + | OpenCV | 3.4.0 | + | gcc | 7.5.0 | + | ffmpeg | 4.3.2 | + +## ３ 代码主要目录介绍 + +本Sample工程名称为Actionrecognition，工程目录如下图所示： + +``` +. +├── data +│ ├── roi +│   │   ├── Climbup +│   │   └── ... +│   └── video +│   │   ├── Alone +│   │   └── ... +├── models +│   ├── ECONet +│   │   └── ... +│   └── yolov3 +│   │   └── ... +├── pipeline +│   ├── plugin_all.pipeline +│   ├── plugin_alone.pipeline +│   ├── plugin_climb.pipeline +│   ├── plugin_outofbed.pipeline +│   ├── plugin_overspeed.pipeline +│   ├── plugin_overstay.pipeline +│   └── plugin_violentaction.pipeline +├── plugins +│   ├── MxpiStackFrame // 堆帧插件 +│   │   ├── CMakeLists.txt +│   │   ├── MxpiStackFrame.cpp +│   │   ├── MxpiStackFrame.h +│   │   ├── BlockingMap.cpp +│   │   ├── BlockingMap.h +│   │   └── build.sh +│   ├── PluginAlone // 单人独处插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginAlone.cpp +│   │   ├── PluginAlone.h +│   │   └── build.sh +│   ├── PluginClimb // 攀高检测插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginClimb.cpp +│   │   ├── PluginClimb.h +│   │   └── build.sh +│   ├── PluginOutOfBed // 离床检测插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginOutOfBed.cpp +│   │   ├── PluginOutOfBed.h +│   │   └── build.sh +│   ├── PluginOverSpeed // 快速移动插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginOverSpeed.cpp +│   │   ├── PluginOverSpeed.h +│   │   └── build.sh +│   ├── PluginOverStay // 逗留超时插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginOverStay.cpp +│   │   ├── PluginOverStay.h +│   │   └── build.sh +│   ├── PluginCounter // 计时插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginCounter.cpp +│   │   ├── PluginCounter.h +│   │   └── build.sh +│   ├── PluginViolentAction // 剧烈运动插件 +│   │   ├── CMakeLists.txt +│   │   ├── Plugin_ViolentAction.cpp +│   │   ├── Plugin_ViolentAction.h +│   │   └── build.sh +├── main.py +├── README.md +└── run.sh +``` + +## 4 软件方案介绍 + +为了完成上述六种应用场景中的行为识别，系统需要检测出同一目标短时间内状态的变化以及是否存在剧烈运动，因此系统中需要包含目标检测、目标跟踪、动作识别与逻辑后处理。其中目标检测模块选取Yolov3，得到行人候选框；目标跟踪模块使用IOU匹配，关联连续帧中的同一目标。将同一目标在连续帧的区域抠图组成视频序列，输入动作识别模块ECONet，模型输出动作类别，判断是否为剧烈运动。逻辑后处理通过判断同一目标在连续帧内的空间位置变化判断难以被定义为运动的其余五种应用场景。系统方案中各模块功能如表1.1 所示。 + +表1.1 系统方案中个模块功能： + +| 序号 | 子系统 | 功能描述 | +| ---- | -------------- | ------------------------------------------------------------ | +| 1 | 初始化配置 | 主要用于初始化资源，如线程数量、共享内存等。 | +| 2 | 视频流 | 从多路IPC相机拉流，并传输入Device进行计算。 | +| 3 | 视频解码 | 通过硬件（DVPP）对视频进行解码，转换为YUV数据进行后续处理。 | +| 4 | 图像预处理 | 在进行基于深度神经网络的图像处理之前，需要将图像缩放到固定的尺寸和格式。 | +| 5 | 目标检测 | 基于深度学习的目标检测算法是该系统的核心模块之一，本方案选用基于Yolov3的目标检测。 | +| 6 | 目标跟踪 | 基于卡尔曼滤波与匈牙利算法的目标跟踪算法是该系统的核心模块之一，本方案选用IOU匹配。 | +| 7 | 图像抠图 | 将同一目标在连续帧所在区域抠图，并组成图像序列，输入动作识别模块。 | +| 8 | 动作识别 | 基于深度学习的动作识别算法是该系统的核心模块之一，本方案选用基于ECONet的动作识别模型。 | +| 9 | 单人独处后处理 | 当连续m帧只出现一个目标ID时，则判断为单人独处并报警，如果单人独处报警之前n帧内已经报警过，则不重复报警。 | +| 10 | 快速移动后处理 | 当同一目标在连续m帧中心点平均位移高于阈值v，则判断为快速移动，如果快速移动报警之前n帧内已经报警过，则不重复报警。 | +| 11 | 逗留超时后处理 | 当同一目标在连续m帧中心点平均位移低于阈值v，则判断为快速移动，如果快速移动报警之前n帧内已经报警过，则不重复报警。 | +| 12 | 离床检测后处理 | 当同一目标在连续m帧内从给定区域roi内离开，则判断为离床，如果离床报警之前n帧内已经报警过，则不重复报警。 | +| 13 | 攀高检测后处理 | 当同一目标在连续m帧内从给定区域roi内中心点上升，并且中心点位移大于阈值h，则判断为离床，如果离床报警之前n帧内已经报警过，则不重复报警。 | +| 14 | 剧烈运动后处理 | 动作识别模块输出类别为关注的动作类别时，则判断为剧烈运动，如果剧烈运动之前n帧内已经报警过，则不重复报警。 | + +## 5 准备 + +**步骤1：** 参考安装教程《mxVision 用户指南》安装 mxVision SDK。 + +**步骤2：** 配置 mxVision SDK 环境变量。 + +`export MX_SDK_HOME=${安装路径}/mxVision ` + +注：本例中mxVision SDK安装路径为 /root/work/MindX_SDK/mxVision。 + +**步骤3：** 推荐在${MX_SDK_HOME}/samples下创建ActionRecognition根目录，在项目根目录下创建目录models `mkdir models`，分别为yolov3和ECONet创建一个文件夹，将两个离线模型及各自的配置文件放入文件夹下。下载地址：[Baiduyun](https://pan.baidu.com/s/1IEWS0IjOFaQrXFdt-4kSRg)(passwd: rdpu)。创建完成后models文件夹下的目录结构如下： + +``` +.models +├── ECONet +│   ├── eco_aipp.cfg // 模型转换aipp配置文件 +│   ├── eco_post.cfg // 模型后处理配置文件 +│   ├── ECONet.om // 离线模型 +│   ├── ucf101.names // label文件 +│   ├── ucf101_best.pb // 冻结pb模型 +│   └── trans_pb2om.sh // 模型转换脚本 +├── yolov3 +│   ├── coco.names // label文件 +│   ├── yolov3_tf_bs1_fp16.om // 离线模型 +│   └── yolov3_tf_bs1_fp16.cfg // 模型后处理配置文件 +``` + +**步骤4：** 编译程序前提需要先交叉编译好第三方依赖库。 + +**步骤5：** 配置环境变量MX_SDK_HOME： + +```cmake +export MX_SDK_HOME=/MindX_SDK/mxVision/ +# 此处MX_SDK_HOME请使用MindX_SDK的实际路径 +``` + +**步骤6**：在插件代码目录下创建build文件夹，使用cmake命令进行编译，生成.so文件。下面以单人独处插件的编译过程作为范例： + +```bash +## 进入目录 /plugins/plugin_Alone +## 创建build目录 +mkdir build +## 使用cmake命令进行编译 +cmake .. +make -j +``` + +或者使用插件代码目录下的build.sh脚本，例： + +```bash +## 前提条件是正确设置export MX_SDK_HOME +chmod +x build.sh +./build.sh +``` + +编译好的插件会自动存放到SDK的插件库中，可以直接在pipiline中使用。 + +**步骤7:** 配置pipeline + +1. 插件参数介绍 + + * MxpiStackFrame + + | 参数名称 | 参数解释 | + | :----------- | -------------------- | + | visionSource | 抠图插件名称 | + | trackSource | 跟踪插件名称 | + | frameNum | 跳帧间隔（为1不跳） | + | timeOut | 某个目标堆帧超时时间 | + | sleepTime | 检查线程休眠时间 | + + * PluginAlone + + | 参数名称 | 参数解释 | + | :------------------ | ---------------------- | + | dataSourceDetection | 目标检测后处理插件名称 | + | dataSourceTrack | 跟踪插件名称 | + | detectThresh | 检测帧数 | + | detectRatio | 警报帧阈值 | + | detectSleep | 警报间隔 | + + * PluginClimb + + | 参数名称 | 参数解释 | + | :------------------ | ---------------------- | + | dataSourceTrack | 跟踪插件名称 | + | dataSourceDetection | 目标检测后处理插件名称 | + | detectRatio | 警报帧阈值 | + | filePath | ROI配置txt文件 | + | detectSleep | 警报间隔 | + | bufferLength | 检测帧数窗口大小 | + | highThresh | 高度阈值 | + + * PluginCounter + + | 参数名称 | 参数解释 | + | :----------------- | ------------ | + | dataSourceTrack | 跟踪插件名称 | + | descriptionMessage | 插件描述信息 | + + * PluginOutOfBed + + | 参数名称 | 参数解释 | + | :------------------ | ---------------------- | + | dataSourceTrack | 跟踪插件名称 | + | dataSourceDetection | 目标检测后处理插件名称 | + | configPath | ROI配置txt文件 | + | detectThresh | 检测帧数窗口大小 | + | detectSleep | 警报间隔 | + | detectRatio | 警报帧阈值 | + + * PluginOverSpeed + + | 参数名称 | 参数解释 | + | :------------------ | ---------------------- | + | dataSourceTrack | 跟踪插件名称 | + | dataSourceDetection | 目标检测后处理插件名称 | + | speedThresh | 速度阈值 | + | frames | 检测帧数窗口大小 | + | detectSleep | 警报间隔 | + + * PluginOverStay + + | 参数名称 | 参数解释 | + | :------------------ | ---------------------- | + | dataSourceTrack | 跟踪插件名称 | + | dataSourceDetection | 目标检测后处理插件名称 | + | stayThresh | 逗留时间阈值 | + | frames | 检测间隔帧数 | + | distanceThresh | 逗留范围 | + | detectRatio | 警报帧阈值 | + | detectSleep | 警报间隔 | + + * PluginViolentAction + + | 参数名称 | 参数解释 | + | :-------------- | --------------------- | + | classSource | 分类后处理插件名称 | + | filePath | 感兴趣动作类别txt文件 | + | detectSleep | 警报间隔 | + | actionThreshold | 动作阈值 | + +2. 配置范例 + + ```json + ## PluginClimb + "mxpi_pluginclimb0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "detectRatio": "0.6", + "filePath": "./data/roi/Climbup/*.txt", + "detectSleep": "30", + "bufferLength": "8", + "highThresh": "10" + }, + "factory": "mxpi_pluginclimb", + "next": "mxpi_dataserialize0" + } + ## Yolov3 + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "./models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }， + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "./models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "./models/yolov3/coco.names", + "postProcessLibPath": "../../lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_distributor0" + }, + ## ECONet + "mxpi_tensorinfer1":{ + "props": { + "dataSource": "mxpi_stackframe0", + "skipModelCheck": "1", + "modelPath": "./models/ECONet/ECONet.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_classpostprocessor0" + }, + ``` + + 根据所需场景，配置pipeline文件，调整路径参数以及插件阈值参数。例如"filePath"字段替换为roi/Climb目录下的感兴趣区域txt文件，“postProcessLibPath”字段是SDK模型后处理插件库路径。 + +3. 将pipeline中“rtspUrl”字段值替换为可用的 rtsp 流源地址（需要自行准备可用的视频流，视频流格式为H264），[自主搭建rtsp视频流教程](###7.3-数据下载与RTSP)。 + +**步骤8：** 在main.py中，修改pipeline路径、对应的流名称以及需要获取结果的插件名称。 + +```python +## 插件位置 +with open("./pipeline/plugin_outofbed.pipeline", 'rb') as f: + pipelineStr = f.read() +## pipeline中的流名称 +streamName = b'classification+detection' +## 想要获取结果的插件名称 +key = b'mxpi_pluginalone0' +``` + +## 6 运行 + +修改 run.sh 文件中的环境路径和项目路径。 + +```bash +export MX_SDK_HOME=${CUR_PATH}/../../.. +## 注意当前目录CUR_PATH与MX_SDK_HOME环境目录的相对位置 +``` + +直接运行 + +```bash +chmod +x run.sh +./run.sh +``` + +## 7 常见问题 +### 7.1 未配置ROI + +#### 问题描述： + +攀高检测与离床检测出现如下报错： +```c++ +terminate called after throwing an instance of 'cv::Exception' + what(): OpenCV(4.2.0) /usr1/workspace/MindX_SDK_Multi_DailyBuild/opensource/opensource-scl7/opencv/modules/imgproc/src/geometry.cpp:103: error: (-215:Assertion failed) total >= 0 && (depth == CV_32S || depth == CV_32F) in function 'pointPolygonTest' +``` +#### 解决方案： +在pipeline处没有配置roi区域的txt文件，进行配置即可。 + +示例： + +```json +"mxpi_pluginoutofbed0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetect": "mxpi_objectpostprocessor0", + "configPath": "./data/roi/OutofBed/*.txt", + "detectThresh": "8", + "detectSleep": "15", + "detectRatio": "0.25" + }, + "factory": "mxpi_pluginoutofbed", + "next": "mxpi_dataserialize0" +} +``` + +### 7.2 模型路径配置 + +#### 问题描述： + +检测过程中用到的模型以及模型后处理插件需配置路径属性。 + +#### 后处理插件配置范例： + +```json +"mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "./models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "./models/yolov3/coco.names", + "postProcessLibPath": "../../../lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" +} +``` + +### 7.3 数据下载与RTSP + +H264视频文件及ROI文件：[Baiduyun](https://pan.baidu.com/s/1Xz378sHh0WeeeYIjnZ-pPQ) ，提取码：as01 ； + +RTSP取流地址（可以从网络摄像机获取，也可通过Live555等工具将本地视频文 件转换为rtsp流）。自主搭建RTSP拉流教程：[live555链接](https://bbs.huaweicloud.com/forum/thread-68720-1-1.html)，需要注意的是在搭建RTSP时，使用./genMakefiles 命令生成编译文件时，输入的参数是根据cofig.<后缀>获取的，与服务器架构等有关。 + +RTSP视频拉流插件配置范例： + +```json +"mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp_Url" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" +} +``` + +其中rtsp_Url的格式是 rtsp:://host:port/Data，host:port/路径映射到mediaServer/目录下，Data为视频文件的路径。 + +RTSP拉流教程：[live555链接](https://bbs.huaweicloud.com/forum/thread-68720-1-1.html)中第七步视频循环推流，按照提示修改cpp文件可以使自主搭建的rtsp循环推流，如果不作更改，则为有限的视频流。 + +## 8 模型转换 + +本项目中用到的模型有：ECONet，yolov3 + +yolov3模型转换及下载参考华为昇腾社区[ModelZoo](https://www.hiascend.com/zh/software/modelzoo/detail/C/210261e64adc42d2b3d84c447844e4c7)； + +ECONet离线模型转换参考 [昇腾Gitee](https://gitee.com/ascend/modelzoo/tree/master/contrib/TensorFlow/Research/cv/econet/ECONet_tf_paper99)：下载冻结pb模型ucf101_best.pb，编辑trans_pb2om.sh文件，将--model 配置为ECONet模型所在目录，--output配置为模型输出路径，--insert_op_conf配置为aipp文件路径，在命令行输入 + +```bash +chmod +x trans_pb2om.sh +./trans_pb2om.sh +``` + +完成ECONet模型转换。模型下载或转换完成后，按照目录结构放置模型。 diff --git a/contrib/ActionRecognition/data/.keep b/contrib/ActionRecognition/data/.keep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/roi/.keep b/contrib/ActionRecognition/data/roi/.keep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/roi/Climbup/.gitkeep b/contrib/ActionRecognition/data/roi/Climbup/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/.gitkeep b/contrib/ActionRecognition/data/roi/OutOfBed/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/roi/ViolentAction/.gitkeep b/contrib/ActionRecognition/data/roi/ViolentAction/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/video/.keep b/contrib/ActionRecognition/data/video/.keep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/video/Alone/.gitkeep b/contrib/ActionRecognition/data/video/Alone/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/video/Climbup/.gitkeep b/contrib/ActionRecognition/data/video/Climbup/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/video/OutOfBed/.gitkeep b/contrib/ActionRecognition/data/video/OutOfBed/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/video/Over_staying/.gitkeep b/contrib/ActionRecognition/data/video/Over_staying/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/video/Speed_up/.gitkeep b/contrib/ActionRecognition/data/video/Speed_up/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/data/video/Violent_action/.gitkeep b/contrib/ActionRecognition/data/video/Violent_action/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/main.py b/contrib/ActionRecognition/main.py new file mode 100644 index 0000000000000000000000000000000000000000..43592e6f1d6fc5e48422a557395115a666f0ca65 --- /dev/null +++ b/contrib/ActionRecognition/main.py @@ -0,0 +1,62 @@ +#!/usr/bin/env python +# coding=utf-8 + +""" + + Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + +""" + +from StreamManagerApi import StreamManagerApi, StringVector + +if __name__ == '__main__': + # init stream manager + streamManagerApi = StreamManagerApi() + ret = streamManagerApi.InitManager() + if ret != 0: + print("Failed to init Stream manager, ret=%s" % str(ret)) + exit() + + # create streams by pipeline config file + with open("./pipeline/plugin_alone.pipeline", 'rb') as f: + pipelineStr = f.read() + ret = streamManagerApi.CreateMultipleStreams(pipelineStr) + if ret != 0: + print("Failed to create Stream, ret=%s" % str(ret)) + exit() + + # Inputs data to a specified stream based on streamName. + streamName = b'detection+tracking' + inPluginId = 0 + + retStr = '' + key = b'mxpi_pluginalone0' + keyVec = StringVector() + keyVec.push_back(key) + while True: + # Obtain the inference result by specifying streamName and uniqueId. + inferResult = streamManagerApi.GetResult(streamName, 0, 10000) + if inferResult is None: + break + if inferResult.errorCode != 0: + print("GetResultWithUniqueId error. errorCode=%d, errorMsg=%s" % ( + inferResult.errorCode, inferResult.data.decode())) + break + retStr = inferResult.data.decode() + print(retStr) + + # destroy streams + streamManagerApi.DestroyAllStreams() + diff --git a/contrib/ActionRecognition/models/ECONet/eco_aipp.cfg b/contrib/ActionRecognition/models/ECONet/eco_aipp.cfg new file mode 100644 index 0000000000000000000000000000000000000000..f01079fbee26c7a20ee5b6b11d1473fd81a8970e --- /dev/null +++ b/contrib/ActionRecognition/models/ECONet/eco_aipp.cfg @@ -0,0 +1,29 @@ +aipp_op { +aipp_mode : static +related_input_rank : 0 +input_format : YUV420SP_U8 +crop : false +csc_switch : true +rbuv_swap_switch : false +matrix_r0c0 : 256 +matrix_r0c1 : 0 +matrix_r0c2 : 359 +matrix_r1c0 : 256 +matrix_r1c1 : -88 +matrix_r1c2 : -183 +matrix_r2c0 : 256 +matrix_r2c1 : 454 +matrix_r2c2 : 0 +input_bias_0 : 0 +input_bias_1 : 128 +input_bias_2 : 128 +mean_chn_0 : 104 +mean_chn_1 : 117 +mean_chn_2 : 128 +min_chn_0 : 0.0 +min_chn_1 : 0.0 +min_chn_2 : 0.0 +var_reci_chn_0 : 1.0 +var_reci_chn_1 : 1.0 +var_reci_chn_2 : 1.0 +} diff --git a/contrib/ActionRecognition/models/ECONet/trans_pb2om.sh b/contrib/ActionRecognition/models/ECONet/trans_pb2om.sh new file mode 100644 index 0000000000000000000000000000000000000000..c12848050bad982a9d35ac1f691371e7be477113 --- /dev/null +++ b/contrib/ActionRecognition/models/ECONet/trans_pb2om.sh @@ -0,0 +1,18 @@ +export install_path=/usr/local/Ascend/ascend-toolkit/latest +export PATH=/usr/local/python3.7.5/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH +export PYTHONPATH=${install_path}/atc/python/site-packages:${install_path}/atc/python/site-packages/auto_tune.egg/auto_tune:${install_path}/atc/python/site-packages/schedule_search.egg:$PYTHONPATH +export LD_LIBRARY_PATH=${install_path}/atc/lib64:${install_path}/acllib/lib64:$LD_LIBRARY_PATH +export ASCEND_OPP_PATH=${install_path}/opp +export SLOG_PRINT_TO_STDOUT=1 + + +/usr/local/Ascend/ascend-toolkit/latest/atc/bin/atc \ + --mode=0 \ + --model=${pbmodel_path} \ + --output=${om_output_path} \ + --soc_version=Ascend310 \ + --input_format=NHWC\ + --input_shape="clip_holder:8,224,224,3" \ + --insert_op_conf=${aipp_path} \ + --framework=3 \ + log=debug > log.txt diff --git a/contrib/ActionRecognition/models/yolov3/.gitkeep b/contrib/ActionRecognition/models/yolov3/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/models/yolov3/.keep b/contrib/ActionRecognition/models/yolov3/.keep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/contrib/ActionRecognition/pipeline/plugin_all.pipeline b/contrib/ActionRecognition/pipeline/plugin_all.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..83a5cf0d9f406f6c5452590367b6583e69023796 --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_all.pipeline @@ -0,0 +1,137 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp_Url" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "./models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "./models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "./models/yolov3/coco.names", + "postProcessLibPath": "../../lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_plugincounter0" + }, + "mxpi_plugincounter0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0" + }, + "factory": "mxpi_plugincounter", + "next": "mxpi_pluginalone0" + }, + "mxpi_pluginalone0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "detectThresh":"8", + "detectRatio":"0.8", + "detectSleep":"270" + }, + "factory": "mxpi_pluginalone", + "next": "mxpi_pluginoverspeed0" + }, + "mxpi_pluginoverspeed0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "speedThresh": "1", + "frames": "8", + "detectSleep": "270" + }, + "factory": "mxpi_pluginoverspeed", + "next": "mxpi_pluginoverstay0" + }, + "mxpi_pluginoverstay0": { + "props": { + "status": "1", + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "stayThresh":"90", + "frames":"90", + "distanceThresh":"100", + "detectRatio":"0.7", + "detectsleep":"90" + }, + "factory": "mxpi_pluginoverstay", + "next": "mxpi_pluginclimb0" + }, + "mxpi_pluginclimb0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "detectRatio": "0.6", + "filePath": "./data/roi/Climbup/*.txt", + "detectSleep": "30", + "bufferLength": "8", + "highThresh": "10" + }, + "factory": "mxpi_pluginclimb", + "next": "mxpi_pluginoutofbed0" + }, + "mxpi_pluginoutofbed0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetect": "mxpi_objectpostprocessor0", + "configPath": "./data/roi/OutofBed/*.txt", + "detectThresh": "8", + "detectSleep": "15", + "detectRatio": "0.25" + }, + "factory": "mxpi_pluginoutofbed", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginoverstay0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} \ No newline at end of file diff --git a/contrib/ActionRecognition/pipeline/plugin_alone.pipeline b/contrib/ActionRecognition/pipeline/plugin_alone.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..c8ed69cd2cebd62e838559892b0aa42877e49641 --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_alone.pipeline @@ -0,0 +1,80 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp_Url" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "./models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "./models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "./models/yolov3/coco.names", + "postProcessLibPath": "../../lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginalone0" + }, + "mxpi_pluginalone0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "detectThresh": "8", + "detectRatio": "0.8", + "detectSleep": "270" + }, + "factory": "mxpi_pluginalone", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginalone0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/plugin_climb.pipeline b/contrib/ActionRecognition/pipeline/plugin_climb.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..f07068b38d8dd43f48621c1aee7adbf714396cac --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_climb.pipeline @@ -0,0 +1,82 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp_Url" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "./models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "./models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "./models/yolov3/coco.names", + "postProcessLibPath": "../../lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginclimb0" + }, + "mxpi_pluginclimb0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "detectRatio": "0.6", + "filePath": "./data/roi/Climbup/*.txt", + "detectSleep": "30", + "bufferLength": "8", + "highThresh": "10" + }, + "factory": "mxpi_pluginclimb", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginclimb0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/plugin_outofbed.pipeline b/contrib/ActionRecognition/pipeline/plugin_outofbed.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..7e8500a5356f8dae41c075fa7256ed9a32543f56 --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_outofbed.pipeline @@ -0,0 +1,81 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp_Url" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "./models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "./models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "./models/yolov3/coco.names", + "postProcessLibPath": "../../lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginoutofbed0" + }, + "mxpi_pluginoutofbed0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "configPath": "./datas/OutofBed/*.txt", + "detectThresh": "8", + "detectSleep": "15", + "detectRatio": "0.25" + }, + "factory": "mxpi_pluginoutofbed", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginoutofbed0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/plugin_overspeed.pipeline b/contrib/ActionRecognition/pipeline/plugin_overspeed.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..8ab1739c87b78a9557d605344abeed37069edc67 --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_overspeed.pipeline @@ -0,0 +1,80 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp_Url" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "./models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "./models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "./models/yolov3/coco.names", + "postProcessLibPath": "../../lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginoverspeed0" + }, + "mxpi_pluginoverspeed0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "speedThresh": "10", + "frames": "8", + "detectSleep": "270" + }, + "factory": "mxpi_pluginoverspeed", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginoverspeed0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/plugin_overstay.pipeline b/contrib/ActionRecognition/pipeline/plugin_overstay.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..bd2c611f63a5bc6b34ab03ecb984c6cac2196c54 --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_overstay.pipeline @@ -0,0 +1,82 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp_Url" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "./models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "./models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "./models/yolov3/coco.names", + "postProcessLibPath": "../../lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginoverstay0" + }, + "mxpi_pluginoverstay0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "stayThresh":"90", + "frames":"90", + "distanceThresh":"100", + "detectRatio":"0.7", + "detectSleep":"90" + }, + "factory": "mxpi_pluginoverstay", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginoverstay0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/plugin_violentaction.pipeline b/contrib/ActionRecognition/pipeline/plugin_violentaction.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..93afbe616a324b21f41f1e407acfb3b3acc5202f --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_violentaction.pipeline @@ -0,0 +1,125 @@ +{ + "detection+action recognition": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp_Url" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "./models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "./models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "./models/yolov3/coco.names", + "postProcessLibPath": "../../lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_distributor0" + }, + "mxpi_distributor0": { + "props": { + "dataSource": "mxpi_objectpostprocessor0", + "classIds": "0" + }, + "factory": "mxpi_distributor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_distributor0_0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_imagecrop0" + }, + "mxpi_imagecrop0": { + "props": { + "dataSource": "mxpi_distributor0_0", + "dataSourceImage": "mxpi_videodecoder0", + "resizeHeight": "224", + "resizeWidth": "224", + "resizeType": "Resizer_KeepAspectRatio_Fit" + }, + "factory": "mxpi_imagecrop", + "next": "mxpi_stackframe0" + }, + "mxpi_stackframe0":{ + "props": { + "visionSource": "mxpi_imagecrop0", + "trackSource": "mxpi_motsimplesort0", + "frameNum": "3" + }, + "factory": "mxpi_stackframe", + "next": "mxpi_tensorinfer1" + }, + "mxpi_tensorinfer1":{ + "props": { + "dataSource": "mxpi_stackframe0", + "skipModelCheck": "1", + "modelPath": "./models/ECONet/ECONet.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_classpostprocessor0" + }, + "mxpi_classpostprocessor0":{ + "props": { + "dataSource": "mxpi_tensorinfer1", + "postProcessConfigPath": "./models/ECONet/eco_post.cfg", + "labelPath":"./models/ECONet/ucf101.names, + "postProcessLibPath":"../../lib/modelpostprocessors/libresnet50postprocess.so" + }, + "factory": "mxpi_classpostprocessor", + "next": "mxpi_violentaction0" + }, + "mxpi_violentaction0":{ + "props": { + "classSource": "mxpi_classpostprocessor0", + "filePath": "./data/roi/ViolentAction/aoi.txt", + "detectSleep": "5" + }, + "factory": "mxpi_violentaction", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_violentaction0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.cpp b/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.cpp new file mode 100644 index 0000000000000000000000000000000000000000..7b44f51e92a7438ef9860078722275c551441586 --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.cpp @@ -0,0 +1,138 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "BlockingMap.h" + +namespace MxPlugins { + BlockingMap::BlockingMap() {} + + BlockingMap::~BlockingMap() {} + + void BlockingMap::Insert(const uint32_t &id, MxBase::MemoryData newData) { + // add std::lock_guard + std::lock_guard guard(mtx_); + // get current timestamp + using Time = std::chrono::high_resolution_clock; + auto currentTime = Time::now(); + std::pair> time_MxpiVisionList; + // set MxpiVisionInfo and MxpiVisionData + auto mxpiVisionList = copyList(newData); + time_MxpiVisionList = std::make_pair(currentTime, mxpiVisionList); + blockingMap_[id] = time_MxpiVisionList; + keys_.insert(id); + } + + void BlockingMap::Update(const uint32_t &id, MxBase::MemoryData newData) { + // add std::lock_guard + std::lock_guard guard(mtx_); + std::pair> + time_MxpiVisionList = blockingMap_[id]; + auto mxpiVisionList = time_MxpiVisionList.second; + // set MxpiVisionInfo and MxpiVisionData + MxTools::MxpiVision *dstMxpivision = mxpiVisionList->add_visionvec(); + MxTools::MxpiVisionInfo *mxpiVisionInfo = dstMxpivision->mutable_visioninfo(); + mxpiVisionInfo->set_format(1); + mxpiVisionInfo->set_height(224); + mxpiVisionInfo->set_width(224); + mxpiVisionInfo->set_heightaligned(224); + mxpiVisionInfo->set_widthaligned(224); + // set MxpiVisionData by MemoryData + MxTools::MxpiVisionData *mxpiVisionData = dstMxpivision->mutable_visiondata(); + mxpiVisionData->set_dataptr((uint64_t) newData.ptrData); + mxpiVisionData->set_datasize(newData.size); + mxpiVisionData->set_deviceid(newData.deviceId); + mxpiVisionData->set_memtype((MxTools::MxpiMemoryType) newData.type); + // visionlist->pair + time_MxpiVisionList = std::make_pair(time_MxpiVisionList.first, mxpiVisionList); + blockingMap_[id] = time_MxpiVisionList; + } + + std::pair> + BlockingMap::Get(const uint32_t &id) { + // add std::lock_guard + std::lock_guard guard(mtx_); + if (blockingMap_.find(id) != blockingMap_.end()) { + return blockingMap_[id]; + } else { + // If can't find the element, manually assign nullptr + std::pair> empty; + using Time = std::chrono::high_resolution_clock; + auto currentTime = Time::now(); + empty = std::make_pair(currentTime, nullptr); + return empty; + } + } + + void BlockingMap::Clear(const uint32_t &id) { + std::lock_guard guard(mtx_); + blockingMap_.erase(id); + keys_.erase(id); + } + + void BlockingMap::Reinsert(const uint32_t &id, std::shared_ptr &mxpiVisionList) { + // add std::lock_guard + std::lock_guard guard(mtx_); + // get current timestamp + using Time = std::chrono::high_resolution_clock; + auto currentTime = Time::now(); + std::pair> time_MxpiVisionList; + // set MxpiVisionInfo and MxpiVisionData + time_MxpiVisionList = std::make_pair(currentTime, mxpiVisionList); + blockingMap_[id] = time_MxpiVisionList; + keys_.insert(id); + } + + std::uint32_t BlockingMap::count(const uint32_t &id) { + std::lock_guard guard(mtx_); + return blockingMap_.count(id); + } + + size_t BlockingMap::Size() const { + return blockingMap_.size(); + } + + std::vector BlockingMap::Keys() { + // id<->key + std::vector keys; + std::lock_guard guard(mtx_); + for (auto iter = keys_.begin(); iter != keys_.end(); iter++) { + keys.push_back(*iter); + } + return keys; + } + + std::shared_ptr BlockingMap::copyList(MxBase::MemoryData newData) { + // new shared_ptr MxpiVisionList; + std::shared_ptr dstMxpiVisionListSptr(new MxTools::MxpiVisionList, + MxTools::g_deleteFuncMxpiVisionList); + MxTools::MxpiVision *dstMxpivision = dstMxpiVisionListSptr->add_visionvec(); + MxTools::MxpiVisionInfo *mxpiVisionInfo = dstMxpivision->mutable_visioninfo(); + mxpiVisionInfo->set_format(1); + mxpiVisionInfo->set_height(224); + mxpiVisionInfo->set_width(224); + mxpiVisionInfo->set_heightaligned(224); + mxpiVisionInfo->set_widthaligned(224); + // set MxpiVisionData by MemoryData + MxTools::MxpiVisionData *mxpiVisionData = dstMxpivision->mutable_visiondata(); + mxpiVisionData->set_dataptr((uint64_t) newData.ptrData); + mxpiVisionData->set_datasize(newData.size); + mxpiVisionData->set_deviceid(newData.deviceId); + mxpiVisionData->set_memtype((MxTools::MxpiMemoryType) newData.type); + return dstMxpiVisionListSptr; + } +} \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.h b/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.h new file mode 100644 index 0000000000000000000000000000000000000000..82260278c5a7acc03732e9fcba1a7883d739a0cb --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.h @@ -0,0 +1,83 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef INC_FACE_BLOCKING_MAP_H +#define INC_FACE_BLOCKING_MAP_H + +#include +#include +#include +#include +#include +#include +#include +#include "MxBase/Tensor/TensorBase/TensorBase.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/Log/Log.h" +#include "MxBase/MemoryHelper/MemoryHelper.h" +#include "MxStream/StreamManager/MxStreamManager.h" +#include "MxTools/Proto/MxpiDataTypeDeleter.h" + +/** + * This is a thread safe map. All functions use std::lock_guard. +*/ + +namespace MxPlugins { + class BlockingMap { + public: + /** + * @api + * @brief Initialize a new mxpivisionlist for this id, and insert the visiondata passed from upstream into the mxpivisionlist. + * @param trackID, MemoryData + */ + void Insert(const uint32_t &id, const MxBase::MemoryData newData); + + /** + * @api + * @brief Update mxpivisionlist and insert the visiondata for this id. + * @param trackID, MemoryData + */ + void Update(const uint32_t &id, const MxBase::MemoryData newData); + + /** + * @api + * @brief Reinsert mxpivisionlist and timestamp for this id. + * @param trackID, MxpivisionList + */ + void Reinsert(const uint32_t &id, std::shared_ptr &mxpiVisionList); + + std::pair> + Get(const uint32_t &id); // get std::pair instance + void Clear(const uint32_t &id); // erase map and id + std::vector Keys(); // save tackId into keys set + size_t Size() const; + + std::uint32_t count(const uint32_t &id); // return the number of elements of this id + std::shared_ptr + copyList(const MxBase::MemoryData newData); // set MxpiVisionInfo and MxpiVisionData + BlockingMap(); + + ~BlockingMap(); + + private: + std::mutex mtx_ = {}; + std::map>> blockingMap_ = {}; + std::set keys_ = {}; + }; +} +#endif + diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/CMakeLists.txt b/contrib/ActionRecognition/plugins/MxpiStackFrame/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..3b681e17d31675fd9e5438ee0c7d512fdb0d15be --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/CMakeLists.txt @@ -0,0 +1,45 @@ +cmake_minimum_required(VERSION 3.10) +project(MxpiStackFrame) + +set(CMAKE_CXX_STANDARD 11) + +set(PLUGIN_NAME "mxpi_stackframe") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options(-fPIC -fstack-protector-all -g -Wl,-z,relro,-z,now,-z -pie -Wall) +add_compile_options(-std=c++11 -Wno-deprecated-declarations) +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") + +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +add_definitions(-DENABLE_DVPP_INTERFACE) + +set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") + +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${MX_SDK_HOME}/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/opencv4) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) + +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} + mxpidatatype + plugintoolkit + mxbase + streammanager + mindxsdk_protobuf + glib-2.0 + gstreamer-1.0 + gobject-2.0 + gstbase-1.0 + gmodule-2.0 + ) diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.cpp b/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.cpp new file mode 100644 index 0000000000000000000000000000000000000000..77a8c29b95ae3e9ec3be068f46876db4a4f349e1 --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.cpp @@ -0,0 +1,377 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "MxpiStackFrame.h" + +using namespace MxBase; +using namespace MxTools; +using namespace MxPlugins; +using namespace std; + +namespace { + bool isStop_ = true; + std::shared_ptr ObjectFrame = std::make_shared(); +} + +APP_ERROR MxpiStackFrame::Init(std::map> &configParamMap) { + LogInfo << "Begin to initialize MxpiStackFrame(" << pluginName_ << ")."; + std::shared_ptr visionSource = std::static_pointer_cast(configParamMap["visionSource"]); + visionsource_ = *visionSource; + + std::shared_ptr trackSource = std::static_pointer_cast(configParamMap["trackSource"]); + tracksource_ = *trackSource; + + std::shared_ptr framenum = std::static_pointer_cast(configParamMap["frameNum"]); + skipFrameNum_ = *framenum; + + std::shared_ptr timeout = std::static_pointer_cast(configParamMap["timeOut"]); + timeout_ = *timeout; + + std::shared_ptr sleeptime = std::static_pointer_cast(configParamMap["sleepTime"]); + sleepTime_ = *sleeptime; + + isStop_ = false; + // crate CheckFrame thread in Init function + CreateThread(); + LogInfo << "End to initialize MxpiStackFrame(" << pluginName_ << ")."; + return APP_ERR_OK; +} + +APP_ERROR MxpiStackFrame::DeInit() { + LogInfo << "Begin to deinitialize MxpiStackFrame(" << pluginName_ << ")."; + // Block the current thread until the thread identified by *this ends its execution + stopFlag_ = true; + if (thread_->joinable()) { + thread_->join(); + } + LogInfo << "End to deinitialize MxpiStackFrame(" << pluginName_ << ")."; + return APP_ERR_OK; +} + +APP_ERROR MxpiStackFrame::CheckDataSource(MxTools::MxpiMetadataManager &mxpiMetadataManager) { + if (mxpiMetadataManager.GetMetadata(visionsource_) == nullptr) { + LogDebug << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "vision metadata is null. please check" + << "Your property visionSource(" << visionsource_ << ")."; + return APP_ERR_METADATA_IS_NULL; + } + if (mxpiMetadataManager.GetMetadata(tracksource_) == nullptr) { + LogDebug << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "track metadata is null. please check" + << "Your property dataSourceFeature(" << tracksource_ << ")."; + return APP_ERR_METADATA_IS_NULL; + } + return APP_ERR_OK; +} + +void MxpiStackFrame::StackFrame(const std::shared_ptr &visionList, + const std::shared_ptr &trackLetList, + std::shared_ptr &blockingMap) { + for (int32_t i = 0; i < (int32_t) trackLetList->trackletvec_size(); i++) { + auto &trackObject = trackLetList->trackletvec(i); + uint32_t trackId = trackObject.trackid(); + // lost object has no header vector; no object ; + if (trackObject.headervec_size() == 0 || visionList->visionvec_size() == 0) { + continue; + } + // get the visionvec by memberid + int32_t memberId = trackObject.headervec(0).memberid(); + // filter out images not cropped + int32_t j = 0; + for (; j < visionList->visionvec_size(); j++) { + int32_t visionMemberId = visionList->visionvec(j).headervec(0).memberid(); + if (visionMemberId == memberId) { + break; + } + } + if (j >= visionList->visionvec_size()) { + continue; + } + auto vision = visionList->visionvec(memberId); + // convert visiondata to memorydata + MxBase::MemoryData memoryData = ConvertMemoryData(vision.visiondata()); + // add the vision to the blockingMap by trackId + if (blockingMap->count(trackId) == 0) { + // Initialize a new Mxpivisionlist + blockingMap->Insert(trackId, memoryData); + } else { + auto time_visionlist = blockingMap->Get(trackId); + LogInfo << "time_visionlist.second->visionvec_size() " << time_visionlist.second->visionvec_size(); + if (time_visionlist.second->visionvec_size() >= 8) { + continue; + } else { + blockingMap->Update(trackId, memoryData); + } + } + } + visionList->clear_visionvec(); +} + +void MxpiStackFrame::CreateThread() { + thread_.reset(new std::thread(&MxpiStackFrame::WatchThread, this)); +} + +void MxpiStackFrame::WatchThread() { + // set current device context ; same as deviceId in pipeline + DeviceContext context = {}; + context.devId = 0; + APP_ERROR ret = DeviceManager::GetInstance()->SetDevice(context); + if (ret != APP_ERR_OK) { + LogError << "SetDevice failed"; + } + while (!stopFlag_) { + std::this_thread::sleep_for(std::chrono::milliseconds(sleepTime_)); + CheckFrames(ObjectFrame); + } +} + +std::shared_ptr MxpiStackFrame::ConvertVisionList2TensorPackageList( + std::shared_ptr &mxpiVisionList) { + // use make_shared with caution + std::shared_ptr tensorPackageList(new MxpiTensorPackageList, + g_deleteFuncMxpiTensorPackageList); + MxBase::MemoryData concatData = {}; + concatData.deviceId = mxpiVisionList->visionvec(0).visiondata().deviceid(); + concatData.type = (MxBase::MemoryData::MemoryType) 1; + concatData.size = + mxpiVisionList->visionvec_size() * (uint32_t) mxpiVisionList->visionvec(0).visiondata().datasize(); + // malloc new memory + APP_ERROR ret = MxBase::MemoryHelper::MxbsMalloc(concatData); + if (ret != APP_ERR_OK) { + LogError << "MxbsMalloc failed"; + LogError << "concatData.size:" << concatData.size << " concatData.type:" << concatData.type + << "concatData.deviceId:" << concatData.deviceId; + return tensorPackageList; + } + for (int i = 0; i < mxpiVisionList->visionvec_size(); i++) { + MxBase::MemoryData memoryData = ConvertMemoryData(mxpiVisionList->visionvec(i).visiondata()); + // memory copy + MxBase::MemoryData newData = {}; + newData.deviceId = mxpiVisionList->visionvec(i).visiondata().deviceid(); + newData.type = (MxBase::MemoryData::MemoryType) 1; + newData.size = (uint32_t) mxpiVisionList->visionvec(i).visiondata().datasize(); + newData.ptrData = (void *) ((uint8_t *) concatData.ptrData + i * newData.size); + // MxbsMallocAndCopy cannot be used here, cause npu memory leak + ret = MxBase::MemoryHelper::MxbsMemcpy(newData, memoryData, memoryData.size); + if (ret != APP_ERR_OK) { + LogError << "MxbsMemcpy failed"; + MxBase::MemoryHelper::Free(concatData); + return tensorPackageList; + } + } + auto tensorPackage = tensorPackageList->add_tensorpackagevec(); + auto tensorVector = tensorPackage->add_tensorvec(); + tensorVector->set_tensordataptr((uint64) concatData.ptrData); + tensorVector->set_tensordatasize(concatData.size); + tensorVector->set_deviceid(concatData.deviceId); + tensorVector->set_memtype((MxpiMemoryType) concatData.type); + // explicitly specify tenspr shape + tensorVector->add_tensorshape(8); + tensorVector->add_tensorshape(224); + tensorVector->add_tensorshape(224); + tensorVector->add_tensorshape(3); + return tensorPackageList; +} + +void MxpiStackFrame::CheckFrames(std::shared_ptr &blockingMap) { + // Get current timestamp + LogInfo << "Begin to check frames"; + using Time = std::chrono::high_resolution_clock; + using Duration = std::chrono::duration; + using Millisecond = std::chrono::duration>; + auto currentTime = Time::now(); + std::vector keys = blockingMap->Keys(); + for (uint32_t key : keys) { // key <-> trackId + auto TimeMxpiVisionList = blockingMap->Get(key); + if (TimeMxpiVisionList.second == nullptr) { + continue; + } + Duration duration = currentTime - TimeMxpiVisionList.first; + double lastTime = std::chrono::duration_cast(duration).count(); + if (lastTime > timeout_) { + LogInfo << "Object:" << key << " is TimeOut"; + blockingMap->Clear(key); + continue; + } + if (TimeMxpiVisionList.second->visionvec_size() == 8) { + // Add MxpiTensorPackageList to metadata and Send data to downstream plugin + const string metaKey = pluginName_; + auto dstMxpiVisionLitsSptr = TimeMxpiVisionList.second; + auto tensorPackageListPtr = ConvertVisionList2TensorPackageList(dstMxpiVisionLitsSptr); + blockingMap->Clear(key); + // sliding window + std::vector slidingWindow = {}; + uint32_t size = dstMxpiVisionLitsSptr->visionvec_size(); + for (uint32_t i = 0; i < size; i++) { + auto mxpiVision = dstMxpiVisionLitsSptr->visionvec(i); + auto mxpiVisionData = mxpiVision.visiondata(); + if (i < size / 2) { + MxBase::MemoryData memoryData = ConvertMemoryData(mxpiVisionData); + APP_ERROR ret = MemoryHelper::MxbsFree(memoryData); + if (ret != APP_ERR_OK) { + LogError << "MxbsFree failed"; + } + } else { + slidingWindow.emplace_back(mxpiVisionData); + } + } + dstMxpiVisionLitsSptr->clear_visionvec(); + auto mxpiVisionList = ConstructMxpiVisionList(slidingWindow); + blockingMap->Reinsert(key, mxpiVisionList); + // CreateDeviceBuffer; need inputParam + auto *outbuffer = MxpiBufferManager::CreateHostBuffer(inputParam); + MxpiMetadataManager mxpiMetadataManager(*outbuffer); + auto ret = mxpiMetadataManager.AddProtoMetadata(metaKey, static_pointer_cast(tensorPackageListPtr)); + if (ret != APP_ERR_OK) { + LogError << ErrorInfo_.str(); + SendMxpiErrorInfo(*outbuffer, pluginName_, ret, ErrorInfo_.str()); + SendData(0, *outbuffer); + } + LogInfo << "Object:" << key << " has stacked enough frames and begin to send data to downstream plugin"; + SendData(0, *outbuffer); + } + } +} + +std::shared_ptr MxpiStackFrame::ConstructMxpiVisionList( + std::vector &slidingWindow) { + std::shared_ptr dstMxpiVisionListSptr(new MxTools::MxpiVisionList, + MxTools::g_deleteFuncMxpiVisionList); + for (auto iter = slidingWindow.begin(); iter != slidingWindow.end(); iter++) { + MxTools::MxpiVision *dstMxpivision = dstMxpiVisionListSptr->add_visionvec(); + MxTools::MxpiVisionInfo *mxpiVisionInfo = dstMxpivision->mutable_visioninfo(); + mxpiVisionInfo->set_format(1); + mxpiVisionInfo->set_height(224); + mxpiVisionInfo->set_width(224); + mxpiVisionInfo->set_heightaligned(224); + mxpiVisionInfo->set_widthaligned(224); + // set MxpiVisionData by MemoryData + MxTools::MxpiVisionData *mxpiVisionData = dstMxpivision->mutable_visiondata(); + mxpiVisionData->set_dataptr((uint64_t) iter->dataptr()); + mxpiVisionData->set_datasize(iter->datasize()); + mxpiVisionData->set_deviceid(iter->deviceid()); + mxpiVisionData->set_memtype(iter->memtype()); + } + return dstMxpiVisionListSptr; +} + +MxBase::MemoryData MxpiStackFrame::ConvertMemoryData(const MxTools::MxpiVisionData &mxpiVisionData) { + MxBase::MemoryData memoryData = {}; + memoryData.deviceId = mxpiVisionData.deviceid(); + memoryData.type = (MxBase::MemoryData::MemoryType) 1; + memoryData.size = (uint32_t) mxpiVisionData.datasize(); + memoryData.ptrData = (void *) mxpiVisionData.dataptr(); + return memoryData; +} + +APP_ERROR MxpiStackFrame::Process(std::vector &mxpiBuffer) { + LogInfo << "Begin to process MxpiStackFrame(" << elementName_ << ")."; + // Get MxpiVisionList and MxpiTrackletList from mxpibuffer + MxpiBuffer *inputMxpiBuffer = mxpiBuffer[0]; + MxpiMetadataManager mxpiMetadataManager(*inputMxpiBuffer); + ErrorInfo_.str(""); + // check data source + APP_ERROR ret = CheckDataSource(mxpiMetadataManager); + if (ret != APP_ERR_OK) { + SendData(0, *inputMxpiBuffer); + return ret; + } + // Get metadata by key + std::shared_ptr vision_metadata = mxpiMetadataManager.GetMetadata(visionsource_); + std::shared_ptr srcVisionListPtr = std::static_pointer_cast(vision_metadata); + std::shared_ptr track_metadata = mxpiMetadataManager.GetMetadata(tracksource_); + std::shared_ptr srcTrackLetListPtr = std::static_pointer_cast(track_metadata); + // Stacking frames by track ID ; Choose skipFrameNum_ to skip frame + // DestroyBuffer (Get buffer to Host if needed) + if (count % skipFrameNum_ == 0) { + count = 1; + LogInfo << "Begin to stack frames"; + StackFrame(srcVisionListPtr, srcTrackLetListPtr, ObjectFrame); + MxpiBufferManager::DestroyBuffer(inputMxpiBuffer); + } else { + count++; + MxpiBufferManager::DestroyBuffer(inputMxpiBuffer); + } + LogInfo << "End to process MxpiStackFrame(" << elementName_ << ")."; + return APP_ERR_OK; +} + +std::vector> MxpiStackFrame::DefineProperties() { + std::vector> properties; + // the cropped image + auto visionsource = std::make_shared>(ElementProperty{ + STRING, + "visionSource", + "imageSource", + "the name of cropped image source", + "default", "NULL", "NULL" + }); + // the tracklet information + auto tracksource = std::make_shared>(ElementProperty{ + STRING, + "trackSource", + "objectSource", + "tracklelist to get the responding id", + "defalut", "NULL", "NULL" + }); + // skip frame number + auto framenum = std::make_shared>(ElementProperty{ + UINT, + "frameNum", + "frameNum", + "the number of skip frame", + 1, 1, 10 + }); + auto timeout = std::make_shared>(ElementProperty{ + DOUBLE, + "timeOut", + "timeOut", + "Time to discard the frames", + 5000., 500., 10000. + }); + auto sleeptime = std::make_shared>(ElementProperty{ + UINT, + "sleepTime", + "sleepTime", + "The Time CheckFrames thread hangs", + 100, 100, 1000 + }); + properties.push_back(visionsource); + properties.push_back(tracksource); + properties.push_back(framenum); + properties.push_back(timeout); + properties.push_back(sleeptime); + return properties; +} + +MxpiPortInfo MxpiStackFrame::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + // Input: {{Mxpivisionlist}, {MxpiTrackLetList}} + std::vector> value = {{"ANY"}}; + GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; +} + +MxpiPortInfo MxpiStackFrame::DefineOutputPorts() { + MxpiPortInfo outputPortInfo; + // Output: {{MxpiTensorPackageList}} + std::vector> value = {{"ANY"}}; + GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; +} + +namespace { + MX_PLUGIN_GENERATE(MxpiStackFrame) +} + diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.h b/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.h new file mode 100644 index 0000000000000000000000000000000000000000..012f0ae4f3412afbfef58517b97024b4ea78c515 --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.h @@ -0,0 +1,146 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * imitations under the License. + */ + +#ifndef MXPLUGINS_MXPIFRAMESTACK_H +#define MXPLUGINS_MXPIFRAMESTACK_H + +#include +#include +#include "opencv2/opencv.hpp" +#include "MxBase/Log/Log.h" +#include "MxBase/ErrorCode/ErrorCode.h" +#include "MxBase/MemoryHelper/MemoryHelper.h" +#include "MxBase/DvppWrapper/DvppWrapper.h" +#include "MxBase/Tensor/TensorBase/TensorBase.h" +#include "MxStream/StreamManager/MxStreamManager.h" +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/buffer/MxpiBufferManager.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxTools/Proto/MxpiDataTypeDeleter.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "BlockingMap.h" + +/** + * This plugin is to stack frames based on detected objects. +*/ + +namespace MxPlugins { + class MxpiStackFrame : public MxTools::MxPluginBase { + public: + /** + * @description: Init configs. + * @param configParamMap: config. + * @return: Error code. + */ + APP_ERROR Init(std::map> &configParamMap) override; + + /** + * @description: DeInit device. + * @return: Error code. + */ + APP_ERROR DeInit() override; + + /** + * @description: MxpiStackFrame plugin process. + * @param mxpiBuffer: data receive from the previous. + * @return: Error code. + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @description: MxpiStackFrame plugin define properties. + * @return: properties. + */ + static std::vector> DefineProperties(); + + /** + * @api + * @brief Define the number and data type of input ports. + * @return MxTools::MxpiPortInfo. + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + /** + * @api + * @brief Define the number and data type of output ports. + * @return MxTools::MxpiPortInfo. + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + + /** + * @api + * @brief Transfer MxpiVisionList to MxpiTensorPackageList. + * @param shared_ptr . + * @return shared_ptr . + */ + static std::shared_ptr + ConvertVisionList2TensorPackageList(std::shared_ptr &mxpiVisionList); + + /** + * @api + * @brief Reconstruct MxpiVisionList according to MxpiVisionData. + * @param std::vector. + * @return shared_ptr . + */ + static std::shared_ptr + ConstructMxpiVisionList(std::vector &slidingWindow); + + /** + * @api + * @brief Convert MxpiVisionData to MemoryData. + * @param MxTools::MxpiVisionData. + * @return MxBase::MemoryData. + */ + static MxBase::MemoryData ConvertMemoryData(const MxTools::MxpiVisionData &mxpiVisionData); + + private: + /** + * @api + * @brief Check metadata. + * @param MxTools::MxpiMetadataManager. + * @return Error Code. + */ + APP_ERROR CheckDataSource(MxTools::MxpiMetadataManager &mxpiMetadataManager); + + /** + * @api + * @brief Stack frames for each human object. + * @param MxpiVisionList, MxpiTrackLetList, BlockingMap + */ + void StackFrame(const std::shared_ptr &visionList, + const std::shared_ptr &trackLetList, + std::shared_ptr &blockingMap); + // Check Thread function; Check whether a object is 8 frames + void CheckFrames( + std::shared_ptr &blockingMap); + void CreateThread(); // create CheckFrame thread + void WatchThread(); // Watch CheckFrame thread + std::string visionsource_ = ""; // cropped image from crop plugin + std::string tracksource_ = ""; // track result + uint32_t skipFrameNum_ = 0; + uint32_t count = 1; + double timeout_ = 500.; //Millisecond + uint32_t sleepTime_ = 0; + std::ostringstream ErrorInfo_; + std::unique_ptr thread_ = nullptr; + bool stopFlag_ = false; + MxTools::InputParam inputParam; // to create buffer + }; +} +#endif \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/build.sh b/contrib/ActionRecognition/plugins/MxpiStackFrame/build.sh new file mode 100644 index 0000000000000000000000000000000000000000..aa342ac93c13a536628dc26c88de95b3f0182a17 --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/build.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.mitations under the License. + +set -e + +current_folder="$( cd "$(dirname "$0")" ;pwd -P )" + +function build_plugin() { + build_path=$current_folder/build + if [ -d "$build_path" ]; then + rm -rf $build_path + else + echo "file $build_path is not exist." + fi + mkdir -p $build_path + cd $build_path + cmake .. + make -j + if [ $? -ne 0 ]; then + echo "Build Failed" + exit -1 + fi + cd .. + exit 0 +} + +build_plugin +exit 0 \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginAlone/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginAlone/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..1849e36805e0393229d577a8e729cff646788e7a --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginAlone/CMakeLists.txt @@ -0,0 +1,28 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginalone) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginalone") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${MX_SDK_HOME}/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.cpp b/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.cpp new file mode 100644 index 0000000000000000000000000000000000000000..bd3b50ed943c6fde13307177458d481f9e3d5e87 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.cpp @@ -0,0 +1,236 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + +#include "MxBase/Log/Log.h" +#include "PluginAlone.h" + +using namespace MxPlugins; +using namespace MxTools; +using namespace std; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins { + APP_ERROR PluginAlone::Init(std::map> &configParamMap) { + LogInfo << "PluginAlone::Init start."; + APP_ERROR ret = APP_ERR_OK; + // Get the property values by key + // Data from track plugin + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + std::shared_ptr dataSourceDetection = + std::static_pointer_cast(configParamMap["dataSourceDetection"]); + detectionsource_ = *dataSourceDetection.get(); + // Description message + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + // Configuration parameter + confthres_ = *std::static_pointer_cast(configParamMap["detectThresh"]); + confratio_ = *std::static_pointer_cast(configParamMap["detectRatio"]); + confsleep_ = *std::static_pointer_cast(configParamMap["detectSleep"]); + + return APP_ERR_OK; + } + + APP_ERROR PluginAlone::DeInit() { + LogInfo << "PluginAlone::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginAlone::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginAlone::Process(std::vector &mxpiBuffer) { + LogInfo << "PluginAlone::Process start"; + MxpiBuffer *buffer = mxpiBuffer[0]; + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginAlone process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginAlone process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + if (metadata == nullptr || Detect == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + // Data processing + if (sleeptime == 0) { + LogInfo << "PluginAlone start"; + int alarm = calculate(data_queue, confthres_, confratio_, srcTrackLetListSptr, srcObjectListSptr); + if (alarm == 1) { + LogInfo << "Alarm alone"; + alarm_count++; + LogInfo << "Alarmed " << alarm_count << " times"; + sleeptime = confsleep_; + data_queue.clear(); + // Send the data to downstream plugin + std::shared_ptr result = std::make_shared(); + result->set_attrname("Alarm alone"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginAlone add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + } + } else { + LogInfo << "PluginAlone start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime--; + } + LogInfo << "PluginAlone end"; + // Send the data to downstream plugin + SendData(0, *buffer); + LogInfo << "PluginAlone::Process end"; + return APP_ERR_OK; + } + + int PluginAlone::calculate(std::vector &data_queue, int confthres_, float confratio_, + std::shared_ptr srcTrackLetListSptr, + std::shared_ptr srcObjectListSptr) { + uint32_t count = 0; + uint32_t thresh_ = (int) (confthres_ * confratio_); + // update data + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0) { + continue; + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if (detectObject.classvec(0).classid() != 0) { + continue; + } + count++; + } + uint32_t result_present = 0; + uint32_t result_total = 0; + if (count == 1) { + result_present = 1; + } + int alarm = 0; + if (data_queue.size() == confthres_) { + for (int i = confthres_ - 1; i >= 0; i--) { + if (data_queue[i] == 1) { + result_total++; + } + if (i != confthres_ - 1) { + data_queue[i + 1] = data_queue[i]; + } + } + data_queue[0] = result_present; + if (result_total >= thresh_) { + alarm = 1; + } + } else { + data_queue.push_back(result_present); + } + return alarm; + } + + std::vector> PluginAlone::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceTrack", "detect", + "the name of previous plugin", "mxpi_motsimplesort0", + "NULL", "NULL"}); + + auto detectsourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceDetection", "name", + "the name of previous plugin", "mxpi_objectpostprocessor0", + "NULL", "NULL"}); + + auto threshProSptr = + std::make_shared>(ElementProperty{ + UINT, "detectThresh", "thresh", + "the number of frame when judging", + 100, 0, 1000}); + + auto ratioProSptr = + std::make_shared>(ElementProperty{ + FLOAT, "detectRatio", "ratio", + "the ratio of judging", + 0.8, 0.0, 1.0}); + + auto sleepProSptr = + std::make_shared>(ElementProperty{ + UINT, "detectSleep", "sleep", + "the time of stop detection", + 8, 0, 300}); + + auto descriptionMessageProSptr = + std::make_shared>(ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginAlone", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(threshProSptr); + properties.push_back(ratioProSptr); + properties.push_back(sleepProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + + // Register the Sample plugin through macro + MxpiPortInfo PluginAlone::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; +} + +namespace { + MX_PLUGIN_GENERATE(PluginAlone) +} diff --git a/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.h b/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.h new file mode 100644 index 0000000000000000000000000000000000000000..6d003540bad7288739102f4cc5a38ff687a649c7 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.h @@ -0,0 +1,103 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + +#ifndef SDKMEMORY_PLUGINALONE_H +#define SDKMEMORY_PLUGINALONE_H + +#include "MxBase/ErrorCode/ErrorCode.h" +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" + +/** +* @api +* @brief Definition of PluginAlone class. +*/ + +namespace MxPlugins { + class PluginAlone : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map> &configParamMap) override; + + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + /** + * @api + * @brief Define the input ports. + * @return MxpiPortInfo + */ + static int calculate(std::vector &data_queue, int confthres_, float confratio_, + std::shared_ptr srcTrackLetListSptr, + std::shared_ptr srcObjectListSptr); + /** + * @api + * @brief Data processing. + * @return int + */ + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer &buffer, + const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + int confthres_; + float confratio_; + int confsleep_; + int sleeptime = 0; + std::vector data_queue; + int alarm_count = 0; + }; +} +#endif // SDKMEMORY_PLUGINALONE_H \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginAlone/build.sh b/contrib/ActionRecognition/plugins/PluginAlone/build.sh new file mode 100644 index 0000000000000000000000000000000000000000..aa342ac93c13a536628dc26c88de95b3f0182a17 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginAlone/build.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.mitations under the License. + +set -e + +current_folder="$( cd "$(dirname "$0")" ;pwd -P )" + +function build_plugin() { + build_path=$current_folder/build + if [ -d "$build_path" ]; then + rm -rf $build_path + else + echo "file $build_path is not exist." + fi + mkdir -p $build_path + cd $build_path + cmake .. + make -j + if [ $? -ne 0 ]; then + echo "Build Failed" + exit -1 + fi + cd .. + exit 0 +} + +build_plugin +exit 0 \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginClimb/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginClimb/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..78d2ddb011eb565062c34f2665407c94f47fb1bc --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginClimb/CMakeLists.txt @@ -0,0 +1,33 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginclimb) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginclimb") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${MX_SDK_HOME}/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) +include_directories(${MX_SDK_HOME}/opensource/include/opencv4) + + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) +message(${TARGET_LIBRARY}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) +target_link_libraries(${TARGET_LIBRARY} opencv_world) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.cpp b/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.cpp new file mode 100644 index 0000000000000000000000000000000000000000..562d514ab66515de4dc2a5d262fec84e85c19ee4 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.cpp @@ -0,0 +1,312 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + +#include "math.h" +#include +#include "MxBase/Log/Log.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "PluginClimb.h" + +using namespace MxPlugins; +using namespace MxTools; +using namespace std; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins { + APP_ERROR PluginClimb::Init(std::map> &configParamMap) { + LogInfo << "PluginClimb::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + + std::shared_ptr dataSourceDetection = + std::static_pointer_cast(configParamMap["dataSourceDetection"]); + detectionsource_ = *dataSourceDetection.get(); + + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + + // Configuration parameter + highthresh_ = *std::static_pointer_cast(configParamMap["highThresh"]); + bufferlength_ = *std::static_pointer_cast(configParamMap["bufferLength"]); + ratio_ = *std::static_pointer_cast(configParamMap["detectRatio"]); + detectsleep_ = *std::static_pointer_cast(configParamMap["detectSleep"]); + filepath_ = *std::static_pointer_cast(configParamMap["filePath"]); + + return APP_ERR_OK; + } + + APP_ERROR PluginClimb::DeInit() { + LogInfo << "PluginClimb::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginClimb::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginClimb::Process(std::vector &mxpiBuffer) { + LogInfo << "PluginClimb::Process start"; + + MxpiBuffer *buffer = mxpiBuffer[0]; + + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + + test_framenum++; + + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginClimb process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginClimb process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + + if (metadata == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + + // update data + if (pathflag == 0) { + LogInfo << "highthresh" << highthresh_; + LogInfo << "bufferlength" << bufferlength_; + LogInfo << "detectsleep" << detectsleep_; + // log in the txt of roi + readTxt(filepath_, roi); + LogInfo << "*********************************"; + LogInfo << "log in txt"; + LogInfo << roi; + LogInfo << "*********************************"; + pathflag = 1; + } + if (sleeptime == 0) { + LogInfo << "PluginClimb start"; + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0) { + continue; + // As long as the person who has appeared will have a trackobject. + // Determine whether this id appears in the current frame. + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if (detectObject.classvec(0).classid() != 0) { + continue; + } + int cnt_x = (detectObject.x0() + detectObject.x1()) / 2; // + int cnt_y = (detectObject.y0() + detectObject.y1()) / 2; + if (trackdata.count(trackObject.trackid()) == 0) { + int up_num = 0; + int in_num = 0; + if (cv::pointPolygonTest(roi, cv::Point(cnt_x, cnt_y), false) > 0) { + in_num += 1; + } + trackdata[trackObject.trackid()].push_back(cnt_x); // store cnt_x + trackdata[trackObject.trackid()].push_back(cnt_y); // store cnt_y + trackdata[trackObject.trackid()].push_back(up_num); // store up_num + trackdata[trackObject.trackid()].push_back(in_num); // store in_num + } else { + int up_num = trackdata[trackObject.trackid()][trackdata[trackObject.trackid()].size() - 2]; + int in_num = trackdata[trackObject.trackid()].back(); + if (cv::pointPolygonTest(roi, cv::Point(cnt_x, cnt_y), false) > 0) { + in_num = trackdata[trackObject.trackid()].back() + 1; + } // if the target is in the area, in_num ++ + if (cnt_y > trackdata[trackObject.trackid()][trackdata[trackObject.trackid()].size() - 3]) { + up_num = trackdata[trackObject.trackid()][trackdata[trackObject.trackid()].size() - 2] + 1; + } // if the target is rising, up_num ++ + trackdata[trackObject.trackid()].push_back(cnt_x); // store cnt_x + trackdata[trackObject.trackid()].push_back(cnt_y); // store cnt_y + trackdata[trackObject.trackid()].push_back(up_num); // store up_num + trackdata[trackObject.trackid()].push_back(in_num); // store in_num + if (trackdata[trackObject.trackid()].size() == 4 * bufferlength_) { + if (trackdata[trackObject.trackid()][4 * bufferlength_ - 2] > (int) (bufferlength_ * ratio_) + && + (trackdata[trackObject.trackid()][4 * bufferlength_ - 1] > (int) (bufferlength_ * ratio_)) + && trackdata[trackObject.trackid()][4 * bufferlength_ - 3] - + trackdata[trackObject.trackid()][1] > highthresh_) { + LogInfo << "Alarm Climb Up"; + alarm_count++; + LogInfo << "Alarmed " << alarm_count << " times"; + trackdata.clear(); + sleeptime = detectsleep_; + // Send the data to downstream plugin + std::shared_ptr result = std::make_shared(); + result->set_attrname("Alarm Climb"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginClimb add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + } else { + // Climbing up judgment failed, delete the last frame of the queue, continue to load data + std::vector tmp(trackdata[trackObject.trackid()].begin() + 4, + trackdata[trackObject.trackid()].end()); + trackdata[trackObject.trackid()] = tmp; + } + } + } + } + } else { + LogInfo << "PluginClimb start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime--; + } + LogInfo << "PluginClimb end"; + + SendData(0, *buffer); + + LogInfo << "PluginClimb::Process end"; + return APP_ERR_OK; + } + + std::vector> PluginClimb::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceTrack", "name", "the name of previous plugin", "mxpi_motsimplesort0", "NULL", + "NULL"}); + + auto detectsourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceDetection", "name", "the name of previous plugin", "mxpi_fairmot0", + "NULL", "NULL"}); + + auto highthreshProSptr = // 高度差阈值 + std::make_shared>(ElementProperty{ + UINT, "highThresh", "name", "the name of previous plugin", 8, + 0, 1000}); + + auto bufferlengthProSptr = // 每个id需要的判断数据数量阈值 + std::make_shared>(ElementProperty{ + UINT, "bufferLength", "name", "the name of previous plugin", 10, + 8, 1000}); + + auto ratioProSptr = // 上升/在墙里 比例参数 + std::make_shared>(ElementProperty{ + FLOAT, "detectRatio", "name", "the name of previous plugin", 0.75, + 0, 1}); + + auto sleepProSptr = // 检测后休止时间 + std::make_shared>(ElementProperty{ + UINT, "detectSleep", "name", " ", 8, 0, + 100}); + + auto filepathProSptr = // 路径参数 + std::make_shared>(ElementProperty{ + STRING, "filePath", "name", "the name of previous plugin", "mxpi_fairmot0", + "NULL", "NULL"}); + + auto descriptionMessageProSptr = + std::make_shared>(ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginClimb", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(highthreshProSptr); + properties.push_back(bufferlengthProSptr); + properties.push_back(ratioProSptr); + properties.push_back(sleepProSptr); + properties.push_back(filepathProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + + // Register the Sample plugin through macro + MxpiPortInfo PluginClimb::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; + + MxpiPortInfo PluginClimb::DefineOutputPorts() { + MxpiPortInfo outputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; + } + // log in the txt of roi + + void PluginClimb::readTxt(string file, vector &roi) { + ifstream infile; + infile.open(file); + string str; + while (getline(infile, str)) { + int i = 0; + cv::Point p; + int count = 1; + while (i < str.length()) { + int j = i; + while (j < str.length() && str[j] != ';') { + j++; + } + string s = str.substr(i, j - i); + if (count == 1) { + p.x = atoi(s.c_str()); + count++; + } else if (count == 2) { + p.y = atoi(s.c_str()); + } + i = j + 1; + } + roi.push_back(p); + } + infile.close(); + } +} + +namespace { + MX_PLUGIN_GENERATE(PluginClimb) +} diff --git a/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.h b/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.h new file mode 100644 index 0000000000000000000000000000000000000000..23fb5889bdd621ba6a0590f17939a6b26f5a1d8b --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.h @@ -0,0 +1,107 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#ifndef SDKMEMORY_PLUGINCLIMB_H +#define SDKMEMORY_PLUGINCLIMB_H + +#include +#include "MxBase/ErrorCode/ErrorCode.h" +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" + +/** +* @api +* @brief Definition of PluginClimb class. +*/ + +namespace MxPlugins { + class PluginClimb : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map> &configParamMap) override; + + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + static MxTools::MxpiPortInfo DefineOutputPorts(); + + static void readTxt(std::string file, std::vector &roi); + + static int distance(int x0, int y0, int x1, int y1); + + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer &buffer, + const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + std::map> trackdata; + int confthresh_; + int confframes_; + int confsleep_; + int confhigh_; + int frame_num = 0; + int sleeptime = 0; + int pathflag = 0; + + int test_framenum = 0; + int highthresh_; + int bufferlength_; + float ratio_; + int detectsleep_; + int alarm_count = 0; + std::string filepath_; + std::vector roi; + }; +} +#endif \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginClimb/build.sh b/contrib/ActionRecognition/plugins/PluginClimb/build.sh new file mode 100644 index 0000000000000000000000000000000000000000..aa342ac93c13a536628dc26c88de95b3f0182a17 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginClimb/build.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.mitations under the License. + +set -e + +current_folder="$( cd "$(dirname "$0")" ;pwd -P )" + +function build_plugin() { + build_path=$current_folder/build + if [ -d "$build_path" ]; then + rm -rf $build_path + else + echo "file $build_path is not exist." + fi + mkdir -p $build_path + cd $build_path + cmake .. + make -j + if [ $? -ne 0 ]; then + echo "Build Failed" + exit -1 + fi + cd .. + exit 0 +} + +build_plugin +exit 0 \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginCounter/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginCounter/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..b31f2122237d22f0ab4b1b0c2b45dffc525ea9f7 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginCounter/CMakeLists.txt @@ -0,0 +1,28 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_plugincounter) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_plugincounter") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${MX_SDK_HOME}/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.cpp b/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.cpp new file mode 100644 index 0000000000000000000000000000000000000000..7d58166de28427043bb5812624425d0a8a07d65f --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.cpp @@ -0,0 +1,154 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + +#include "math.h" +#include "MxBase/Log/Log.h" +#include "MxTools/Proto/MxpiDumpData.pb.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "PluginCounter.h" + +using namespace MxPlugins; +using namespace MxTools; +using namespace std; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins { + APP_ERROR PluginCounter::Init(std::map> &configParamMap) { + LogInfo << "PluginCounter::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + + return APP_ERR_OK; + } + + APP_ERROR PluginCounter::DeInit() { + LogInfo << "PluginCounter::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginCounter::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginCounter::Process(std::vector &mxpiBuffer) { + LogInfo << "PluginCounter::Process start"; + MxpiBuffer *buffer = mxpiBuffer[0]; + + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + frame++; + LogInfo << "当前帧号 :" << frame; + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginCounter process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginCounter process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + + if (metadata == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + // Send the data to downstream plugin + std::shared_ptr result = std::make_shared(); + result->set_attrname("PluginCounter"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginCounter add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + // Send the data to downstream plugin + SendData(0, *buffer); + LogInfo << "PluginCounter::Process end"; + return APP_ERR_OK; + } + + std::vector> PluginCounter::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceTrack", "name", "the name of previous plugin", "mxpi_motsimplesort0", "NULL", + "NULL"}); + + auto descriptionMessageProSptr = + std::make_shared>(ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginCounter", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + // Register the Sample plugin through macro + MxpiPortInfo PluginCounter::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; + + MxpiPortInfo PluginCounter::DefineOutputPorts() { + MxpiPortInfo outputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; + } +} +namespace { + MX_PLUGIN_GENERATE(PluginCounter) +} diff --git a/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.h b/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.h new file mode 100644 index 0000000000000000000000000000000000000000..25200b202f964da70781ecd32f47682738002d6b --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.h @@ -0,0 +1,96 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#ifndef SDKMEMORY_PLUGINCOUNTER_H +#define SDKMEMORY_PLUGINCOUNTER_H + +#include "MxBase/ErrorCode/ErrorCode.h" +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" + +/** +* @api +* @brief Definition of PluginCounter class. +*/ + +namespace MxPlugins { + class PluginCounter : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map > &configParamMap) override; + + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector > DefineProperties(); + + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + /** + * @api + * @brief Define the input ports. + * @return MxpiPortInfo + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + /** + * @api + * @brief Define the output ports. + * @return MxpiPortInfo + */ + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer &buffer, + const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + std::map> trackdata; + int frame = 0; + }; +} +#endif // SDKMEMORY_PLUGINCOUNTER_H \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginCounter/build.sh b/contrib/ActionRecognition/plugins/PluginCounter/build.sh new file mode 100644 index 0000000000000000000000000000000000000000..aa342ac93c13a536628dc26c88de95b3f0182a17 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginCounter/build.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.mitations under the License. + +set -e + +current_folder="$( cd "$(dirname "$0")" ;pwd -P )" + +function build_plugin() { + build_path=$current_folder/build + if [ -d "$build_path" ]; then + rm -rf $build_path + else + echo "file $build_path is not exist." + fi + mkdir -p $build_path + cd $build_path + cmake .. + make -j + if [ $? -ne 0 ]; then + echo "Build Failed" + exit -1 + fi + cd .. + exit 0 +} + +build_plugin +exit 0 \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOutOfBed/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginOutOfBed/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..0b4ef89ba468bd4817e43e733029c54fb570f550 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOutOfBed/CMakeLists.txt @@ -0,0 +1,33 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginoutofbed) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginoutofbed") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${MX_SDK_HOME}/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) +include_directories(${MX_SDK_HOME}/opensource/include/opencv4) + + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) +message(${TARGET_LIBRARY}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) +target_link_libraries(${TARGET_LIBRARY} opencv_world) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.cpp b/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.cpp new file mode 100644 index 0000000000000000000000000000000000000000..aaa46cc0c104b22ab1c9dad8e2e4f83a54564df2 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.cpp @@ -0,0 +1,288 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include "MxBase/Log/Log.h" +#include "PluginOutOfBed.h" + +using namespace MxPlugins; +using namespace MxTools; +using namespace std; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins { + APP_ERROR PluginOutOfBed::Init(std::map> &configParamMap) { + LogInfo << "PluginOutOfBed::Init start."; + APP_ERROR ret = APP_ERR_OK; + // Get the property values by key + // Data from track plugin + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + // Data from detect plugin + std::shared_ptr detectsourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceDetect"]); + detectionsource_ = *detectsourcePropSptr.get(); + // Description message + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + // Configuration parameter + confthres_ = *std::static_pointer_cast(configParamMap["detectThresh"]); + confratio_ = *std::static_pointer_cast(configParamMap["detectRatio"]); + confsleep_ = *std::static_pointer_cast(configParamMap["detectSleep"]); + configpath = *std::static_pointer_cast(configParamMap["configPath"]); + return APP_ERR_OK; + } + + APP_ERROR PluginOutOfBed::DeInit() { + LogInfo << "PluginOutOfBed::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginOutOfBed::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginOutOfBed::Process(std::vector &mxpiBuffer) { + MxpiBuffer *buffer = mxpiBuffer[0]; + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + framesnum++; + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginOutOfBed process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "FairMOT_Climb process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + if (metadata == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + bool alarm = OutOfBedProcess(srcTrackLetListSptr, srcObjectListSptr); + LogInfo << "frame:" << framesnum; + LogInfo << "PluginOutOfBed end"; + APP_ERROR ret; + if (alarm) { + std::shared_ptr result = std::make_shared(); + result->set_attrname("Alarm outOfBed"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, + result); + } + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginOutOfBed add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + // Send the data to downstream plugin + MxTools::MxPluginBase::SendData(0, *buffer); + return APP_ERR_OK; + } + + std::vector> PluginOutOfBed::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared> + (ElementProperty{ + STRING, "dataSourceTrack", "name", "the name of previous plugin", + "mxpi_motsimplesort0", "NULL", + "NULL"}); + auto detectsourceProSptr = + std::make_shared> + (ElementProperty{ + STRING, "dataSourceDetect", "name", "the name of previous plugin", + "mxpi_motsimplesort0", "NULL", + "NULL"}); + auto threshProSptr = + std::make_shared> + (ElementProperty{ + UINT, "detectThresh", "name", "123", 8, 0, + 100}); + auto ratioProSptr = + std::make_shared> + (ElementProperty{ + FLOAT, "detectRatio", "name", " ", 1.0, 0.0, + 1.0}); + auto sleepProSptr = + std::make_shared> + (ElementProperty{ + UINT, "detectSleep", "name", " ", 8, 0, + 100}); + auto descriptionMessageProSptr = + std::make_shared> + (ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is FairMOT_Alone", "NULL", "NULL"}); + auto configPathSptr = + std::make_shared> + (ElementProperty{ + STRING, "configPath", "message", "Description mesasge of plugin", + "This is FairMOT_Alone", "NULL", "NULL"}); + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(threshProSptr); + properties.push_back(ratioProSptr); + properties.push_back(sleepProSptr); + properties.push_back(configPathSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + + MxpiPortInfo PluginOutOfBed::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + } + + // Process the txt and get the location of the bed + void PluginOutOfBed::readTxt(std::string file, std::vector &roi) { + ifstream infile; + infile.open(file); + string str; + while (getline(infile, str)) { + int i = 0; + cv::Point p; + int count = 1; + while (i < str.length()) { + int j = i; + while (j < str.length() && str[j] != ';') { + j++; + } + string s = str.substr(i, j - i); + if (count == 1) { + p.x = atoi(s.c_str()); + count++; + } else if (count == 2) { + p.y = atoi(s.c_str()); + } + i = j + 1; + } + roi.push_back(p); + } + infile.close(); + } + + // out of bed + bool PluginOutOfBed::OutOfBed(std::vector queue) { + int length = queue.size(); + int count_H = 0, count_T = 0; + for (int i = 0; i < length / 2; i++) { + if (queue[i]) { + count_H++; + } + } + for (int j = length - 1; j >= length / 2; j--) { + if (!queue[j]) { + count_T++; + } + } + + if (count_H > length * confratio_ && count_T > length * confratio_) { + return 1; + } + return 0; + } + + bool PluginOutOfBed::OutOfBedProcess(std::shared_ptr srcTrackLetListSptr, + std::shared_ptr srcObjectListSptr) { + bool flag = false; + if (sleeptime == 0) { + LogInfo << "PluginOutOfBed start"; + cv::Point2f center; + // read txt and get location of bed + if (pathflag) { + readTxt(configpath, bed); + pathflag = false; + } + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0) { + continue; + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if (detectObject.classvec(0).classid() != 0) { + continue; + } + if (trackdata.count(trackObject.trackid()) == 0) { + center.x = (detectObject.x0() + detectObject.x1()) / 2; + center.y = (detectObject.y0() + detectObject.y1()) / 2; + trackdata[trackObject.trackid()].push_back(cv::pointPolygonTest(bed, center, false) > 0); + } else { + center.x = (detectObject.x0() + detectObject.x1()) / 2; + center.y = (detectObject.y0() + detectObject.y1()) / 2; + trackdata[trackObject.trackid()].push_back(cv::pointPolygonTest(bed, center, false) > 0); + if (trackdata[trackObject.trackid()].size() >= confthres_) { + if (OutOfBed(trackdata[trackObject.trackid()])) { + LogInfo << "Alarm OutOfBed"; + alarmcount++; + LogInfo << "Alarmed" << alarmcount << "times"; + flag = true; + sleeptime = confsleep_; + frames = 0; + trackdata.clear(); + break; + } else { + trackdata[trackObject.trackid()].erase(trackdata[trackObject.trackid()].begin()); + } + } + } + } + frames++; + } else { + LogInfo << "PluginOutOfBed start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime--; + } + return flag; + } + +} +namespace { + MX_PLUGIN_GENERATE(PluginOutOfBed) +} diff --git a/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.h b/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.h new file mode 100644 index 0000000000000000000000000000000000000000..89bce28c23ddfb0b0fd719d6d208838c222e5459 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.h @@ -0,0 +1,122 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef MXPIFAIRMOT_FAIRMOT_OUTOFBED_H +#define MXPIFAIRMOT_FAIRMOT_OUTOFBED_H + +#include +#include +#include +#include "MxBase/ErrorCode/ErrorCode.h" +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" + +/** +* @api +* @brief Definition of PluginOutOfBed class. +*/ + +namespace MxPlugins { + class PluginOutOfBed : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map> &configParamMap) override; + + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + /** + * @api + * @brief read ROI + * @param ROI file path + */ + void readTxt(std::string file, std::vector &roi); + + /** + * @api + * @brief Out of bed detection. + * @param queue + * @return bool + */ + bool OutOfBed(std::vector queue); + + /** + * @api + * @brief Out of bed process. + * @param srcTrackLetListSptr ,srcTrackLetListSptr + * @return bool + */ + bool OutOfBedProcess(std::shared_ptr srcTrackLetListSptr, + std::shared_ptr srcObjectListSptr); + + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer &buffer, const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::string configpath; + std::ostringstream ErrorInfo_; + uint confthres_; + float confratio_; + uint confsleep_; + uint sleeptime = 0; + std::map> trackdata; + std::vector bed; + uint frames; + uint framesnum = 0; + bool pathflag = true; + uint alarmcount = 0; + }; + +} +#endif // MXPIFAIRMOT_FAIRMOT_OUTOFBED_H diff --git a/contrib/ActionRecognition/plugins/PluginOutOfBed/build.sh b/contrib/ActionRecognition/plugins/PluginOutOfBed/build.sh new file mode 100644 index 0000000000000000000000000000000000000000..aa342ac93c13a536628dc26c88de95b3f0182a17 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOutOfBed/build.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.mitations under the License. + +set -e + +current_folder="$( cd "$(dirname "$0")" ;pwd -P )" + +function build_plugin() { + build_path=$current_folder/build + if [ -d "$build_path" ]; then + rm -rf $build_path + else + echo "file $build_path is not exist." + fi + mkdir -p $build_path + cd $build_path + cmake .. + make -j + if [ $? -ne 0 ]; then + echo "Build Failed" + exit -1 + fi + cd .. + exit 0 +} + +build_plugin +exit 0 \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOverSpeed/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginOverSpeed/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..fbe215e5e7386004864c740a75fe0884c2b6279a --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverSpeed/CMakeLists.txt @@ -0,0 +1,28 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginoverspeed) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginoverspeed") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${MX_SDK_HOME}/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.cpp b/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.cpp new file mode 100644 index 0000000000000000000000000000000000000000..0a47f80d5473889c4d59a358503b975b3dd27aaa --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.cpp @@ -0,0 +1,261 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#include "PluginOverSpeed.h" +#include "MxBase/Log/Log.h" +#include "math.h" + +using namespace MxPlugins; +using namespace MxTools; +using namespace MxPlugins; +using namespace std; + +namespace { + const std::string SAMPLE_KEY = "MxpiObjectList"; + const int minframe = 4; + +} + +namespace MxPlugins { + APP_ERROR PluginOverSpeed::Init(std::map> &configParamMap) { + LogInfo << "PluginOverSpeed::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + + std::shared_ptr dataSourceDetection = + std::static_pointer_cast(configParamMap["dataSourceDetection"]); + detectionsource_ = *dataSourceDetection.get(); + + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + + //Configuration parameter + confthresh_ = *std::static_pointer_cast(configParamMap["speedThresh"]); + confframes_ = *std::static_pointer_cast(configParamMap["frames"]); + confsleep_ = *std::static_pointer_cast(configParamMap["detectSleep"]); + + return APP_ERR_OK; + } + + APP_ERROR PluginOverSpeed::DeInit() { + LogInfo << "PluginOverSpeed::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginOverSpeed::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginOverSpeed::Process(std::vector &mxpiBuffer) { + LogInfo << "PluginOverSpeed::Process start"; + MxpiBuffer *buffer = mxpiBuffer[0]; + + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + + frame++; + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginOverSpeed process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginOverSpeed process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + + if (metadata == nullptr || Detect == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + + //data processing + if (sleeptime == 0) { + LogInfo << "PluginOverSpeed start"; + int alarm = calculate(trackdata, confframes_, frame_num, confthresh_, srcTrackLetListSptr, + srcObjectListSptr); + if (alarm == 1) { + LogInfo << "Alarm OverSpeed"; + alarm_count++; + LogInfo << "Alarmed " << alarm_count << " times"; + sleeptime = confsleep_; + + std::shared_ptr result = std::make_shared(); + result->set_attrname("Alarm OverSpeed"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginOverSpeed add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + } + } else { + LogInfo << "PluginOverSpeed start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime--; + } + // LogInfo << "frame : " << frame; + LogInfo << "PluginOverSpeed end"; + // Send the data to downstream plugin + SendData(0, *buffer); + + LogInfo << "PluginOverSpeed::Process end"; + return APP_ERR_OK; + } + + int PluginOverSpeed::calculate(std::map> &trackdata, int confframes_, int &frame_num, + int confthresh_, + std::shared_ptr srcTrackLetListSptr, + std::shared_ptr srcObjectListSptr) { + int alarm = 0; + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0) { + continue; + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if (detectObject.classvec(0).classid() != 0) { + continue; + } + if (trackdata.count(trackObject.trackid()) == 0) { + trackdata[trackObject.trackid()].push_back((int) ((detectObject.x0() + detectObject.x1()) / 2)); + trackdata[trackObject.trackid()].push_back((int) ((detectObject.y0() + detectObject.y1()) / 2)); + trackdata[trackObject.trackid()][2] = 0; + trackdata[trackObject.trackid()][3] = 0; + } else { + trackdata[trackObject.trackid()][2] += 1; + int x0 = trackdata[trackObject.trackid()][0]; + int y0 = trackdata[trackObject.trackid()][1]; + int x1 = (int) ((detectObject.x0() + detectObject.x1()) / 2); + int y1 = (int) ((detectObject.y0() + detectObject.y1()) / 2); + trackdata[trackObject.trackid()][3] += distance(x0, y0, x1, y1); + trackdata[trackObject.trackid()][0] = (int) ((detectObject.x0() + detectObject.x1()) / 2); + trackdata[trackObject.trackid()][1] = (int) ((detectObject.y0() + detectObject.y1()) / 2); + } + } + frame_num++; + //calculate data + if (frame_num == confframes_) { + frame_num = 0; + for (auto iter = trackdata.begin(); iter != trackdata.end(); iter++) { + if (trackdata[iter->first][2] == 0 || trackdata[iter->first][2] < minframe) { + continue; + } + if (((int) (trackdata[iter->first][3] / trackdata[iter->first][2])) > confthresh_) { + alarm = 1; + break; + } + } + trackdata.clear(); + } + return alarm; + } + + std::vector> PluginOverSpeed::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceTrack", "name", "the source of tack data", "mxpi_motsimplesort0", + "NULL", "NULL"}); + + auto detectsourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceDetection", "name", "the source of detect data", "mxpi_objectpostprocessor0", + "NULL", "NULL"}); + + auto threshProSptr = + std::make_shared>(ElementProperty{ + UINT, "speedThresh", "thresh", "the thresh of overspeed", 10, + 1, 1000}); + + auto frameProSptr = + std::make_shared>(ElementProperty{ + UINT, "frames", "frame", "the number of frame when judging", 8, + 8, 100}); + + auto sleepProSptr = + std::make_shared>(ElementProperty{ + UINT, "detectSleep", "sleep", "the time of stop detection", 8, 0, + 300}); + + auto descriptionMessageProSptr = + std::make_shared>(ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginOverSpeed", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(threshProSptr); + properties.push_back(frameProSptr); + properties.push_back(sleepProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + // Register the Sample plugin through macro + + MxpiPortInfo PluginOverSpeed::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; + + MxpiPortInfo PluginOverSpeed::DefineOutputPorts() { + MxpiPortInfo outputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; + } + + int PluginOverSpeed::distance(int x0, int y0, int x1, int y1) { + int distance = int(sqrt((x1 - x0) * (x1 - x0) + (y1 - y0) * (y1 - y0))); + return distance; + } +} +namespace { + MX_PLUGIN_GENERATE(PluginOverSpeed) +} diff --git a/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.h b/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.h new file mode 100644 index 0000000000000000000000000000000000000000..7b58a3257704685f2459838b904b4af1e3a954ed --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.h @@ -0,0 +1,120 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + +#ifndef SDKMEMORY_PLUGINOVERSPEED_H +#define SDKMEMORY_PLUGINOVERSPEED_H + +#include "MxBase/ErrorCode/ErrorCode.h" +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" + +/** +* @api +* @brief Definition of PluginOverSpeed class. +*/ + +namespace MxPlugins { + class PluginOverSpeed : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map> &configParamMap) override; + + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + /** + * @api + * @brief Define the input ports. + * @return MxpiPortInfo + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + + /** + * @api + * @brief Define the output ports. + * @return MxpiPortInfo + */ + static int distance(int x0, int y0, int x1, int y1); + + /** + * @api + * @brief Calculate the distance between two points. + * @param int + * @return int + */ + static int + calculate(std::map> &trackdata, int confframes_, int &frame_num, int confthresh_, + std::shared_ptr srcTrackLetListSptr, + std::shared_ptr srcObjectListSptr); + /** + * @api + * @brief Data processing. + * @return int + */ + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer &buffer, const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + std::map> trackdata; + int confthresh_; + int confframes_; + int confsleep_; + int frame_num = 0; + int sleeptime = 0; + int frame = 0; + int alarm_count = 0; + }; +} +#endif // SDKMEMORY_PLUGINOVERSPEED_H \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOverSpeed/build.sh b/contrib/ActionRecognition/plugins/PluginOverSpeed/build.sh new file mode 100644 index 0000000000000000000000000000000000000000..aa342ac93c13a536628dc26c88de95b3f0182a17 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverSpeed/build.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.mitations under the License. + +set -e + +current_folder="$( cd "$(dirname "$0")" ;pwd -P )" + +function build_plugin() { + build_path=$current_folder/build + if [ -d "$build_path" ]; then + rm -rf $build_path + else + echo "file $build_path is not exist." + fi + mkdir -p $build_path + cd $build_path + cmake .. + make -j + if [ $? -ne 0 ]; then + echo "Build Failed" + exit -1 + fi + cd .. + exit 0 +} + +build_plugin +exit 0 \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOverStay/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginOverStay/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..c6f0906a8d5d94aa187c2699e915c372b771fcfe --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverStay/CMakeLists.txt @@ -0,0 +1,28 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginoverstay) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginoverstay") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${MX_SDK_HOME}/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.cpp b/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.cpp new file mode 100644 index 0000000000000000000000000000000000000000..ca51797e1665813aab146c54e9dd2e18923584c4 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.cpp @@ -0,0 +1,273 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + +#include "math.h" +#include "MxBase/Log/Log.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "PluginOverStay.h" + +using namespace MxPlugins; +using namespace MxTools; +using namespace std; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins { + + APP_ERROR PluginOverStay::Init(std::map> &configParamMap) { + LogInfo << "PluginOverStay::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + + std::shared_ptr dataSourceDetection = + std::static_pointer_cast(configParamMap["dataSourceDetection"]); + detectionsource_ = *dataSourceDetection.get(); + + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + + // Configuration parameter + confthresh_ = *std::static_pointer_cast(configParamMap["stayThresh"]); + confframes_ = *std::static_pointer_cast(configParamMap["frames"]); + confsleep_ = *std::static_pointer_cast(configParamMap["detectSleep"]); + confdistance_ = *std::static_pointer_cast(configParamMap["distanceThresh"]); + confratio_ = *std::static_pointer_cast(configParamMap["detectRatio"]); + + return APP_ERR_OK; + } + + APP_ERROR PluginOverStay::DeInit() { + LogInfo << "PluginOverStay::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginOverStay::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginOverStay::Process(std::vector &mxpiBuffer) { + + LogInfo << "PluginOverStay::Process start"; + MxpiBuffer *buffer = mxpiBuffer[0]; + + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + + frame++; + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginOverStay process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginOverStay process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + + if (metadata == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + + // data processing + if (sleeptime == 0) { + LogInfo << "PluginOverStay start"; + // update data + int alarm = calculate(trackdata, confframes_, frame_num, confthresh_, confratio_, confdistance_, + srcTrackLetListSptr, srcObjectListSptr); + if (alarm == 1) { + LogInfo << "Alarm Overstay"; + alarm_count++; + LogInfo << "Alarmed " << alarm_count << " times"; + sleeptime = confsleep_; + + std::shared_ptr result = std::make_shared(); + result->set_attrname("Alarm Overstay"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginOverStay add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + } + } else { + LogInfo << "PluginOverStay start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime--; + } + LogInfo << "PluginOverStay end"; + // Send the data to downstream plugin + SendData(0, *buffer); + LogInfo << "PluginOverStay::Process end"; + return APP_ERR_OK; + } + + int PluginOverStay::calculate(std::map> &trackdata, int confframes_, int &frame_num, + int confthresh_, int confratio_, int confdistance_, + std::shared_ptr srcTrackLetListSptr, + std::shared_ptr srcObjectListSptr) { + int alarm = 0; + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0) { + continue; + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if (detectObject.classvec(0).classid() != 0) { + continue; + } + if (trackdata.count(trackObject.trackid()) == 0) { + trackdata[trackObject.trackid()].push_back((int) ((detectObject.x0() + detectObject.x1()) / 2)); + trackdata[trackObject.trackid()].push_back((int) ((detectObject.y0() + detectObject.y1()) / 2)); + trackdata[trackObject.trackid()][2] = 1; + } else { + int dis = distance(trackdata[trackObject.trackid()][0], + trackdata[trackObject.trackid()][1], + (int) ((detectObject.x0() + detectObject.x1()) / 2), + (int) ((detectObject.y0() + detectObject.y1()) / 2)); + if (dis < confdistance_) { + trackdata[trackObject.trackid()][2]++; + } else { + trackdata[trackObject.trackid()][2] = 1; + } + trackdata[trackObject.trackid()][0] = (int) ((detectObject.x0() + detectObject.x1()) / 2); + trackdata[trackObject.trackid()][1] = (int) ((detectObject.y0() + detectObject.y1()) / 2); + } + } + frame_num++; + // calculate data + if (frame_num == confframes_) { + frame_num = 0; + for (auto iter = trackdata.begin(); iter != trackdata.end(); iter++) { + if (trackdata[iter->first][2] >= (int) (confratio_ * confthresh_)) { + alarm = 1; + break; + } + } + trackdata.clear(); + } + return alarm; + } + + std::vector> PluginOverStay::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceTrack", "name", "the name of previous plugin", "mxpi_motsimplesort0", "NULL", + "NULL"}); + + auto detectsourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceDetection", "name", "the name of previous plugin", "mxpi_fairmot0", + "NULL", "NULL"}); + + auto threshProSptr = // 逗留时间阈值 + std::make_shared>(ElementProperty{ + UINT, "stayThresh", "name", "the name of previous plugin", 10, + 10, 1000}); + + auto distanceProSptr = // 移动距离阈值 + std::make_shared>(ElementProperty{ + UINT, "distanceThresh", "name", "the name of previous plugin", 6, + 0, 100}); + + auto frameProSptr = // 多少帧检测一次 + std::make_shared>(ElementProperty{ + UINT, "frames", "name", "the name of previous plugin", 8, + 8, 100}); + + auto sleepProSptr = // 检测后休止时间 + std::make_shared>(ElementProperty{ + UINT, "detectSleep", "name", " ", 8, 0, + 100}); + + auto ratioProSptr = // 检出比例 + std::make_shared>(ElementProperty{ + FLOAT, "detectRatio", "name", " ", 0.8, 0.1, + 1.0}); + + auto descriptionMessageProSptr = + std::make_shared>(ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginOverStay", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(threshProSptr); + properties.push_back(frameProSptr); + properties.push_back(sleepProSptr); + properties.push_back(distanceProSptr); + properties.push_back(ratioProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + + MxpiPortInfo PluginOverStay::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; + + MxpiPortInfo PluginOverStay::DefineOutputPorts() { + MxpiPortInfo outputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; + } + + int PluginOverStay::distance(int x0, int y0, int x1, int y1) { + int distance = int(sqrt((x1 - x0) * (x1 - x0) + (y1 - y0) * (y1 - y0))); + return distance; + } +} +namespace { + MX_PLUGIN_GENERATE(PluginOverStay) +} diff --git a/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.h b/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.h new file mode 100644 index 0000000000000000000000000000000000000000..3ec2e7e184c23171aff8fe2a35d33ef6658cd687 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.h @@ -0,0 +1,121 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#ifndef SDKMEMORY_PLUGINOVERSTAY_H +#define SDKMEMORY_PLUGINOVERSTAY_H + +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/ErrorCode/ErrorCode.h" + +/** +* @api +* @brief Definition of PluginOverStay class. +*/ +namespace MxPlugins { + class PluginOverStay : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map> &configParamMap) override; + + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + /** + * @api + * @brief Define the input ports. + * @return MxpiPortInfo + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + + /** + * @api + * @brief Define the output ports. + * @return MxpiPortInfo + */ + static int distance(int x0, int y0, int x1, int y1); + + /** + * @api + * @brief Calculate the distance between two points. + * @param int + * @return int + */ + static int + calculate(std::map> &trackdata, int confframes_, int &frame_num, int confthresh_, + int confratio_, int confdistance_, + std::shared_ptr srcTrackLetListSptr, + std::shared_ptr srcObjectListSptr); + /** + * @api + * @brief Data processing. + * @return int + */ + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer &buffer, const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + std::map> trackdata; + int confthresh_; + int confframes_; + int confsleep_; + int confdistance_; + float confratio_; + int frame_num = 0; + int sleeptime = 0; + int alarm_count = 0; + int frame = 0; + }; +} +#endif // SDKMEMORY_PLUGINOVERSTAY_H \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOverStay/build.sh b/contrib/ActionRecognition/plugins/PluginOverStay/build.sh new file mode 100644 index 0000000000000000000000000000000000000000..aa342ac93c13a536628dc26c88de95b3f0182a17 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverStay/build.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.mitations under the License. + +set -e + +current_folder="$( cd "$(dirname "$0")" ;pwd -P )" + +function build_plugin() { + build_path=$current_folder/build + if [ -d "$build_path" ]; then + rm -rf $build_path + else + echo "file $build_path is not exist." + fi + mkdir -p $build_path + cd $build_path + cmake .. + make -j + if [ $? -ne 0 ]; then + echo "Build Failed" + exit -1 + fi + cd .. + exit 0 +} + +build_plugin +exit 0 \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginViolentAction/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginViolentAction/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..dd6ead89b2aced97841f12ce524fa3a0cc250d89 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginViolentAction/CMakeLists.txt @@ -0,0 +1,46 @@ +cmake_minimum_required(VERSION 3.10) +project(Plugin_ViolentAction) + +set(CMAKE_CXX_STANDARD 11) + +set(PLUGIN_NAME "mxpi_violentaction") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options(-fPIC -fstack-protector-all -g -Wl,-z,relro,-z,now,-z -pie -Wall) +add_compile_options(-std=c++11 -Wno-deprecated-declarations) +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") + +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +add_definitions(-DENABLE_DVPP_INTERFACE) + + +set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${MX_SDK_HOME}/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/opencv4) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) + + +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} + mxpidatatype + plugintoolkit + mxbase + streammanager + mindxsdk_protobuf + glib-2.0 + gstreamer-1.0 + gobject-2.0 + gstbase-1.0 + gmodule-2.0 + ) diff --git a/contrib/ActionRecognition/plugins/PluginViolentAction/Plugin_ViolentAction.cpp b/contrib/ActionRecognition/plugins/PluginViolentAction/Plugin_ViolentAction.cpp new file mode 100644 index 0000000000000000000000000000000000000000..28d0770d595a59f80cfe5909aba16b31e8d4a0b0 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginViolentAction/Plugin_ViolentAction.cpp @@ -0,0 +1,210 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "MxBase/Log/Log.h" +#include "Plugin_ViolentAction.h" + +using namespace MxPlugins; +using namespace MxTools; +using namespace MxBase; +using namespace std; + +APP_ERROR PluginViolentAction::Init(std::map> &configParamMap) { + LogInfo << "Begin to initialize PluginViolentAction(" << pluginName_ << ")."; + // Get the property values by key + std::shared_ptr classSource = std::static_pointer_cast(configParamMap["classSource"]); + classSource_ = *classSource; + + std::shared_ptr filePath = std::static_pointer_cast(configParamMap["filePath"]); + filePath_ = *filePath; + + std::shared_ptr detectSleep = std::static_pointer_cast(configParamMap["detectSleep"]); + detectSleep_ = *detectSleep; + + std::shared_ptr actionThreshold = std::static_pointer_cast( + configParamMap["actionThreshold"]); + actionThreshold_ = *actionThreshold; + LogInfo << "End to initialize PluginViolentAction(" << pluginName_ << ")."; + return APP_ERR_OK; +} + +APP_ERROR PluginViolentAction::DeInit() { + LogInfo << "Begin to deinitialize PluginViolentAction(" << pluginName_ << ")."; + LogInfo << "End to deinitialize PluginViolentAction(" << pluginName_ << ")."; + return APP_ERR_OK; +} + +APP_ERROR PluginViolentAction::CheckDataSource(MxTools::MxpiMetadataManager &mxpiMetadataManager) { + if (mxpiMetadataManager.GetMetadata(classSource_) == nullptr) { + LogDebug << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "class metadata is null. please check" + << "Your property classSource(" << classSource_ << ")."; + return APP_ERR_METADATA_IS_NULL; + } + return APP_ERR_OK; +} + +std::shared_ptr +PluginViolentAction::ActionMatch(std::shared_ptr &mxpiClassList) { + // set MxpiAttributeList + MxpiClass mxpiClass = mxpiClassList->classvec(0); + int32_t classId = mxpiClass.classid(); + std::string className = mxpiClass.classname(); + float confidence = mxpiClass.confidence(); + std::shared_ptr mxpiAttributeList = std::make_shared(); + MxpiAttribute *mxpiAttribute = mxpiAttributeList->add_attributevec(); + mxpiAttribute->set_confidence(confidence); + mxpiAttribute->set_attrid(classId); + mxpiAttribute->set_attrname(className); + if (sleepTime_ == 0) { + auto iterator = find(aoi.begin(), aoi.end(), className); + if (iterator != aoi.end()) { + // mark the times of alarms for statistics + if (confidence < actionThreshold_) { + // confidence is too low + alarmInformation = "Low confidence"; + mxpiAttribute->set_attrvalue(alarmInformation); + return mxpiAttributeList; + } else { + alarm_count++; + sleepTime_ = detectSleep_; + alarmInformation = "Alarm Violent Action"; + mxpiAttribute->set_attrvalue(alarmInformation); + return mxpiAttributeList; + } + } else { + // no interested action + alarmInformation = "No Alarm"; + mxpiAttribute->set_attrvalue(alarmInformation); + return mxpiAttributeList; + } + } else { + // alarm sleep + alarmInformation = "Alarmed in a short period of time"; + mxpiAttribute->set_attrvalue(alarmInformation); + sleepTime_--; + return mxpiAttributeList; + } +} + +void PluginViolentAction::ReadTxt(std::string file, std::vector &aoi) { + // read txt file + std::ifstream infile; + infile.open(file.data()); + assert(infile.is_open()); + std::string str; + while (getline(infile, str)) { + // Remove the ending symbol + str.erase(str.end() - 1); + aoi.emplace_back(str); + } + infile.close(); +} + +APP_ERROR PluginViolentAction::Process(std::vector &mxpiBuffer) { + LogInfo << "Begin to process PluginViolentAction(" << elementName_ << ")."; + // Get MxpiClassList from MxpiBuffer + MxpiBuffer *inputMxpiBuffer = mxpiBuffer[0]; + MxpiMetadataManager mxpiMetadataManager(*inputMxpiBuffer); + ErrorInfo_.str(""); + // check data source + APP_ERROR ret = CheckDataSource(mxpiMetadataManager); + if (ret != APP_ERR_OK) { + SendData(0, *inputMxpiBuffer); + return ret; + } + // Get metadata by key + std::shared_ptr class_metadata = mxpiMetadataManager.GetMetadata(classSource_); + std::shared_ptr srcClassListPtr = std::static_pointer_cast(class_metadata); + // update data; Read the action of interest file + // set pathflag to make file IO only once + if (pathflag == 0) { + ReadTxt(filePath_, aoi); + pathflag = 1; + } + // Match the upstream action to the action of interest and alarm. + auto attributeListPtr = ActionMatch(srcClassListPtr); + ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, static_pointer_cast(attributeListPtr)); + if (ret != APP_ERR_OK) { + LogError << ErrorInfo_.str(); + SendMxpiErrorInfo(*inputMxpiBuffer, pluginName_, ret, ErrorInfo_.str()); + SendData(0, *inputMxpiBuffer); + } + // Send the data to downstream plugin + SendData(0, *inputMxpiBuffer); + LogInfo << "End to process PluginViolentAction(" << elementName_ << ")."; + return APP_ERR_OK; +} + +std::vector> PluginViolentAction::DefineProperties() { + std::vector> properties; + // Get the action category from previous plugin + auto classsource = std::make_shared>(ElementProperty{ + STRING, + "classSource", + "labelSource", + "Recognized Action Class", + "default", "NULL", "NULL" + }); + // The action of interest file path + auto filepath = std::make_shared>(ElementProperty{ + STRING, + "filePath", + "Action of interest file path", + "the path of predefined violent action classes file", + "NULL", "NULL", "NULL" + }); + // sleep time after alarm + auto detectsleep = std::make_shared>(ElementProperty{ + UINT, + "detectSleep", + "process sleep time", + "sleep some time to avoid frequent alarms ", + 8, 0, 100 + }); + // threshold + auto actionthreshold = std::make_shared>(ElementProperty{ + FLOAT, + "actionThreshold", + "actionThreshold", + "filter low threshold action", + 0.3, 0.0, 1.0 + }); + properties.push_back(classsource); + properties.push_back(filepath); + properties.push_back(detectsleep); + properties.push_back(actionthreshold); + return properties; +} + +MxpiPortInfo PluginViolentAction::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + // Input: {{MxpiClassList}} + std::vector> value = {{"ANY"}}; + GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; +} + +MxpiPortInfo PluginViolentAction::DefineOutputPorts() { + MxpiPortInfo outputPortInfo; + // Output: {{MxpiAttributeList}} + std::vector> value = {{"ANY"}}; + GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; +} + +namespace { + MX_PLUGIN_GENERATE(PluginViolentAction) +} diff --git a/contrib/ActionRecognition/plugins/PluginViolentAction/Plugin_ViolentAction.h b/contrib/ActionRecognition/plugins/PluginViolentAction/Plugin_ViolentAction.h new file mode 100644 index 0000000000000000000000000000000000000000..a7369977da21d4c352249963b507aaafc0524df5 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginViolentAction/Plugin_ViolentAction.h @@ -0,0 +1,103 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef SDKMEMORY_PLUGINVIOLENTACTTON_H +#define SDKMEMORY_PLUGINVIOLENTACTTON_H + +#include "MxBase/ErrorCode/ErrorCode.h" +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" + + +/** + * This plug is to recognize whether the object's action is a Violent Action and alarm. +*/ + +namespace MxPlugins { + class PluginViolentAction : public MxTools::MxPluginBase { + public: + /** + * @description: Init configs. + * @param configParamMap: config. + * @return: Error code. + */ + APP_ERROR Init(std::map> &configParamMap) override; + + /** + * @description: DeInit device. + * @return: Error code. + */ + APP_ERROR DeInit() override; + + /** + * @description: Plugin_ViolentAction plugin process. + * @param mxpiBuffer: data receive from the previous. + * @return: Error code. + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @description: Plugin_ViolentAction plugin define properties. + * @return: properties. + */ + static std::vector> DefineProperties(); + + /** + * @api + * @brief Define the number and data type of input ports. + * @return MxTools::MxpiPortInfo. + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + /** + * @api + * @brief Define the number and data type of output ports. + * @return MxTools::MxpiPortInfo. + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + + private: + /** + * @api + * @brief Check metadata. + * @param MxTools::MxpiMetadataManager. + * @return Error Code. + */ + APP_ERROR CheckDataSource(MxTools::MxpiMetadataManager &mxpiMetadataManager); + + /** + * @api + * @brief Match the recognized action class to the action of interest and alarm. + * @return MxTools::MxpiAttributeList. + */ + std::shared_ptr ActionMatch(std::shared_ptr &mxpiClassList); + + void ReadTxt(std::string file, std::vector &aoi); // read the action of interest txt file + std::string classSource_ = ""; // previous plugin MxpiClassList + std::string filePath_ = ""; + std::ostringstream ErrorInfo_; // Error Code + std::vector aoi = {}; + std::string alarmInformation = ""; + int pathflag = 0; // flag to mark file IO only once + uint32_t sleepTime_ = 0; + uint32_t detectSleep_ = 0; + int alarm_count = 0; + float actionThreshold_ = 0.0; + }; +} +#endif \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginViolentAction/build.sh b/contrib/ActionRecognition/plugins/PluginViolentAction/build.sh new file mode 100644 index 0000000000000000000000000000000000000000..aa342ac93c13a536628dc26c88de95b3f0182a17 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginViolentAction/build.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License.mitations under the License. + +set -e + +current_folder="$( cd "$(dirname "$0")" ;pwd -P )" + +function build_plugin() { + build_path=$current_folder/build + if [ -d "$build_path" ]; then + rm -rf $build_path + else + echo "file $build_path is not exist." + fi + mkdir -p $build_path + cd $build_path + cmake .. + make -j + if [ $? -ne 0 ]; then + echo "Build Failed" + exit -1 + fi + cd .. + exit 0 +} + +build_plugin +exit 0 \ No newline at end of file diff --git a/contrib/ActionRecognition/run.sh b/contrib/ActionRecognition/run.sh new file mode 100644 index 0000000000000000000000000000000000000000..0a331c7f78334fa01d26df2b75884264647e6bd9 --- /dev/null +++ b/contrib/ActionRecognition/run.sh @@ -0,0 +1,36 @@ +#!/bin/bash +# +# Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +set -e + +CUR_PATH=$(cd "$(dirname "$0")" || { warn "Failed to check path/to/run.sh" ; exit ; } ; pwd) + +# Simple log helper functions +info() { echo -e "\033[1;34m[INFO ][MxStream] $1\033[1;37m" ; } +warn() { echo >&2 -e "\033[1;31m[WARN ][MxStream] $1\033[1;37m" ; } + +export MX_SDK_HOME=${CUR_PATH}/../.. +export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:${LD_LIBRARY_PATH} +export GST_PLUGIN_SCANNER=${MX_SDK_HOME}/opensource/libexec/gstreamer-1.0/gst-plugin-scanner +export GST_PLUGIN_PATH=${MX_SDK_HOME}/opensource/lib/gstreamer-1.0:${MX_SDK_HOME}/lib/plugins +export GST_DEBUG=3 + +#to set PYTHONPATH, import the StreamManagerApi.py +export PYTHONPATH=$PYTHONPATH:${MX_SDK_HOME}/python + +python3.7 main.py +exit 0 \ No newline at end of file diff --git a/contrib/build_all.sh b/contrib/build_all.sh index a72ab33153802240c05a1fafc4901f0d55093135..e4f192f4c2925aca22ea15f04fc433bef3a8607b 100644 --- a/contrib/build_all.sh +++ b/contrib/build_all.sh @@ -16,7 +16,15 @@ set -e current_folder="$( cd "$(dirname "$0")" ;pwd -P )" + SAMPLE_FOLDER=( +/ActionRecognition/plugins/MxpiStackFrame/ +/ActionRecognition/plugins/PluginAlone/ +/ActionRecognition/plugins/PluginClimb/ +/ActionRecognition/plugins/PluginCounter/ +/ActionRecognition/plugins/PluginOutOfBed/ +/ActionRecognition/plugins/PluginOverSpeed/ +/ActionRecognition/plugins/PluginViolentAction/ )