diff --git a/contrib/ActionRecognition/README.md b/contrib/ActionRecognition/README.md new file mode 100644 index 0000000000000000000000000000000000000000..306ceba7afc6eb923e0833ba3bdfc176c9a2b135 --- /dev/null +++ b/contrib/ActionRecognition/README.md @@ -0,0 +1,304 @@ +# ActionRecgnition + +## 1 介绍 + +本开发样例演示动作识别系统 ActionRecgnition，供用户参考。 +本系统基于mxVision SDK进行开发，以昇腾Atlas300卡为主要的硬件平台，主要应用于单人独处、逗留超时、快速移动、剧烈运动、离床检测、攀高检测六种应用场景。 + +1. 单人独处：识别出单个人独处场景后报警。 +2. 逗留超时：识别出单人或多人在区域内长时间逗留的情况并发出警报。 +3. 快速移动：检测出视频中单人或多人行进速度大于阈值的情况，并发出警报。 +4. 剧烈运动：检测到视频流中有剧烈运动并进行报警。 +5. 离床检测：检测出视频中行人离开指定区域的情况并报警。 +6. 攀高检测：检测出行人中心点向上移动的情况，并发出警报。 + +## 2 环境依赖 + +* 支持的硬件形态和操作系统版本 + + | 硬件形态 | 操作系统版本 | + | ------------------------------------- | -------------- | + | x86_64+Atlas 300I 推理卡（型号3010） | Ubuntu 18.04.1 | + | x86_64+Atlas 300I 推理卡 （型号3010） | CentOS 7.6 | + | ARM+Atlas 300I 推理卡 （型号3000） | Ubuntu 18.04.1 | + | ARM+Atlas 300I 推理卡 （型号3000） | CentOS 7.6 | + +* 软件依赖 + + | 软件名称 | 版本 | + | -------- | ----- | + | cmake | 3.5.+ | + | mxVision | 2.0.2 | + | Python | 3.7.5 | + | OpenCV | 4.3.0 | + | gcc | 7.5.0 | + | ffmpeg | 4.3.2 | + +## ３ 代码主要目录介绍 + +本Sample工程名称为Actionrecognition，工程目录如下图1.2所示： + +``` +. +├── pipeline +│   ├── plugin_all.pipeline +│   ├── plugin_alone.pipeline +│   ├── plugin_climb.pipeline +│   ├── plugin_outofbed.pipeline +│   ├── plugin_overspeed.pipeline +│   └── plugin_overstay.pipeline +├── plugins +│   ├── PluginAlone//单人独处插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginAlone.cpp +│   │   └── PluginAlone.h +│   ├── PluginClimb//攀高检测插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginClimb.cpp +│   │   └── PluginClimb.h +│   ├── PluginOutOfBed//离床检测插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginOutOfBed.cpp +│   │   └── PluginOutOfBede.h +│   ├── PluginOverSpeed//快速移动插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginOverSpeed.cpp +│   │   └── PluginOverSpeed.h +│   ├── PluginOverStay//逗留超时插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginOverStay.cpp +│   │   └── PluginOverStay.h +│   ├── PluginCounter//计时插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginCounter.cpp +│   │   └── PluginCounter.h +│   ├── PluginViolentAction//剧烈运动插件 +│   │   ├── CMakeLists.txt +│   │   ├── PluginViolentAction.cpp +│   │   └── PluginViolentAction.h +│   ├── MxpiStackFrame//堆帧插件 +│   │   ├── CMakeLists.txt +│   │   ├── MxpiStackFrame.cpp +│   │   ├── MxpiStackFrame.h +│   │   ├── BlockingMap.cpp +│   │   └── BlockingMap.h +├── models +│   ├── yolov3 +│   └── ecolite +├── data +│ ├── video +│   └── roi +├── main.py +├── run.sh +└── README.md +``` + +## 4 软件方案介绍 + +为了完成上述六种应用场景中的行为识别，系统需要检测出同一目标短时间内状态的变化以及是否存在剧烈运动，因此系统中需要包含目标检测、目标跟踪、动作识别与逻辑后处理。其中目标检测模块选取Yolov3，得到行人候选框；目标跟踪模块使用IOU匹配，关联连续帧中的同一目标。将同一目标在连续帧的区域抠图组成视频序列，输入动作识别模块，模型输出动作类别，判断是否为剧烈运动。逻辑后处理通过判断同一目标在连续帧内的空间位置变化判断难以被定义为运动的其余五种应用场景。系统方案中各模块功能如表1.1 所示。 + +表1.1 系统方案中个模块功能： + +| 序号 | 子系统 | 功能描述 | +| ---- | -------------- | ------------------------------------------------------------ | +| 1 | 初始化配置 | 主要用于初始化资源，如线程数量、共享内存等。 | +| 2 | 视频流 | 从多路IPC相机拉流，并传输入Device进行计算。 | +| 3 | 视频解码 | 通过硬件（DVPP）对视频进行解码，转换为YUV数据进行后续处理。 | +| 4 | 图像预处理 | 在进行基于深度神经网络的图像处理之前，需要将图像缩放到固定的尺寸和格式。 | +| 5 | 目标检测 | 基于深度学习的目标检测算法是该系统的核心模块之一，本方案选用基于Yolov3的目标检测。 | +| 6 | 目标跟踪 | 基于卡尔曼滤波与匈牙利算法的目标跟踪算法是该系统的核心模块之一，本方案选用IOU匹配。 | +| 7 | 图像抠图 | 将同一目标在连续帧所在区域抠图，并组成图像序列，输入动作识别模块。 | +| 8 | 动作识别 | 基于深度学习的动作识别算法是该系统的核心模块之一，本方案选用基于ECOLite的动作识别。 | +| 9 | 单人独处后处理 | 当连续m帧只出现一个目标ID时，则判断为单人独处并报警，如果单人独处报警之前n帧内已经报警过，则不重复报警。 | +| 10 | 快速移动后处理 | 当同一目标在连续m帧中心点平均位移高于阈值v，则判断为快速移动，如果快速移动报警之前n帧内已经报警过，则不重复报警。 | +| 11 | 逗留超时后处理 | 当同一目标在连续m帧中心点平均位移低于阈值v，则判断为快速移动，如果快速移动报警之前n帧内已经报警过，则不重复报警。 | +| 12 | 离床检测后处理 | 当同一目标在连续m帧内从给定区域roi内离开，则判断为离床，如果离床报警之前n帧内已经报警过，则不重复报警。 | +| 13 | 攀高检测后处理 | 当同一目标在连续m帧内从给定区域roi内中心点上升，并且中心点位移大于阈值h，则判断为离床，如果离床报警之前n帧内已经报警过，则不重复报警。 | +| 14 | 剧烈运动后处理 | 动作识别模块输出类别为关注的动作类别时，则判断为剧烈运动，如果剧烈运动之前n帧内已经报警过，则不重复报警。 | + +## 5 准备 + +**步骤1：** 参考安装教程《mxVision 用户指南》安装 mxVision SDK。 + +**步骤2：** 配置 mxVision SDK 环境变量。 + +`export MX_SDK_HOME=${安装路径}/mxVision ` + +注：本例中mxVision SDK安装路径为 /root/work/MindX_SDK。 + +**步骤3：** 创建ActionRecognition根目录，在项目根目录下创建目录models `mkdir models`，分别为yolov3和ecolite创建一个文件夹，将两个模型( 可在华为ModelZoo下载模型 )及各自的配置文件放入文件夹下，创建完成后models文件夹下的目录结构如下： + +``` +.models +├── yolov3 +│   ├── yolov3.om +│   ├── coco.names// label文件 +│   ├── aipp_yolov3.aippconfig// 非必需，模型转换需要 +├── ecolite +│   ├── ecolite.om +│   ├── ucf101.names//label文件 +│   ├── ecolite.aippconfig// 非必需，模型转换需要 +``` + +**步骤4：** 编译程序前提需要先交叉编译好第三方依赖库。 + +**步骤5：**修改代码目录中：```plugins/```下的各个插件中```cmake.txt```文件中的路径为实际安装依赖包的路径。修改代码路径如下所示： + +```cmake +set(MX_SDK_HOME /root/work/mxVision-2.0.2/) +#请修改为实际路径 +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY /root/work/Mind_SDK/mxVision-2.0.2/lib/plugins) +#请修改为实际路径 +``` + +**步骤6**：在插件目录下创建build文件夹，使用cmake命令进行编译，生成.so文件。下面以单人独处插件的编译过程作为范例： + +```bash +## 进入目录 /plugins/plugin_Alone +## 创建build目录 +mkdir build +## 使用cmake命令进行编译 +cmake .. +make -j +``` + +编译好的插件会自动存放到SDK的插件库中，可以直接编入pipiline中使用。 + +**步骤7:** 配置pipeline + +1. 插件参数介绍 + + * PluginAlone + + | 参数名称 | 参数解释 | + | :-------------- | ------------ | + | dataSourceTrack | 跟踪插件名称 | + | detectThresh | 检测帧数 | + | detectRatio | 警报帧阈值 | + | detectSleep | 警报间隔 | + + * PluginClimb + + | 参数名称 | 参数解释 | + | :------------------ | ---------------------- | + | dataSourceTrack | 跟踪插件名称 | + | dataSourceDetection | 目标检测后处理插件名称 | + | detectRatio | 警报帧阈值 | + | filePath | ROI配置txt文件 | + | detectSleep | 警报间隔 | + | bufferLength | 检测帧数窗口大小 | + | highThresh | 高度阈值 | + + * PluginOutOfBed + + | 参数名称 | 参数解释 | + | :------------------ | ---------------------- | + | dataSourceTrack | 跟踪插件名称 | + | dataSourceDetection | 目标检测后处理插件名称 | + | configPath | ROI配置文件 | + | detectThresh | 检测帧数窗口大小 | + | detectSleep | 警报间隔 | + | detectRatio | 警报帧阈值 | + + * PluginOverSpeed + + | 参数名称 | 参数解释 | + | :------------------ | ---------------------- | + | dataSourceTrack | 跟踪插件名称 | + | dataSourceDetection | 目标检测后处理插件名称 | + | speedThresh | 速度阈值 | + | frames | 检测帧数窗口大小 | + | detectSleep | 警报间隔 | + + * PluginOverStay + + | 参数名称 | 参数解释 | + | :------------------ | ---------------------- | + | dataSourceTrack | 跟踪插件名称 | + | dataSourceDetection | 目标检测后处理插件名称 | + | stayThresh | 逗留时间阈值 | + | frames | 检测间隔帧数 | + | distanceThresh | 逗留范围 | + | detectRatio | 警报帧阈值 | + | detectSleep | 警报间隔 | + + * MxpiStackFrame + + | 参数名称 | 参数解释 | + | :----------- | -------------------- | + | visionSource | 抠图插件名称 | + | trackSource | 跟踪插件名称 | + | frameNum | 跳帧间隔（为1不跳） | + | timeOut | 某个目标堆帧超时时间 | + | sleepTime | 检查线程休眠时间 | + | ratio | 警报帧阈值 | + | detectsleep | 警报间隔 | + +2. 配置范例 + + ```bash + ## PluginAlone + "mxpi_pluginalone0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "detectThresh":"8", + "detectRatio":"0.8", + "detectSleep":"270" + }, + "factory": "mxpi_pluginalone", + "next": "mxpi_dataserialize0" + } + ``` + + 根据所需场景，配置pipeline文件，调整路径参数以及插件阈值参数。 + +3. 将所有“rtspUrl”字段值替换为可用的 rtsp 流源地址（需要自行准备可用的视频流，视频流格式为H264）。 + +**步骤8：** 在main.py中，更改pipeline路径。 + +## 6 运行 + +直接运行 + +```bash +./bash run.sh +``` + +## 7 常见问题 +### 7.1 安装opencv失败 +#### 问题描述： + +编译失败： + +``` +undefine reference to 'gst_***' +... +collect2: error: ld returned l exit status +``` + +#### 解决方案： + +```bash +mkdir build +cd build +cmake -DCMAKE_BUILD_TYPE=RELEASE \ +-DENABLE_NEON=OFF \ +-DCMAKE_INSTALL_PREFIX=/usr/local/opencv \ +-DCMAKE_CXX_FLAGS="-march=armv8-a" \ +-DWITH_WEBP=OFF \ +-DWITH_GSTREAMER=OFF \ +-DBUILD_opencv_world=ON .. +make -j8 +``` + +### 7.2 未配置ROI + +#### 问题描述： + +攀高检测与离床检测出现如下报错： +```bash +terminate called after throwing an instance of 'cv::Exception' + what(): OpenCV(4.2.0) /usr1/workspace/MindX_SDK_Multi_DailyBuild/opensource/opensource-scl7/opencv/modules/imgproc/src/geometry.cpp:103: error: (-215:Assertion failed) total >= 0 && (depth == CV_32S || depth == CV_32F) in function 'pointPolygonTest' +``` +#### 解决方案： +在pipeline处没有配置roi区域的txt文件，进行配置即可。 \ No newline at end of file diff --git a/contrib/ActionRecognition/data/roi/Climbup/E001_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E001_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..2712969674270f4fcb7bf8396498a9d9cfb54d96 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E001_points.txt @@ -0,0 +1,5 @@ +620;14 +599;1041 +1080;1050 +1089;22 +620;18 diff --git a/contrib/ActionRecognition/data/roi/Climbup/E002_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E002_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..b4d82cc373a8fc5b1b8c54ce627486d7d7801b6a --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E002_points.txt @@ -0,0 +1,5 @@ +576;15 +560;1045 +1009;1052 +1027;24 +574;19 diff --git a/contrib/ActionRecognition/data/roi/Climbup/E003_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E003_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..67a10eed6350dce842185db1e9815bedc5be117f --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E003_points.txt @@ -0,0 +1,5 @@ +706;10 +700;1035 +1149;1053 +1156;15 +705;13 diff --git a/contrib/ActionRecognition/data/roi/Climbup/E004_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E004_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..838058f813a89a12b8ca6df62ae0e828925396bd --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E004_points.txt @@ -0,0 +1,5 @@ +772;8 +775;1032 +1181;1044 +1190;9 +773;11 diff --git a/contrib/ActionRecognition/data/roi/Climbup/E005_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E005_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..14702dec624c72eb45064ddc4c8dca31d3c5aa04 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E005_points.txt @@ -0,0 +1,5 @@ +779;26 +766;1038 +1167;1050 +1176;37 +777;30 diff --git a/contrib/ActionRecognition/data/roi/Climbup/E006_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E006_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..29690030dd3d96fa9c026cec1deaa0750130be84 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E006_points.txt @@ -0,0 +1,5 @@ +699;94 +720;1038 +1092;1055 +1080;90 +697;96 diff --git a/contrib/ActionRecognition/data/roi/Climbup/E007_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E007_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..94e84e6ef4859955ea16f37da9cf2cb33ef7bc8f --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E007_points.txt @@ -0,0 +1,6 @@ +686;93 +696;1044 +1035;1050 +1090;1051 +1071;85 +685;95 diff --git a/contrib/ActionRecognition/data/roi/Climbup/E008_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E008_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..1a8bd77280d7f741fb552faae4e57f96499e309b --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E008_points.txt @@ -0,0 +1,5 @@ +718;41 +703;933 +1409;938 +1418;45 +717;42 diff --git a/contrib/ActionRecognition/data/roi/Climbup/E009_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E009_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..c5180d48b67a832bfceea7ff1e2d3dfe260ed272 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E009_points.txt @@ -0,0 +1,5 @@ +635;15 +679;948 +1411;941 +1383;14 +635;16 diff --git a/contrib/ActionRecognition/data/roi/Climbup/E010_points.txt b/contrib/ActionRecognition/data/roi/Climbup/E010_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..dd7b942da6e21519dab159e93053af026a2a3ce2 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/Climbup/E010_points.txt @@ -0,0 +1,5 @@ +592;9 +590;1052 +1084;1061 +1103;9 +588;12 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D001_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D001_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..1be6d1e963ebd39969930067599d43049878aaef --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D001_points.txt @@ -0,0 +1,5 @@ +887;5 +881;1006 +1415;1012 +1418;16 +884;11 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D002_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D002_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..d56892d8807c6eeeffc3241412eed707df4c027a --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D002_points.txt @@ -0,0 +1,5 @@ +710;4 +717;1009 +1583;1013 +1558;17 +711;7 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D003_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D003_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..6be9b9338e0adbccf466ab104eba7c09ff86f331 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D003_points.txt @@ -0,0 +1,5 @@ +818;7 +803;1015 +1310;1012 +1317;13 +818;8 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D004_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D004_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..bffca6ac5f5bc05994335f3dad0f8443124e7952 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D004_points.txt @@ -0,0 +1,5 @@ +918;16 +933;1012 +1405;1016 +1411;7 +918;21 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D005_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D005_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..8309e6d3e2fcacc92796dab6a204961360dfce65 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D005_points.txt @@ -0,0 +1,5 @@ +762;5 +762;1013 +1436;1013 +1375;17 +762;11 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D006_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D006_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..d1951d4cd9fe24e6feede9f5b3750475852f8adc --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D006_points.txt @@ -0,0 +1,5 @@ +952;7 +943;1011 +1460;1012 +1479;17 +951;13 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D007_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D007_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..087318abb4b5b11d837cba0699d10a0351de4c16 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D007_points.txt @@ -0,0 +1,5 @@ +899;5 +841;1013 +1460;1009 +1453;25 +897;10 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D008_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D008_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..9cf67e5a7cd3070a112676df37da122dfdd4b8bf --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D008_points.txt @@ -0,0 +1,5 @@ +792;10 +812;1006 +1359;1015 +1402;13 +793;13 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D009_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D009_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..934f87b05803d014ade378a09b251f421fa3747e --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D009_points.txt @@ -0,0 +1,5 @@ +885;5 +842;1059 +1887;1066 +1899;17 +885;5 diff --git a/contrib/ActionRecognition/data/roi/OutOfBed/D010_points.txt b/contrib/ActionRecognition/data/roi/OutOfBed/D010_points.txt new file mode 100644 index 0000000000000000000000000000000000000000..8b958d58112eddc9e74c613c43df61eb9d06c1f4 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/OutOfBed/D010_points.txt @@ -0,0 +1,5 @@ +933;5 +856;1065 +1879;1065 +1896;19 +933;7 diff --git a/contrib/ActionRecognition/data/roi/ViolentAction/aoi.txt b/contrib/ActionRecognition/data/roi/ViolentAction/aoi.txt new file mode 100644 index 0000000000000000000000000000000000000000..d134df338b603048954551662c312130a8841875 --- /dev/null +++ b/contrib/ActionRecognition/data/roi/ViolentAction/aoi.txt @@ -0,0 +1,10 @@ +BodyWeightSquats +Hammering +HandstandPushups +HighJump +JumpRope +LongJump +PullUps +Punch +PushUps +SumoWrestling \ No newline at end of file diff --git a/contrib/ActionRecognition/data/video/Alone/A_001.264 b/contrib/ActionRecognition/data/video/Alone/A_001.264 new file mode 100644 index 0000000000000000000000000000000000000000..1f4ae003dacedf93cf3d9755e9df7ab2c4f9c15d Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_001.264 differ diff --git a/contrib/ActionRecognition/data/video/Alone/A_002.264 b/contrib/ActionRecognition/data/video/Alone/A_002.264 new file mode 100644 index 0000000000000000000000000000000000000000..e4e6f3f81778536a8d883d46a425cf48c931ba39 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_002.264 differ diff --git a/contrib/ActionRecognition/data/video/Alone/A_003.264 b/contrib/ActionRecognition/data/video/Alone/A_003.264 new file mode 100644 index 0000000000000000000000000000000000000000..170e7f86be78100a944c22b6ff61c886165cbd86 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_003.264 differ diff --git a/contrib/ActionRecognition/data/video/Alone/A_004.264 b/contrib/ActionRecognition/data/video/Alone/A_004.264 new file mode 100644 index 0000000000000000000000000000000000000000..582b9d5c1fbf33e1e71303b4747dae4d6a678a48 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_004.264 differ diff --git a/contrib/ActionRecognition/data/video/Alone/A_005.264 b/contrib/ActionRecognition/data/video/Alone/A_005.264 new file mode 100644 index 0000000000000000000000000000000000000000..7cefb85dce26cdb3bd3d799e49babb8129d55192 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_005.264 differ diff --git a/contrib/ActionRecognition/data/video/Alone/A_006.264 b/contrib/ActionRecognition/data/video/Alone/A_006.264 new file mode 100644 index 0000000000000000000000000000000000000000..c609650c8b02eefdb35d68407d57ae6c6428c8c5 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_006.264 differ diff --git a/contrib/ActionRecognition/data/video/Alone/A_007.264 b/contrib/ActionRecognition/data/video/Alone/A_007.264 new file mode 100644 index 0000000000000000000000000000000000000000..d17efed2d74a2e17ad7e5c7e9184f56d7226ec24 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_007.264 differ diff --git a/contrib/ActionRecognition/data/video/Alone/A_008.264 b/contrib/ActionRecognition/data/video/Alone/A_008.264 new file mode 100644 index 0000000000000000000000000000000000000000..507114a968696ae7a587b75864b6c3f0f4481cd0 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_008.264 differ diff --git a/contrib/ActionRecognition/data/video/Alone/A_009.264 b/contrib/ActionRecognition/data/video/Alone/A_009.264 new file mode 100644 index 0000000000000000000000000000000000000000..35c77fa4c4e247e68701c9b1be332acae86664e8 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_009.264 differ diff --git a/contrib/ActionRecognition/data/video/Alone/A_010.264 b/contrib/ActionRecognition/data/video/Alone/A_010.264 new file mode 100644 index 0000000000000000000000000000000000000000..7e40db4fa2f245df5cd55bd7c5bf0d5b597d08c2 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Alone/A_010.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_001.264 b/contrib/ActionRecognition/data/video/Climbup/E_001.264 new file mode 100644 index 0000000000000000000000000000000000000000..f1e9b8af4fb1bf89ba7d06b924c6cd6ca8902259 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_001.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_002.264 b/contrib/ActionRecognition/data/video/Climbup/E_002.264 new file mode 100644 index 0000000000000000000000000000000000000000..d3b19f64c0fa69fa40c76f3a2d50e071f2090030 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_002.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_003.264 b/contrib/ActionRecognition/data/video/Climbup/E_003.264 new file mode 100644 index 0000000000000000000000000000000000000000..8ad346ded46160e5ace00fbbd75a7c76c6efbdf1 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_003.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_004.264 b/contrib/ActionRecognition/data/video/Climbup/E_004.264 new file mode 100644 index 0000000000000000000000000000000000000000..8268cccab497eed9b7a2daad3de0fb177a616d73 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_004.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_005.264 b/contrib/ActionRecognition/data/video/Climbup/E_005.264 new file mode 100644 index 0000000000000000000000000000000000000000..239665906c1335f94d8f7b1001900e5a27169d15 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_005.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_006.264 b/contrib/ActionRecognition/data/video/Climbup/E_006.264 new file mode 100644 index 0000000000000000000000000000000000000000..f2baceb38a2ebfe30fbfc195d572c4cbe46d94dc Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_006.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_007.264 b/contrib/ActionRecognition/data/video/Climbup/E_007.264 new file mode 100644 index 0000000000000000000000000000000000000000..e9c6aebf0c7e19dd526d42cb17ee4e2022326358 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_007.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_008.264 b/contrib/ActionRecognition/data/video/Climbup/E_008.264 new file mode 100644 index 0000000000000000000000000000000000000000..eddbf18e106189571808eb2986f1610dfabdccd6 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_008.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_009.264 b/contrib/ActionRecognition/data/video/Climbup/E_009.264 new file mode 100644 index 0000000000000000000000000000000000000000..e2ef043c8a5a46626e73f9de105fa47a31c1eeaf Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_009.264 differ diff --git a/contrib/ActionRecognition/data/video/Climbup/E_010.264 b/contrib/ActionRecognition/data/video/Climbup/E_010.264 new file mode 100644 index 0000000000000000000000000000000000000000..cb2641e7fb4b1d770ca6b2e101e751e6a4ae1d37 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Climbup/E_010.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_001.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_001.264 new file mode 100644 index 0000000000000000000000000000000000000000..07fef8226395a5a47d7e7d43cb46249e1f20541c Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_001.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_002.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_002.264 new file mode 100644 index 0000000000000000000000000000000000000000..7ecbbdfa7b09b95be54417516be098d169eede3b Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_002.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_003.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_003.264 new file mode 100644 index 0000000000000000000000000000000000000000..cb295534eb62ff2a5f49ac7d2170016292c867dc Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_003.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_004.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_004.264 new file mode 100644 index 0000000000000000000000000000000000000000..c4647efb09e0c307ea2fcdebd6bd81570dae8aa4 Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_004.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_005.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_005.264 new file mode 100644 index 0000000000000000000000000000000000000000..a768321da692bd2672bba158383f09d4410e26d2 Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_005.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_006.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_006.264 new file mode 100644 index 0000000000000000000000000000000000000000..9d74a8a9e90afdab02dc727d23a6e347dde5cb17 Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_006.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_007.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_007.264 new file mode 100644 index 0000000000000000000000000000000000000000..fabeaca72c5eebb3949bf7867c93a8b1c2325874 Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_007.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_008.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_008.264 new file mode 100644 index 0000000000000000000000000000000000000000..cedb2c985a8546badde8cb8349d07c5f34285cfc Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_008.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_009.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_009.264 new file mode 100644 index 0000000000000000000000000000000000000000..a2bb3998eddf9476c0d33786ed978e0e1acc58ce Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_009.264 differ diff --git a/contrib/ActionRecognition/data/video/OutOfBed/D_010.264 b/contrib/ActionRecognition/data/video/OutOfBed/D_010.264 new file mode 100644 index 0000000000000000000000000000000000000000..ea91fb4d7b9a1649e17df95f4942626f4a4c1e64 Binary files /dev/null and b/contrib/ActionRecognition/data/video/OutOfBed/D_010.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_001.264 b/contrib/ActionRecognition/data/video/Over_staying/B_001.264 new file mode 100644 index 0000000000000000000000000000000000000000..caffadce509b3b571e92a378b955122607278e79 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_001.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_002.264 b/contrib/ActionRecognition/data/video/Over_staying/B_002.264 new file mode 100644 index 0000000000000000000000000000000000000000..4b19de36eb2fb41c9302ffc0c18c6d422ad8bf94 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_002.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_003.264 b/contrib/ActionRecognition/data/video/Over_staying/B_003.264 new file mode 100644 index 0000000000000000000000000000000000000000..d7057a603613f478dcee04b724b187a957a7c616 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_003.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_004.264 b/contrib/ActionRecognition/data/video/Over_staying/B_004.264 new file mode 100644 index 0000000000000000000000000000000000000000..79fab4eef343e85116d370b9a7c3e703f7604994 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_004.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_005.264 b/contrib/ActionRecognition/data/video/Over_staying/B_005.264 new file mode 100644 index 0000000000000000000000000000000000000000..36133652c6140d713bb574e863585d7c56ec7b89 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_005.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_006.264 b/contrib/ActionRecognition/data/video/Over_staying/B_006.264 new file mode 100644 index 0000000000000000000000000000000000000000..d0be5eefab5a8da867edc4748a50361c940413c9 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_006.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_007.264 b/contrib/ActionRecognition/data/video/Over_staying/B_007.264 new file mode 100644 index 0000000000000000000000000000000000000000..0f025a90ac8863f16c6b31dbc5605606345c014e Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_007.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_008.264 b/contrib/ActionRecognition/data/video/Over_staying/B_008.264 new file mode 100644 index 0000000000000000000000000000000000000000..ea9483e1f5e5a8d3ddce22f975df888c5209f1ec Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_008.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_009.264 b/contrib/ActionRecognition/data/video/Over_staying/B_009.264 new file mode 100644 index 0000000000000000000000000000000000000000..976377106e39a63170a944807282223ae56b667e Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_009.264 differ diff --git a/contrib/ActionRecognition/data/video/Over_staying/B_010.264 b/contrib/ActionRecognition/data/video/Over_staying/B_010.264 new file mode 100644 index 0000000000000000000000000000000000000000..0d2dafa807b4b6d3b4fd31e15eefdaf5b2637f20 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Over_staying/B_010.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_001.264 b/contrib/ActionRecognition/data/video/Speed_up/C_001.264 new file mode 100644 index 0000000000000000000000000000000000000000..68b4170dab6f71dc454e5be83243fe58203cd88e Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_001.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_002.264 b/contrib/ActionRecognition/data/video/Speed_up/C_002.264 new file mode 100644 index 0000000000000000000000000000000000000000..6237b0c9659fea008f1ef07363cbadfe03ecac9d Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_002.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_003.264 b/contrib/ActionRecognition/data/video/Speed_up/C_003.264 new file mode 100644 index 0000000000000000000000000000000000000000..d5752ee67685344c59fd8f4bcdcd1ad7019acf9d Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_003.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_004.264 b/contrib/ActionRecognition/data/video/Speed_up/C_004.264 new file mode 100644 index 0000000000000000000000000000000000000000..08a616815e751d61e587ab51ea27cd2f351d335e Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_004.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_005.264 b/contrib/ActionRecognition/data/video/Speed_up/C_005.264 new file mode 100644 index 0000000000000000000000000000000000000000..22be958969e0ea0917871a8d6b3da7869fb516c3 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_005.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_006.264 b/contrib/ActionRecognition/data/video/Speed_up/C_006.264 new file mode 100644 index 0000000000000000000000000000000000000000..96b15d1efa01963d5692eca31bbb90d645dd2547 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_006.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_007.264 b/contrib/ActionRecognition/data/video/Speed_up/C_007.264 new file mode 100644 index 0000000000000000000000000000000000000000..36192b92b4b924b36fbeef97e334557cf6fa39b9 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_007.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_008.264 b/contrib/ActionRecognition/data/video/Speed_up/C_008.264 new file mode 100644 index 0000000000000000000000000000000000000000..f5c07e829a958e930ee1c25b3f50c273dc59d443 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_008.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_009.264 b/contrib/ActionRecognition/data/video/Speed_up/C_009.264 new file mode 100644 index 0000000000000000000000000000000000000000..b8f661213585b9986e13776ba66492ecb7ad966e Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_009.264 differ diff --git a/contrib/ActionRecognition/data/video/Speed_up/C_010.264 b/contrib/ActionRecognition/data/video/Speed_up/C_010.264 new file mode 100644 index 0000000000000000000000000000000000000000..193f533ba954500114c34f303104a95b9232f365 Binary files /dev/null and b/contrib/ActionRecognition/data/video/Speed_up/C_010.264 differ diff --git a/contrib/ActionRecognition/main.py b/contrib/ActionRecognition/main.py new file mode 100644 index 0000000000000000000000000000000000000000..73607f516e612144fd87290adf4e875700d1bba6 --- /dev/null +++ b/contrib/ActionRecognition/main.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python +# coding=utf-8 + +# /* +# * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +# * +# * Licensed under the Apache License, Version 2.0 (the "License"); +# * you may not use this file except in compliance with the License. +# * You may obtain a copy of the License at +# * +# * http://www.apache.org/licenses/LICENSE-2.0 +# * +# * Unless required by applicable law or agreed to in writing, software +# * distributed under the License is distributed on an "AS IS" BASIS, +# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# * See the License for the specific language governing permissions and +# * limitations under the License. +# */ + +# import StreamManagerApi.py +from StreamManagerApi import * + +if __name__ == '__main__': + # init stream manager + streamManagerApi = StreamManagerApi() + ret = streamManagerApi.InitManager() + if ret != 0: + print("Failed to init Stream manager, ret=%s" % str(ret)) + exit() + + # create streams by pipeline config file + with open("../pipeline/stayovertime.pipeline", 'rb') as f: + pipelineStr = f.read() + ret = streamManagerApi.CreateMultipleStreams(pipelineStr) + if ret != 0: + print("Failed to create Stream, ret=%s" % str(ret)) + exit() + + # Inputs data to a specified stream based on streamName. + # streamName is the same as the name in pipeline. + streamName = b'detection+tracking' + inPluginId = 0 + + retStr = '' + key = b'mxpi_motsimplesort0' + keyVec = StringVector() + keyVec.push_back(key) + while True: + # Obtain the inference result by specifying streamName and uniqueId. + inferResult = streamManagerApi.GetResult(streamName, 0) + if inferResult == None: + exit() + if inferResult.errorCode != 0: + print("GetResultWithUniqueId error. errorCode=%d, errorMsg=%s" % (inferResult.errorCode, inferResult.data.decode())) + continue + retStr = inferResult.data.decode() + print(retStr) + + # destroy streams + streamManagerApi.DestroyAllStreams() \ No newline at end of file diff --git a/contrib/ActionRecognition/pipeline/plugin_all.pipeline b/contrib/ActionRecognition/pipeline/plugin_all.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..2961a2a3e931c204c3c9435ff4a34520acefcce6 --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_all.pipeline @@ -0,0 +1,136 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp://192.168.88.108:8554/datas/over_staying264/IMG_1142.264" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "../models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "../models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "../models/yolov3/coco.names", + "postProcessLibPath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_plugincounter0" + }, + "mxpi_plugincounter0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0" + }, + "factory": "mxpi_plugincounter", + "next": "mxpi_pluginalone0" + }, + "mxpi_pluginalone0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "detectThresh":"8", + "detectRatio":"0.8", + "detectSleep":"270" + }, + "factory": "mxpi_pluginalone", + "next": "mxpi_pluginoverspeed0" + }, + "mxpi_pluginoverspeed0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "speedThresh": "1", + "frames": "8", + "detectSleep": "270" + }, + "factory": "mxpi_pluginoverspeed", + "next": "mxpi_pluginoverstay0" + }, + "mxpi_pluginoverstay0": { + "props": { + "status": "1", + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "staythresh":"90", + "frames":"90", + "distancethresh":"100", + "ratio":"0.7", + "detectsleep":"90" + }, + "factory": "mxpi_pluginoverstay", + "next": "mxpi_pluginclimb0" + }, + "mxpi_pluginclimb0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "detectRatio": "0.6", + "filePath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/samples/mxVision/pipeline/datas/climb_up/IMG_1153.MP4_points.txt", + "detectSleep": "30", + "bufferLength": "8", + "highThresh": "10" + }, + "factory": "mxpi_pluginclimb", + "next": "mxpi_pluginoutofbed0" + }, + "mxpi_pluginoutofbed0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetect": "mxpi_objectpostprocessor0", + "configPath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/samples/mxVision/pipeline/datas/get_up/IMG_1040.MP4_points.txt", + "detectThresh": "8", + "detectSleep": "15", + "detectRatio": "0.25" + }, + "factory": "mxpi_pluginoutofbed", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginoverstay0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} \ No newline at end of file diff --git a/contrib/ActionRecognition/pipeline/plugin_alone.pipeline b/contrib/ActionRecognition/pipeline/plugin_alone.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..f08664c51a1bbdcd53ddddce157e25c0515be116 --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_alone.pipeline @@ -0,0 +1,79 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp://192.168.88.108:8554/datas/alone264/IMG_0973.264" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "../models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "../models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "../models/yolov3/coco.names", + "postProcessLibPath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginalone0" + }, + "mxpi_pluginalone0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "detectThresh":"8", + "detectRatio":"0.8", + "detectSleep":"270" + }, + "factory": "mxpi_pluginalone", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginalone0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/plugin_climb.pipeline b/contrib/ActionRecognition/pipeline/plugin_climb.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..10293daa261a49f7502d0a6bc16e4bbdf782a871 --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_climb.pipeline @@ -0,0 +1,82 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp://192.168.88.108:8554/datas/climb_up264/IMG_1154.264" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "../models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "../models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "../models/yolov3/coco.names", + "postProcessLibPath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginclimb0" + }, + "mxpi_pluginclimb0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "detectRatio": "0.6", + "filePath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/samples/mxVision/pipeline/datas/climb_up/IMG_1153.MP4_points.txt", + "detectSleep": "30", + "bufferLength": "8", + "highThresh": "10" + }, + "factory": "mxpi_pluginclimb", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginclimb0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/plugin_outofbed.pipeline b/contrib/ActionRecognition/pipeline/plugin_outofbed.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..96c5323d9151ae15eed91435ef61d46b61d0d4a4 --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_outofbed.pipeline @@ -0,0 +1,81 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp://192.168.88.108:8554/datas/get_up264/IMG_1040.264" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "../models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "../models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "../models/yolov3/coco.names", + "postProcessLibPath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginoutofbed0" + }, + "mxpi_pluginoutofbed0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetect": "mxpi_objectpostprocessor0", + "configPath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/samples/mxVision/pipeline/datas/get_up/IMG_1040.MP4_points.txt", + "detectThresh": "8", + "detectSleep": "15", + "detectRatio": "0.25" + }, + "factory": "mxpi_pluginoutofbed", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginoutofbed0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/plugin_overspeed.pipeline b/contrib/ActionRecognition/pipeline/plugin_overspeed.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..64f5b2ef7abbd00ba769ac0021005c57d6381e8c --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_overspeed.pipeline @@ -0,0 +1,80 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp://192.168.88.108:8554/datas/speed_up264/IMG_1141.264" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "../models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "../models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "../models/yolov3/coco.names", + "postProcessLibPath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginoverspeed0" + }, + "mxpi_pluginoverspeed0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "speedThresh": "10", + "frames": "8", + "detectSleep": "270" + }, + "factory": "mxpi_pluginoverspeed", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginoverspeed0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/plugin_overstay.pipeline b/contrib/ActionRecognition/pipeline/plugin_overstay.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..f1cbd3c592abcabcb9bd891cf9c19cdb36f3406a --- /dev/null +++ b/contrib/ActionRecognition/pipeline/plugin_overstay.pipeline @@ -0,0 +1,82 @@ +{ + "detection+tracking": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp://192.168.88.108:8554/datas/over_staying264/IMG_1142.264" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "../models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "../models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "../models/yolov3/coco.names", + "postProcessLibPath": "/home/song4/work/Mind_SDK/mxManufacture-0.2/lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_pluginoverstay0" + }, + "mxpi_pluginoverstay0": { + "props": { + "dataSourceTrack": "mxpi_motsimplesort0", + "dataSourceDetection": "mxpi_objectpostprocessor0", + "stayThresh":"90", + "frames":"90", + "distanceThresh":"100", + "detectRatio":"0.7", + "detectSleep":"90" + }, + "factory": "mxpi_pluginoverstay", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_pluginoverstay0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/pipeline/violentaction.pipeline b/contrib/ActionRecognition/pipeline/violentaction.pipeline new file mode 100644 index 0000000000000000000000000000000000000000..cd07c7b7f73cba70cf00ce0f01880eee6ef92eed --- /dev/null +++ b/contrib/ActionRecognition/pipeline/violentaction.pipeline @@ -0,0 +1,117 @@ +{ + "detection+tracking+action recognition": { + "stream_config": { + "deviceId": "0" + }, + "mxpi_rtspsrc0": { + "props": { + "rtspUrl": "rtsp://192.168.88.108:8554/datas/over_staying264/IMG_1142.264" + }, + "factory": "mxpi_rtspsrc", + "next": "mxpi_videodecoder0" + }, + "mxpi_videodecoder0":{ + "props": { + "vdecChannelId": "0" + }, + "factory": "mxpi_videodecoder", + "next": "mxpi_imageresize0" + }, + "mxpi_imageresize0":{ + "props": { + "dataSource": "mxpi_videodecoder0", + "resizeHeight": "416", + "resizeWidth": "416" + }, + "factory": "mxpi_imageresize", + "next": "mxpi_tensorinfer0" + }, + "mxpi_tensorinfer0":{ + "props": { + "dataSource": "mxpi_imageresize0", + "modelPath": "../models/yolov3/yolov3_tf_bs1_fp16.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_objectpostprocessor0" + }, + "mxpi_objectpostprocessor0": { + "props": { + "dataSource": "mxpi_tensorinfer0", + "funcLanguage":"c++", + "postProcessConfigPath": "../models/yolov3/yolov3_tf_bs1_fp16.cfg", + "labelPath": "../models/yolov3/coco.names", + "postProcessLibPath": "/home/song5/work/MindX_SDK/mxVision-2.0.2/lib/modelpostprocessors/libyolov3postprocess.so" + }, + "factory": "mxpi_objectpostprocessor", + "next": "mxpi_motsimplesort0" + }, + "mxpi_motsimplesort0": { + "props": { + "dataSourceDetection": "mxpi_objectpostprocessor0" + }, + "factory": "mxpi_motsimplesort", + "next": "mxpi_imagecrop0" + }, + "mxpi_imagecrop0": { + "props": { + "dataSource": "mxpi_objectpostprocessor0", + "dataSourceImage": "mxpi_videodecoder0", + "resizeHeight":"224", + "resizeWidth":"224", + "resizeType":"Resizer_Stretch" + }, + "factory": "mxpi_imagecrop", + "next": "mxpi_stackframe0" + }, + "mxpi_stackframe0":{ + "props": { + "visionSource": "mxpi_imagecrop0", + "trackSource": "mxpi_motsimplesort0", + "frameNum":"3" + }, + "factory": "mxpi_stackframe", + "next": "mxpi_tensorinfer1" + }, + "mxpi_tensorinfer1":{ + "props": { + "dataSource": "mxpi_stackframe0", + "skipModelCheck": "1", + "modelPath": "../models/ecolite/ecolite_batch_8_node.om" + }, + "factory": "mxpi_tensorinfer", + "next": "mxpi_classpostprocessor0" + }, + "mxpi_classpostprocessor0":{ + "props": { + "dataSource": "mxpi_tensorinfer1", + "postProcessConfigPath": "../models/ecolite/ecolite_post.cfg", + "labelPath":"../models/ecolite/ucf101.names", + "postProcessLibPath":"../../../lib/modelpostprocessors/libresnet50postprocess.so" + }, + "factory": "mxpi_classpostprocessor", + "next": "mxpi_violentaction0" + }, + "mxpi_violentaction0":{ + "props": { + "classSource": "mxpi_classpostprocessor0", + "filePath": "/home/song5/work/MindX_SDK/mxVision-2.0.2/samples/mxVision/pipeline/datas/action-of-interest/aoi.txt", + "detectSleep": "5" + }, + "factory": "mxpi_violentaction", + "next": "mxpi_dataserialize0" + }, + "mxpi_dataserialize0": { + "props": { + "outputDataKeys": "mxpi_violentaction0" + }, + "factory": "mxpi_dataserialize", + "next": "appsink0" + }, + "appsink0": { + "props": { + "blocksize": "4096000" + }, + "factory": "appsink" + } + } +} diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.cpp b/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.cpp new file mode 100644 index 0000000000000000000000000000000000000000..e600d5d441f32db711a933e230b14bee9619c73b --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.cpp @@ -0,0 +1,121 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "BlockingMap.h" + +namespace MxPlugins { + BlockingMap::BlockingMap() {} + + BlockingMap::~BlockingMap() {} + + void BlockingMap::Insert(const uint32_t &id, MxBase::MemoryData newData) { + // add std::lock_guard + std::lock_guard guard(mtx_); + // get current timestamp + using Time = std::chrono::high_resolution_clock; + auto currentTime = Time::now(); + std::pair> time_MxpiVisionList; + // set MxpiVisionInfo and MxpiVisionData + auto mxpiVisionList = copyList(newData); + time_MxpiVisionList = std::make_pair(currentTime, mxpiVisionList); + blockingMap_[id] = time_MxpiVisionList; + keys_.insert(id); + } + + void BlockingMap::Update(const uint32_t &id, MxBase::MemoryData newData) { + // add std::lock_guard + std::lock_guard guard(mtx_); + std::pair> time_MxpiVisionList = blockingMap_[id]; + auto mxpiVisionList = time_MxpiVisionList.second; + // set MxpiVisionInfo and MxpiVisionData + MxpiVision *dstMxpivision = mxpiVisionList->add_visionvec(); + MxpiVisionInfo *mxpiVisionInfo = dstMxpivision->mutable_visioninfo(); + mxpiVisionInfo->set_format(1); + mxpiVisionInfo->set_height(224); + mxpiVisionInfo->set_width(224); + mxpiVisionInfo->set_heightaligned(224); + mxpiVisionInfo->set_widthaligned(224); + // set MxpiVisionData by MemoryData + MxpiVisionData *mxpiVisionData = dstMxpivision->mutable_visiondata(); + mxpiVisionData->set_dataptr((uint64_t) newData.ptrData); + mxpiVisionData->set_datasize(newData.size); + mxpiVisionData->set_deviceid(newData.deviceId); + mxpiVisionData->set_memtype((MxTools::MxpiMemoryType) newData.type); + //visionlist->pair + time_MxpiVisionList = std::make_pair(time_MxpiVisionList.first, mxpiVisionList); + blockingMap_[id] = time_MxpiVisionList; + } + + std::pair> + BlockingMap::Get(const uint32_t &id) { + // add std::lock_guard + std::lock_guard guard(mtx_); + if (blockingMap_.find(id) != blockingMap_.end()) { + return blockingMap_[id]; + } else { + // If can't find the element, manually assign nullptr + std::pair> empty; + using Time = std::chrono::high_resolution_clock; + auto currentTime = Time::now(); + empty = std::make_pair(currentTime, nullptr); + return empty; + } + } + + void BlockingMap::Clear(const uint32_t &id) { + std::lock_guard guard(mtx_); + blockingMap_.erase(id); + keys_.erase(id); + } + + std::uint32_t BlockingMap::count(const uint32_t &id) { + std::lock_guard guard(mtx_); + return blockingMap_.count(id); + } + + size_t BlockingMap::Size() const { + return blockingMap_.size(); + } + + std::vector BlockingMap::Keys() { + // id<->key + std::vector keys; + std::lock_guard guard(mtx_); + for (auto iter = keys_.begin(); iter != keys_.end(); iter++) { + keys.push_back(*iter); + } + return keys; + } + + std::shared_ptr BlockingMap::copyList(MxBase::MemoryData newData) { + // new shared_ptr MxpiVisionList; + std::shared_ptr dstMxpiVisionListSptr(new MxpiVisionList, g_deleteFuncMxpiVisionList); + MxpiVision *dstMxpivision = dstMxpiVisionListSptr->add_visionvec(); + MxpiVisionInfo *mxpiVisionInfo = dstMxpivision->mutable_visioninfo(); + mxpiVisionInfo->set_format(1); + mxpiVisionInfo->set_height(224); + mxpiVisionInfo->set_width(224); + mxpiVisionInfo->set_heightaligned(224); + mxpiVisionInfo->set_widthaligned(224); + // set MxpiVisionData by MemoryData + MxpiVisionData *mxpiVisionData = dstMxpivision->mutable_visiondata(); + mxpiVisionData->set_dataptr((uint64_t) newData.ptrData); + mxpiVisionData->set_datasize(newData.size); + mxpiVisionData->set_deviceid(newData.deviceId); + mxpiVisionData->set_memtype((MxTools::MxpiMemoryType) newData.type); + return dstMxpiVisionListSptr; + } +} \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.h b/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.h new file mode 100644 index 0000000000000000000000000000000000000000..5fc41177a71c7c17782a7ebcfe6005eb5e26fe34 --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/BlockingMap.h @@ -0,0 +1,75 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef INC_FACE_BLOCKING_MAP_H +#define INC_FACE_BLOCKING_MAP_H + +#include +#include +#include +#include +#include +#include +#include +#include "MxBase/Tensor/TensorBase/TensorBase.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/Log/Log.h" +#include "MxBase/MemoryHelper/MemoryHelper.h" +#include "MxStream/StreamManager/MxStreamManager.h" +#include "MxTools/Proto/MxpiDataTypeDeleter.h" + +/** + * This is a thread safe map. All functions use std::lock_guard. +*/ + +using namespace MxTools; + +namespace MxPlugins { + class BlockingMap { + public: + /** + * @api + * @brief Initialize a new mxpivisionlist for this id, and insert the visiondata passed from upstream into the mxpivisionlist. + * @param trackID, MemoryData + */ + void Insert(const uint32_t &id, const MxBase::MemoryData newData); + + /** + * @api + * @brief Update mxpivisionlist and insert the visiondata for this id. + * @param trackID, MemoryData + */ + void Update(const uint32_t &id, const MxBase::MemoryData newData); + + std::pair> Get(const uint32_t &id); // get std::pair instance + void Clear(const uint32_t &id); // erase map and id + std::vector Keys(); // save tackId into keys set + size_t Size() const; + std::uint32_t count(const uint32_t &id); // return the number of elements of this id + std::shared_ptr copyList(const MxBase::MemoryData newData); // set MxpiVisionInfo and MxpiVisionData + BlockingMap(); + + ~BlockingMap(); + + private: + std::mutex mtx_ = {}; + // > + std::map>> blockingMap_ = {}; + std::set keys_ = {}; + }; +} +#endif + diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/CMakeLists.txt b/contrib/ActionRecognition/plugins/MxpiStackFrame/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..ea431017e5e5d30dc1f7929f6ffe8ff10da897e2 --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/CMakeLists.txt @@ -0,0 +1,45 @@ +cmake_minimum_required(VERSION 3.10) +project(MxpiStackFrame) + +set(CMAKE_CXX_STANDARD 11) + +set(PLUGIN_NAME "mxpi_stackframe") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options(-fPIC -fstack-protector-all -g -Wl,-z,relro,-z,now,-z -pie -Wall) +add_compile_options(-std=c++11 -Wno-deprecated-declarations) +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") + +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +add_definitions(-DENABLE_DVPP_INTERFACE) + + +set(MX_SDK_HOME /home/song5/work/MindX_SDK/mxVision-2.0.2) + +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY "/home/song5/work/MindX_SDK/mxVision-2.0.2/lib/plugins/") + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/opencv4) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) + + +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + + +add_library(${TARGET_LIBRARY} SHARED MxpiStackFrame.cpp MxpiStackFrame.h BlockingMap.cpp BlockingMap.h) +target_link_libraries(${TARGET_LIBRARY} + mxpidatatype + plugintoolkit + mxbase + streammanager + mindxsdk_protobuf + glib-2.0 + gstreamer-1.0 + gobject-2.0 + gstbase-1.0 + gmodule-2.0 + ) diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.cpp b/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.cpp new file mode 100644 index 0000000000000000000000000000000000000000..1a82652d14eb3d8afacc4fe43035d8d45f73ab6b --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.cpp @@ -0,0 +1,324 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "MxpiStackFrame.h" + +using namespace MxBase; +using namespace MxTools; +using namespace MxPlugins; +using namespace std; + +namespace { + bool isStop_ = true; + std::shared_ptr ObjectFrame = std::make_shared(); +} + +APP_ERROR MxpiStackFrame::Init(std::map> &configParamMap) { + LogInfo << "Begin to initialize MxpiStackFrame(" << pluginName_ << ")."; + std::shared_ptr visionSource = std::static_pointer_cast(configParamMap["visionSource"]); + visionsource_ = *visionSource; + + std::shared_ptr trackSource = std::static_pointer_cast(configParamMap["trackSource"]); + tracksource_ = *trackSource; + + std::shared_ptr framenum = std::static_pointer_cast(configParamMap["frameNum"]); + skipFrameNum_ = *framenum; + + std::shared_ptr timeout = std::static_pointer_cast(configParamMap["timeOut"]); + timeout_ = *timeout; + + std::shared_ptr sleeptime = std::static_pointer_cast(configParamMap["sleepTime"]); + sleepTime_ = *sleeptime; + + isStop_ = false; + // crate CheckFrame thread in Init function + CreateThread(); + LogInfo << "End to initialize MxpiStackFrame(" << pluginName_ << ")."; + return APP_ERR_OK; +} + +APP_ERROR MxpiStackFrame::DeInit() { + LogInfo << "Begin to deinitialize MxpiStackFrame(" << pluginName_ << ")."; + // Block the current thread until the thread identified by *this ends its execution + stopFlag_ = true; + if (thread_->joinable()) { + thread_->join(); + } + LogInfo << "End to deinitialize MxpiStackFrame(" << pluginName_ << ")."; + return APP_ERR_OK; +} + +APP_ERROR MxpiStackFrame::CheckDataSource(MxTools::MxpiMetadataManager &mxpiMetadataManager) { + if (mxpiMetadataManager.GetMetadata(visionsource_) == nullptr) { + LogDebug << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "vision metadata is null. please check" + << "Your property visionSource(" << visionsource_ << ")."; + return APP_ERR_METADATA_IS_NULL; + } + if (mxpiMetadataManager.GetMetadata(tracksource_) == nullptr) { + LogDebug << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "track metadata is null. please check" + << "Your property dataSourceFeature(" << tracksource_ << ")."; + return APP_ERR_METADATA_IS_NULL; + } + return APP_ERR_OK; +} + +void MxpiStackFrame::StackFrame(const std::shared_ptr &visionList, + const std::shared_ptr &trackLetList, + std::shared_ptr &blockingMap) { + for (int32_t i = 0; i < (int32_t) trackLetList->trackletvec_size(); i++) { + auto &trackObject = trackLetList->trackletvec(i); + uint32_t trackId = trackObject.trackid(); + // lost object has no header vector; no object ; + if (trackObject.headervec_size() == 0 || visionList->visionvec_size() == 0) { + continue; + } + // get the visionvec by memberid + int32_t memberId = trackObject.headervec(0).memberid(); + auto vision = visionList->visionvec(memberId); + // transfer visiondata to memorydata + MxBase::MemoryData memoryData = {}; + memoryData.deviceId = vision.visiondata().deviceid(); + memoryData.type = (MxBase::MemoryData::MemoryType) 1; + memoryData.size = (uint32_t) vision.visiondata().datasize(); + memoryData.ptrData = (void *) vision.visiondata().dataptr(); + // add the vision to the blockingMap by trackId + if (blockingMap->count(trackId) == 0) { + // Initialize a new Mxpivisionlist + blockingMap->Insert(trackId, memoryData); + } else { + auto time_visionlist = blockingMap->Get(trackId); + LogInfo << "time_visionlist.second->visionvec_size() " << time_visionlist.second->visionvec_size(); + if (time_visionlist.second->visionvec_size() >= 8) { + continue; + } else { + blockingMap->Update(trackId, memoryData); + } + } + } + visionList->clear_visionvec(); +} + +void MxpiStackFrame::CreateThread() { + thread_.reset(new std::thread(&MxpiStackFrame::WatchThread, this)); +} + +void MxpiStackFrame::WatchThread() { + // set current device context ; same as deviceId in pipeline + DeviceContext context = {}; + context.devId = 0; + APP_ERROR ret = DeviceManager::GetInstance()->SetDevice(context); + if (ret != APP_ERR_OK) { + LogError << "SetDevice failed"; + } + while (!stopFlag_) { + std::this_thread::sleep_for(std::chrono::milliseconds(sleepTime_)); + CheckFrames(ObjectFrame); + } +} + +std::shared_ptr MxpiStackFrame::ConvertVisionList2TensorPackageList( + std::shared_ptr &mxpiVisionList) { + // use make_shared with caution + std::shared_ptr tensorPackageList(new MxpiTensorPackageList, + g_deleteFuncMxpiTensorPackageList); + MxBase::MemoryData concatData = {}; + concatData.deviceId = mxpiVisionList->visionvec(0).visiondata().deviceid(); + concatData.type = (MxBase::MemoryData::MemoryType) 1; + concatData.size = + mxpiVisionList->visionvec_size() * (uint32_t) mxpiVisionList->visionvec(0).visiondata().datasize(); + // malloc new memory + APP_ERROR ret = MxBase::MemoryHelper::MxbsMalloc(concatData); + if (ret != APP_ERR_OK) { + LogError << "MxbsMalloc failed"; + LogError << "concatData.size:" << concatData.size << " concatData.type:" << concatData.type + << "concatData.deviceId:" << concatData.deviceId; + return tensorPackageList; + } + for (int i = 0; i < mxpiVisionList->visionvec_size(); i++) { + MxBase::MemoryData memoryData = {}; + memoryData.deviceId = mxpiVisionList->visionvec(i).visiondata().deviceid(); + memoryData.type = (MxBase::MemoryData::MemoryType) 1; + memoryData.size = (uint32_t) mxpiVisionList->visionvec(i).visiondata().datasize(); + memoryData.ptrData = (void *) mxpiVisionList->visionvec(i).visiondata().dataptr(); + // memory copy + MxBase::MemoryData newData = {}; + newData.deviceId = mxpiVisionList->visionvec(i).visiondata().deviceid(); + newData.type = (MxBase::MemoryData::MemoryType) 1; + newData.size = (uint32_t) mxpiVisionList->visionvec(i).visiondata().datasize(); + newData.ptrData = (void *) ((uint8_t *) concatData.ptrData + i * newData.size); + // MxbsMallocAndCopy cannot be used here, cause npu memory leak + ret = MxBase::MemoryHelper::MxbsMemcpy(newData, memoryData, memoryData.size); + if (ret != APP_ERR_OK) { + LogError << "MxbsMemcpy failed"; + MxBase::MemoryHelper::Free(concatData); + return tensorPackageList; + } + } + auto tensorPackage = tensorPackageList->add_tensorpackagevec(); + auto tensorVector = tensorPackage->add_tensorvec(); + tensorVector->set_tensordataptr((uint64) concatData.ptrData); + tensorVector->set_tensordatasize(concatData.size); + tensorVector->set_deviceid(concatData.deviceId); + tensorVector->set_memtype((MxpiMemoryType) concatData.type); + // explicitly specify tenspr shape + tensorVector->add_tensorshape(8); + tensorVector->add_tensorshape(224); + tensorVector->add_tensorshape(224); + tensorVector->add_tensorshape(3); + return tensorPackageList; +} + +void MxpiStackFrame::CheckFrames(std::shared_ptr &blockingMap) { + // Get current timestamp + LogInfo << "Begin to check frames"; + using Time = std::chrono::high_resolution_clock; + using Duration = std::chrono::duration; + using Millisecond = std::chrono::duration>; + auto currentTime = Time::now(); + std::vector keys = blockingMap->Keys(); + for (uint32_t key : keys) { // key <-> trackId + auto TimeMxpiVisionList = blockingMap->Get(key); + if (TimeMxpiVisionList.second == nullptr) { + continue; + } + Duration duration = currentTime - TimeMxpiVisionList.first; + double lastTime = std::chrono::duration_cast(duration).count(); + if (lastTime > timeout_) { + LogInfo << "Object:" << key << " is TimeOut"; + blockingMap->Clear(key); + continue; + } + if (TimeMxpiVisionList.second->visionvec_size() == 8) { + // Add MxpiTensorPackageList to metadata and Send data to downstream plugin + const string metaKey = pluginName_; + auto dstMxpiVisionLitsSptr = TimeMxpiVisionList.second; + auto tensorPackageListPtr = ConvertVisionList2TensorPackageList(dstMxpiVisionLitsSptr); + blockingMap->Clear(key); + // CreateDeviceBuffer; need inputParam + auto *outbuffer = MxpiBufferManager::CreateHostBuffer(inputParam); + MxpiMetadataManager mxpiMetadataManager(*outbuffer); + auto ret = mxpiMetadataManager.AddProtoMetadata(metaKey, static_pointer_cast(tensorPackageListPtr)); + if (ret != APP_ERR_OK) { + LogError << ErrorInfo_.str(); + SendMxpiErrorInfo(*outbuffer, pluginName_, ret, ErrorInfo_.str()); + SendData(0, *outbuffer); + } + LogInfo << "Object:" << key << " has stacked enough frames and begin to send data to downstream plugin"; + SendData(0, *outbuffer); + } + } +} + +APP_ERROR MxpiStackFrame::Process(std::vector &mxpiBuffer) { + LogInfo << "Begin to process MxpiStackFrame(" << elementName_ << ")."; + // Get MxpiVisionList and MxpiTrackletList from mxpibuffer + MxpiBuffer *inputMxpiBuffer = mxpiBuffer[0]; + MxpiMetadataManager mxpiMetadataManager(*inputMxpiBuffer); + ErrorInfo_.str(""); + // check data source + APP_ERROR ret = CheckDataSource(mxpiMetadataManager); + if (ret != APP_ERR_OK) { + SendData(0, *inputMxpiBuffer); + return ret; + } + // Get metadata by key + std::shared_ptr vision_metadata = mxpiMetadataManager.GetMetadata(visionsource_); + std::shared_ptr srcVisionListPtr = std::static_pointer_cast(vision_metadata); + std::shared_ptr track_metadata = mxpiMetadataManager.GetMetadata(tracksource_); + std::shared_ptr srcTrackLetListPtr = std::static_pointer_cast(track_metadata); + // Stacking frames by track ID ; Choose skipFrameNum_ to skip frame + // DestroyBuffer (Get buffer to Host if needed) + if (count % skipFrameNum_ == 0) { + count = 1; + LogInfo << "Begin to stack frames"; + StackFrame(srcVisionListPtr, srcTrackLetListPtr, ObjectFrame); + MxpiBufferManager::DestroyBuffer(inputMxpiBuffer); + } else { + count++; + MxpiBufferManager::DestroyBuffer(inputMxpiBuffer); + } + LogInfo << "End to process MxpiStackFrame(" << elementName_ << ")."; + return APP_ERR_OK; +} + +std::vector> MxpiStackFrame::DefineProperties() { + std::vector> properties; + // the cropped image + auto visionsource = std::make_shared>(ElementProperty{ + STRING, + "visionSource", + "imageSource", + "the name of cropped image source", + "default", "NULL", "NULL" + }); + // the tracklet information + auto tracksource = std::make_shared>(ElementProperty{ + STRING, + "trackSource", + "objectSource", + "tracklelist to get the responding id", + "defalut", "NULL", "NULL" + }); + // skip frame number + auto framenum = std::make_shared>(ElementProperty{ + UINT, + "frameNum", + "frameNum", + "the number of skip frame", + 1, 1, 5 + }); + auto timeout = std::make_shared>(ElementProperty{ + DOUBLE, + "timeOut", + "timeOut", + "Time to discard the frames", + 800., 100., 2000. + }); + auto sleeptime = std::make_shared>(ElementProperty{ + UINT, + "sleepTime", + "sleepTime", + "The Time CheckFrames thread hangs", + 100, 100, 1000 + }); + properties.push_back(visionsource); + properties.push_back(tracksource); + properties.push_back(framenum); + properties.push_back(timeout); + properties.push_back(sleeptime); + return properties; +} + +MxpiPortInfo MxpiStackFrame::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + // Input: {{Mxpivisionlist}, {MxpiTrackLetList}} + std::vector> value = {{"ANY"}}; + GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; +} + +MxpiPortInfo MxpiStackFrame::DefineOutputPorts() { + MxpiPortInfo outputPortInfo; + // Output: {{MxpiTensorPackageList}} + std::vector> value = {{"ANY"}}; + GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; +} + +namespace { + MX_PLUGIN_GENERATE(MxpiStackFrame) +} + diff --git a/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.h b/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.h new file mode 100644 index 0000000000000000000000000000000000000000..b49f84c0aa9bca91a3022d338b89f005244539f7 --- /dev/null +++ b/contrib/ActionRecognition/plugins/MxpiStackFrame/MxpiStackFrame.h @@ -0,0 +1,127 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * imitations under the License. + */ + +#ifndef MXPLUGINS_MXPIFRAMESTACK_H +#define MXPLUGINS_MXPIFRAMESTACK_H + +#include "MxBase/Tensor/TensorBase/TensorBase.h" +#include "MxBase/ErrorCode/ErrorCode.h" +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/buffer/MxpiBufferManager.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include +#include "MxBase/Log/Log.h" +#include "opencv2/opencv.hpp" +#include "MxBase/MemoryHelper/MemoryHelper.h" +#include "MxBase/DvppWrapper/DvppWrapper.h" +#include "MxStream/StreamManager/MxStreamManager.h" +#include "MxTools/Proto/MxpiDataTypeDeleter.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include +#include "BlockingMap.h" + +/** + * This plugin is to stack frames based on detected objects. +*/ + +namespace MxPlugins { + class MxpiStackFrame : public MxTools::MxPluginBase { + public: + /** + * @description: Init configs. + * @param configParamMap: config. + * @return: Error code. + */ + APP_ERROR Init(std::map> &configParamMap) override; + + /** + * @description: DeInit device. + * @return: Error code. + */ + APP_ERROR DeInit() override; + + /** + * @description: MxpiStackFrame plugin process. + * @param mxpiBuffer: data receive from the previous. + * @return: Error code. + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @description: MxpiStackFrame plugin define properties. + * @return: properties. + */ + static std::vector> DefineProperties(); + + /** + * @api + * @brief Define the number and data type of input ports. + * @return MxTools::MxpiPortInfo. + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + /** + * @api + * @brief Define the number and data type of output ports. + * @return MxTools::MxpiPortInfo. + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + + /** + * @api + * @brief Transfer MxpiVisionList to MxpiTensorPackageList. + * @param shared_ptr . + * @return shared_ptr . + */ + static std::shared_ptr + ConvertVisionList2TensorPackageList(std::shared_ptr &mxpiVisionList); + + private: + /** + * @api + * @brief Check metadata. + * @param MxTools::MxpiMetadataManager. + * @return Error Code. + */ + APP_ERROR CheckDataSource(MxTools::MxpiMetadataManager &mxpiMetadataManager); + + /** + * @api + * @brief Stack frames for each human object. + * @param MxpiVisionList, MxpiTrackLetList, BlockingMap + */ + void StackFrame(const std::shared_ptr &visionList, + const std::shared_ptr &trackLetList, + std::shared_ptr &blockingMap); + void CheckFrames(std::shared_ptr &blockingMap); // Check Thread function; Check whether a object is 8 frames + void CreateThread(); // create CheckFrame thread + void WatchThread(); // Watch CheckFrame thread + std::string visionsource_ = ""; // cropped image from crop plugin + std::string tracksource_ = ""; // track result + uint32_t skipFrameNum_ = 0; + uint32_t count = 1; + double timeout_ = 500.; //Millisecond + uint32_t sleepTime_ = 0; + std::ostringstream ErrorInfo_; + std::unique_ptr thread_ = nullptr; + bool stopFlag_ = false; + InputParam inputParam; // to create buffer + }; +} +#endif \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginAlone/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginAlone/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..0124020c83e1f0fb0695da150f9c2fdca6d7a79d --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginAlone/CMakeLists.txt @@ -0,0 +1,28 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginalone) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginalone") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME /home/song5/work/MindX_SDK/mxVision-2.0.2/) +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY /home/song5/work/MindX_SDK/mxVision-2.0.2/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.cpp b/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.cpp new file mode 100644 index 0000000000000000000000000000000000000000..d6c7f8600700c5c1b78a373c81e868615371ceae --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.cpp @@ -0,0 +1,244 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#include "PluginAlone.h" +#include "MxBase/Log/Log.h" +using namespace MxPlugins; +using namespace MxTools; +using namespace std; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} +namespace MxPlugins { + APP_ERROR PluginAlone::Init(std::map > &configParamMap) { + LogInfo << "PluginAlone::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + // Data from track plugin + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + std::shared_ptr dataSourceDetection = + std::static_pointer_cast(configParamMap["dataSourceDetection"]); + detectionsource_ = *dataSourceDetection.get(); + //Description message + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + //Configuration parameter + confthres_ = *std::static_pointer_cast(configParamMap["detectThresh"]); + confratio_ = *std::static_pointer_cast(configParamMap["detectRatio"]); + confsleep_ = *std::static_pointer_cast(configParamMap["detectSleep"]); + + return APP_ERR_OK; + } + + APP_ERROR PluginAlone::DeInit() { + LogInfo << "PluginAlone::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginAlone::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginAlone::Process(std::vector &mxpiBuffer) { + LogInfo << "PluginAlone::Process start"; + MxpiBuffer *buffer = mxpiBuffer[0]; + + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + frame++; + + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginAlone process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginAlone process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + shared_ptr att = mxpiMetadataManager.GetMetadata(("mxpi_plugincounter0")); + + if (metadata == nullptr || Detect == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr mxpiAttribute = std::static_pointer_cast(att); + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + + //Data processing + if (sleeptime == 0) { + LogInfo << "PluginAlone start"; + uint32_t count = 0; + uint32_t thresh_ = (int) (confthres_ * confratio_); + //update data + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0) { + continue; + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if (detectObject.classvec(0).classid() != 0) { + continue; + } + count++; + } + uint32_t result_present = 0; + uint32_t result_total = 0; + uint32_t alarm = 0; + if (count == 1) { + result_present = 1; + } + //calculate data + if (data_queue.size() == confthres_) { + for (int i = confthres_ - 1; i >= 0; i--) { + if (data_queue[i] == 1) { + result_total++; + } + if (i != confthres_ - 1) { + data_queue[i + 1] = data_queue[i]; + } + } + data_queue[0] = result_present; + if (result_total >= thresh_) { + alarm = 1; + } + } else { + data_queue.push_back(result_present); + } + if (alarm == 1) { + LogInfo << "Alarm alone"; + alarm_count++; + LogInfo << "Alarmed" << alarm_count << "times"; + sleeptime = confsleep_; + data_queue.clear(); + + // Send the data to downstream plugin + std::shared_ptr result = std::make_shared(); + result->set_attrname("Alarm alone"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginAlone add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + } + } else { + LogInfo << "PluginAlone start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime--; + } + // LogInfo << "frame : " << frame; + LogInfo << "PluginAlone end"; + // Send the data to downstream plugin + SendData(0, *buffer); + LogInfo << "PluginAlone::Process end"; + return APP_ERR_OK; + } + + std::vector > PluginAlone::DefineProperties() { + // Define an A to store properties + std::vector > properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared < ElementProperty < string >> (ElementProperty < string > { + STRING, "dataSourceTrack", "detect", + "the name of previous plugin", "mxpi_motsimplesort0", + "NULL", "NULL"}); + + auto detectsourceProSptr = + std::make_shared < ElementProperty < string >> (ElementProperty < string > { + STRING, "dataSourceDetection", "name", + "the name of previous plugin", "mxpi_objectpostprocessor0", + "NULL", "NULL"}); + + auto threshProSptr = + std::make_shared < ElementProperty < uint32_t >> (ElementProperty < uint32_t > { + UINT, "detectThresh", "thresh", + "the number of frame when judging", + 100, 0, 1000}); + + auto ratioProSptr = + std::make_shared < ElementProperty < float >> (ElementProperty < float > { + FLOAT, "detectRatio", "ratio", + "the ratio of judging", + 0.8, 0.0, 1.0}); + + auto sleepProSptr = + std::make_shared < ElementProperty < uint32_t >> (ElementProperty < uint32_t > { + UINT, "detectSleep", "sleep", + "the time of stop detection", + 8, 0, 300}); + + // auto previousPluginProSptr = + // std::make_shared>(ElementProperty{ + // STRING, "previousPlugin", "previous", "the name of previous plugin", "mxpi_objectpostprocessor0", + // "NULL", "NULL"}); + + auto descriptionMessageProSptr = + std::make_shared < ElementProperty < string >> (ElementProperty < string > { + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginAlone", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(threshProSptr); + properties.push_back(ratioProSptr); + properties.push_back(sleepProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + + // Register the Sample plugin through macro + MxpiPortInfo PluginAlone::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector > value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; +} + +namespace { + MX_PLUGIN_GENERATE(PluginAlone) +} diff --git a/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.h b/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.h new file mode 100644 index 0000000000000000000000000000000000000000..7507cc1cae65d0d4d169a63b53ff5993d4d6705e --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginAlone/PluginAlone.h @@ -0,0 +1,86 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#ifndef SDKMEMORY_PluginAlone_H +#define SDKMEMORY_PluginAlone_H +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/ErrorCode/ErrorCode.h" +/** +* @api +* @brief Definition of PluginAlone class. +*/ +namespace MxPlugins { + class PluginAlone : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map>& configParamMap) override; + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector& mxpiBuffer) override; + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + /** + * @api + * @brief Define the input ports. + * @return MxpiPortInfo + */ + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer& buffer, + const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + int confthres_; + float confratio_; + int confsleep_; + int sleeptime = 0; + std::vector data_queue; + int frame = 0; + int alarm_count = 0; + }; +} +#endif //SDKMEMORY_PluginAlone_H \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginClimb/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginClimb/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..1c9b9dfbc5ccd9e0407fd7ab4e916b3d1a382573 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginClimb/CMakeLists.txt @@ -0,0 +1,33 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginclimb) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginclimb") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME /home/song5/work/MindX_SDK/mxVision-2.0.2/) +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY /home/song5/work/MindX_SDK/mxVision-2.0.2/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) +include_directories(${MX_SDK_HOME}/opensource/include/opencv4) + + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) +message(${TARGET_LIBRARY}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) +target_link_libraries(${TARGET_LIBRARY} opencv_world) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.cpp b/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.cpp new file mode 100644 index 0000000000000000000000000000000000000000..9a76f7ba86bab18abb69aa1ccd37dc99eaba07b1 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.cpp @@ -0,0 +1,317 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#include "PluginClimb.h" +#include "MxBase/Log/Log.h" +#include "math.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include +using namespace MxPlugins; +using namespace MxTools; +using namespace std; +//extern struct MxpiPortInfo; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins { + APP_ERROR PluginClimb::Init(std::map > &configParamMap) { + + LogInfo << "PluginClimb::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + + std::shared_ptr dataSourceDetection = + std::static_pointer_cast(configParamMap["dataSourceDetection"]); + detectionsource_ = *dataSourceDetection.get(); + + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + + //Configuration parameter + highthresh_ = *std::static_pointer_cast(configParamMap["highThresh"]); + bufferlength_ = *std::static_pointer_cast(configParamMap["bufferLength"]); + ratio_ = *std::static_pointer_cast(configParamMap[" DetectRatio"]); + detectsleep_ = *std::static_pointer_cast(configParamMap["detectSleep"]); + filepath_ = *std::static_pointer_cast(configParamMap["filePath"]); + + return APP_ERR_OK; + } + + APP_ERROR PluginClimb::DeInit() { + LogInfo << "PluginClimb::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginClimb::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginClimb::Process(std::vector &mxpiBuffer) { + + LogInfo << "PluginClimb::Process start"; + + MxpiBuffer *buffer = mxpiBuffer[0]; + + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + + test_framenum++; + + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginClimb process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginClimb process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + + if (metadata == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + + //update data + if (pathflag == 0) { + LogInfo << "highthresh" << highthresh_; + LogInfo << "bufferlength" << bufferlength_; + LogInfo << "detectsleep" << detectsleep_; + readTxt(filepath_, roi); //log in the txt of roi + LogInfo << "*********************************"; + LogInfo << "log in txt"; + LogInfo << roi; + LogInfo << "*********************************"; + pathflag = 1; + } + if (sleeptime == 0) { + LogInfo << "PluginClimb start"; + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0) { + continue; + //As long as the person who has appeared will have a trackobject. + // Determine whether this id appears in the current frame. + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if (detectObject.classvec(0).classid() != 0) { + continue; + } + int cnt_x = (detectObject.x0() + detectObject.x1()) / 2; // + int cnt_y = (detectObject.y0() + detectObject.y1()) / 2; + if (trackdata.count(trackObject.trackid()) == 0) { + int up_num = 0; + int in_num = 0; + if (cv::pointPolygonTest(roi, cv::Point(cnt_x, cnt_y), false) > 0) { + in_num += 1; + } + trackdata[trackObject.trackid()].push_back(cnt_x); // store cnt_x + trackdata[trackObject.trackid()].push_back(cnt_y); //store cnt_y + trackdata[trackObject.trackid()].push_back(up_num); //store up_num + trackdata[trackObject.trackid()].push_back(in_num); //store in_num + + } else { + int up_num = trackdata[trackObject.trackid()][trackdata[trackObject.trackid()].size() - 2]; + int in_num = trackdata[trackObject.trackid()].back(); + if (cv::pointPolygonTest(roi, cv::Point(cnt_x, cnt_y), false) > 0) { + in_num = trackdata[trackObject.trackid()].back() + 1; + } // if the target is in the area, in_num ++ + + if (cnt_y > trackdata[trackObject.trackid()][trackdata[trackObject.trackid()].size() - 3]) { + up_num = trackdata[trackObject.trackid()][trackdata[trackObject.trackid()].size() - 2] + 1; + } // if the target is rising, up_num ++ + + trackdata[trackObject.trackid()].push_back(cnt_x); // store cnt_x + trackdata[trackObject.trackid()].push_back(cnt_y); // store cnt_y + trackdata[trackObject.trackid()].push_back(up_num); //store up_num + trackdata[trackObject.trackid()].push_back(in_num); //store in_num + + if (trackdata[trackObject.trackid()].size() == 4 * bufferlength_) { + if (trackdata[trackObject.trackid()][4 * bufferlength_ - 2] > (int) (bufferlength_ * ratio_) + && + (trackdata[trackObject.trackid()][4 * bufferlength_ - 1] > (int) (bufferlength_ * ratio_)) + && trackdata[trackObject.trackid()][4 * bufferlength_ - 3] - + trackdata[trackObject.trackid()][1] > highthresh_) { + LogInfo << "Alarm Climb Up"; + alarm_count++; + LogInfo << "Alarmed " << alarm_count << " times"; + trackdata.clear(); + sleeptime = detectsleep_; + + // Send the data to downstream plugin + std::shared_ptr result = std::make_shared(); + result->set_attrname("Alarm Climb"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginClimb add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + } else { + //Climbing up judgment failed, delete the last frame of the queue, continue to load data + std::vector tmp(trackdata[trackObject.trackid()].begin() + 4, + trackdata[trackObject.trackid()].end()); + trackdata[trackObject.trackid()] = tmp; + } + } + } + } + } else { + LogInfo << "PluginClimb start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime--; + } + LogInfo << "PluginClimb end"; + + SendData(0, *buffer); + + LogInfo << "PluginClimb::Process end"; + return APP_ERR_OK; + } + + std::vector > PluginClimb::DefineProperties() { + // Define an A to store properties + std::vector > properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared < ElementProperty < string >> (ElementProperty < string > { + STRING, "dataSourceTrack", "name", "the name of previous plugin", "mxpi_motsimplesort0", "NULL", + "NULL"}); + + auto detectsourceProSptr = + std::make_shared < ElementProperty < string >> (ElementProperty < string > { + STRING, "dataSourceDetection", "name", "the name of previous plugin", "mxpi_fairmot0", + "NULL", "NULL"}); + + auto highthreshProSptr = //高度差阈值 + std::make_shared < ElementProperty < int >> (ElementProperty < int > { + UINT, "highThresh", "name", "the name of previous plugin", 8, + 0, 1000}); + + auto bufferlengthProSptr = //每个id需要的判断数据数量阈值 + std::make_shared < ElementProperty < int >> (ElementProperty < int > { + UINT, "bufferLength", "name", "the name of previous plugin", 10, + 8, 1000}); + + auto ratioProSptr = //上升/在墙里 比例参数 + std::make_shared < ElementProperty < float >> (ElementProperty < float > { + FLOAT, "detectRatio", "name", "the name of previous plugin", 0.75, + 0, 1}); + + auto sleepProSptr = //检测后休止时间 + std::make_shared < ElementProperty < int >> (ElementProperty < int > { + UINT, "detectSleep", "name", " ", 8, 0, + 100}); + + auto filepathProSptr = //路径参数 + std::make_shared < ElementProperty < string >> (ElementProperty < string > { + STRING, "filePath", "name", "the name of previous plugin", "mxpi_fairmot0", + "NULL", "NULL"}); + + auto descriptionMessageProSptr = + std::make_shared < ElementProperty < string >> (ElementProperty < string > { + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginClimb", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(highthreshProSptr); + properties.push_back(bufferlengthProSptr); + properties.push_back(ratioProSptr); + properties.push_back(sleepProSptr); + properties.push_back(filepathProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + + // Register the Sample plugin through macro + MxpiPortInfo PluginClimb::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector > value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; + + MxpiPortInfo PluginClimb::DefineOutputPorts() { + MxpiPortInfo outputPortInfo; + std::vector > value = {{"ANY"}}; + MxPluginBase::GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; + } + //log in the txt of roi + + void PluginClimb::readTxt(string file, vector &roi) { + ifstream infile; + infile.open(file); + string str; + while (getline(infile, str)) { + int i = 0; + cv::Point p; + int count = 1; + while (i < str.length()) { + int j = i; + while (j < str.length() && str[j] != ';') { + j++; + } + string s = str.substr(i, j - i); + if (count == 1) { + p.x = atoi(s.c_str()); + count++; + } else if (count == 2) { + p.y = atoi(s.c_str()); + } + i = j + 1; + } + roi.push_back(p); + } + infile.close(); + } +} + +namespace { + MX_PLUGIN_GENERATE(PluginClimb) +} diff --git a/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.h b/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.h new file mode 100644 index 0000000000000000000000000000000000000000..df9009959ce307a6b69f495a9c4fc7318d433f83 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginClimb/PluginClimb.h @@ -0,0 +1,95 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#ifndef SDKMEMORY_PluginClimb_H +#define SDKMEMORY_PluginClimb_H +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/ErrorCode/ErrorCode.h" +#include +/** +* @api +* @brief Definition of PluginClimb class. +*/ +namespace MxPlugins { + class PluginClimb : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map>& configParamMap) override; + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector& mxpiBuffer) override; + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + static MxTools::MxpiPortInfo DefineOutputPorts(); + static void readTxt(std::string file, std::vector &roi); + static int distance(int x0, int y0, int x1, int y1); + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer& buffer, + const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + std::map> trackdata; + int confthresh_; + int confframes_; + int confsleep_; + int confhigh_; + int frame_num = 0; + int sleeptime = 0; + int pathflag = 0; + + int test_framenum = 0; + int highthresh_; + int bufferlength_; + float ratio_; + int detectsleep_; + int alarm_count = 0; + std::string filepath_; + std::vector roi; + }; +} +#endif //SDKMEMORY_FairMOT_OverSpeed_H \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginCounter/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginCounter/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..e18c521bf655e1feac693efcd18de803f762a724 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginCounter/CMakeLists.txt @@ -0,0 +1,28 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_plugincounter) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_plugincounter") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME /home/song5/work/MindX_SDK/mxVision-2.0.2/) +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY /home/song5/work/MindX_SDK/mxVision-2.0.2/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.cpp b/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.cpp new file mode 100644 index 0000000000000000000000000000000000000000..ec2a99ed3296365470722f00ee8594f3ab6465c3 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.cpp @@ -0,0 +1,163 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#include "PluginCounter.h" +#include "MxBase/Log/Log.h" +#include "math.h" +#include "MxTools/Proto/MxpiDumpData.pb.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +using namespace MxPlugins; +using namespace MxTools; +using namespace std; +//extern struct MxpiPortInfo; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins { + + APP_ERROR PluginCounter::Init(std::map> &configParamMap) { + + LogInfo << "PluginCounter::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + + return APP_ERR_OK; + } + + APP_ERROR PluginCounter::DeInit() { + LogInfo << "PluginCounter::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginCounter::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginCounter::Process(std::vector &mxpiBuffer) { + + LogInfo << "PluginCounter::Process start"; + MxpiBuffer *buffer = mxpiBuffer[0]; + + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + + frame++; + LogInfo << "当前帧号 :" << frame; + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginCounter process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginCounter process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + + if (metadata == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + // Send the data to downstream plugin + std::shared_ptr result = std::make_shared(); + result->set_attrname("PluginCounter"); + LogInfo << result->DebugString(); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + + //APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, static_pointer_cast(srcTrackLetListSptr)); + + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginCounter add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + // Send the data to downstream plugin + SendData(0, *buffer); + + LogInfo << "PluginCounter::Process end"; + return APP_ERR_OK; + } + + std::vector> PluginCounter::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceTrack", "name", "the name of previous plugin", "mxpi_motsimplesort0", "NULL", + "NULL"}); + + auto descriptionMessageProSptr = + std::make_shared>(ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginCounter", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + // Register the Sample plugin through macro + + MxpiPortInfo PluginCounter::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; + + MxpiPortInfo PluginCounter::DefineOutputPorts() + { + MxpiPortInfo outputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; + } +} +namespace { + MX_PLUGIN_GENERATE(PluginCounter) +} diff --git a/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.h b/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.h new file mode 100644 index 0000000000000000000000000000000000000000..b23925e0a1756837ea159c602d448860025ee08d --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginCounter/PluginCounter.h @@ -0,0 +1,87 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#ifndef SDKMEMORY_PluginCounter_H +#define SDKMEMORY_PluginCounter_H +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/ErrorCode/ErrorCode.h" +/** +* @api +* @brief Definition of PluginCounter class. +*/ +namespace MxPlugins { + class PluginCounter : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map>& configParamMap) override; + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector& mxpiBuffer) override; + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + /** + * @api + * @brief Define the input ports. + * @return MxpiPortInfo + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + /** + * @api + * @brief Define the output ports. + * @return MxpiPortInfo + */ + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer& buffer, + const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + std::map> trackdata; + int frame = 0; + }; +} +#endif //SDKMEMORY_PluginCounter_H \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOutOfBed/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginOutOfBed/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..09efe1cb92f39ba7e347d4f6a5ca87e2e2593a68 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOutOfBed/CMakeLists.txt @@ -0,0 +1,34 @@ + +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginoutofbed) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginoutofbed") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME /home/song5/work/MindX_SDK/mxVision-2.0.2/) +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY /home/song5/work/MindX_SDK/mxVision-2.0.2/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) +include_directories(${MX_SDK_HOME}/opensource/include/opencv4) + + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) +message(${TARGET_LIBRARY}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) +target_link_libraries(${TARGET_LIBRARY} opencv_world) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.cpp b/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.cpp new file mode 100644 index 0000000000000000000000000000000000000000..4da0e31bc7a154aedd2e741857021e5e08f803b9 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.cpp @@ -0,0 +1,277 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "PluginOutOfBed.h" +#include "MxBase/Log/Log.h" +#include +using namespace MxPlugins; +using namespace MxTools; +using namespace std; + + + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins{ + APP_ERROR PluginOutOfBed::Init(std::map> &configParamMap) { + LogInfo << "PluginOutOfBed::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + // Data from track plugin + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + + // Data from detect plugin + std::shared_ptr detectsourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceDetect"]); + detectionsource_ = *detectsourcePropSptr.get(); + + //Description message + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + + //Configuration parameter + confthres_ = *std::static_pointer_cast(configParamMap["detectThresh"]); + confratio_ = *std::static_pointer_cast(configParamMap["detectRatio"]); + confsleep_ = *std::static_pointer_cast(configParamMap["detectSleep"]); + configpath = *std::static_pointer_cast(configParamMap["configPath"]); + return APP_ERR_OK; + } + + APP_ERROR PluginOutOfBed::DeInit() { + LogInfo << "PluginOutOfBed::DeInit end."; + return APP_ERR_OK; + } + APP_ERROR PluginOutOfBed::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + APP_ERROR PluginOutOfBed::Process(std::vector &mxpiBuffer) { + MxpiBuffer *buffer = mxpiBuffer[0]; + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + + framesnum++; + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginOutOfBed process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "FairMOT_Climb process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + + + if (metadata == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + + if(sleeptime == 0) { + LogInfo << "PluginOutOfBed start"; + cv::Point2f center; + //read txt and get location of bed + if(pathflag){ + readTxt(configpath,bed); + pathflag = false; + } + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0) { + continue; + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if(detectObject.classvec(0).classid() != 0)continue; + if (trackdata.count(trackObject.trackid()) == 0) { + center.x = (detectObject.x0()+detectObject.x1())/2; + center.y = (detectObject.y0()+detectObject.y1())/2; + trackdata[trackObject.trackid()].push_back(cv::pointPolygonTest(bed, center, false)>0); + }else{ + center.x = (detectObject.x0()+detectObject.x1())/2; + center.y = (detectObject.y0()+detectObject.y1())/2; + trackdata[trackObject.trackid()].push_back(cv::pointPolygonTest(bed, center, false)>0); + if(trackdata[trackObject.trackid()].size() >= confthres_){ + if(OutOfBed(trackdata[trackObject.trackid()])) { + LogInfo << "Alarm OutOfBed"; + alarmcount++; + LogInfo << "Alarmed" << alarmcount << "times"; + sleeptime = confsleep_; + frames = 0; + trackdata.clear(); + break; + }else + { + trackdata[trackObject.trackid()].erase(trackdata[trackObject.trackid()].begin()); + } + } + } + } + frames++; + }else + { + LogInfo << "PluginOutOfBed start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime --; + } + LogInfo << "frame:"<(srcTrackLetListSptr)); + + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginOutOfBed add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + + // Send the data to downstream plugin + MxTools::MxPluginBase::SendData(0, *buffer); + return APP_ERR_OK; + } + std::vector> PluginOutOfBed::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceTrack", "name", "the name of previous plugin", "mxpi_motsimplesort0", "NULL", + "NULL"}); + auto detectsourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceDetect", "name", "the name of previous plugin", "mxpi_motsimplesort0", "NULL", + "NULL"}); + + auto threshProSptr = + std::make_shared>(ElementProperty{ + UINT, "detectThresh", "name", "123", 8, 0, + 100}); + + auto ratioProSptr = + std::make_shared>(ElementProperty{ + FLOAT, "detectRatio", "name", " ", 1.0, 0.0, + 1.0}); + + auto sleepProSptr = + std::make_shared>(ElementProperty{ + UINT, "detectSleep", "name", " ", 8, 0, + 100}); + + auto descriptionMessageProSptr = + std::make_shared>(ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is FairMOT_Alone", "NULL", "NULL"}); + + auto configPathSptr = + std::make_shared>(ElementProperty{ + STRING, "configPath", "message", "Description mesasge of plugin", + "This is FairMOT_Alone", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(threshProSptr); + properties.push_back(ratioProSptr); + properties.push_back(sleepProSptr); + properties.push_back(configPathSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + MxpiPortInfo PluginOutOfBed::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + } + //Process the txt and get the location of the bed + void PluginOutOfBed::readTxt(std::string file, std::vector &roi) + { + ifstream infile; + infile.open(file); + string str; + while (getline(infile, str)) + { + int i = 0; + cv::Point p; + int count = 1; + while (i < str.length()) + { + int j = i; + while (j < str.length() && str[j] != ';') + j++; + + string s = str.substr(i, j - i); + if (count == 1) + { + p.x = atoi(s.c_str()); + count++; + } + else if (count == 2) + { + p.y = atoi(s.c_str()); + } + i = j + 1; + } + roi.push_back(p); + } + infile.close(); + } + // out of bed + bool PluginOutOfBed::OutOfBed(std::vector queue) { + int length = queue.size(); + int count_H = 0,count_T = 0; + for (int i =0;i= length/2;j--){ + if(!queue[j]) count_T++; + } + + if(count_H > length * confratio_ && count_T > length * confratio_) return 1; + return 0; + } +} +namespace { + MX_PLUGIN_GENERATE(PluginOutOfBed) +} diff --git a/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.h b/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.h new file mode 100644 index 0000000000000000000000000000000000000000..655505afbc8182102a4bcb75e71706cead9a9827 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOutOfBed/PluginOutOfBed.h @@ -0,0 +1,101 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef MXPIFAIRMOT_FAIRMOT_OUTOFBED_H +#define MXPIFAIRMOT_FAIRMOT_OUTOFBED_H +#include +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/ErrorCode/ErrorCode.h" +#include +#include +/** +* @api +* @brief Definition of PluginOutOfBed class. +*/ +namespace MxPlugins{ + class PluginOutOfBed : public MxTools::MxPluginBase{ + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map> &configParamMap) override; + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + /** + * @api + * @brief read ROI + * @param ROI file path + */ + void readTxt(std::string file, std::vector &roi); + /** + * @api + * @brief Out of bed detection. + * @param queue + * @return bool + */ + bool OutOfBed(std::vector queue); + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer& buffer, const std::string pluginName,const MxTools::MxpiErrorInfo mxpiErrorInfo); + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::string configpath; + std::ostringstream ErrorInfo_; + uint confthres_; + float confratio_; + uint confsleep_; + uint sleeptime = 0; + std::map> trackdata; + std::vector bed; + uint frames; + uint framesnum = 0; + bool pathflag = true; + uint alarmcount = 0; + }; + +} +#endif //MXPIFAIRMOT_FAIRMOT_OUTOFBED_H diff --git a/contrib/ActionRecognition/plugins/PluginOverSpeed/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginOverSpeed/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..82dea7e83c757fd94a237ce55da0c76aa956bd8c --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverSpeed/CMakeLists.txt @@ -0,0 +1,28 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginoverspeed) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginoverspeed") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME /home/song5/work/MindX_SDK/mxVision-2.0.2/) +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY /home/song5/work/MindX_SDK/mxVision-2.0.2/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.cpp b/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.cpp new file mode 100644 index 0000000000000000000000000000000000000000..f4d6c05497af5111a0000361132ed0253852ec0d --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.cpp @@ -0,0 +1,250 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#include "PluginOverSpeed.h" +#include "MxBase/Log/Log.h" +#include "math.h" +using namespace MxPlugins; +using namespace MxTools; +using namespace MxPlugins; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins { + + APP_ERROR PluginOverSpeed::Init(std::map> &configParamMap) { + LogInfo << "PluginOverSpeed::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + + std::shared_ptr dataSourceDetection = + std::static_pointer_cast(configParamMap["dataSourceDetection"]); + detectionsource_ = *dataSourceDetection.get(); + + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + + //Configuration parameter + confthresh_ = *std::static_pointer_cast(configParamMap["speedthresh"]); + confframes_ = *std::static_pointer_cast(configParamMap["frames"]); + confsleep_ = *std::static_pointer_cast(configParamMap["detectsleep"]); + + return APP_ERR_OK; + } + + APP_ERROR PluginOverSpeed::DeInit() { + LogInfo << "PluginOverSpeed::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginOverSpeed::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginOverSpeed::Process(std::vector &mxpiBuffer) { + LogInfo << "PluginOverSpeed::Process start"; + MxpiBuffer *buffer = mxpiBuffer[0]; + + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + + frame++; + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginOverSpeed process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginOverSpeed process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + + if (metadata == nullptr || Detect == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + + //data processing + if(sleeptime == 0) { + LogInfo << "PluginOverSpeed start"; + //update data + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0){ + continue; + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if(detectObject.classvec(0).classid() != 0){ + continue; + } + if (trackdata.count(trackObject.trackid()) == 0) { + trackdata[trackObject.trackid()].push_back((int) ((detectObject.x0() + detectObject.x1()) / 2)); + trackdata[trackObject.trackid()].push_back((int) ((detectObject.y0() + detectObject.y1()) / 2)); + trackdata[trackObject.trackid()][2] = 0; + trackdata[trackObject.trackid()][3] = 0; + } else { + trackdata[trackObject.trackid()][2] += 1; + int x0 = trackdata[trackObject.trackid()][0]; + int y0 = trackdata[trackObject.trackid()][1]; + int x1 = (int) ((detectObject.x0() + detectObject.x1()) / 2); + int y1 = (int) ((detectObject.y0() + detectObject.y1()) / 2); + trackdata[trackObject.trackid()][3] += distance(x0,y0,x1,y1); + trackdata[trackObject.trackid()][0] = (int) ((detectObject.x0() + detectObject.x1()) / 2); + trackdata[trackObject.trackid()][1] = (int) ((detectObject.y0() + detectObject.y1()) / 2); + } + } + frame_num++; + + //calculate data + if (frame_num == confframes_) { + frame_num = 0; + map>::iterator iter; + for (iter = trackdata.begin(); iter != trackdata.end(); iter++) { + if(trackdata[iter->first][2] == 0 || trackdata[iter->first][2] < 4){ + continue; + } + if (((int) (trackdata[iter->first][3] / trackdata[iter->first][2])) > confthresh_) { + LogInfo << "Alarm OverSpeed"; + alarm_count++; + LogInfo << "Alarmed " << alarm_count << " times"; + sleeptime = confsleep_; + + std::shared_ptr result = std::make_shared(); + result->set_attrname("Alarm OverSpeed"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginOverSpeed add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + break; + } + } + trackdata.clear(); + } + }else{ + LogInfo << "PluginOverSpeed start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime--; + } + // LogInfo << "frame : " << frame; + LogInfo << "PluginOverSpeed end"; + + // Send the data to downstream plugin + SendData(0, *buffer); + + LogInfo << "PluginOverSpeed::Process end"; + return APP_ERR_OK; + } + + std::vector> PluginOverSpeed::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceTrack", "name", "the source of tack data", "mxpi_motsimplesort0", + "NULL","NULL"}); + + auto detectsourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceDetection", "name", "the source of detect data", "mxpi_objectpostprocessor0", + "NULL", "NULL"}); + + auto threshProSptr = + std::make_shared>(ElementProperty{ + UINT, "speedthresh", "thresh", "the thresh of overspeed", 10, + 1, 1000}); + + auto frameProSptr = + std::make_shared>(ElementProperty{ + UINT, "frames", "frame", "the number of frame when judging", 8, + 8, 100}); + + auto sleepProSptr = + std::make_shared>(ElementProperty{ + UINT, "detectsleep", "sleep", "the time of stop detection", 8, 0, + 300}); + + auto descriptionMessageProSptr = + std::make_shared>(ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginOverSpeed", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(threshProSptr); + properties.push_back(frameProSptr); + properties.push_back(sleepProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + // Register the Sample plugin through macro + + MxpiPortInfo PluginOverSpeed::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; + + MxpiPortInfo PluginOverSpeed::DefineOutputPorts() + { + MxpiPortInfo outputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; + } + + int PluginOverSpeed::distance(int x0, int y0, int x1, int y1){ + int distance = int(sqrt((x1 - x0) * (x1 - x0) + (y1 - y0) * (y1 - y0))); + return distance; + } +} +namespace { + MX_PLUGIN_GENERATE(PluginOverSpeed) +} diff --git a/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.h b/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.h new file mode 100644 index 0000000000000000000000000000000000000000..b34df58b0752f5977933ec650511ecd2d8f5e0b6 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverSpeed/PluginOverSpeed.h @@ -0,0 +1,101 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#ifndef SDKMEMORY_PluginOverSpeed_H +#define SDKMEMORY_PluginOverSpeed_H +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/ErrorCode/ErrorCode.h" +/** +* @api +* @brief Definition of PluginOverSpeed class. +*/ + + +namespace MxPlugins { + class PluginOverSpeed : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map>& configParamMap) override; + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector& mxpiBuffer) override; + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + /** + * @api + * @brief Define the input ports. + * @return MxpiPortInfo + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + /** + * @api + * @brief Define the output ports. + * @return MxpiPortInfo + */ + static int distance(int x0, int y0, int x1, int y1); + /** + * @api + * @brief Calculate the distance between two points. + * @param int + * @return int + */ + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer& buffer, const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + std::map> trackdata; + int confthresh_; + int confframes_; + int confsleep_; + int frame_num = 0; + int sleeptime = 0; + int frame = 0; + int alarm_count = 0; + }; +} +#endif //SDKMEMORY_PluginOverSpeed_H \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOverStay/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginOverStay/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..caec67f14385307bcb951da0209dc628bb4783ad --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverStay/CMakeLists.txt @@ -0,0 +1,28 @@ +cmake_minimum_required(VERSION 3.5.2) +project(mxpi_pluginoverstay) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +set(PLUGIN_NAME "mxpi_pluginoverstay") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") +add_definitions(-DENABLE_DVPP_INTERFACE) + +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations) + +set(MX_SDK_HOME /home/song5/work/MindX_SDK/mxVision-2.0.2/) +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY /home/song5/work/MindX_SDK/mxVision-2.0.2/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + +file(GLOB PLUGIN_SRC ./*.cpp) +message(${PLUGIN_SRC}) + +add_library(${TARGET_LIBRARY} SHARED ${PLUGIN_SRC}) +target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0) +target_link_libraries(${TARGET_LIBRARY} mxpidatatype plugintoolkit mxbase mindxsdk_protobuf) \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.cpp b/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.cpp new file mode 100644 index 0000000000000000000000000000000000000000..f3d3ae64babe1b09d8899c5f2527efe39576653a --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.cpp @@ -0,0 +1,267 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#include "PluginOverStay.h" +#include "MxBase/Log/Log.h" +#include "math.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +using namespace MxPlugins; +using namespace MxTools; +using namespace std; +//extern struct MxpiPortInfo; + +namespace { + const string SAMPLE_KEY = "MxpiObjectList"; +} + +namespace MxPlugins { + + APP_ERROR PluginOverStay::Init(std::map> &configParamMap) { + LogInfo << "PluginOverStay::Init start."; + APP_ERROR ret = APP_ERR_OK; + + // Get the property values by key + std::shared_ptr tracksourcePropSptr = + std::static_pointer_cast(configParamMap["dataSourceTrack"]); + tracksource_ = *tracksourcePropSptr.get(); + + std::shared_ptr dataSourceDetection = + std::static_pointer_cast(configParamMap["dataSourceDetection"]); + detectionsource_ = *dataSourceDetection.get(); + + std::shared_ptr descriptionMessageProSptr = + std::static_pointer_cast(configParamMap["descriptionMessage"]); + descriptionMessage_ = *descriptionMessageProSptr.get(); + + //Configuration parameter + confthresh_ = *std::static_pointer_cast(configParamMap["staythresh"]); + confframes_ = *std::static_pointer_cast(configParamMap["frames"]); + confsleep_ = *std::static_pointer_cast(configParamMap["detectsleep"]); + confdistance_ = *std::static_pointer_cast(configParamMap["distancethresh"]); + confratio_ = *std::static_pointer_cast(configParamMap["ratio"]); + + return APP_ERR_OK; + } + + APP_ERROR PluginOverStay::DeInit() { + LogInfo << "PluginOverStay::DeInit end."; + return APP_ERR_OK; + } + + APP_ERROR PluginOverStay::SetMxpiErrorInfo(MxpiBuffer &buffer, const std::string pluginName, + const MxpiErrorInfo mxpiErrorInfo) { + APP_ERROR ret = APP_ERR_OK; + // Define an object of MxpiMetadataManager + MxpiMetadataManager mxpiMetadataManager(buffer); + ret = mxpiMetadataManager.AddErrorInfo(pluginName, mxpiErrorInfo); + if (ret != APP_ERR_OK) { + LogError << "Failed to AddErrorInfo."; + return ret; + } + ret = SendData(0, buffer); + return ret; + } + + APP_ERROR PluginOverStay::Process(std::vector &mxpiBuffer) { + + LogInfo << "PluginOverStay::Process start"; + MxpiBuffer *buffer = mxpiBuffer[0]; + + MxpiMetadataManager mxpiMetadataManager(*buffer); + MxpiErrorInfo mxpiErrorInfo; + ErrorInfo_.str(""); + auto errorInfoPtr = mxpiMetadataManager.GetErrorInfo(); + + frame++; + if (errorInfoPtr != nullptr) { + ErrorInfo_ << GetError(APP_ERR_COMM_FAILURE, pluginName_) << "PluginOverStay process is not implemented"; + mxpiErrorInfo.ret = APP_ERR_COMM_FAILURE; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + LogError << "PluginOverStay process is not implemented"; + return APP_ERR_COMM_FAILURE; + } + // Get the data from buffer + shared_ptr metadata = mxpiMetadataManager.GetMetadata(tracksource_); + shared_ptr Detect = mxpiMetadataManager.GetMetadata(detectionsource_); + + if (metadata == nullptr) { + ErrorInfo_ << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "Metadata is NULL, failed"; + mxpiErrorInfo.ret = APP_ERR_METADATA_IS_NULL; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return APP_ERR_METADATA_IS_NULL; // self define the error code + } + // check whether the proto struct name is MxpiObjectList + google::protobuf::Message *msg = (google::protobuf::Message *) metadata.get(); + + // get trackletlist + std::shared_ptr srcTrackLetListSptr = std::static_pointer_cast(metadata); + std::shared_ptr srcObjectListSptr = std::static_pointer_cast(Detect); + + //data processing + if(sleeptime == 0) { + LogInfo << "PluginOverStay start"; + //update data + for (uint32_t i = 0; i < (uint32_t) srcTrackLetListSptr->trackletvec_size(); i++) { + auto &trackObject = srcTrackLetListSptr->trackletvec(i); + if (trackObject.headervec_size() == 0){ + continue; + } + auto &detectObject = srcObjectListSptr->objectvec(trackObject.headervec(0).memberid()); + if(detectObject.classvec(0).classid() != 0){ + continue; + } + if (trackdata.count(trackObject.trackid()) == 0) { + + trackdata[trackObject.trackid()].push_back((int) ((detectObject.x0() + detectObject.x1()) / 2)); + trackdata[trackObject.trackid()].push_back((int) ((detectObject.y0() + detectObject.y1()) / 2)); + trackdata[trackObject.trackid()][2] = 1; + } else { + + int dis = distance(trackdata[trackObject.trackid()][0], + trackdata[trackObject.trackid()][1], + (int) ((detectObject.x0() + detectObject.x1()) / 2), + (int) ((detectObject.y0() + detectObject.y1()) / 2)); + if(dis < confdistance_){ + trackdata[trackObject.trackid()][2]++; + } + else trackdata[trackObject.trackid()][2] = 1; + trackdata[trackObject.trackid()][0] = (int) ((detectObject.x0() + detectObject.x1()) / 2); + trackdata[trackObject.trackid()][1] = (int) ((detectObject.y0() + detectObject.y1()) / 2); + } + } + frame_num++; + + //calculate data + if (frame_num == confframes_) { + frame_num = 0; + map>::iterator iter; + for (iter = trackdata.begin(); iter != trackdata.end(); iter++) { + if (trackdata[iter->first][2] >= (int)(confratio_ * confthresh_)) { + LogInfo << "Alarm Overstay"; + alarm_count++; + LogInfo << "Alarmed " << alarm_count << " times"; + sleeptime = confsleep_; + + std::shared_ptr result = std::make_shared(); + result->set_attrname("Alarm Overstay"); + APP_ERROR ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, result); + if (ret != APP_ERR_OK) { + ErrorInfo_ << GetError(ret, pluginName_) << "PluginOverStay add metadata failed."; + mxpiErrorInfo.ret = ret; + mxpiErrorInfo.errorInfo = ErrorInfo_.str(); + SetMxpiErrorInfo(*buffer, pluginName_, mxpiErrorInfo); + return ret; + } + + break; + } + } + trackdata.clear(); + } + }else{ + LogInfo << "PluginOverStay start"; + LogInfo << "Alarmed in a short period of time"; + sleeptime--; + } +// LogInfo << "frame : " << frame; + LogInfo << "PluginOverStay end"; + + // Send the data to downstream plugin + SendData(0, *buffer); + + LogInfo << "PluginOverStay::Process end"; + return APP_ERR_OK; + } + + std::vector> PluginOverStay::DefineProperties() { + // Define an A to store properties + std::vector> properties; + // Set the type and related information of the properties, and the key is the name + auto tracksourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceTrack", "name", "the name of previous plugin", "mxpi_motsimplesort0", "NULL", + "NULL"}); + + auto detectsourceProSptr = + std::make_shared>(ElementProperty{ + STRING, "dataSourceDetection", "name", "the name of previous plugin", "mxpi_fairmot0", + "NULL", "NULL"}); + + auto threshProSptr = //逗留时间阈值 + std::make_shared>(ElementProperty{ + UINT, "staythresh", "name", "the name of previous plugin", 10, + 10, 1000}); + + auto distanceProSptr = //移动距离阈值 + std::make_shared>(ElementProperty{ + UINT, "distancethresh", "name", "the name of previous plugin", 6, + 0, 100}); + + auto frameProSptr = //多少帧检测一次 + std::make_shared>(ElementProperty{ + UINT, "frames", "name", "the name of previous plugin", 8, + 8, 100}); + + auto sleepProSptr = //检测后休止时间 + std::make_shared>(ElementProperty{ + UINT, "detectsleep", "name", " ", 8, 0, + 100}); + + auto ratioProSptr = //检出比例 + std::make_shared>(ElementProperty{ + FLOAT, "ratio", "name", " ", 0.8, 0.1, + 1.0}); + + auto descriptionMessageProSptr = + std::make_shared>(ElementProperty{ + STRING, "descriptionMessage", "message", "Description mesasge of plugin", + "This is PluginOverStay", "NULL", "NULL"}); + + properties.push_back(tracksourceProSptr); + properties.push_back(detectsourceProSptr); + properties.push_back(threshProSptr); + properties.push_back(frameProSptr); + properties.push_back(sleepProSptr); + properties.push_back(distanceProSptr); + properties.push_back(ratioProSptr); + properties.push_back(descriptionMessageProSptr); + return properties; + } + + MxpiPortInfo PluginOverStay::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; + }; + + MxpiPortInfo PluginOverStay::DefineOutputPorts() + { + MxpiPortInfo outputPortInfo; + std::vector> value = {{"ANY"}}; + MxPluginBase::GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; + } + + int PluginOverStay::distance(int x0, int y0, int x1, int y1){ + int distance = int(sqrt((x1 - x0) * (x1 - x0) + (y1 - y0) * (y1 - y0))); + return distance; + } +} +namespace { + MX_PLUGIN_GENERATE(PluginOverStay) +} diff --git a/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.h b/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.h new file mode 100644 index 0000000000000000000000000000000000000000..7e84f56b18b556829df858ab8a28e62d8cee80fe --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginOverStay/PluginOverStay.h @@ -0,0 +1,101 @@ +/* +* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ +#ifndef SDKMEMORY_PluginOverStay_H +#define SDKMEMORY_PluginOverStay_H +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/ErrorCode/ErrorCode.h" +/** +* @api +* @brief Definition of PluginOverStay class. +*/ +namespace MxPlugins { + class PluginOverStay : public MxTools::MxPluginBase { + public: + /** + * @api + * @brief Initialize configure parameter. + * @param configParamMap + * @return APP_ERROR + */ + APP_ERROR Init(std::map>& configParamMap) override; + /** + * * @api + * @brief DeInitialize configure parameter. + * @return APP_ERROR + */ + APP_ERROR DeInit() override; + /** + * @api + * @brief Process the data of MxpiBuffer. + * @param mxpiBuffer + * @return APP_ERROR + */ + APP_ERROR Process(std::vector& mxpiBuffer) override; + /** + * @api + * @brief Definition the parameter of configure properties. + * @return std::vector> + */ + static std::vector> DefineProperties(); + /** + * @api + * @brief Get the number of class id and confidence from model inference. + * @param key + * @param buffer + * @return APP_ERROR + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + /** + * @api + * @brief Define the input ports. + * @return MxpiPortInfo + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + /** + * @api + * @brief Define the output ports. + * @return MxpiPortInfo + */ + static int distance(int x0, int y0, int x1, int y1); + /** + * @api + * @brief Calculate the distance between two points. + * @param int + * @return int + */ + private: + APP_ERROR SetMxpiErrorInfo(MxTools::MxpiBuffer& buffer, const std::string pluginName, + const MxTools::MxpiErrorInfo mxpiErrorInfo); + std::string tracksource_; + std::string detectionsource_; + std::string descriptionMessage_; + std::ostringstream ErrorInfo_; + std::map> trackdata; + int confthresh_; + int confframes_; + int confsleep_; + int confdistance_; + float confratio_; + int frame_num = 0; + int sleeptime = 0; + int alarm_count = 0; + int frame = 0; + }; +} +#endif //SDKMEMORY_PluginOverStay_H \ No newline at end of file diff --git a/contrib/ActionRecognition/plugins/PluginViolentAction/CMakeLists.txt b/contrib/ActionRecognition/plugins/PluginViolentAction/CMakeLists.txt new file mode 100644 index 0000000000000000000000000000000000000000..a4e8dc932a59b6efa3779660f294eac0b2700c75 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginViolentAction/CMakeLists.txt @@ -0,0 +1,44 @@ +cmake_minimum_required(VERSION 3.10) +project(Plugin_ViolentAction) + +set(CMAKE_CXX_STANDARD 11) + +set(PLUGIN_NAME "mxpi_violentaction") +set(TARGET_LIBRARY ${PLUGIN_NAME}) + +add_compile_options(-fPIC -fstack-protector-all -g -Wl,-z,relro,-z,now,-z -pie -Wall) +add_compile_options(-std=c++11 -Wno-deprecated-declarations) +add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}") + +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) +add_definitions(-DENABLE_DVPP_INTERFACE) + + +set(MX_SDK_HOME /home/song5/work/MindX_SDK/mxVision-2.0.2/) +set(CMAKE_LIBRARY_OUTPUT_DIRECTORY /home/song5/work/MindX_SDK/mxVision-2.0.2/lib/plugins/) + +include_directories(${MX_SDK_HOME}/include) +include_directories(${MX_SDK_HOME}/opensource/include) +include_directories(${MX_SDK_HOME}/opensource/include/gstreamer-1.0) +include_directories(${MX_SDK_HOME}/opensource/include/opencv4) +include_directories(${MX_SDK_HOME}/opensource/include/glib-2.0) +include_directories(${MX_SDK_HOME}/opensource/lib/glib-2.0/include) + + +link_directories(${MX_SDK_HOME}/lib) +link_directories(${MX_SDK_HOME}/opensource/lib) + + +add_library(${TARGET_LIBRARY} SHARED PluginViolentAction.cpp PluginViolentAction.h) +target_link_libraries(${TARGET_LIBRARY} + mxpidatatype + plugintoolkit + mxbase + streammanager + mindxsdk_protobuf + glib-2.0 + gstreamer-1.0 + gobject-2.0 + gstbase-1.0 + gmodule-2.0 + ) diff --git a/contrib/ActionRecognition/plugins/PluginViolentAction/PluginViolentAction.cpp b/contrib/ActionRecognition/plugins/PluginViolentAction/PluginViolentAction.cpp new file mode 100644 index 0000000000000000000000000000000000000000..6c15dd84377d368543c1261131d6d34e29b93c0b --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginViolentAction/PluginViolentAction.cpp @@ -0,0 +1,195 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "PluginViolentAction.h" +#include "MxBase/Log/Log.h" +#include + +using namespace MxPlugins; +using namespace MxTools; +using namespace MxBase; +using namespace std; + +namespace { + const std::string SAMPLE_KEY = "MxpiClassList"; +} + +APP_ERROR PluginViolentAction::Init(std::map> &configParamMap) { + LogInfo << "Begin to initialize PluginViolentAction(" << pluginName_ << ")."; + // Get the property values by key + std::shared_ptr classSource = std::static_pointer_cast(configParamMap["classSource"]); + classSource_ = *classSource; + + std::shared_ptr filePath = std::static_pointer_cast(configParamMap["filePath"]); + filePath_ = *filePath; + + std::shared_ptr detectSleep = std::static_pointer_cast(configParamMap["detectSleep"]); + detectSleep_ = *detectSleep; + LogInfo << "End to initialize PluginViolentAction(" << pluginName_ << ")."; + return APP_ERR_OK; +} + +APP_ERROR PluginViolentAction::DeInit() { + LogInfo << "Begin to deinitialize PluginViolentAction(" << pluginName_ << ")."; + LogInfo << "End to deinitialize PluginViolentAction(" << pluginName_ << ")."; + return APP_ERR_OK; +} + +APP_ERROR PluginViolentAction::CheckDataSource(MxTools::MxpiMetadataManager &mxpiMetadataManager) { + if (mxpiMetadataManager.GetMetadata(classSource_) == nullptr) { + LogDebug << GetError(APP_ERR_METADATA_IS_NULL, pluginName_) << "class metadata is null. please check" + << "Your property classSource(" << classSource_ << ")."; + return APP_ERR_METADATA_IS_NULL; + } + return APP_ERR_OK; +} + +std::shared_ptr +PluginViolentAction::ActionMatch(std::shared_ptr &mxpiClassList) { + // set MxpiAttributeList + MxpiClass mxpiClass = mxpiClassList->classvec(0); + int32_t classId = mxpiClass.classid(); + std::string className = mxpiClass.classname(); + float confidence = mxpiClass.confidence(); + std::shared_ptr mxpiAttributeList = std::make_shared(); + MxpiAttribute *mxpiAttribute = mxpiAttributeList->add_attributevec(); + mxpiAttribute->set_confidence(confidence); + mxpiAttribute->set_attrid(classId); + mxpiAttribute->set_attrname(className); + if (sleepTime_ == 0) { + auto iterator = find(aoi.begin(), aoi.end(), className); + if (iterator != aoi.end()) { + // mark the times of alarms for statistics + alarm_count ++; + sleepTime_ = detectSleep_; + alarmInformation = "Alarm Violent Action"; + mxpiAttribute->set_attrvalue(alarmInformation); + return mxpiAttributeList; + } else { + // no interested action + alarmInformation = "No Alarm"; + mxpiAttribute->set_attrvalue(alarmInformation); + return mxpiAttributeList; + } + } else { + // alarm sleep + alarmInformation = "Alarmed in a short period of time"; + mxpiAttribute->set_attrvalue(alarmInformation); + sleepTime_--; + return mxpiAttributeList; + } +} + +void PluginViolentAction::ReadTxt(std::string file, std::vector &aoi) { + // read txt file + std::ifstream infile; + infile.open(file.data()); + assert(infile.is_open()); + std::string str; + while (getline(infile, str)) { + // Remove the ending symbol + str.erase(str.end()-1); + aoi.emplace_back(str); + } + infile.close(); +} + +APP_ERROR PluginViolentAction::Process(std::vector &mxpiBuffer) { + LogInfo << "Begin to process PluginViolentAction(" << elementName_ << ")."; + // Get MxpiClassList from MxpiBuffer + MxpiBuffer *inputMxpiBuffer = mxpiBuffer[0]; + MxpiMetadataManager mxpiMetadataManager(*inputMxpiBuffer); + ErrorInfo_.str(""); + // check data source + APP_ERROR ret = CheckDataSource(mxpiMetadataManager); + if (ret != APP_ERR_OK) { + SendData(0, *inputMxpiBuffer); + return ret; + } + // Get metadata by key + std::shared_ptr class_metadata = mxpiMetadataManager.GetMetadata(classSource_); + std::shared_ptr srcClassListPtr = std::static_pointer_cast(class_metadata); + // update data; Read the action of interest file + // set pathflag to make file IO only once + if (pathflag == 0) { + ReadTxt(filePath_, aoi); + pathflag = 1; + } + // Match the upstream action to the action of interest and alarm. + auto attributeListPtr = ActionMatch(srcClassListPtr); + ret = mxpiMetadataManager.AddProtoMetadata(pluginName_, static_pointer_cast(attributeListPtr)); + if (ret != APP_ERR_OK) { + LogError << ErrorInfo_.str(); + SendMxpiErrorInfo(*inputMxpiBuffer, pluginName_, ret, ErrorInfo_.str()); + SendData(0, *inputMxpiBuffer); + } + // Send the data to downstream plugin + SendData(0, *inputMxpiBuffer); + LogInfo << "End to process PluginViolentAction(" << elementName_ << ")."; + return APP_ERR_OK; +} + +std::vector> PluginViolentAction::DefineProperties() { + std::vector> properties; + // Get the action category from previous plugin + auto classsource = std::make_shared>(ElementProperty{ + STRING, + "classSource", + "labelSource", + "Recognized Action Class", + "default", "NULL", "NULL" + }); + // The action of interest file path + auto filepath = std::make_shared>(ElementProperty{ + STRING, + "filePath", + "Action of interest file path", + "the path of predefined violent action classes file", + "NULL", "NULL", "NULL" + }); + // sleep time after alarm + auto detectsleep = std::make_shared>(ElementProperty{ + UINT, + "detectSleep", + "process sleep time", + "sleep some time to avoid frequent alarms ", + 8, 0, 100 + }); + properties.push_back(classsource); + properties.push_back(filepath); + properties.push_back(detectsleep); + return properties; +} + +MxpiPortInfo PluginViolentAction::DefineInputPorts() { + MxpiPortInfo inputPortInfo; + // Input: {{MxpiClassList}} + std::vector> value = {{"ANY"}}; + GenerateStaticInputPortsInfo(value, inputPortInfo); + return inputPortInfo; +} + +MxpiPortInfo PluginViolentAction::DefineOutputPorts() { + MxpiPortInfo outputPortInfo; + // Output: {{MxpiAttributeList}} + std::vector> value = {{"ANY"}}; + GenerateStaticOutputPortsInfo(value, outputPortInfo); + return outputPortInfo; +} + +namespace { + MX_PLUGIN_GENERATE(PluginViolentAction) +} diff --git a/contrib/ActionRecognition/plugins/PluginViolentAction/PluginViolentAction.h b/contrib/ActionRecognition/plugins/PluginViolentAction/PluginViolentAction.h new file mode 100644 index 0000000000000000000000000000000000000000..8cb037a2306bb8c0af3350ab5162167ec015f277 --- /dev/null +++ b/contrib/ActionRecognition/plugins/PluginViolentAction/PluginViolentAction.h @@ -0,0 +1,100 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef SDKMEMORY_PluginViolentAction_H +#define SDKMEMORY_PluginViolentAction_H + +#include "MxTools/PluginToolkit/base/MxPluginGenerator.h" +#include "MxTools/PluginToolkit/base/MxPluginBase.h" +#include "MxTools/PluginToolkit/metadata/MxpiMetadataManager.h" +#include "MxTools/Proto/MxpiDataType.pb.h" +#include "MxBase/ErrorCode/ErrorCode.h" + +/** + * This plug is to recognize whether the object's action is a Violent Action and alarm. +*/ +using namespace MxTools; + +namespace MxPlugins { + class PluginViolentAction : public MxTools::MxPluginBase { + public: + /** + * @description: Init configs. + * @param configParamMap: config. + * @return: Error code. + */ + APP_ERROR Init(std::map> &configParamMap) override; + + /** + * @description: DeInit device. + * @return: Error code. + */ + APP_ERROR DeInit() override; + + /** + * @description: Plugin_ViolentAction plugin process. + * @param mxpiBuffer: data receive from the previous. + * @return: Error code. + */ + APP_ERROR Process(std::vector &mxpiBuffer) override; + + /** + * @description: Plugin_ViolentAction plugin define properties. + * @return: properties. + */ + static std::vector> DefineProperties(); + + /** + * @api + * @brief Define the number and data type of input ports. + * @return MxTools::MxpiPortInfo. + */ + static MxTools::MxpiPortInfo DefineInputPorts(); + + /** + * @api + * @brief Define the number and data type of output ports. + * @return MxTools::MxpiPortInfo. + */ + static MxTools::MxpiPortInfo DefineOutputPorts(); + + private: + /** + * @api + * @brief Check metadata. + * @param MxTools::MxpiMetadataManager. + * @return Error Code. + */ + APP_ERROR CheckDataSource(MxpiMetadataManager &mxpiMetadataManager); + /** + * @api + * @brief Match the recognized action class to the action of interest and alarm. + * @return MxTools::MxpiAttributeList. + */ + std::shared_ptr ActionMatch(std::shared_ptr &mxpiClassList); + void ReadTxt(std::string file, std::vector &aoi); // read the action of interest txt file + std::string classSource_ = ""; // previous plugin MxpiClassList + std::string filePath_ = ""; + std::ostringstream ErrorInfo_; // Error Code + std::vector aoi = {}; + std::string alarmInformation = ""; + int pathflag = 0; // flag to mark file IO only once + uint32_t sleepTime_ = 0; + uint32_t detectSleep_ = 0; + int alarm_count = 0; + }; +} +#endif //SDKMEMORY_PluginViolentAction_H \ No newline at end of file diff --git a/contrib/ActionRecognition/run.sh b/contrib/ActionRecognition/run.sh new file mode 100644 index 0000000000000000000000000000000000000000..08be6068e2aa9b79f8b01068196b48ef848be1a3 --- /dev/null +++ b/contrib/ActionRecognition/run.sh @@ -0,0 +1,37 @@ +#!/bin/bash + +#/* + #* Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + #* + #* Licensed under the Apache License, Version 2.0 (the "License"); + #* you may not use this file except in compliance with the License. + #* You may obtain a copy of the License at + #* + #* http://www.apache.org/licenses/LICENSE-2.0 + #* + #* Unless required by applicable law or agreed to in writing, software + #* distributed under the License is distributed on an "AS IS" BASIS, + #* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + #* See the License for the specific language governing permissions and + #* limitations under the License. + #*/ + +set -e + +CUR_PATH=$(cd "$(dirname "$0")" || { warn "Failed to check path/to/run.sh" ; exit ; } ; pwd) + +# Simple log helper functions +info() { echo -e "\033[1;34m[INFO ][MxStream] $1\033[1;37m" ; } +warn() { echo >&2 -e "\033[1;31m[WARN ][MxStream] $1\033[1;37m" ; } + +export MX_SDK_HOME=${CUR_PATH}/../../.. +export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:${LD_LIBRARY_PATH} +export GST_PLUGIN_SCANNER=${MX_SDK_HOME}/opensource/libexec/gstreamer-1.0/gst-plugin-scanner +export GST_PLUGIN_PATH=${MX_SDK_HOME}/opensource/lib/gstreamer-1.0:${MX_SDK_HOME}/lib/plugins +export GST_DEBUG=3 + +#to set PYTHONPATH, import the StreamManagerApi.py +export PYTHONPATH=$PYTHONPATH:${MX_SDK_HOME}/python + +python3.7 main.py +exit 0 \ No newline at end of file