diff --git a/samples/built-in/tracking/deepsort/README.md b/samples/built-in/tracking/deepsort/README.md index 9003ca109136e0c8fd3bf307e7397b3a1febf8d2..08730601873040970d8dec37bef6735bcd791ae0 100644 --- a/samples/built-in/tracking/deepsort/README.md +++ b/samples/built-in/tracking/deepsort/README.md @@ -21,7 +21,7 @@ # 概述 -DeepSORT（Deep Simple Online and Realtime Tracking）是一种经典的多目标追踪算法，在原始 SORT 算法基础上引入了深度外观特征（ReID），有效降低了 ID Switch 率。。该模型在 MOTA 数据集上进行了验证。 +DeepSORT（Deep Simple Online and Realtime Tracking）是一种经典的多目标追踪算法，在原始 SORT 算法基础上引入了深度外观特征（ReID），有效降低了 ID Switch 率。该模型在 MOT16 数据集上进行了验证。 - 参考实现： @@ -44,8 +44,8 @@ DeepSORT（Deep Simple Online and Realtime Tracking）是一种经典的多目 | 输出数据 | 数据类型 | 大小 | 数据排布格式 | | -------- | -------- | ----------- | ----------- | | feature_map_1 | FP32 | 1 x 3 x 80 x 80 x 85 | NCHW | - | feature_map_2 | FP32 | 1 x 3 x 80 x 80 x 85 | NCHW | - | feature_map_3 | FP32 | 1 x 3 x 80 x 80 x 85 | NCHW | + | feature_map_2 | FP32 | 1 x 3 x 40 x 40 x 85 | NCHW | + | feature_map_3 | FP32 | 1 x 3 x 20 x 20 x 85 | NCHW | ### ReID - 输入数据 | 输入数据 | 数据类型 | 大小 | 数据排布格式 | @@ -156,7 +156,7 @@ mkdir -p model ``` 比如 ``` - cmake ../src -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=../../../../common/cmake/toolchain_aarch64_ohos.cmake -DSOC_VERSION=SS928V100 + cmake ../src -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=../../../../common/cmake/toolchain_aarch64_linux.cmake -DSOC_VERSION=SS928V100 ``` 3. 执行**make**命令，生成的可执行文件main在“./out“目录下。 @@ -175,7 +175,7 @@ mkdir -p model 4. 切换到可执行文件main所在的目录，运行可执行文件。测试图片上模型推理命令参考： ``` - ./main model/yolov5s.om model/reid_net.om datasets/MOT16/train/MOT16-09/img1 + ./main ../model/yolov5s.om ../model/reid_net.om ../datasets/MOT16/train/MOT16-09/img1 ``` 备注：若需要在数据集上进行精度评估，需要参考[安装依赖](#section183221994410)、[准备数据集](#section183221994411)和[精度&性能评估](#section741711594518)章节。 @@ -188,26 +188,53 @@ pip3 install -r requirements.txt ## 准备数据集 - 该模型通常使用 [MOA16数据集](https://motchallenge.net/data/MOT16/) 进行训练和验证。 + 该模型通常使用 [MOT16数据集](https://motchallenge.net/data/MOT16/) 进行训练和验证。 - 在 `samples/built-in/tracking/deepsort/` 目录下创建 `datasets` 文件夹（或建立软链接），将下载的数据集拷贝到该目录并进行解压。 + 在 `samples/built-in/tracking/deepsort/` 目录下创建 `datasets` 文件夹（或建立软链接），将下载的数据集拷贝到该目录并进行解压，解压结果如下： + + ``` + datasets + └── MOT16 + ├── test + │ ├── MOT16-01 + │ ├── MOT16-03 + │ ├── MOT16-06 + │ ├── MOT16-07 + │ ├── MOT16-08 + │ ├── MOT16-12 + │ └── MOT16-14 + └── train + ├── MOT16-02 + ├── MOT16-04 + ├── MOT16-05 + ├── MOT16-09 + ├── MOT16-10 + ├── MOT16-11 + └── MOT16-13 + ``` ## 模型转化 -使用 Ultralytics 导出 ONNX，再使用 ATC 工具转为 OM 模型。 +使用 `export_onnx.py` 导出 ONNX，再使用 ATC 工具转为 OM 模型。 1. 获取 DeepSort 源码 ```bash git clone https://github.com/ZQPei/deep_sort_pytorch.git + git reset 4f910afc16860ff05c6be408b120a49524bd4f68 cd deep_sort_pytorch patch -p1 < ../deepsort_py.patch cd .. ``` 2. 导出onnx。 + 下载权重文件： + YOLO: https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5s.pt + 下载后归档在`deep_sort_pytorch/detector/YOLOv5` + ReID: https://drive.google.com/drive/folders/1xhG0kRH1EX5B9_Iz8gQJb7UNnn_riXi6 + 下载后归档在`deep_sort_pytorch/deep_sort/deep/checkpoint` 在 `samples/built-in/tracking/deepsort/deep_sort_pytorch` 目录下执行，生成 `model/*.onnx` 文件。 @@ -217,56 +244,59 @@ pip3 install -r requirements.txt python export_onnx.py --output_dir ../model ``` -2. 生成模型校准数据。 +3. 生成模型校准数据。 - 选取几张图片生成模型校准数据，引用的图片数据默认在samples/samples_GPL/built-in/yolov8s-obb/data/file_list.json中描述（当前仅包含3张示例图片，用于量化校准已足够）。校准数据文件默认保存在out/preprocess/bin目录下。 + 选取几张图片生成模型校准数据，引用的图片数据默认在 datasets/MOT16/train。校准数据文件默认保存在datasets/prep_data_aipp目录下。 ```sh cd samples/built-in/tracking/deepsort/script python generate_real_bin.py ``` -3. 使用 ATC 工具将 ONNX 模型转 OM 模型。 +4. 使用 ATC 工具将 ONNX 模型转 OM 模型。 Hi3403V100 SVP_NNN 上的 om 模型转换命令 ```bash cd samples/built-in/tracking/deepsort - atc --compile_mode=6 --framework=5 --model=model/yolov5s.onnx --input_shape=images:1,3,640,640 --output=model/yolov5s --soc_version=SS928V100 --image_list=./datasets/prep_data_aipp/yolo_real.bin + atc --framework=5 --model=model/yolov5s.onnx --input_shape=images:1,3,640,640 --output=model/yolov5s --soc_version=SS928V100 --image_list=./datasets/prep_data_aipp/yolo_real.bin - atc --compile_mode=6 --framework=5 --model=model/reid_net.onnx --input_shape=input:1,3,128,64 --output=model/reid_net --soc_version=SS928V100 --image_list=./datasets/prep_data_aipp/reid_real.bin + atc --framework=5 --model=model/reid_net.onnx --input_shape=input:1,3,128,64 --output=model/reid_net --soc_version=SS928V100 --image_list=./datasets/prep_data_aipp/reid_real.bin ``` Hi3403V100 NNN上的 om 模型转换命令 ```bash cd samples/built-in/tracking/deepsort - atc --framework=5 --model==model/yolov5s.onnx --input_shape=images:1,3,640,640 --output=/home/hispark/code/deepsort/ZQpei/deep_sort_pytorch/om_output/yolov5s --enable_single_stream=true --soc_version=OPTG + atc --framework=5 --model=model/yolov5s.onnx --input_shape=images:1,3,640,640 --output=model/yolov5s --enable_single_stream=true --soc_version=OPTG - atc --framework=5 --model=/home/hispark/code/deepsort/ZQpei/deep_sort_pytorch/onnx_output/reid_net.onnx --input_shape=input:1,3,128,64 --output=/home/hispark/code/deepsort/ZQpei/deep_sort_pytorch/om_output/reid_net --enable_single_stream=true --soc_version=OPTG + atc --framework=5 --model=model/reid_net.onnx --input_shape=input:1,3,128,64 --output=model/reid_net --enable_single_stream=true --soc_version=OPTG ``` 运行成功后生成 `model/yolov5s.om` 和 `model/reid_net.om` 模型文件。 ## 精度&性能评估 -1. 修改配置文件 `cfg.txt`（可选用于调整置信度等参数）。 +1. 修改配置文件 `data/deepsort.json`（可选用于调整置信度等参数）。 2. 运行推理。（**your_path**是板端文件系统具体的路径前缀） ```bash cd ${your_path}/modelzoo/samples/built-in/tracking/deepsort/out export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${your_path}/modelzoo/samples/samples_GPL/opensource/opencv/lib" - ./main model/yolov5s.om model/reid_net.om datasets/MOT16/train/MOT16-09/img1 + for folder in MOT16-02 MOT16-04 MOT16-05 MOT16-09 MOT16-10 MOT16-11 MOT16-13; do + ./main ../model/yolov5s.om ../model/reid_net.om ../datasets/MOT16/train/$folder/img1 + done ``` 图片推理结果会保存在 `out` 目录下。 > **注意**：图片路径下图片文件名需要按照时间升序排列。 + 3. 精度验证。 在服务器使用 `script/evaluate_mot.py` 进行精度评估。 ```bash # 指定结果目录和 GT 目录 - python evaluate_mot.py --ts_dir ${your_path}/modelzoo/samples/built-in/tracking/deepsort/out --gt_dir ${your_path}/modelzoo/samples/built-in/tracking/deepsort/datasets/MOT16 + python evaluate_mot.py --ts_dir ${your_path}/modelzoo/samples/built-in/tracking/deepsort/out --gt_dir ${your_path}/modelzoo/samples/built-in/tracking/deepsort/datasets/MOT16/train # 只评估单个序列 python evaluate_mot.py --gt /path/to/gt.txt --ts /path/to/result.txt ``` @@ -290,14 +320,14 @@ pip3 install -r requirements.txt ``` IDF1 IDP IDR Rcll Prcn GT MT PT ML FP FN IDs FM MOTA MOTP IDt IDa IDm - MOT16-02 12.0% 24.3% 8.0% 25.4% 77.2% 54 5 18 31 1341 13295 496 499 15.1% 0.238 204 289 20 - MOT16-04 22.2% 35.8% 16.1% 40.9% 90.8% 83 7 39 37 1965 28111 1167 1229 34.3% 0.208 417 711 25 - MOT16-05 18.1% 21.5% 15.7% 57.8% 79.1% 125 22 81 22 1043 2879 470 390 35.6% 0.252 332 188 75 - MOT16-09 29.8% 32.3% 27.6% 64.2% 75.0% 25 5 18 2 1125 1882 305 284 37.0% 0.209 75 211 4 - MOT16-10 23.1% 32.6% 17.9% 44.3% 80.9% 54 7 24 23 1285 6860 509 598 29.7% 0.247 121 363 15 - MOT16-11 30.9% 36.8% 26.6% 59.0% 81.5% 69 11 32 26 1232 3761 436 399 40.8% 0.171 69 333 10 - MOT16-13 12.2% 23.1% 8.3% 30.4% 84.7% 107 6 48 53 629 7966 521 552 20.4% 0.260 195 340 36 - OVERALL 20.9% 31.7% 15.6% 41.3% 84.1% 517 63 260 194 8620 64754 3904 3951 30.0% 0.219 1413 2435 185 + MOT16-02 12.7% 25.6% 8.4% 25.3% 76.9% 54 5 19 30 1357 13323 489 512 14.9% 0.236 174 308 18 + MOT16-04 25.5% 41.1% 18.5% 40.8% 90.8% 83 6 39 38 1975 28163 1188 1210 34.1% 0.206 352 783 21 + MOT16-05 21.1% 25.0% 18.3% 57.9% 79.1% 125 20 84 21 1046 2870 455 402 35.9% 0.249 287 223 73 + MOT16-09 29.2% 31.6% 27.1% 64.6% 75.4% 25 6 17 2 1109 1862 301 290 37.8% 0.214 76 204 6 + MOT16-10 30.9% 44.6% 23.7% 43.5% 82.0% 54 7 22 25 1177 6956 499 604 29.9% 0.247 93 385 15 + MOT16-11 33.5% 40.0% 28.7% 59.1% 82.3% 69 9 34 26 1165 3753 457 431 41.4% 0.171 62 357 8 + MOT16-13 13.0% 25.2% 8.7% 29.5% 85.3% 107 4 49 54 581 8069 563 594 19.5% 0.258 163 395 31 + OVERALL 23.8% 36.3% 17.7% 41.1% 84.4% 517 57 264 196 8410 64996 3952 4043 29.9% 0.218 1207 2655 172 ``` 4. 推理耗时和 FPS(ReID)。 @@ -307,7 +337,7 @@ pip3 install -r requirements.txt ```sh cd ${your_path}/modelzoo/samples/built-in/tracking/deepsort/out - ./main model/yolov5s.om model/reid_net.om datasets/MOT16/train/MOT16-09/img1 data/deepsort.json data/deepsort.json + ./main ../model/yolov5s.om ../model/reid_net.om ../datasets/MOT16/train/MOT16-09/img1 ../data/deepsort.json ``` Hi3403V100 SVP_NNN平台上性能结果： @@ -328,5 +358,5 @@ pip3 install -r requirements.txt | 芯片型号 | Batch Size | 数据集 | MOTA | 性能（fps） | | ----------- | ---------- | -------- | ------------------ | ------------------ | -| Hi3403V100 SVP_NNN | 1 | MOTA16 | 30.0% | 693.28 | -| Hi3403V100 NNN | 1 | MOTA16 | 29.9% | 70.49 | \ No newline at end of file +| Hi3403V100 SVP_NNN | 1 | MOT16 | 30.0% | 693.28 | +| Hi3403V100 NNN | 1 | MOT16 | 29.9% | 70.49 | \ No newline at end of file diff --git a/samples/built-in/tracking/deepsort/deepsort.json b/samples/built-in/tracking/deepsort/deepsort.json new file mode 100644 index 0000000000000000000000000000000000000000..213a9324124fc2c5344377c7da9cb2f638a23942 --- /dev/null +++ b/samples/built-in/tracking/deepsort/deepsort.json @@ -0,0 +1,146 @@ +{ + "modelName": "DeepSORT", + "modelDesc": "DeepSORT（Deep Simple Online and Realtime Tracking）是一种经典的多目标追踪算法，在原始 SORT 算法基础上引入了深度外观特征（ReID），有效降低了 ID Switch 率。该模型在 MOT16 数据集上进行了验证", + "modelRepository": "https://gitee.com/HiSpark/modelzoo/tree/master/samples/built-in/tracking/deepsort", + "modelParameter": { + "输入": "128*64", + "参数量": "11.164M", + "计算量": "2.253GFLOPs" + }, + "isBeta": false, + "betaDesc": "", + "modelFeedback": "https://developers.hisilicon.com/forum/0155201230363076006", + "modelUsageScenes": [ + { + "task": "Vision", + "tags": [ + "多目标跟踪" + ] + } + ], + "modelFrame": [ + "PyTorch" + ], + "modelDatasets": { + "desc": "MOT16", + "link": "https://motchallenge.net/data/MOT16/" + }, + "srcModelLicense": [ + { + "desc": "源模型", + "link": "https://github.com/ZQPei/deep_sort_pytorch/blob/master/LICENSE" + } + ], + "modelLicense": [ + { + "desc": "部署模型", + "link": "https://gitee.com/HiSpark/modelzoo/blob/master/samples/built-in/tracking/deepsort/LICENSE" + } + ], + "quickStart": { + "md": "https://gitee.com/HiSpark/modelzoo/blob/master/samples/built-in/tracking/deepsort/README.md" + }, + "modelChipset": [ + { + "chipset": "Hi3403V100 SVP_NNN", + "tools": [ + { + "name": "CANN工具", + "link": "https://hispark-obs.obs.cn-east-3.myhuaweicloud.com/SVP_NNN_PC_V1.0.6.0.tgz", + "desc": "AI异构计算架构，承上启下，提升计算效率的关键平台" + }, + { + "name": "编译工具链", + "link": "https://gitee.com/HiSpark/pegasus/blob/Beta-v0.9.1/docs/Hi3403V100%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA%E6%8C%87%E5%8D%97/Hi3403V100%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA%E6%8C%87%E5%8D%97.md#241%E5%AE%89%E8%A3%85clang%E4%BA%A4%E5%8F%89%E7%BC%96%E8%AF%91%E5%99%A8", + "desc": "高效编译，精准适配AI性能优化，应用流畅运行" + }, + { + "name": "SDK", + "link": "https://gitee.com/HiSpark/ss928v100_clang/tree/Beta-v0.9.1/ ", + "desc": "稳定、易用的设计，支撑客户快速产品量产" + } + ], + "os": [ + "OpenHarmony", + "Linux" + ], + "performance": [ + { + "quantMode": "a8w8", + "detail": [ + { + "performance": 1.14, + "performanceUnit": "耗时（ms）", + "performanceDesc": "" + }, + { + "performance": 693.28, + "performanceUnit": "性能（fps）", + "performanceDesc": "" + }, + { + "performance": 8.003, + "performanceUnit": "单帧内存带宽（MB）", + "performanceDesc": "" + }, + { + "performance": 59.863, + "performanceUnit": "内存（MB）", + "performanceDesc": "" + } + ] + } + ] + }, + { + "chipset": "Hi3403V100 NNN", + "tools": [ + { + "name": "CANN工具包", + "link": "", + "desc": "5.30.t11.7.b140 (请联系FAE获取)" + }, + { + "name": "编译工具链", + "link": "", + "desc": "aarch64-mix210-linux-gcc (请联系FAE获取)" + }, + { + "name": "SDK", + "link": "", + "desc": "SPC022 (请联系FAE获取)" + } + ], + "os": [ + "Linux" + ], + "performance": [ + { + "quantMode": "f16", + "detail": [ + { + "performance": 14.19, + "performanceUnit": "耗时（ms）", + "performanceDesc": "" + }, + { + "performance": 70.49 , + "performanceUnit": "性能（fps）", + "performanceDesc": "" + }, + { + "performance": 30.732, + "performanceUnit": "单帧内存带宽（MB）", + "performanceDesc": "" + }, + { + "performance": 157.676, + "performanceUnit": "内存（MB）", + "performanceDesc": "" + } + ] + } + ] + } + ] +} \ No newline at end of file diff --git a/samples/built-in/tracking/deepsort/requirements.txt b/samples/built-in/tracking/deepsort/requirements.txt index 6b5686e2f66df5a4058d058d838e59aa3542228f..304b32c818832d9c861426a61566cf4b6b36d788 100644 --- a/samples/built-in/tracking/deepsort/requirements.txt +++ b/samples/built-in/tracking/deepsort/requirements.txt @@ -1,72 +1,46 @@ -apturl==0.5.2 -asn1crypto==0.24.0 -attrs==22.2.0 -Brlapi==0.6.6 -certifi==2018.1.18 -chardet==3.0.4 -click==6.7 -colorama==0.3.7 -command-not-found==0.3 -cryptography==2.1.4 -cupshelpers==1.0 -decorator==5.1.1 -defer==1.0.6 -distro==1.9.0 -distro-info===0.18ubuntu0.18.04.1 -httplib2==0.9.2 -idna==2.6 -keyring==10.6.0 -keyrings.alt==3.0 -language-selector==0.1 -launchpadlib==1.10.6 -lazr.restfulclient==0.13.5 -lazr.uri==1.0.3 -louis==3.5.0 -macaroonbakery==1.1.3 -Mako==1.0.7 -MarkupSafe==1.0 +certifi==2026.2.25 +charset-normalizer==3.4.6 +contourpy==1.1.1 +cycler==0.12.1 +decorator==5.2.1 +exceptiongroup==1.3.1 +flake8==7.1.2 +flake8_import_order==0.19.2 +fonttools==4.57.0 +future==1.0.0 +idna==3.11 +importlib_resources==6.4.5 +iniconfig==2.1.0 +kiwisolver==1.4.7 +matplotlib==3.7.5 +mccabe==0.7.0 +motmetrics==1.2.0 mpmath==1.3.0 -netifaces==0.10.4 -numpy==1.19.5 -oauth==1.0.1 -olefile==0.45.1 -opencv-python==4.12.0.88 -packaging==21.3 -pexpect==4.2.1 -Pillow==5.1.0 -protobuf==3.0.0 -pycairo==1.16.2 -pycrypto==2.6.1 -pycups==1.9.73 -PyGObject==3.26.1 -pymacaroons==0.13.0 -PyNaCl==1.1.2 +numpy==1.23.5 +opencv-python==4.8.1.78 +packaging==26.0 +pandas==1.5.3 +Pillow==9.0.1 +pluggy==1.5.0 +py-cpuinfo==9.0.0 +pycodestyle==2.12.1 +pyflakes==3.2.0 pyparsing==3.1.4 -pyRFC3339==1.0 -python-apt==1.6.6 -python-dateutil==2.6.1 -python-debian==0.1.32 -pytz==2018.3 -pyxdg==0.25 -PyYAML==3.12 -reportlab==3.4.0 -requests==2.18.4 -requests-unixsocket==0.1.5 -scikit-build==0.16.7 -SecretStorage==2.3.1 -simplejson==3.13.2 -six==1.11.0 -ssh-import-id==5.7 -sympy==1.9 -system-service==0.3 -systemd-python==234 -typing_extensions==4.1.1 -ubuntu-drivers-common==0.0.0 -ubuntu-pro-client==8001 -ufw==0.36 -unattended-upgrades==0.1 -urllib3==1.22 -usb-creator==0.3.3 -wadllib==1.3.2 -xkit==0.0.0 -zope.interface==4.3.2 +pytest==8.3.5 +pytest-benchmark==4.0.0 +python-dateutil==2.9.0.post0 +pytz==2026.1.post1 +PyYAML==6.0.3 +requests==2.31.0 +scipy==1.10.1 +seaborn==0.11.2 +six==1.17.0 +sympy==1.13.3 +tomli==2.4.1 +torch==1.12.1 +torchvision==0.13.1 +tqdm==4.65.2 +typing_extensions==4.13.2 +urllib3==2.2.3 +xmltodict==0.15.0 +zipp==3.20.2 diff --git a/samples/built-in/tracking/deepsort/src/CMakeLists.txt b/samples/built-in/tracking/deepsort/src/CMakeLists.txt index 45f2a7f6b7db891a1dfd72db35e38c6b7e5de82f..a3822795117049d44f1a35df18a6c3e470749c89 100644 --- a/samples/built-in/tracking/deepsort/src/CMakeLists.txt +++ b/samples/built-in/tracking/deepsort/src/CMakeLists.txt @@ -53,7 +53,7 @@ include_directories( find_library(OpenCV_WORLD_SHARED_LIB NAMES libopencv_world.so.412 - PATHS "../../../../opensource/opencv/lib/aarch64_ohos" + PATHS ${OPENCV_LIB_PATH} NO_DEFAULT_PATH )