30、OAK摄像头使用官方的yolox进行初训练和测试

基本思想：想法是使用OAK深度相机进行目标检测和测距，整个平台在树莓派上运行，同时使用超声波测距测试，对比一下OAK测试距离是否准确，先测试自训练一下yolox，实测一下效果，代码例程是官方的，资料有点零散，只是做了整理和测试

链接：https://pan.baidu.com/s/1vF7zbdyqpsfwmAV8Xmo9MQ
提取码：th5e

第一步：先下载个模型测试一下免费计算机视觉预训练模型合集 – OAKChina

配置环境

3.1. USB 系列 — DepthAI Docs 0.3.0.0 documentation

1.4.4. Ubuntu — DepthAI Docs 0.3.0.0 documentation

测试命令

buntu@ubuntu:~/depthai$ pip install sentry-sdk
untu@ubuntu:~/depthai$ python3 depthai_demo.py -usbs usb2

测试画面

第二步、运行demo yolox，先测试一下模型

ubuntu@ubuntu:~$ git clone https://gitee.com/oakchina/depthai-examples.git
ubuntu@ubuntu:~$ cd depthai-examples/
ubuntu@ubuntu:~/depthai-examples$ cd yolox/
ubuntu@ubuntu:~/depthai-examples/yolox$ python3 yolox.py
Available devices:
[0] 184430102113691200 [X_LINK_UNBOOTED]
Creating pipeline...
Creating Color Camera...
Starting pipeline...

第三步：训练自己的yolox模型，置换官网的example-yolox原来的模型

ubuntu@ubuntu-Super-Server:~/sxj731533730$ git clone https://github.com/Megvii-BaseDetection/YOLOX
Cloning into 'YOLOX'...
remote: Enumerating objects: 1656, done.
remote: Total 1656 (delta 0), reused 0 (delta 0), pack-reused 1656
Receiving objects: 100% (1656/1656), 6.02 MiB | 3.62 MiB/s, done.
Resolving deltas: 100% (972/972), done.
ubuntu@ubuntu-Super-Server:~/sxj731533730$ cd YOLOX
ubuntu@ubuntu-Super-Server:~/sxj731533730/YOLOX$ git checkout -b v0.1.0
Switched to a new branch 'v0.1.0'

准备好数据集，然后开始训练

1）将voc数据集转成coco数据集
https://sxj731533730.blog.csdn.net/article/details/90046780

2）修改配置文件

ubuntu@ubuntu-Super-Server:~/sxj731533730/YOLOX$ vim /home/ubuntu/sxj731533730/YOLOX/exps/example/custom# Define yourself dataset pathself.data_dir = "/home/ubuntu/sxj731533730/YOLOX/datasets"self.train_ann = "instances_train2017.json"self.val_ann = "instances_val2017.json"self.num_classes = 24

修改类别信息

ubuntu@ubuntu-Super-Server:~/sxj731533730/YOLOX$ vim yolox/data/datasets/coco_classes.py

修改类别数量，这个和训练无关，和转onnx有关

/home/ubuntu/sxj731533730/YOLOX/yolox/exp/yolox_base.py
self.num_classes = 24

类别

COCO_CLASSES=('Y0','Y1','Y2','Y3','R0','R1','R2','R3','R4','R5','R6','R7','R8','R9','G0','G1','G2','G3','G4','G5','G6','G7','G8','G9',)

3)数据集的目录 train2017/val2017 存放图片

ubuntu@ubuntu-Super-Server:~/sxj731533730/YOLOX$ tree -L 1 datasets/
datasets/
├── annotations
├── README.md
├── train2017
├── val2017
└── voc2coco.py3 directories, 2 filesubuntu@ubuntu-Super-Server:~/sxj731533730/YOLOX$ tree -L 1 datasets/annotations/
datasets/annotations/
├── instances_train2017.json
└── instances_val2017.json0 directories, 2 files

4）开始训练模型

ubuntu@ubuntu-Super-Server:~/sxj731533730/YOLOX$ python3 tools/train.py -f exps/example/custom/nano.py -d 1 -b 32 --fp16 -o -c weights/yolox_nano.pth
Unable to init server: Could not connect: Connection refused
Unable to init server: Could not connect: Connection refused(train.py:785661): Gdk-CRITICAL **: 23:27:50.391: gdk_cursor_new_for_display: assertion 'GDK_IS_DISPLAY (display)' failed
2022-05-17 23:27:50 | INFO     | yolox.core.trainer:130 - args: Namespace(batch_size=32, cache=False, ckpt='weights/yolox_nano.pth', devices=1, dist_backend='nccl', dist_url=None, exp_file='exps/example/custom/nano.py', experiment_name='nano', fp16=True, logger='tensorboard', machine_rank=0, name=None, num_machines=1, occupy=True, opts=[], resume=False, start_epoch=None)
2022-05-17 23:27:50 | INFO     | yolox.core.trainer:131 - exp value:

训练结束

2022-05-19 03:11:26 | INFO     | yolox.core.trainer:342 -
Average forward time: 0.75 ms, Average NMS time: 0.88 ms, Average inference time: 1.63 msAverage Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.750Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.995Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.928Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.752Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.400Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.768Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.787Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.787Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.788Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.4002022-05-19 03:11:26 | INFO     | yolox.core.trainer:352 - Save weights to ./YOLOX_outputs/nano
2022-05-19 03:11:26 | INFO     | yolox.core.trainer:352 - Save weights to ./YOLOX_outputs/nano
2022-05-19 03:11:27 | INFO     | yolox.core.trainer:195 - Training of experiment is done and the best AP is 75.03

第四步：开始转模型pth-onnx-openvino-oak

1)转模型pth模型到onnx

ubuntu@ubuntu-Super-Server:~/sxj731533730/YOLOX$ python3 export_onnx.py --output-name yolox_nano.onnx -n yolox-nano -c /home/ubuntu/sxj731533730/YOLOX/YOLOX_outputs/nano/best_ckpt.pth
2022-05-19 03:23:04.048 | INFO     | __main__:main:64 - args value: Namespace(batch_size=1, ckpt='/home/ubuntu/sxj731533730/YOLOX/YOLOX_outputs/nano/best_ckpt.pth', decode_in_inference=False, dynamic=False, exp_file=None, experiment_name=None, input='images', name='yolox-nano', no_onnxsim=False, opset=11, opts=[], output='output', output_name='yolox_nano.onnx')
2022-05-19 03:23:04.167 | INFO     | __main__:main:88 - loading checkpoint done.
2022-05-19 03:23:11.454 | INFO     | __main__:main:101 - generated onnx model named yolox_nano.onnx
2022-05-19 03:23:11.837 | INFO     | __main__:main:117 - generated simplified onnx model named yolox_nano.onnx

测试一下onnx

ubuntu@ubuntu-Super-Server:~/sxj731533730/YOLOX$ python3 demo/ONNXRuntime/onnx_inference.py -m  yolox_nano.onnx  -i test.jpg -o  result -s 0.3 --input_shape 416,416

转换模型到OpenVINO，首先配置一下PC端的OpenVINO的环境

2)配置一下OpenVINO

ubuntu@ubuntu:~/Downloads$ axel -n 100 https://registrationcenter-download.intel.com/akdlm/IRC_NAS/18096/l_openvino_toolkit_p_2021.4.689.tgz
ubuntu@ubuntu:~/Downloads$ cd l_openvino_toolkit_p_2021.4.689/
ubuntu@ubuntu:~/Downloads/l_openvino_toolkit_p_2021.4.689$ sudo ./install_GUI.sh
ubuntu@ubuntu:~$ cd /opt/intel/openvino_2021/install_dependencies/
ubuntu@ubuntu:/opt/intel/openvino_2021/install_dependencies$ sudo -E ./install_openvino_dependencies.sh
ubuntu@ubuntu:/opt/intel/openvino_2021/bin$ sudo vim ~/.bashrc
在末尾添加
source /opt/intel/openvino_2021/bin/setupvars.sh
ubuntu@ubuntu:/opt/intel/openvino_2021/bin$ source ~/.bashrc
[setupvars.sh] OpenVINO environment initialized
ubuntu@ubuntu:/opt/intel/openvino_2021/bin$ cd /opt//intel/openvino_2021/deployment_tools/model_optimizer//install_prerequisites/
ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/model_optimizer/install_prerequisites$ sudo ./install_prerequisites.sh

测试检测分类

ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/demo$ sudo ./demo_squeezenet_download_convert_run.sh Top 10 results:Image /opt/intel/openvino_2021.4.689/deployment_tools/demo/car.pngclassid probability label
------- ----------- -----
817     0.6853030   sports car, sport car
479     0.1835197   car wheel
511     0.0917197   convertible
436     0.0200694   beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon
751     0.0069604   racer, race car, racing car
656     0.0044177   minivan
717     0.0024739   pickup, pickup truck
581     0.0017788   grille, radiator grille
468     0.0013083   cab, hack, taxi, taxicab
661     0.0007443   Model T[ INFO ] Execution successful[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool###################################################Demo completed successfully.

3）测试图片

ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/demo$ sudo ./demo_security_barrier_camera.sh

测试图片效果

转模型

ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/model_optimizer$ sudo python3 mo.py --input_shape [1,3,416,416] --input_model ~/YOLOX/yolox_nano.onnx --data_type FP16ubuntu@ubuntu:~/YOLOX$ python3 demo/OpenVINO/python/openvino_inference.py  -m /opt/intel/openvino_2021/deployment_tools/model_optimizer/yolox_nano.xml -i test.jpg  -o demo_output  -s 0.2
/home/ubuntu/.local/lib/python3.8/site-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.interpolation: int = Image.BILINEAR,
/home/ubuntu/.local/lib/python3.8/site-packages/torchvision/transforms/functional_pil.py:295: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.interpolation: int = Image.NEAREST,
/home/ubuntu/.local/lib/python3.8/site-packages/torchvision/transforms/functional_pil.py:328: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.interpolation: int = Image.BICUBIC,
/home/ubuntu/.local/lib/python3.8/site-packages/thop/utils.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop workingfrom collections import Iterable
[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: /opt/intel/openvino_2021/deployment_tools/model_optimizer/yolox_nano.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode

转blob模型

ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/model_optimizer$ ls
extensions             mo.py                   requirements_tf.txt
install_prerequisites  mo_tf.py                requirements.txt
mo                     requirements_caffe.txt  setup.py
mo_caffe.py            requirements_kaldi.txt  version.txt
mo_kaldi.py            requirements_mxnet.txt  yolox_tiny.bin
mo_mxnet.py            requirements_onnx.txt   yolox_tiny.mapping
mo_onnx.py             requirements_tf2.txt    yolox_tiny.xml
ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/tools$ sudo chmod 77 compile_tool/
ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/tools/compile_tool$ ./compile_tool -m /opt/intel/openvino_2021/deployment_tools/model_optimizer/yolox_nano.xml -ip U8 -d MYRIAD -VPU_NUMBER_OF_SHAVES 4 -VPU_NUMBER_OF_CMX_SLICES 4
Inference Engine: IE version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4Network inputs:images : U8 / NCHW
Network outputs:output : FP16 / CHW
[Warning][VPU][Config] Deprecated option was used : VPU_MYRIAD_PLATFORM
Output file yolox_nano.blob can't be opened for writin
#如果无法写入文件， 就给整体目录一个777权限
ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/tools/compile_tool$ ./compile_tool -m /opt/intel/openvino_2021/deployment_tools/model_optimizer/yolox_nano.xml -ip U8 -d MYRIAD -VPU_NUMBER_OF_SHAVES 4 -VPU_NUMBER_OF_CMX_SLICES 4
Inference Engine: IE version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4Network inputs:images : U8 / NCHW
Network outputs:output : FP16 / CHW
[Warning][VPU][Config] Deprecated option was used : VPU_MYRIAD_PLATFORM
Done. LoadNetwork time elapsed: 5232 ms
ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/tools/compile_tool$ ls
compile_tool  README.md  yolox_nano.blob

第五步：测试一下模型

1）下载官网的example代码

[setupvars.sh] OpenVINO environment initialized
ubuntu@ubuntu:~$ git clone https://github.com/OAKChina/depthai-examples.git

模型替换，将yolox中yolox_nano.onnx复制到models目录下，将ubuntu@ubuntu:/opt/intel/openvino_2021/deployment_tools/tools/compile_tool 产生的blob文件复制到model文件下，并且重命名为对应的名字yolox_nano_openvino_2021.4_shave.blob

2）删除这个均值和方差处理，因为我做转换没有使用，所以这里也不用了

#optimizer_params=[
#    "--scale_values=[58.395, 57.12 , 57.375]",
#    "--mean_values=[123.675, 116.28 , 103.53]",
#],

测试摄像头实时检测目标

/usr/bin/python3.8 /home/ubuntu/sxj/depthai-examples/yolox/yolox.py
Available devices:
[0] 184430102113691200 [X_LINK_UNBOOTED]
Creating pipeline...
Creating Color Camera...
[184430102113691200] [6.858] [NeuralNetwork(0)] [warning] Network compiled for 4 shaves, maximum available 13, compiling for 6 shaves likely will yield in better performance
Starting pipeline...

测试图片

使用官方的os镜像，测试也没啥问题，模型我就不放了

参考：

2、Python Labelme标注的json与LabelImg标注的xml 数文件相互转换、以及YOLO数据集 VOC数据格式、coco数据集（仅适用矩形框）_sxj731533730的博客-CSDN博客_labelme转labelimg

linux安装OpenVINO及配置_超哥--的博客-CSDN博客_linux openvino安装

OAK和 yolox完美结合，实现对足球、篮球、排球的识别和测量 - 知乎

YOLOX/demo/ONNXRuntime at main · Megvii-BaseDetection/YOLOX · GitHub

免费计算机视觉预训练模型合集 – OAKChina

30、OAK摄像头使用官方的yolox进行初训练和测试相关推荐

OpenVINO2021.4+YOLOX目标检测模型部署测试
点击上方"3D视觉工坊",选择"星标" 干货第一时间送达引言本文基于YOLOX的ONNX模型分别测试了YOLOX-Small与YOLOX-Tiny版本的模型 ...
YOLOX部署优化训练
用给一个小不知名小板卡部署一下不过这里用python 就当个玩具吧 YOLOX将近两年来目标检测领域的各个角度的优秀进展与YOLO进行了巧妙地集成组合并且重回Anchor Free的怀抱.本文详细的 ...
YOLOX——Win10下训练自定义VOC数据集
前言上个博客试着把yolox在win下的环境配置,然后就要试着训练自己的数据集,我这里用的是VOC格式的数据集,训练环境是win10 x64,CUDA11.0 cudnn 8.1 GPU 是GTX ...
Google官方网络框架Volley实战——QQ吉凶测试，南无阿弥陀佛！
Google官方网络框架Volley实战--QQ吉凶测试,南无阿弥陀佛! 这次我们用第三方的接口来做一个QQ吉凶的测试项目,代码依然是比较的简单无图无真相直接撸代码了,详细解释都已经写在注释里了 ...
ZBrush 4R8将于6月30停止预售，官方：预售期内8折优惠，你还在犹豫？
近日Pixologi发布了ZBrush 4R8,一直神龙见首不见尾的ZBrush 4R8终于在万众齐呼下落下帷幕,此次的ZBrush 4R8主要包括:Vector displacement Maps, ...
从140元到30元，BM官方电报群与成员轻松互动：EOS没死
据51BB8财经最新数据,EOS当前报价4.68美元,折合人民币32.31元. 这是什么概念?在4月末5月初,借超级节点(BP)竞选和主网上线的推波助澜,EOS的报价曾一度飙涨到140元人民币左右,从 ...
60分钟入门PyTorch，官方教程手把手教你训练第一个深度学习模型（附链接）
来源:机器之心本文约800字,建议阅读5分钟. 本文介绍了官方教程入门PyTorch的技巧训练. 近期的一份调查报告显示:PyTorch 已经力压 TensorFlow 成为各大顶会的主流深度学习框 ...
60分钟入门PyTorch，官方教程手把手教你训练第一个深度学习模型
点击我爱计算机视觉标星,更快获取CVML新技术本文转载自机器之心. 近期的一份调查报告显示:PyTorch 已经力压 TensorFlow 成为各大顶会的主流深度学习框架.想发论文,不学 PyTor ...
一根绳子从一头烧需30时分钟_小学生一分钟跳绳满分训练指南
小学生一分钟跳绳测试项目是体育老师和众多家庭的痛,大多数的孩子1分钟跳不到60个.而跳绳又是体质健康测试里占分比较高的一个项目,所以很多宝妈/宝爸们很是焦虑.因此,我们根据平时儿童跳绳训练的经验和网上 ...

30、OAK摄像头使用官方的yolox进行初训练和测试

30、OAK摄像头使用官方的yolox进行初训练和测试相关推荐

最新文章

热门文章