基于LPRNet的车牌识别算法移植与测试

课程全程将在[SOPHGO（算能）云平台上进行。

本次课程将介绍:

（1）SOPHGO（算能）云平台环境搭建

（2）LPRNet算法

（3）通过BMNNSDK 2.7.0进行LPRNet模型转换和量化

（4）实现算法的移植

（5）部署和测试

（6）相关链接

（1） SOPHGO（算能）云平台环境搭建

a. 开通云平台账号

参考： https://cloud.sophgo.com/tpu.pdf

b. 开发环境初始化

1) 进入命令行模式，进去默认在/home/sophgo目录

2) 切换成 root 权限

sudo -i

3) 安装驱动

cd /home/sophgo/bmnnsdk2-bm1684_v2.7.0/scripts

./install_driver_pcie.sh

ls /dev/bm*

如果可以看到以下设备节点，表示驱动安装成功：

4) 加载Docker，并初始化环境

cd /home/sophgo/
apt install unzip
unzip bmnnsdk2-bm1684-ubuntu-docker-py37.zip
cd bmnnsdk2-bm1684-ubuntu-docker-py37/
docker load -i bmnnsdk2-bm1684-ubuntu.docker

5) 通过脚本创建Docker容器

cd /home/sophgo/bmnnsdk2-bm1684_v2.7.0/
# 执行脚本创建Docker容器
./docker_run_bmnnsdk.sh

# 自动进入Docker容器
# 进行环境初始化
cd scripts/

# 更新 pip
/usr/local/bin/python3 -m pip install --upgrade pip

# 安装 nntc
./install_lib.sh nntc

# 执行脚本 envsetup_pcie.sh 设置环境变量
source ./envsetup_pcie.sh

# 安装sophon包
pip3 install /workspace/lib/sail/python3/pcie/py37/sophon-2.7.0-py3-none-any.whl

6) Docker 常用命令

# 启动 Docker容器
docker start <CONTAINER ID>

# 查看正在运行的 Docker容器
docker ps

# 进入 Docker 容器 docker exec [OPTIONS] CONTAINER COMMAND [ARG...]
docker exec -it <CONTAINER ID> bash

7) 通过云空间文件系统拷贝代码到Docker容器中

打开云空间文件系统：

上传文件：文件会被存储在服务器的 /tmp 目录下**

拷贝文件到Docker容器中：
Docker容器的workspace目录被映射到 /home/sophgo/bmnnsdk2-bm1684_v2.7.0/

cp /tmp/<待拷贝文件> /home/sophgo/bmnnsdk2-bm1684_v2.7.0/

（2）LPRNet简介

论文链接：[https://arxiv.org/abs/1806.10447v1](https://arxiv.org/abs/1806.10447v1)

代码仓库链接：[https://github.com/sirius-ai/LPRNet_Pytorch]

LPRNet 全称为 License Plate Recognition via Deep Neural Networks，是由Intel于2018年发表的一种轻量级卷积神经网络，可实现端到端的车牌识别。其优点主要包括以下三点：

1. LPRNet不需要字符预先分割，能够端到端进行模型训练和推理，支持可变长字符车牌识别，车牌识别的准确率高、算法实时性强。
2. LPRNet是第一个没有使用RNN的实时轻量级OCR算法，能够在各种设备上运行，包括嵌入式设备。
3. LPRNet的鲁棒性强，在视角和摄像畸变、光照条件恶劣、视角变化等复杂的情况下，仍表现出较好的识别效果。

（3）模型转换与量化

a. 准备模型与数据

1) 从Github上获取SOPHON示例项目：git clone -b 2.7.0 https://github.com/sophon-ai-algo/examples.git

LPRNet 示例项目路径：/examples/simple/lprnet**

附：也可通过本地直接拷贝该项目到云平台（参考 (1).b.7) ) 通过云空间文件系统拷贝代码到Docker容器中）**

2) 下载相关数据集和模型

当前路径为: /examples/simple/lprnet

打开script文件夹
cd script
运行 download.sh 脚本文件下载数据，数据模型文件被保存在/examples/simple/lprnet/data下
./download.sh

3) 训练好的Pytorch模型转换为JIT模型

由于 BMNNSDK2 中的PyTorch模型编译工具BMNETP只接受PyTorch的JIT模型(TorchScript模型)，需要用户自行将训练好的Pytorch模型进行转换。

在SOPHON示例项目/examples/simple/lprnet/data/models文件夹中有已经转换好的 LPRNet_model.torchscript 模型

示例：将训练好的 Final_LPRNet_model.pth 转换为 LPRNet_model.torchscript

从Github上获取LPRNet源码：git clone https://github.com/sirius-ai/LPRNet_Pytorch.git

import torch

from model.LPRNet import LPRNet

lpr_max_len=8
phase=False
class_num=68
dropout_rate=0

PATH_TO_PT_MODEL = 'Final_LPRNet_model.pth'
PATH_TO_JIT_MODEL = 'LPRNet_model.torchscript'

LPRNet 为官方源码 https://github.com/sirius-ai/LPRNet_Pytorch/blob/master/model/LPRNet.py
lprnet = LPRNet(lpr_max_len, phase, class_num, dropout_rate)

在CPU上加载网络模型
lprnet.eval().load_state_dict(torch.load(PATH_TO_PT_MODEL, map_location=torch.device('cpu')))
jit.trace
model = torch.jit.trace(lprnet.forward, torch.rand(1, 3, 24, 94))
保存JIT模型
torch.jit.save(model, PATH_TO_JIT_MODEL)

得到 LPRNet_model.torchscript 后，需要通过 BMNNSDK2 中的PyTorch模型编译工具BMNETP和量化工具生成：FP32 BModel 和 INT8 BModel

4) 生成FP32 BModel

BMNETP是针对pytorch的模型编译器，可以把pytorch模型直接编译成BMRuntime所需的执行指令。https://doc.sophgo.com/docs/docs_latest_release/nntc/html/usage/bmnetp.html#pytorch

将 LPRNet_model.torchscript 转换为 FP32 BModel的命令：

python3 -m bmnetp --net_name=lprnet \
--target=BM1684 \
--opt=1 \
--cmp=true \
--shapes=[1,3,24,94] \
--model=<LPRNet_model.torchscript的路径> \
--outdir=<输出文件夹的路径> \
--dyn=false

附：可通过运行scripts/gen_fp32bmodel.sh脚本文件（注意修改脚本文件中的模型所在路径），在data/models/fp32bmodel/下生成lprnet_fp32_1b4b.bmodel。

# 运行scripts/gen_fp32bmodel.sh脚本文件
./gen_fp32bmodel.sh

*选做5) 生成INT8 BModel(注意： LPRNet模型量化需在20220317之后版本的BMNNSDK2 SDK中进行！)

参考文档：https://doc.sophgo.com/docs/2.7.0/docs_latest_release/calibration-tools/html/module/chapter4.html#int8umodel

具体步骤：

方法一：一键量化 (注意模型所在的路径)

python3 -m ufw.cali.cali_model --net_name lprnet \
--model ../data/models/LPRNet_model.pt \
--cali_image_path ../data/images/test_md5 \
--cali_image_preprocess 'resize_h=24,resize_w=94;mean_value=127.5:127.5:127.5,scale=0.0078125,bgr2rgb=True' \
--input_shapes '[1,3,24,94]' \
--cali_iterations=450 \
--postprocess_and_calc_score_class=feature_similarity

方法二：分步量化 (注意模型所在的路径)

分步量化步骤：

1. 生成量化数据集lmdb

2. 原模型转 FP32 UModel

3. 量化 Int8 UModel

4. bmnetu 编译 Int8 BModel

可以通过运行scripts文件夹中的gen_int8bmodel.sh脚本文件（注意修改脚本文件中的模型所在路径）

6) 查看得到的BModel的信息

# 以lprnet_fp32_1b4b.bmodel为例

# bm_model.bin --info xxxx.bmodel
bm_model.bin --info lprnet_fp32_1b4b.bmodel

# 将得到模型信息：
bmodel version: B.2.2
chip: BM1684
create time: Fri Apr 15 09:25:42 2022

==========================================
net 0: [lprnet] static
------------
stage 0:
input: x.1, [1, 3, 24, 94], float32, scale: 1
output: 237, [1, 68, 18], float32, scale: 1
------------
stage 1:
input: x.1, [4, 3, 24, 94], float32, scale: 1
output: 237, [4, 68, 18], float32, scale: 1

device mem size: 5246160 (coeff: 1941776, instruct: 188928, runtime: 3115456)
host mem size: 0 (coeff: 0, runtime: 0)

# bmrt_test --bmodel xxxx.bmodel
bmrt_test --bmodel lprnet_fp32_1b4b.bmodel

# 将得到模型信息：
[BMRT][deal_with_options:1398] INFO:Loop num: 1
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1018] INFO:Loading bmodel from [lprnet_fp32_1b4b.bmodel]. Thanks for your patience...
[BMRT][load_bmodel:982] INFO:pre net num: 0, load net num: 1
[BMRT][bmrt_test:758] INFO:==> running network #0, name: lprnet, loop: 0
[BMRT][bmrt_test:1001] INFO:net[lprnet] stage[0], launch total time is 1697 us (npu 1579 us, cpu 118 us)
[BMRT][bmrt_test:1004] INFO:+++ The network[lprnet] stage[0] output_data +++
[BMRT][print_array:700] INFO:output data #0 shape: [1 68 18 ] < -145.928 -94.5729 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 ... > len=1224
[BMRT][bmrt_test:1049] INFO:load input time(s): 0.000073
[BMRT][bmrt_test:1050] INFO:calculate time(s): 0.001713
[BMRT][bmrt_test:1051] INFO:get output time(s): 0.000106
[BMRT][bmrt_test:1052] INFO:compare time(s): 0.000211
[BMRT][bmrt_test:1001] INFO:net[lprnet] stage[1], launch total time is 3485 us (npu 3399 us, cpu 86 us)
[BMRT][bmrt_test:1004] INFO:+++ The network[lprnet] stage[1] output_data +++
[BMRT][print_array:700] INFO:output data #0 shape: [4 68 18 ] < -145.928 -94.573 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 -119.698 ... > len=4896
[BMRT][bmrt_test:1049] INFO:load input time(s): 0.000155
[BMRT][bmrt_test:1050] INFO:calculate time(s): 0.003489
[BMRT][bmrt_test:1051] INFO:get output time(s): 0.000069
[BMRT][bmrt_test:1052] INFO:compare time(s): 0.000085

（4）算法移植

a. Python示例程序:

文件位置：/examples/simple/lprnet/python

由于python例程需要用到sail库，需要安装Sophon Inference:

pip3 install /workspace/lib/sail/python3/pcie/py37/sophon-x.x.x-py3-none-any.whl

lprnet_cv_cv_sail.py 代码

# -*- coding: utf-8 -*-
import os
import time
import cv2
import numpy as np
import argparse
#from pyrsistent import T
import sophon.sail as sail
import logging
logging.basicConfig(level=logging.DEBUG)

CHARS = ['京', '沪', '津', '渝', '冀', '晋', '蒙', '辽', '吉', '黑',
'苏', '浙', '皖', '闽', '赣', '鲁', '豫', '鄂', '湘', '粤',
'桂', '琼', '川', '贵', '云', '藏', '陕', '甘', '青', '宁',
'新',
'0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'J', 'K',
'L', 'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'U', 'V',
'W', 'X', 'Y', 'Z', 'I', 'O', '-'
]

CHARS_DICT = {i:char for i, char in enumerate(CHARS)}

# input: x.1, [1, 3, 24, 96], float32, scale: 1
class LPRNet(object):
def __init__(self, opt, img_size = [94, 24]):
self.batch_size = opt.batch_size

# load bmodel
model_path = opt.bmodel
print("using model {}".format(model_path))
self.net = sail.Engine(model_path, opt.tpu_id, sail.IOMode.SYSIO)
self.graph_name = self.net.get_graph_names()[0]
self.input_name = self.net.get_input_names(self.graph_name)[0]
#self.scale = self.net.get_input_scale(self.graph_name, self.input_name)
#self.scale *= 0.0078125
print("load bmodel success!")
self.img_size = img_size
self.dt = 0.0

# 图片预处理
def preprocess(self, img):
h, w, _ = img.shape
if h != self.img_size[1] or w != self.img_size[0]:
img = cv2.resize(img, self.img_size)
img = img.astype('float32')
img -= 127.5
img *= 0.0078125
img = np.transpose(img, (2, 0, 1))

# CHW to NCHW format
#img = np.expand_dims(img, axis=0)
# Convert the img to row-major order, also known as "C order":
#img = np.ascontiguousarray(img)
return img

def predict(self, tensor):
input_data = {self.input_name: np.array(tensor, dtype=np.float32)}
t0 = time.time()
# 使用sail.engine.process进行推理
outputs = self.net.process(self.graph_name, input_data)
self.dt += time.time() - t0
return list(outputs.values())[0]

# 后处理
def postprocess(self, outputs):
res = list()
#outputs = list(outputs.values())[0]
outputs = np.argmax(outputs, axis = 1)
for output in outputs:
no_repeat_blank_label = list()
pre_c = output[0]
if pre_c != len(CHARS) - 1:
no_repeat_blank_label.append(CHARS_DICT[pre_c])
for c in output:
if (pre_c == c) or (c == len(CHARS) - 1):
if c == len(CHARS) - 1:
pre_c = c
continue
no_repeat_blank_label.append(CHARS_DICT[c])
pre_c = c
res.append(''.join(no_repeat_blank_label))

return res

def __call__(self, img_list):
res_list = []
img_num = len(img_list)
img_input_list = []
for img in img_list:
img = self.preprocess(img)
img_input_list.append(img)

for beg_img_no in range(0, img_num, self.batch_size):
end_img_no = min(img_num, beg_img_no + self.batch_size)
if beg_img_no + self.batch_size > img_num:
for ino in range(beg_img_no, end_img_no):
img_input = np.expand_dims(img_input_list[ino], axis=0)
outputs = self.predict(img_input)
res = self.postprocess(outputs)
res_list.extend(res)
else:
img_input = np.stack(img_input_list[beg_img_no:end_img_no])
#print("=================")
#print(img_input)
outputs = self.predict(img_input)
#print(outputs)
res = self.postprocess(outputs)
res_list.extend(res)

return res_list

def get_time(self):
return self.dt

def main(opt):
lprnet = LPRNet(opt)
if os.path.isfile(opt.img_path):
# 图片解码
src_img = cv2.imdecode(np.fromfile(opt.img_path, dtype=np.uint8), -1)
#src_img = cv2.imread(opt.img_path)
#print(src_img[0])
#print(cv2.getBuildInformation())
res = lprnet([src_img])
logging.info("img:{}, res:{}".format(opt.img_path, res[0]))
else:
img_list = []
t1 = time.time()
for img_name in os.listdir(opt.img_path):
img_file = os.path.join(opt.img_path, img_name)
# 图片解码
src_img = cv2.imdecode(np.fromfile(img_file, dtype=np.uint8), -1)
img_list.append(src_img)

res_list = lprnet(img_list)

t2 = time.time()

Tp = 0
for i, img_name in enumerate(os.listdir(opt.img_path)):
logging.info("img:{}, res:{}".format(img_name, res_list[i]))
if opt.mode == 'val':
label = img_name.split('.')[0]
if res_list[i] == label:
Tp += 1
else:
logging.info("***wrong***")
#logging.info("img:{}, res:{}".format(img_name, res_list[i]))

if opt.mode == 'val':
cn = len(os.listdir(opt.img_path))
logging.info("ACC = %.4f" % (Tp / cn))


logging.info("------------------ Inference Time Info ----------------------")
inference_time = lprnet.get_time() / len(img_list)
logging.info("inference_time(ms): {:.2f}".format(inference_time * 1000))
total_time = t2 - t1
logging.info("total_time(ms): {:.2f}, img_num: {}".format(total_time * 1000, len(img_list)))
average_latency = total_time / len(img_list)
qps = 1 / average_latency
logging.info("average latency time(ms): {:.2f}, QPS: {:2f}".format(average_latency * 1000, qps))

def parse_opt():
parser = argparse.ArgumentParser(prog=__file__)
parser.add_argument('--mode', type=str, default='val')
parser.add_argument('--img_path', type=str, default='data/images/test', help='input image path')
parser.add_argument('--bmodel', type=str, default='scripts/fp32bmodel/lprnet_fp32.bmodel', help='input model path')
parser.add_argument("--batch_size", type=int, default=4)
parser.add_argument('--tpu_id', type=int, default=0, help='tpu id')
opt = parser.parse_args()
return opt

if __name__ == '__main__':
opt = parse_opt()
main(opt)

lprnet_sail_bmcv_sail.py 代码

# -*- coding: utf-8 -*-
import time
import os
import cv2
import numpy as np
import argparse
import sophon.sail as sail
import logging
logging.basicConfig(level=logging.DEBUG)

CHARS_DICT = {i:char for i, char in enumerate(CHARS)}

# input: x.1, [1, 3, 24, 96], float32, scale: 1
class LPRNet(object):
def __init__(self, opt):
self.batch_size = opt.batch_size
# load bmodel
model_path = opt.bmodel
print("using model {}".format(model_path))
self.net = sail.Engine(model_path, opt.tpu_id, sail.IOMode.SYSIO)
print("load bmodel success!")
self.graph_name = self.net.get_graph_names()[0]
self.input_name = self.net.get_input_names(self.graph_name)[0]
self.output_name = self.net.get_output_names(self.graph_name)[0]

self.input_shape = [self.batch_size, 3, 24, 94]
self.input_shapes = {self.input_name: self.input_shape}
self.output_shape = [self.batch_size, 68, 18]
self.input_dtype= self.net.get_input_dtype(self.graph_name, self.input_name)
self.output_dtype = self.net.get_output_dtype(self.graph_name, self.output_name)

# get handle to create input and output tensors
self.handle = self.net.get_handle()
self.input = sail.Tensor(self.handle, self.input_shape, self.input_dtype, False, False)
self.output = sail.Tensor(self.handle, self.output_shape, self.output_dtype, True, True)
self.input_tensors = {self.input_name: self.input}
self.output_tensors = {self.output_name: self.output}
# set io_mode
# self.net.set_io_mode(self.graph_name, sail.IOMode.SYSO)
# init bmcv for preprocess
self.bmcv = sail.Bmcv(self.handle)
self.img_dtype = self.bmcv.get_bm_image_data_format(self.input_dtype)

self.scale = self.net.get_input_scale(self.graph_name, self.input_name)
self.ab = [x * self.scale * 0.0078125 for x in [1, -127.5, 1, -127.5, 1, -127.5]]

self.dt = 0.0

# 图片解码
def decode_bmcv(self, img_file):
decoder = sail.Decoder(img_file)
img = decoder.read(self.handle)
return img

# 预处理
def preprocess_bmcv(self, input, output):
tmp = self.bmcv.vpp_resize(input, 94, 24)

self.bmcv.convert_to(tmp, output, ((self.ab[0], self.ab[1]), \
(self.ab[2], self.ab[3]), \
(self.ab[4], self.ab[5])))
self.bmcv.bm_image_to_tensor(output, self.input)

def predict(self):
t0 = time.time()
# 使用sail.engine.process进行推理
self.net.process(self.graph_name, self.input_tensors, self.input_shapes, self.output_tensors)
self.dt += time.time() - t0
real_output_shape = self.net.get_output_shape(self.graph_name, self.output_name)
outputs = self.output.asnumpy(real_output_shape)
return outputs

# 后处理
def postprocess(self, outputs):
res = list()
#outputs = list(outputs.values())[0]
outputs = np.argmax(outputs, axis = 1)
for output in outputs:
no_repeat_blank_label = list()
pre_c = output[0]
if pre_c != len(CHARS) - 1:
no_repeat_blank_label.append(CHARS_DICT[pre_c])
for c in output:
if (pre_c == c) or (c == len(CHARS) - 1):
if c == len(CHARS) - 1:
pre_c = c
continue
no_repeat_blank_label.append(CHARS_DICT[c])
pre_c = c
res.append(''.join(no_repeat_blank_label))

return res

def __call__(self, img_file_list):
img_num = len(img_file_list)
res_list = []
for beg_img_no in range(0, img_num, self.batch_size):
end_img_no = min(img_num, beg_img_no + self.batch_size)
if (beg_img_no + self.batch_size > img_num) or (self.batch_size != 4):
for ino in range(beg_img_no, end_img_no):
img0 = sail.BMImage()
img1 = sail.BMImage(self.handle, self.input_shape[2], self.input_shape[3], \
sail.Format.FORMAT_BGR_PLANAR, self.img_dtype)
# 图片解码
decoder = sail.Decoder(img_file_list[ino])
ret = decoder.read(self.handle, img0)
self.preprocess_bmcv(img0, img1)
outputs = self.predict()
res = self.postprocess(outputs)
res_list.extend(res)
else:
imgs_0 = sail.BMImageArray4D()
imgs_1 = sail.BMImageArray4D(self.handle, self.input_shape[2], self.input_shape[3], \
sail.Format.FORMAT_BGR_PLANAR, self.img_dtype)
for j in range(4):
# 图片解码
decoder = sail.Decoder(img_file_list[beg_img_no + j])
ret = decoder.read_(self.handle, imgs_0[j])
self.preprocess_bmcv(imgs_0, imgs_1)
outputs = self.predict()
res = self.postprocess(outputs)
res_list.extend(res)

return res_list

def get_time(self):
return self.dt

def main(opt):
lprnet = LPRNet(opt)
if os.path.isfile(opt.img_path):
res_list = lprnet([opt.img_path])
logging.info("img:{}, res:{}".format(opt.img_path, res_list[0]))
else:
Tp = 0
img_file_list = [os.path.join(opt.img_path, img_name) for img_name in os.listdir(opt.img_path)]

t1 = time.time()
res_list = lprnet(img_file_list)
t2 = time.time()

for i, img_name in enumerate(os.listdir(opt.img_path)):
logging.info("img:{}, res:{}".format(img_name, res_list[i]))
if opt.mode == 'val':
label = img_name.split('.')[0]
if res_list[i] == label:
Tp += 1

if opt.mode == 'val':
cn = len(os.listdir(opt.img_path))
logging.info("ACC = %.4f" % (Tp / cn))

logging.info("------------------ Inference Time Info ----------------------")
inference_time = lprnet.get_time() / len(img_file_list)
logging.info("inference_time(ms): {:.2f}".format(inference_time * 1000))
total_time = t2 - t1
logging.info("total_time(ms): {:.2f}, img_num: {}".format(total_time * 1000, len(img_file_list)))
average_latency = total_time / len(img_file_list)
qps = 1 / average_latency
logging.info("average latency time(ms): {:.2f}, QPS: {:2f}".format(average_latency * 1000, qps))

def parse_opt():
parser = argparse.ArgumentParser(prog=__file__)
#parser.add_argument('--img-size', type=int, default=[94, 24], help='inference size (pixels)')
parser.add_argument('--mode', type=str, default='val')
parser.add_argument('--img_path', type=str, default='/workspace/projects/LPRNet/data/images/test', help='input image path')
parser.add_argument('--bmodel', type=str, default='/workspace/bmnnsdk2-bm1684_v2.7.0_0317/examples/LPRNet/scripts/int8bmodel/lprnet_int8.bmodel', help='input model path')
parser.add_argument("--batch_size", type=int, default=1)
parser.add_argument('--tpu_id', type=int, default=0, help='tpu id')
#parser.add_argument('--format', type=str, default="fp32", help='model format fp32 or fix8b')
opt = parser.parse_args()
return opt

if __name__ == '__main__':
opt = parse_opt()
main(opt)
```

b. C++示例程序:

文件位置：/examples/simple/lprnet/cpp

（5）部署测试

a. Python示例程序运行（以lprnet_cv_cv_sail.py使用FP32 BModel为例）

注意：以下命令均在路径 examples/simple/lprnet/ 下运行, 使用不同示例程序仅需更换名称即可**

1) 测试单张图片

# 运行命令
python3 python/lprnet_cv_cv_sail.py --mode test --img_path data/images/test.jpg --bmodel data/models/fp32bmodel/lprnet_fp32_1b4b.bmodel --batch_size 1 --tpu_id 0

# 程序将输出：
...
using model data/models/fp32bmodel/lprnet_fp32_1b4b.bmodel
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1018] INFO:Loading bmodel from [data/models/fp32bmodel/lprnet_fp32_1b4b.bmodel]. Thanks for your patience...
[BMRT][load_bmodel:982] INFO:pre net num: 0, load net num: 1
load bmodel success!
INFO:root:img:data/images/test.jpg, res:皖A22B20
...

2) 测试整个文件夹

# 运行命令
python3 python/lprnet_cv_cv_sail.py --mode test --img_path data/images/test --bmodel data/models/fp32bmodel/lprnet_fp32_1b4b.bmodel --batch_size 4 --tpu_id 0

# 程序将输出整个文件夹的测试结果：
using model data/models/fp32bmodel/lprnet_fp32_1b4b.bmodel
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1018] INFO:Loading bmodel from [data/models/fp32bmodel/lprnet_fp32_1b4b.bmodel]. Thanks for your patience...
[BMRT][load_bmodel:982] INFO:pre net num: 0, load net num: 1
load bmodel success!
INFO:root:img:皖A225J1.jpg, res:皖A225J1
INFO:root:img:皖AMZ210.jpg, res:皖AMZ210
INFO:root:img:皖AAS656.jpg, res:皖AAS656
INFO:root:img:皖AC9165.jpg, res:皖AC9165
INFO:root:img:苏AP48A8.jpg, res:苏AP48A8
...
INFO:root:------------------ Inference Time Info ----------------------
INFO:root:inference_time(ms): 1.92
INFO:root:total_time(ms): 2147.03, img_num: 1000
INFO:root:average latency time(ms): 2.15, QPS: 465.759859

3) 测试整个文件夹，并计算准确率

# 运行命令
python3 python/lprnet_cv_cv_sail.py --mode val --img_path data/images/test --bmodel data/models/fp32bmodel/lprnet_fp32_1b4b.bmodel --batch_size 4 --tpu_id 0

# 程序将输出整个文件夹的测试结果：
...
INFO:root:img:皖NXT689.jpg, res:皖NXT689
INFO:root:img:皖AJ0J22.jpg, res:皖5M9B
INFO:root:***wrong***
INFO:root:img:皖AGX988.jpg, res:皖AGX988
INFO:root:img:皖AL8Y77.jpg, res:皖AL8Y77
INFO:root:img:皖AL136E.jpg, res:皖AL1365
INFO:root:***wrong***
INFO:root:img:皖A160R9.jpg, res:皖A160R9
INFO:root:ACC = 0.9000
INFO:root:------------------ Inference Time Info ----------------------
INFO:root:inference_time(ms): 1.89
INFO:root:total_time(ms): 2101.56, img_num: 1000
INFO:root:average latency time(ms): 2.10, QPS: 475.837511
...

b. C++示例程序运行（以lprnet_cv_cv_bmrt使用FP32 BModel为例）

文件位置：/examples/simple/lprnet/cpp/lprnet_cv_cv_bmrt

注意：以下命令均在路径 examples/simple/lprnet/cpp/lprnet_cv_cv_bmrt 下运行, 使用不同示例程序需更换至对应文件夹

第一步：编译

make -f Makefile.pcie

第二步：运行

1) 测试单张图片

# 运行命令
./lprnet_cv_cv_bmrt.pcie test ../../data/images/test.jpg ../../data/models/fp32bmodel/lprnet_fp32_1b.bmodel 0

# 程序将输出：
set device id:0
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1018] INFO:Loading bmodel from [../../data/models/fp32bmodel/lprnet_fp32_1b.bmodel]. Thanks for your patience...
[BMRT][load_bmodel:982] INFO:pre net num: 0, load net num: 1
> Load model lprnet successfully
input scale:1.000000
output scale:1.000000
fp32 input
fp32 output
input count:6768
Open /dev/bm-sophon0 successfully, device index = 0, jpu fd = 13, vpp fd = 13
../../data/images/test.jpg pred: 皖A22B20

2) 测试整个文件夹

# 运行命令
./lprnet_cv_cv_bmrt.pcie test ../../data/images/test/ ../../data/models/fp32bmodel/lprnet_fp32_4b.bmodel 0

# 程序将输出整个文件夹的测试结果：
set device id:0
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1018] INFO:Loading bmodel from [../../data/models/fp32bmodel/lprnet_fp32_4b.bmodel]. Thanks for your patience...
[BMRT][load_bmodel:982] INFO:pre net num: 0, load net num: 1
> Load model lprnet successfully
input scale:1.000000
output scale:1.000000
fp32 input
fp32 output
input count:27072
Open /dev/bm-sophon0 successfully, device index = 0, jpu fd = 14, vpp fd = 14
皖A225J1.jpg pred:皖A225J1
皖AMZ210.jpg pred:皖AMZ210
皖AAS656.jpg pred:皖AAS656
皖AC9165.jpg pred:皖AC9165
苏AP48A8.jpg pred:苏AP48A8
...

3) 测试整个文件夹，并计算准确率

# 运行命令
./lprnet_cv_cv_bmrt.pcie val ../../data/images/test/ ../../data/models/fp32bmodel/lprnet_fp32_4b.bmodel 0

# 程序将输出整个文件夹的测试结果：

set device id:0
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1018] INFO:Loading bmodel from [../../data/models/fp32bmodel/lprnet_fp32_4b.bmodel]. Thanks for your patience...
[BMRT][load_bmodel:982] INFO:pre net num: 0, load net num: 1
> Load model lprnet successfully
input scale:1.000000
output scale:1.000000
fp32 input
fp32 output
input count:27072
Open /dev/bm-sophon0 successfully, device index = 0, jpu fd = 14, vpp fd = 14
皖A225J1.jpg pred:皖A225J1
皖AMZ210.jpg pred:皖AMZ210
...
皖AL8Y77.jpg pred:皖AL8Y77
皖AL136E.jpg pred:皖AL1365
皖A160R9.jpg pred:皖A160R9
===========================
Acc = 895/1000=0.895000

（6）相关链接

样例开源仓库：https://github.com/sophon-ai-algo/examples

BM1684 BMNNSDK2入门文档：https://sophgo-doc.gitbook.io/bmnnsdk2-bm1684/

编译工具用户开发手册： https://doc.sophgo.com/docs/2.7.0/docs_latest_release/nntc/html/index.html

量化工具用户开发手册： https://doc.sophgo.com/docs/2.7.0/docs_latest_release/calibration-tools/html/index.html

算能量化工具介绍及使用说明：https://www.bilibili.com/video/BV1DA4y1S75p?spm_id_from=333.999.0.0

官网视频教程：https://developer.sophgo.com/site/index/course/all/all.html

官网文档中心：https://developer.sophgo.com/site/index/document/all/all.html

官网下载中心：https://developer.sophgo.com/site/index/material/all/all.html

官网论坛：https://developer.sophgo.com/forum/view/43.html