注册 登录
电子工程世界-论坛 返回首页 EEWORLD首页 频道 EE大学堂 下载中心 Datasheet 专题
LitchiCheng的个人空间 https://home.eeworld.com.cn/space-uid-1003127.html [收藏] [复制] [分享] [RSS]
日志

rk3399使用阿里推理引擎MNN使用cpu和gpu进行benchmark,OpenCL效果不佳?

已有 132 次阅读2024-3-10 11:24 |个人分类:图像识别

背景

MNN是阿里开源的推理引擎,今天测试一下在rk3399平台上的benchmark怎么样?

alibaba/MNN: MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba (github.com)

首先git clone

git clone git@github.com:alibaba/MNN.git

创建build目录

cd MNN
mkdir build
cd build

cmake配置

注意交叉编译器以及opencl库的使用方式,是使用系统opencl库还是使用wrap进行dlopen加载

cmake .. \
-DCMAKE_BUILD_TYPE=Release \
-DMNN_BUILD_DEMO=ON \
-DMNN_BUILD_BENCHMARK=true \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSTEM_VERSION=1 \
-DCMAKE_SYSTEM_PROCESSOR=aarch64 \
-DMNN_OPENCL=ON \
-DMNN_USE_SYSTEM_LIB=ON \
-DCMAKE_C_COMPILER=${cross_compile_toolchain}/bin/aarch64-linux-gnu-gcc \
-DCMAKE_CXX_COMPILER=${cross_compile_toolchain}/bin/aarch64-linux-gnu-g++

make -j32

部署

然后将build目录下的libMNN.so以及benchmark.out和上级目录下的benchmark的model放到一起,同时libMNN.so需要放到rk3399的lib目录下

sudo cp libMNN.so /lib
sudo cp -rf ../benchmark/model .

然后运行benchmark测试,第二个参数:loop测试次数,第4个参数:0代表使用cpu,3代表使用opencl

cpu测试

firefly@firefly:~/MNN$ sudo ./benchmark.out models/ 1 0 0clear
MNN benchmark
Forward type: CPU thread=4 precision=2 sparsity=0 sparseBlockOC=1 testQuantizedModel=0
--------> Benchmarking... loop = 1, warmup = 0
[-INFO-]: precision=2, use fp16 inference if your device supports and open MNN_ARM82=ON.
The device support i8sdot:0, support fp16:0, support i8mm: 0
[ - ] SqueezeNetV1.0.mnn          max =   86.128 ms  min =   86.128 ms  avg =   86.128 ms
[ - ] MobileNetV2_224.mnn         max =   42.041 ms  min =   42.041 ms  avg =   42.041 ms
[ - ] inception-v3.mnn            max =  505.111 ms  min =  505.111 ms  avg =  505.111 ms
[ - ] mobilenetV3.mnn             max =   13.533 ms  min =   13.533 ms  avg =   13.533 ms
[ - ] nasnet.mnn                  max =  145.489 ms  min =  145.489 ms  avg =  145.489 ms
[ - ] mobilenet-v1-1.0.mnn        max =   66.624 ms  min =   66.624 ms  avg =   66.624 ms
[ - ] squeezenetv1.1.mnn          max =   40.437 ms  min =   40.437 ms  avg =   40.437 ms
[ - ] resnet-v2-50.mnn            max =  308.836 ms  min =  308.836 ms  avg =  308.836 ms

gpu测试

firefly@firefly:~/MNN$ sudo ./benchmark.out models/ 1 0 3
MNN benchmark
Forward type: OpenCL thread=4 precision=2 sparsity=0 sparseBlockOC=1 testQuantizedModel=0
--------> Benchmarking... loop = 1, warmup = 0
[-INFO-]: precision=2, use fp16 inference if your device supports and open MNN_ARM82=ON.
The device support i8sdot:0, support fp16:0, support i8mm: 0
arm_release_ver of this libmali is 'r18p0-01rel0', rk_so_ver is '4'.[ - ] SqueezeNetV1.0.mnn          max =  159.619 ms  min =  159.619 ms  avg =  159.619 ms
[ - ] MobileNetV2_224.mnn         max =  126.671 ms  min =  126.671 ms  avg =  126.671 ms
[ - ] inception-v3.mnn            max =  800.436 ms  min =  800.436 ms  avg =  800.436 ms
[ - ] mobilenetV3.mnn             max =   61.661 ms  min =   61.661 ms  avg =   61.661 ms
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
[ - ] nasnet.mnn                  max =  140.189 ms  min =  140.189 ms  avg =  140.189 ms
[ - ] mobilenet-v1-1.0.mnn        max =   98.918 ms  min =   98.918 ms  avg =   98.918 ms
[ - ] squeezenetv1.1.mnn          max =  121.158 ms  min =  121.158 ms  avg =  121.158 ms
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
Map error scalePtrCL == nullptr
Map error biasPtrCL == nullptr
[ - ] resnet-v2-50.mnn            max =  428.075 ms  min =  428.075 ms  avg =  428.075 ms

结论

可以看到,gpu使用上很慢且存在算子的问题,实际上在rk3568上测试opencl很流畅且没有问题,这里留下问题,之后探究

 

本文来自论坛,点击查看完整帖子内容。

评论 (0 个评论)

facelist doodle 涂鸦板

您需要登录后才可以评论 登录 | 注册

热门文章