hollyedward的个人空间动态-电子工程世界

hollyedward

动态发布点评好友关于

2024-12-16
发表了主题帖：《RISC-V 体系结构编程与实践（第2版）》阅读体会一--初遇

感谢EEWORLD提供的《RISC-V 体系结构编程与实践（第2版）》书籍测评机会。准备先熟悉下模拟器环境，开始学习各种指令
2024-11-20
回复了主题帖：读书入围名单：《RISC-V 体系结构编程与实践（第2版）》

个人信息无误，确认可以完成阅读计划和打卡任务
2024-10-01
发表了主题帖：【2024 DigiKey 创意大赛】 1 得捷物料开箱贴

总算收到了得捷寄来的物料，接下来会尽快上手开始开发项目
2024-06-01
回复了主题帖：【AI挑战营第二站】算法工程化部署打包成SDK

onnx是一种保存模型的开放格式，适配很多框架，可以相互转换pth格式啥的 rknn时rockchip针对其下设备npu优化的神经网络推理框架，支持常用的算子 #AI挑战营第二站# 基于RKNN toolkit的模型转换与部署 - ARM技术 - 电子工程世界-论坛 (eeworld.com.cn)
回复了主题帖：【AI挑战营终点站】应用落地：部署手写数字识别应用到幸狐RV1106开发板

打卡，完善中#AI挑战营终点站# luckfox手写体识别部署rknn - ARM技术 - 电子工程世界-论坛 (eeworld.com.cn)
2024-05-31
发表了主题帖： #AI挑战营终点站# luckfox手写体识别部署rknn

本帖最后由 hollyedward 于 2024-5-31 22:19 编辑镜像烧录按照官方教程，将最新的开发板镜像烧录https://wiki.luckfox.com/zh/Luckfox-Pico/Luckfox-Pico-quick-start 需要安装驱动，使用烧录软件。安装SDK 有几种安装方式，也有docker打包的环境镜像在服务器上的虚拟机内下载官方的sdk 相关文件目录首先要，导出编译链地址至环境变量然后编译的时候cmake才能找到 export LUCKFOX_SDK_PATH=/home/user/luckfox-pico 执行命令对项目进行编译，生成最后的可执行文件 // 创建build目录 mkdir build cd build // 生成makefile文件 cmake .. // 使用make工具进行编译 make && make install 烧录镜像通过ssh连接开发板 ssh root@172.32.0.93 用scp将可执行文件和相关库传至开发板 scp -r ./luckfox_mnist_demo root@172.32.0.93:/root 给可执行文件或者文件夹赋权 chmod -R 777 ./luckfox_mnist_demo 运行脚本关闭系统默认 rkipc 程序 RkLunch-stop.sh 运行可执行文件 ./luck_mnist_rtsp_demo ./model.rknn 使用开源软件vlc拉取视频流参考官方的一些例程通过 Rockchip 多媒体处理平台 (RKMPPI) 捕获和流式传输 H.264 视频。都有相应的接口。 //h264_frame VENC_STREAM_S stFrame; stFrame.pstPack = (VENC_PACK_S *)malloc(sizeof(VENC_PACK_S)); VIDEO_FRAME_INFO_S h264_frame; VIDEO_FRAME_INFO_S stVpssFrame; // rkaiq init RK_BOOL multi_sensor = RK_FALSE; const char *iq_dir = "/etc/iqfiles"; rk_aiq_working_mode_t hdr_mode = RK_AIQ_WORKING_MODE_NORMAL; //hdr_mode = RK_AIQ_WORKING_MODE_ISP_HDR2; SAMPLE_COMM_ISP_Init(0, hdr_mode, multi_sensor, iq_dir); SAMPLE_COMM_ISP_Run(0); // rkmpi init if (RK_MPI_SYS_Init() != RK_SUCCESS) { RK_LOGE("rk mpi sys init fail!"); return -1; } // rtsp init rtsp_demo_handle g_rtsplive = NULL; rtsp_session_handle g_rtsp_session; g_rtsplive = create_rtsp_demo(554); g_rtsp_session = rtsp_new_session(g_rtsplive, "/live/0"); rtsp_set_video(g_rtsp_session, RTSP_CODEC_ID_VIDEO_H264, NULL, 0); rtsp_sync_video_ts(g_rtsp_session, rtsp_get_reltime(), rtsp_get_ntptime()); // vi init vi_dev_init(); vi_chn_init(0, width, height); // vpss init vpss_init(0, width, height); // bind vi to vpss MPP_CHN_S stSrcChn, stvpssChn; stSrcChn.enModId = RK_ID_VI; stSrcChn.s32DevId = 0; stSrcChn.s32ChnId = 0; stvpssChn.enModId = RK_ID_VPSS; stvpssChn.s32DevId = 0; stvpssChn.s32ChnId = 0; 参考官方opencv处理帧的代码 while(1) { // get vpss frame s32Ret = RK_MPI_VPSS_GetChnFrame(0,0, &stVpssFrame,-1); if(s32Ret == RK_SUCCESS) { void *data = RK_MPI_MB_Handle2VirAddr(stVpssFrame.stVFrame.pMbBlk); cv::Mat frame(height,width,CV_8UC3, data); // Preprocess the image cv::Mat gray_frame; preprocess(frame, gray_frame); // Run inference int digit = run_inference(gray_frame); // Draw rectangle around detected area (for demonstration, let's assume the area is at the center) int rect_size = 100; cv::Rect rect((frame.cols - rect_size) / 2, (frame.rows - rect_size) / 2, rect_size, rect_size); cv::rectangle(frame, rect, cv::Scalar(0, 255, 0), 2); // Draw the inference result on the frame char fps_text[16]; sprintf(fps_text, "Digit: %d", digit); cv::putText(frame, fps_text, cv::Point(40, 40), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(0, 255, 0), 2); // Display the frame cv::imshow("Detection", frame); cv::waitKey(1); // Encode and send the frame RK_MPI_VENC_SendFrame(0, &stVpssFrame,-1); s32Ret = RK_MPI_VENC_GetStream(0, &stFrame, -1); if(s32Ret == RK_SUCCESS) { if(g_rtsplive && g_rtsp_session) { void *pData = RK_MPI_MB_Handle2VirAddr(stFrame.pstPack->pMbBlk); rtsp_tx_video(g_rtsp_session, (uint8_t *)pData, stFrame.pstPack->u32Len, stFrame.pstPack->u64PTS); rtsp_do_event(g_rtsplive); } RK_U64 nowUs = TEST_COMM_GetNowUs(); fps = (float) 1000000 / (float)(nowUs - stVpssFrame.stVFrame.u64PTS); } // Release frame s32Ret = RK_MPI_VPSS_ReleaseChnFrame(0, 0, &stVpssFrame); if (s32Ret != RK_SUCCESS) { RK_LOGE("RK_MPI_VI_ReleaseChnFrame fail %x", s32Ret); } s32Ret = RK_MPI_VENC_ReleaseStream(0, &stFrame); if (s32Ret != RK_SUCCESS) { RK_LOGE("RK_MPI_VENC_ReleaseStream fail %x", s32Ret); } } int sX,sY,eX,eY; int width = 720; int height = 480; char fps_text[16]; float fps = 0; memset(fps_text,0,16); 所用分辨不需要太高，太高反而影响速度所以我们所需要做的就是，将摄像头获取到的帧进行前处理，处理为28*28的输入。一开始使用原来的模型推理精度较低，需要使用更深层以及带卷积的网络层。以及在前处理部分，需要将摄像头获取的图像，尽可能处理到和数据集里的一致，这样就能提高推理的精确度。这也是为何用更粗的线条会提高识别率，而细的笔迹识别率较低一点，所以在前处理的代码可以加入一些膨胀操作，边缘不用很锐化，做平滑过渡的处理。使用top命令查看手写体任务所占内存内存只占了三分之一的样子，所以还是有很大优化空间的。后续可以加入多线程的识别。代码处理框架参考了论坛内大佬的开源，处理流程都大差不差，主要就是前处理和后处理的区别，很多嵌入式ai板卡跑模型一般都会给好你处理框架以及调用npu啥的驱动代码，自己需要补充和优化前后处理部分的代码。 [localvideo]e531c2ba4846faded3eb65f712a21f86[/localvideo]
2024-05-30
发表了主题帖： #AI挑战营第二站# 基于RKNN toolkit的模型转换与验证（续，含代码）

本帖最后由 hollyedward 于 2024-5-30 03:37 编辑接上一篇文章，可能我使用的服务器的cpu并不支持某些操作，由于rknn也没有开源的sdk，不知道怎么调试，所以我直接租用别的服务器。首先，安装环境，步骤不用赘述模型的导出首先是导出脚本 # filename: onnx2rknn.py import numpy as np from rknn.api import RKNN if __name__ == '__main__': # 模型部署平台 platform = 'rv1106' #训练模拟时输入图片大小 Width = 28 Height = 28 # 此处改为自己的模型地址 MODEL_PATH = '/home/ljl/mnist/mnist_cnn_model.onnx' # 导出模型地址 RKNN_MODEL_PATH = '/home/ljl/mnist/mnist_cnn_model.rknn' # 创建RKNN对象并在屏幕打印详细的日志信息 rknn = RKNN(verbose=True) # 模型配置 # mean_values: 输入图像像素均值 # std_values: 输入图像像素标准差 # target_platform: 目标部署平台 # 本模型训练时输入图象为单通道 rknn.config(mean_values=[0], std_values=[255], target_platform=platform) # 模型加载 print('--> Loading model') ret = rknn.load_onnx(MODEL_PATH) if ret != 0: print('load model failed!') exit(ret) print('done') # 构建 RKNN 模型 print('--> Building model') #do_quantization：是否对模型进行量化。默认值为 True ret = rknn.build(do_quantization=True, dataset="./data.txt") if ret != 0: print('build model failed.') exit(ret) print('done') # 导出模型 ret = rknn.export_rknn(RKNN_MODEL_PATH) #释放RKNN模型 rknn.release() 然后要将测试的图片放在文件夹里，路径写在data.txt 文件中首先要获取mnist的图片用于测试这里提供导出脚本 # filename: generate_data.py import torch import torchvision import torchvision.transforms as transforms import torchvision.datasets as datasets from torch.utils.data import DataLoader import cv2 import os import numpy as np # Test set test_set = datasets.MNIST('dataset/', train=False, transform=transforms.ToTensor(), download=True) test_loader = DataLoader(dataset=test_set, batch_size=1, shuffle=True) def mnist_save_png(): for data, i in test_loader: with torch.no_grad(): image = data.squeeze().numpy() # Remove unnecessary transpose # Optional: If you need to move channel dimension to the last position # image = np.transpose(image, (1, 2, 0)) image = cv2.GaussianBlur(image, (9, 9), 0) # image *= 255 # Scale image to 0-255 range index = i.numpy()[0] if not os.path.exists('./mnist_image/'): os.mkdir('./mnist_image/') # 每张图片只保存一次 if not os.path.exists('./mnist_image/' + str(index) + '.png'): cv2.imwrite('./mnist_image/' + str(index) + '.png', image) if __name__ == '__main__': mnist_save_png() 导出后的效果，分辨率为28 * 28，代码也对应前面文章的这里只是保存了10张图片，如果要更多图片测试需要修改代码准备好后运行脚本即可模型转换打印的过程。感觉rknn至少比全志系的好用些（又不开源，还不允许个人使用），虽然不是开源的。 I rknn-toolkit2 version: 2.0.0b0+9bab5682 --> Loading model I It is recommended onnx opset 19, but your onnx model opset is 17! I Model converted from pytorch, 'opset_version' should be set 19 in torch.onnx.export for successful convert! I Loading : 100%|██████████████████████████████████████████████████| 5/5 [00:00<00:00, 24966.10it/s] done --> Building model D base_optimize ... D base_optimize done. D D fold_constant ... D fold_constant done. D D correct_ops ... D correct_ops done. D D fuse_ops ... W build: Can not find 'idx' to insert, default insert to 0! D fuse_ops results: D replace_reshape_gemm_by_conv: remove node = ['/Reshape', '/fc1/Gemm'], add node = ['/fc1/Gemm_2conv', '/fc1/Gemm_2conv_reshape'] D swap_reshape_relu: remove node = ['/fc1/Gemm_2conv_reshape', '/Relu'], add node = ['/Relu', '/fc1/Gemm_2conv_reshape'] D convert_gemm_by_conv: remove node = ['/fc2/Gemm'], add node = ['/fc2/Gemm_2conv_reshape1', '/fc2/Gemm_2conv', '/fc2/Gemm_2conv_reshape2'] D fuse_two_reshape: remove node = ['/fc1/Gemm_2conv_reshape'] D remove_invalid_reshape: remove node = ['/fc2/Gemm_2conv_reshape1'] D fold_constant ... D fold_constant done. D fuse_ops done. D D sparse_weight ... D sparse_weight done. D I GraphPreparing : 100%|████████████████████████████████████████████| 4/4 [00:00<00:00, 5403.29it/s] I Quantizating : 100%|███████████████████████████████████████████████| 4/4 [00:00<00:00, 283.34it/s] D D quant_optimizer ... D quant_optimizer results: D adjust_relu: ['/Relu'] D quant_optimizer done. D W build: The default input dtype of 'onnx::Reshape_0' is changed from 'float32' to 'int8' in rknn model for performance! Please take care of this change when deploy rknn model with Runtime API! W build: The default output dtype of '15' is changed from 'float32' to 'int8' in rknn model for performance! Please take care of this change when deploy rknn model with Runtime API! I rknn building ... I RKNN: [00:09:32.440] compress = 0, conv_eltwise_activation_fuse = 1, global_fuse = 1, multi-core-model-mode = 7, output_optimize = 1, layout_match = 1, enable_argb_group = 0 I RKNN: librknnc version: 2.0.0b0 (35a6907d79@2024-03-24T02:34:11) D RKNN: [00:09:32.440] RKNN is invoked D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNExtractCustomOpAttrs D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNExtractCustomOpAttrs D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNSetOpTargetPass D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNSetOpTargetPass D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNBindNorm D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNBindNorm D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNAddFirstConv D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNAddFirstConv D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNEliminateQATDataConvert D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNEliminateQATDataConvert D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNTileGroupConv D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNTileGroupConv D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNTileFcBatchFuse D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNTileFcBatchFuse D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNAddConvBias D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNAddConvBias D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNTileChannel D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNTileChannel D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNPerChannelPrep D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNPerChannelPrep D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNBnQuant D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNBnQuant D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNFuseOptimizerPass D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNFuseOptimizerPass D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNTurnAutoPad D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNTurnAutoPad D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNInitRNNConst D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNInitRNNConst D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNInitCastConst D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNInitCastConst D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNMultiSurfacePass D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNMultiSurfacePass D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNReplaceConstantTensorPass D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNReplaceConstantTensorPass D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNSubgraphManager D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNSubgraphManager D RKNN: [00:09:32.443] >>>>>> start: OpEmit D RKNN: [00:09:32.443] <<<<<<<< end: OpEmit D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNLayoutMatchPass I RKNN: [00:09:32.443] AppointLayout: t->setNativeLayout(64), tname:[/fc1/Gemm_output_0_new] I RKNN: [00:09:32.443] AppointLayout: t->setNativeLayout(64), tname:[15_conv] I RKNN: [00:09:32.443] AppointLayout: t->setNativeLayout(0), tname:[15] D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNLayoutMatchPass D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNAddSecondaryNode D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNAddSecondaryNode D RKNN: [00:09:32.443] >>>>>> start: OpEmit D RKNN: [00:09:32.443] finish initComputeZoneMap D RKNN: [00:09:32.443] <<<<<<<< end: OpEmit D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNSubGraphMemoryPlanPass D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNSubGraphMemoryPlanPass D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNProfileAnalysisPass D RKNN: [00:09:32.443] node: Reshape:/fc2/Gemm_2conv_reshape2, Target: NPU D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNProfileAnalysisPass D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNOperatorIdGenPass D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNOperatorIdGenPass D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNWeightTransposePass W RKNN: [00:09:32.444] Warning: Tensor /fc2/Gemm_2conv_reshape2_shape need paramter qtype, type is set to float16 by default! W RKNN: [00:09:32.444] Warning: Tensor /fc2/Gemm_2conv_reshape2_shape need paramter qtype, type is set to float16 by default! D RKNN: [00:09:32.444] <<<<<<<< end: rknn::RKNNWeightTransposePass D RKNN: [00:09:32.444] >>>>>> start: rknn::RKNNCPUWeightTransposePass D RKNN: [00:09:32.444] <<<<<<<< end: rknn::RKNNCPUWeightTransposePass D RKNN: [00:09:32.444] >>>>>> start: rknn::RKNNModelBuildPass D RKNN: [00:09:32.446] <<<<<<<< end: rknn::RKNNModelBuildPass D RKNN: [00:09:32.446] >>>>>> start: rknn::RKNNModelRegCmdbuildPass D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ D RKNN: [00:09:32.446] Network Layer Information Table D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ D RKNN: [00:09:32.446] ID OpType DataType Target InputShape OutputShape Cycles(DDR/NPU/Total) RW(KB) FullName D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ D RKNN: [00:09:32.446] 0 InputOperator INT8 CPU \ (1,1,28,28) 0/0/0 0 InputOperator:onnx::Reshape_0 D RKNN: [00:09:32.446] 1 ConvRelu INT8 NPU (1,1,28,28),(50,1,28,28),(50) (1,50,1,1) 6585/12544/12544 39 Conv:/fc1/Gemm_2conv D RKNN: [00:09:32.446] 2 Conv INT8 NPU (1,50,1,1),(10,50,1,1),(10) (1,10,1,1) 138/64/138 0 Conv:/fc2/Gemm_2conv D RKNN: [00:09:32.446] 3 Reshape INT8 NPU (1,10,1,1),(2) (1,10) 7/0/7 0 Reshape:/fc2/Gemm_2conv_reshape2 D RKNN: [00:09:32.446] 4 OutputOperator INT8 CPU (1,10) \ 0/0/0 0 OutputOperator:15 D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ D RKNN: [00:09:32.446] <<<<<<<< end: rknn::RKNNModelRegCmdbuildPass D RKNN: [00:09:32.446] >>>>>> start: rknn::RKNNFlatcModelBuildPass D RKNN: [00:09:32.446] Export Mini RKNN model to /tmp/tmpkbgrb68z/check.rknn D RKNN: [00:09:32.446] >>>>>> end: rknn::RKNNFlatcModelBuildPass D RKNN: [00:09:32.446] >>>>>> start: rknn::RKNNMemStatisticsPass D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------ D RKNN: [00:09:32.446] Feature Tensor Information Table D RKNN: [00:09:32.446] --------------------------------------------------------------------------------------------+--------------------------------- D RKNN: [00:09:32.446] ID User Tensor DataType DataFormat OrigShape NativeShape | [Start End) Size D RKNN: [00:09:32.446] --------------------------------------------------------------------------------------------+--------------------------------- D RKNN: [00:09:32.446] 1 ConvRelu onnx::Reshape_0 INT8 NC1HWC2 (1,1,28,28) (1,1,28,28,1) | 0x00027500 0x00027880 0x00000380 D RKNN: [00:09:32.446] 2 Conv /fc1/Gemm_output_0_new INT8 NC1HWC2 (1,50,1,1) (1,4,1,1,16) | 0x00027880 0x000278c0 0x00000040 D RKNN: [00:09:32.446] 3 Reshape 15_conv INT8 NC1HWC2 (1,10,1,1) (1,1,1,1,16) | 0x00027500 0x00027510 0x00000010 D RKNN: [00:09:32.446] 4 OutputOperator 15 INT8 UNDEFINED (1,10) (1,10) | 0x00027580 0x000275c0 0x00000040 D RKNN: [00:09:32.446] --------------------------------------------------------------------------------------------+--------------------------------- D RKNN: [00:09:32.446] ----------------------------------------------------------------------------------------------------- D RKNN: [00:09:32.446] Const Tensor Information Table D RKNN: [00:09:32.446] -------------------------------------------------------------------+--------------------------------- D RKNN: [00:09:32.446] ID User Tensor DataType OrigShape | [Start End) Size D RKNN: [00:09:32.446] -------------------------------------------------------------------+--------------------------------- D RKNN: [00:09:32.446] 1 ConvRelu fc1.weight INT8 (50,1,28,28) | 0x00000000 0x00026480 0x00026480 D RKNN: [00:09:32.446] 1 ConvRelu fc1.bias INT32 (50) | 0x00026480 0x00026680 0x00000200 D RKNN: [00:09:32.446] 2 Conv fc2.weight INT8 (10,50,1,1) | 0x00026680 0x00026900 0x00000280 D RKNN: [00:09:32.446] 2 Conv fc2.bias INT32 (10) | 0x00026900 0x00026980 0x00000080 D RKNN: [00:09:32.446] 3 Reshape /fc2/Gemm_2conv_reshape2_shape INT64 (2) | 0x00026980*0x000269c0 0x00000040 D RKNN: [00:09:32.446] -------------------------------------------------------------------+--------------------------------- D RKNN: [00:09:32.446] ---------------------------------------- D RKNN: [00:09:32.446] Total Internal Memory Size: 0.9375KB D RKNN: [00:09:32.446] Total Weight Memory Size: 154.438KB D RKNN: [00:09:32.446] ---------------------------------------- D RKNN: [00:09:32.446] <<<<<<<< end: rknn::RKNNMemStatisticsPass I rknn buiding done. done 模型验证我们需要检验模型的输出是否正确貌似simulator不支持rknn模型验证报错日志：官方提供的demo教程也是使用load_onnx,https://wiki.luckfox.com/zh/Luckfox-Pico/Luckfox-Pico-RKNN-Test 所以其实可以在导出rknn模型的时候，也可以验证模型脚本如下 # filename: rknn_mnist_test.py import numpy as np import cv2 from rknn.api import RKNN # Model conversion parameters MODEL_PATH = '/root/test/my_model.onnx' # Path to the ONNX model RKNN_MODEL_PATH = '/root/test/my_model.rknn' # Path to save the RKNN model # Model inference parameters input_size = (28, 28) # Define the input size (same as your model's input) data_file = 'data.txt' # Path to the data file (containing image paths and labels) rknn = RKNN(verbose=True) # Create RKNN object with verbose logging rknn.config(mean_values=[0], std_values=[255], target_platform='rv1106') # Set configuration parameters ret = rknn.load_onnx(MODEL_PATH) if ret != 0: print('Load ONNX model failed!') exit(ret) print('done') print('--> Building RKNN model') ret = rknn.build(do_quantization=True, dataset="./data.txt") if ret != 0: print('Build model failed.') exit(ret) print('done') # Model export (optional) #导出rknn模型 ret = rknn.export_rknn(RKNN_MODEL_PATH) # Model inference print('--> Performing inference on data') rknn.init_runtime() # Initialize RKNN runtime with open(data_file, 'r') as f: lines = f.readlines() for line in lines: # Get image path and label image_path = line.strip() # Read the image image = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE) # Preprocess the image image = image.astype(np.float32) # image 这时候读取出来是 (28, 28), 需要增加维度 image = np.expand_dims(image, axis=[0,1]) # Run inference outputs = rknn.inference([image], data_format = 'nchw') #(1, 1, 28, 28) 对应nchw,批次，通道，长，宽 print(f"Ineference Output: {outputs}") # Check inference results if outputs is not None: predicted_label = np.argmax(outputs) print(f"Image: {image_path}") print(f"Predicted label: {predicted_label}") else: print(f"Inference failed for image: {image_path}") # Release RKNN resources rknn.release() 打印的结果可以看到推理的结果是正确的
2024-05-28
回复了主题帖：【安信可BW16-Kit开发板】温湿度与光强传感器接入测试

hollyedward 发表于 2024-5-28 16:54 你使用默认的例程试试，这个就是sdk自带的例程，应该没有问题的。还有就是传感器的针脚确定没接错吧，有 ... 你可以试试别的例程，pa26接个别的pwm，gpio调试的，或者啥
回复了主题帖：【安信可BW16-Kit开发板】温湿度与光强传感器接入测试

HuangDarren 发表于 2024-5-26 14:59 應該不是傳感器的問題，我有再新買一個傳感器但同樣不能用，請問要如何排查針腳？有其他替代針腳可以使用 ... 你使用默认的例程试试，这个就是sdk自带的例程，应该没有问题的。还有就是传感器的针脚确定没接错吧，有的不同厂商针脚顺序不一样，看看它的原理图。其他的就没啥了啊，除非是硬件问题可能
2024-05-26
回复了主题帖：【安信可BW16-Kit开发板】温湿度与光强传感器接入测试

HuangDarren 发表于 2024-5-24 13:56 除了PA26以外其他的腳位可以接DATA嗎？我在想會不會是腳位出現故障才無法讀取到傳感器代码肯定没问题的，我都试过了，你可以排查下是不是板子针脚还是传感器的问题
2024-05-21
回复了主题帖：【安信可BW16-Kit开发板】温湿度与光强传感器接入测试

HuangDarren 发表于 2024-5-20 23:53 這是我連接的方式與Output結果(使用您帖子上面相同的Code)，因為私信不能傳圖片所以只能 ... 接线没问题吗，接3.3v看看，还有数据线接PA26
2024-05-17
回复了主题帖：【安信可BW16-Kit开发板】温湿度与光强传感器接入测试

HuangDarren 发表于 2024-5-15 17:03 您好，我想請問為甚麼我照著上面的步驟連接bw16-kit(腳位PA26)跟DHT11但是卻無法讀取到數值?如果可以的話， ... 可以
2024-05-08
回复了主题帖：入围名单公布：嵌入式工程师AI挑战营（初阶），获RV1106 Linux 板+摄像头的名单

本帖最后由 hollyedward 于 2024-5-9 15:02 编辑个人信息已确认，领取板卡，可继续完成&分享挑战营第二站和第三站任务。
2024-05-05
发表了主题帖： #AI挑战营第二站# 基于RKNN toolkit的模型转换与部署

本帖最后由 hollyedward 于 2024-5-7 01:07 编辑 RKNN Toolkit2 是由瑞芯微电子 (Rockchip) 开发的一套用于深度学习模型优化和推理的工具。它主要面向在瑞芯微SoC上进行AI应用开发，但也可以用于PC平台进行模型的转换、量化、推理等操作。 Rockchip RK3566/RK3568 系列， RK3588 系列， K3562 系列， RV1103/RV1106 系列芯片搭载神经网络处理器 NPU，利用 RKNN 可以帮助用户快速部署 AI 模型到 Rockchip 芯片上使用 NPU 硬件加速模型推理。为了使用 RKNPU，用户首先需要在 x86 计算机上使用 RKNN-Toolkit2 工具，将训练好的模型转换为 RKNN 格式的模型，然后在开发板上使用 RKNN C API 或 Python API进行推断。所需工具： RKNN-Toolkit2 是一个软件开发工具包，供用户在 PC 和 Rockchip NPU 平台上执行模型转换、推断和性能评估。 RKNN-Toolkit-Lite2 为 Rockchip NPU 平台提供了 Python 编程接口，帮助用户部署 RKNN 模型并加速实施AI应用。 RKNN Runtime 为 Rockchip NPU 平台提供了 C/C++ 编程接口，帮助用户部署 RKNN 模型并加速实施AI应用。 RKNPU 内核驱动负责与 NPU 硬件交互。 rknn 软件框图环境搭建查看系统版本 cat /proc/version ubuntu版本为22.04，对应的环境python版本为3.10 安装miniconda 创建虚拟环境 # 下载 RKNN-Toolkit2 仓库 git clone https://github.com/airockchip/rknn-toolkit2.git -b v1.6.0 安装项目需要的依赖文件，使用python 3.10对应的配置文件 pip install -r ./rknn-toolkit2/packages/requirements_cp310-1.6.0.txt 需要下载的依赖安装 rknn toolkit2 pip install ./rknn-toolkit2/packages/rknn_toolkit2-1.6.0+81f21f4d-cp38-cp38-linux_x86_64.whl 遇到的问题待解决：在服务器的ubuntu系统（版本22.04）上不能正常使用rknn-toolkit2 换版本都试过了没有用 py38+rknn1.6.0 py39+rknn1.6.0 py310+rknn1.6.0 py310+rknn2.0.0 以上均报错Illegal instruction (core dumped) 这是系统原因吗？这里执行自带的yolov5 example也没有用 python test.py I rknn-toolkit2 version: 2.0.0b0+9bab5682 --> Config model done --> Loading model I It is recommended onnx opset 19, but your onnx model opset is 12! I Model converted from pytorch, 'opset_version' should be set 19 in torch.onnx.export for successful convert! I Loading : 0%| | 0/125 [00:00<?, ?it/I Loading : 100%|██████████████████████████████████████████████| 125/125 [00:00<00:00, 11880.53it/s] done --> Building model D base_optimize ... D base_optimize done. D D fold_constant ... D fold_constant done. D D correct_ops ... D correct_ops done. D D fuse_ops ... D fuse_ops done. D D sparse_weight ... Illegal instruction (core dumped)