NNTK_NLY的个人空间动态-电子工程世界

动态发布点评好友关于

2024-05-29
回复了主题帖：【AI挑战营第二站】算法工程化部署打包成SDK

#AI挑战营第二站#RV1106的rknn模型转换、仿真推理
发表了主题帖： #AI挑战营第二站#RV1106的rknn模型转换、仿真推理

本帖最后由 NNTK_NLY 于 2024-5-29 20:41 编辑 1.环境搭建不再赘述，参考https://bbs.eeworld.com.cn/thread-1280038-1-1.html 2.模型转换 rknn = RKNN(verbose=True) #Pre-process config print('--> Config model') rknn.config(mean_values=[[28]], std_values=[[28]],target_platform='rv1106') print('done') # Load model print('--> Loading model') ret = rknn.load_onnx(model='mnist.onnx') if ret != 0: print('Load model failed!') exit(ret) print('done') # Build model print('--> Building model') ret = rknn.build(do_quantization=True, dataset='./data.txt',rknn_batch_size=1) if ret != 0: print('Build model failed!') exit(ret) print('done') # Export rknn model print('--> Export rknn model') ret = rknn.export_rknn('mnist.rknn') if ret != 0: print('Export rknn model failed!') exit(ret) print('done') # Release rknn.release() 3.紧接着进行模型仿真推理 # Set inputs img = cv2.imread('8.png') img = cv2.resize(img, (28, 28)) #cv2.imwrite('2r.jpg', img) img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) img = np.expand_dims(img, 0) img = np.expand_dims(img, 3) # Init runtime environment print('--> Init runtime environment') ret = rknn.init_runtime() if ret != 0: print('Init runtime environment failed!') exit(ret) print('done') # Inference print('--> Running model') outputs = rknn.inference(inputs=[img]) #Post Process print('--> PostProcess') with open('./synset.txt', 'r') as f: labels = [l.rstrip() for l in f] scores = softmax(outputs[0]) # print the top-5 inferences class scores = np.squeeze(scores) a = np.argsort(scores)[::-1] print('-----TOP 5-----') for i in a[0:5]: print('[%d] score=%.2f class="%s"' % (i, scores[i], labels[i])) print('done') # Release rknn.release() 注意：如需仿真推理，第二步2.模型转换中先不释放rknn实体 # Release rknn.release() 4.模型转换log I rknn-toolkit2 version: 2.0.0b0+9bab5682 --> Config model done --> Loading model I It is recommended onnx opset 19, but your onnx model opset is 10! I Model converted from pytorch, 'opset_version' should be set 19 in torch.onnx.export for successful convert! I Loading : 100%|██████████████████████████████████████████████████| 9/9 [00:00<00:00, 16897.38it/s] done --> Building model D base_optimize ... D base_optimize done. D D fold_constant ... D fold_constant done. D D correct_ops ... D correct_ops done. D D fuse_ops ... D fuse_ops results: D replace_reshape_gemm_by_conv: remove node = ['/Reshape', '/fc1/Gemm'], add node = ['/fc1/Gemm_2conv', '/fc1/Gemm_2conv_reshape'] D swap_reshape_relu: remove node = ['/fc1/Gemm_2conv_reshape', '/Relu_2'], add node = ['/Relu_2', '/fc1/Gemm_2conv_reshape'] D convert_gemm_by_conv: remove node = ['/fc2/Gemm'], add node = ['/fc2/Gemm_2conv_reshape1', '/fc2/Gemm_2conv', '/fc2/Gemm_2conv_reshape2'] D fuse_two_reshape: remove node = ['/fc1/Gemm_2conv_reshape'] D remove_invalid_reshape: remove node = ['/fc2/Gemm_2conv_reshape1'] D fold_constant ... D fold_constant done. D fuse_ops done. D D sparse_weight ... D sparse_weight done. D I GraphPreparing : 100%|██████████████████████████████████████████| 10/10 [00:00<00:00, 4456.81it/s] I Quantizating : 100%|████████████████████████████████████████████| 10/10 [00:00<00:00, 1337.34it/s] D D quant_optimizer ... D quant_optimizer results: D adjust_relu: ['/Relu_2', '/Relu_1', '/Relu'] D quant_optimizer done. D W build: The default input dtype of 'input' is changed from 'float32' to 'int8' in rknn model for performance! Please take care of this change when deploy rknn model with Runtime API! W build: The default output dtype of 'output' is changed from 'float32' to 'int8' in rknn model for performance! Please take care of this change when deploy rknn model with Runtime API! I rknn building ... I RKNN: [18:11:22.305] compress = 0, conv_eltwise_activation_fuse = 1, global_fuse = 1, multi-core-model-mode = 7, output_optimize = 1, layout_match = 1, enable_argb_group = 0 I RKNN: librknnc version: 2.0.0b0 (35a6907d79@2024-03-24T02:34:11) D RKNN: [18:11:22.308] RKNN is invoked D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNExtractCustomOpAttrs D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNExtractCustomOpAttrs D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNSetOpTargetPass D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNSetOpTargetPass D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNBindNorm D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNBindNorm D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNAddFirstConv D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNAddFirstConv D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNEliminateQATDataConvert D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNEliminateQATDataConvert D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNTileGroupConv D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNTileGroupConv D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNTileFcBatchFuse D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNTileFcBatchFuse D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNAddConvBias D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNAddConvBias D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNTileChannel D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNTileChannel D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNPerChannelPrep D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNPerChannelPrep D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNBnQuant D RKNN: [18:11:22.318] <<<<<<<< end: rknn::RKNNBnQuant D RKNN: [18:11:22.318] >>>>>> start: rknn::RKNNFuseOptimizerPass D RKNN: [18:11:22.319] <<<<<<<< end: rknn::RKNNFuseOptimizerPass D RKNN: [18:11:22.319] >>>>>> start: rknn::RKNNTurnAutoPad D RKNN: [18:11:22.319] <<<<<<<< end: rknn::RKNNTurnAutoPad D RKNN: [18:11:22.319] >>>>>> start: rknn::RKNNInitRNNConst D RKNN: [18:11:22.319] <<<<<<<< end: rknn::RKNNInitRNNConst D RKNN: [18:11:22.319] >>>>>> start: rknn::RKNNInitCastConst D RKNN: [18:11:22.319] <<<<<<<< end: rknn::RKNNInitCastConst D RKNN: [18:11:22.319] >>>>>> start: rknn::RKNNMultiSurfacePass D RKNN: [18:11:22.319] <<<<<<<< end: rknn::RKNNMultiSurfacePass D RKNN: [18:11:22.319] >>>>>> start: rknn::RKNNReplaceConstantTensorPass D RKNN: [18:11:22.319] <<<<<<<< end: rknn::RKNNReplaceConstantTensorPass D RKNN: [18:11:22.319] >>>>>> start: rknn::RKNNSubgraphManager D RKNN: [18:11:22.319] <<<<<<<< end: rknn::RKNNSubgraphManager D RKNN: [18:11:22.319] >>>>>> start: OpEmit D RKNN: [18:11:22.319] <<<<<<<< end: OpEmit D RKNN: [18:11:22.319] >>>>>> start: rknn::RKNNLayoutMatchPass I RKNN: [18:11:22.319] AppointLayout: t->setNativeLayout(64), tname:[/Relu_output_0] I RKNN: [18:11:22.319] AppointLayout: t->setNativeLayout(64), tname:[/pool/MaxPool_output_0] I RKNN: [18:11:22.319] AppointLayout: t->setNativeLayout(64), tname:[/Relu_1_output_0] I RKNN: [18:11:22.319] AppointLayout: t->setNativeLayout(64), tname:[/pool_1/MaxPool_output_0] I RKNN: [18:11:22.319] AppointLayout: t->setNativeLayout(64), tname:[/fc1/Gemm_output_0_new] I RKNN: [18:11:22.319] AppointLayout: t->setNativeLayout(64), tname:[output_conv] I RKNN: [18:11:22.319] AppointLayout: t->setNativeLayout(0), tname:[output] D RKNN: [18:11:22.319] <<<<<<<< end: rknn::RKNNLayoutMatchPass D RKNN: [18:11:22.319] >>>>>> start: rknn::RKNNAddSecondaryNode D RKNN: [18:11:22.319] <<<<<<<< end: rknn::RKNNAddSecondaryNode D RKNN: [18:11:22.319] >>>>>> start: OpEmit D RKNN: [18:11:22.319] finish initComputeZoneMap D RKNN: [18:11:22.320] <<<<<<<< end: OpEmit D RKNN: [18:11:22.320] >>>>>> start: rknn::RKNNSubGraphMemoryPlanPass D RKNN: [18:11:22.320] <<<<<<<< end: rknn::RKNNSubGraphMemoryPlanPass D RKNN: [18:11:22.320] >>>>>> start: rknn::RKNNProfileAnalysisPass D RKNN: [18:11:22.320] node: Reshape:/fc2/Gemm_2conv_reshape2, Target: NPU D RKNN: [18:11:22.320] <<<<<<<< end: rknn::RKNNProfileAnalysisPass D RKNN: [18:11:22.320] >>>>>> start: rknn::RKNNOperatorIdGenPass D RKNN: [18:11:22.320] <<<<<<<< end: rknn::RKNNOperatorIdGenPass D RKNN: [18:11:22.320] >>>>>> start: rknn::RKNNWeightTransposePass W RKNN: [18:11:22.331] Warning: Tensor /fc2/Gemm_2conv_reshape2_shape need paramter qtype, type is set to float16 by default! W RKNN: [18:11:22.331] Warning: Tensor /fc2/Gemm_2conv_reshape2_shape need paramter qtype, type is set to float16 by default! D RKNN: [18:11:22.331] <<<<<<<< end: rknn::RKNNWeightTransposePass D RKNN: [18:11:22.331] >>>>>> start: rknn::RKNNCPUWeightTransposePass D RKNN: [18:11:22.331] <<<<<<<< end: rknn::RKNNCPUWeightTransposePass D RKNN: [18:11:22.331] >>>>>> start: rknn::RKNNModelBuildPass D RKNN: [18:11:22.336] <<<<<<<< end: rknn::RKNNModelBuildPass D RKNN: [18:11:22.336] >>>>>> start: rknn::RKNNModelRegCmdbuildPass D RKNN: [18:11:22.336] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ D RKNN: [18:11:22.336] Network Layer Information Table D RKNN: [18:11:22.336] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ D RKNN: [18:11:22.336] ID OpType DataType Target InputShape OutputShape Cycles(DDR/NPU/Total) RW(KB) FullName D RKNN: [18:11:22.336] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ D RKNN: [18:11:22.336] 0 InputOperator INT8 CPU \ (1,1,28,28) 0/0/0 0 InputOperator:input D RKNN: [18:11:22.336] 1 ConvRelu INT8 NPU (1,1,28,28),(32,1,3,3),(32) (1,32,28,28) 4429/7056/7056 2 Conv:/conv1/Conv D RKNN: [18:11:22.336] 2 MaxPool INT8 NPU (1,32,28,28) (1,32,14,14) 5092/0/5092 24 MaxPool:/pool/MaxPool D RKNN: [18:11:22.336] 3 ConvRelu INT8 NPU (1,32,14,14),(64,32,3,3),(64) (1,64,14,14) 6131/7488/7488 24 Conv:/conv2/Conv D RKNN: [18:11:22.336] 4 MaxPool INT8 NPU (1,64,14,14) (1,64,7,7) 2546/0/2546 12 MaxPool:/pool_1/MaxPool D RKNN: [18:11:22.336] 5 ConvRelu INT8 NPU (1,64,7,7),(128,64,7,7),(128) (1,128,1,1) 65865/12544/65865 396 Conv:/fc1/Gemm_2conv D RKNN: [18:11:22.336] 6 Conv INT8 NPU (1,128,1,1),(10,128,1,1),(10) (1,10,1,1) 252/64/252 1 Conv:/fc2/Gemm_2conv D RKNN: [18:11:22.336] 7 Reshape INT8 NPU (1,10,1,1),(2) (1,10) 7/0/7 0 Reshape:/fc2/Gemm_2conv_reshape2 D RKNN: [18:11:22.336] 8 OutputOperator INT8 CPU (1,10) \ 0/0/0 0 OutputOperator:output D RKNN: [18:11:22.336] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ D RKNN: [18:11:22.337] <<<<<<<< end: rknn::RKNNModelRegCmdbuildPass D RKNN: [18:11:22.337] >>>>>> start: rknn::RKNNFlatcModelBuildPass D RKNN: [18:11:22.337] Export Mini RKNN model to /tmp/tmphfyahgcw/check.rknn D RKNN: [18:11:22.337] >>>>>> end: rknn::RKNNFlatcModelBuildPass D RKNN: [18:11:22.337] >>>>>> start: rknn::RKNNMemStatisticsPass D RKNN: [18:11:22.337] --------------------------------------------------------------------------------------------------------------------------------- D RKNN: [18:11:22.337] Feature Tensor Information Table D RKNN: [18:11:22.337] -----------------------------------------------------------------------------------------------+--------------------------------- D RKNN: [18:11:22.337] ID User Tensor DataType DataFormat OrigShape NativeShape | [Start End) Size D RKNN: [18:11:22.337] -----------------------------------------------------------------------------------------------+--------------------------------- D RKNN: [18:11:22.337] 1 ConvRelu input INT8 NC1HWC2 (1,1,28,28) (1,1,28,28,1) | 0x00069000 0x00069380 0x00000380 D RKNN: [18:11:22.337] 2 MaxPool /Relu_output_0 INT8 NC1HWC2 (1,32,28,28) (1,2,28,28,16) | 0x00069380 0x0006f580 0x00006200 D RKNN: [18:11:22.337] 3 ConvRelu /pool/MaxPool_output_0 INT8 NC1HWC2 (1,32,14,14) (1,2,14,14,16) | 0x0006f580 0x00070e00 0x00001880 D RKNN: [18:11:22.337] 4 MaxPool /Relu_1_output_0 INT8 NC1HWC2 (1,64,14,14) (1,4,14,14,16) | 0x00069000 0x0006c100 0x00003100 D RKNN: [18:11:22.337] 5 ConvRelu /pool_1/MaxPool_output_0 INT8 NC1HWC2 (1,64,7,7) (1,4,7,7,16) | 0x0006c100 0x0006ce00 0x00000d00 D RKNN: [18:11:22.337] 6 Conv /fc1/Gemm_output_0_new INT8 NC1HWC2 (1,128,1,1) (1,8,1,1,16) | 0x00069000 0x00069080 0x00000080 D RKNN: [18:11:22.337] 7 Reshape output_conv INT8 NC1HWC2 (1,10,1,1) (1,1,1,1,16) | 0x00069080 0x00069090 0x00000010 D RKNN: [18:11:22.337] 8 OutputOperator output INT8 UNDEFINED (1,10) (1,10) | 0x00069040 0x00069080 0x00000040 D RKNN: [18:11:22.337] -----------------------------------------------------------------------------------------------+--------------------------------- D RKNN: [18:11:22.337] ----------------------------------------------------------------------------------------------------- D RKNN: [18:11:22.337] Const Tensor Information Table D RKNN: [18:11:22.337] -------------------------------------------------------------------+--------------------------------- D RKNN: [18:11:22.337] ID User Tensor DataType OrigShape | [Start End) Size D RKNN: [18:11:22.337] -------------------------------------------------------------------+--------------------------------- D RKNN: [18:11:22.337] 1 ConvRelu conv1.weight INT8 (32,1,3,3) | 0x00000000 0x00000480 0x00000480 D RKNN: [18:11:22.337] 1 ConvRelu conv1.bias INT32 (32) | 0x00000480 0x00000580 0x00000100 D RKNN: [18:11:22.337] 3 ConvRelu conv2.weight INT8 (64,32,3,3) | 0x00000580 0x00004d80 0x00004800 D RKNN: [18:11:22.337] 3 ConvRelu conv2.bias INT32 (64) | 0x00004d80 0x00004f80 0x00000200 D RKNN: [18:11:22.337] 5 ConvRelu fc1.weight INT8 (128,64,7,7) | 0x00004f80 0x00066f80 0x00062000 D RKNN: [18:11:22.337] 5 ConvRelu fc1.bias INT32 (128) | 0x00066f80 0x00067380 0x00000400 D RKNN: [18:11:22.337] 6 Conv fc2.weight INT8 (10,128,1,1) | 0x00067380 0x00067880 0x00000500 D RKNN: [18:11:22.337] 6 Conv fc2.bias INT32 (10) | 0x00067880 0x00067900 0x00000080 D RKNN: [18:11:22.337] 7 Reshape /fc2/Gemm_2conv_reshape2_shape INT64 (2) | 0x00067900*0x00067940 0x00000040 D RKNN: [18:11:22.337] -------------------------------------------------------------------+--------------------------------- D RKNN: [18:11:22.337] ---------------------------------------- D RKNN: [18:11:22.337] Total Internal Memory Size: 31.5KB D RKNN: [18:11:22.337] Total Weight Memory Size: 414.312KB D RKNN: [18:11:22.337] ---------------------------------------- D RKNN: [18:11:22.337] <<<<<<<< end: rknn::RKNNMemStatisticsPass I rknn buiding done. done --> Export rknn model done 5.仿真推理log --> Init runtime environment I Target is None, use simulator! done --> Running model W inference: The 'data_format' is not set, and its default value is 'nhwc'! I GraphPreparing : 100%|██████████████████████████████████████████| 12/12 [00:00<00:00, 4513.64it/s] I SessionPreparing : 100%|████████████████████████████████████████| 12/12 [00:00<00:00, 1368.27it/s] --> PostProcess -----TOP 5----- [8] score=1.00 class="8 8" [3] score=0.00 class="3 3" [9] score=0.00 class="9 9" [2] score=0.00 class="2 2" [6] score=0.00 class="6 6" done 6.模型转换工程目录下文件 7.data.txt内容 ./2.png 8.synset.txt内容 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 9.仿真推理所用图片： 10.附github工程链接https://github.com/Ainit-NNTK/mnist_onnx_export_rknn.git
2024-05-28
回复了主题帖：【AI挑战营终点站】应用落地：部署手写数字识别应用到幸狐RV1106开发板

先打卡一下，后续优化https://bbs.eeworld.com.cn/forum.php?mod=viewthread&tid=1283213&page=1&extra=#pid3333154
发表了主题帖： #AI挑战营终点站#RV1106手写数字识别，基于opencv-mobile和ncnn、rtsp部署

本帖最后由 NNTK_NLY 于 2024-5-28 21:17 编辑 1.onnx模型转ncnn模型 https://convertmodel.com/ 2.下载opencv-mobile预编译包 https://github.com/nihui/opencv-mobile/releases/download/v26/opencv-mobile-4.9.0-luckfox-pico.zip 3.为luckfox-pico编译ncnn git clone https://github.com/Tencent/ncnn.git cd ./ncnn./toolchains vi luckfox-pico.cmake 填入 set(CMAKE_SYSTEM_NAME Linux) set(CMAKE_SYSTEM_PROCESSOR arm) if(DEFINED ENV{TOOLCHAIN_ROOT_PATH}) file(TO_CMAKE_PATH $ENV{TOOLCHAIN_ROOT_PATH} TOOLCHAIN_ROOT_PATH) else() message(FATAL_ERROR "TOOLCHAIN_ROOT_PATH env must be defined") endif() set(TOOLCHAIN_ROOT_PATH ${TOOLCHAIN_ROOT_PATH} CACHE STRING "root path to toolchain") set(CMAKE_C_COMPILER "/mnt/sdd1/soc/toolchains/luckfox-pico/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-gcc") set(CMAKE_CXX_COMPILER "/mnt/sdd1/soc/toolchains/luckfox-pico/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-g++") set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) set(CMAKE_C_FLAGS "-march=armv7-a -mfloat-abi=hard -mfpu=neon") set(CMAKE_CXX_FLAGS "-march=armv7-a -mfloat-abi=hard -mfpu=neon") # cache flags set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS}" CACHE STRING "c flags") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}" CACHE STRING "c++ flags") 其中set(CMAKE_C_COMPILER "/mnt/sdd1/soc/toolchains/luckfox-pico/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-gcc") set(CMAKE_CXX_COMPILER "/mnt/sdd1/soc/toolchains/luckfox-pico/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-g++") 按实际修改回到ncnn根目录 mkdir -p build-luckfox-pico cmake -DCMAKE_TOOLCHAIN_FILE=../toolchains/luckfox-pico.cmake .. make -j4 make install 3.参考 git clone https://github.com/luckfox-eng29/luckfox_pico_rtsp_opencv cd ./luckfox_pico_rtsp_opencv cp ncnn/build-luckfox-pico/install/* ./ncnn_install 改CMakeLists.txt set(CMAKE_C_COMPILER "/path/to/luckfox-pico/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-gcc") set(CMAKE_CXX_COMPILER "/path/to//luckfox-pico/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-g++") 4.编写main.cpp opencv获取摄像头帧 void *data = RK_MPI_MB_Handle2VirAddr(stVpssFrame.stVFrame.pMbBlk); cv::Mat frame(height, width, CV_8UC3, data); opencv框选数字 cv::Rect digit_rect = find_digit_contour(frame); digit_rect.x = std::max(0, digit_rect.x - 10); digit_rect.y = std::max(0, digit_rect.y - 50); digit_rect.width = std::min(frame.cols - digit_rect.x, digit_rect.width + 20); digit_rect.height = std::min(frame.rows - digit_rect.y, digit_rect.height + 100); cv::Rect find_digit_contour(const cv::Mat &image) { cv::Mat gray, blurred, edged; cv::cvtColor(image, gray, cv::COLOR_BGR2GRAY); cv::GaussianBlur(gray, blurred, cv::Size(5, 5), 0); cv::Canny(blurred, edged, 50, 150); std::vector<std::vector<cv::Point>> contours; cv::findContours(edged, contours, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_SIMPLE); if (contours.empty()) { return cv::Rect(); } // 找到最大的轮廓 auto largest_contour = std::max_element(contours.begin(), contours.end(), [](const std::vector<cv::Point> &a, const std::vector<cv::Point> &b) { return cv::contourArea(a) < cv::contourArea(b); }); return cv::boundingRect(*largest_contour); } opencv截取数字的区域 cv::Mat digit_region = frame(digit_rect); opencv转灰度图并resize到28*28 cv::cvtColor(digit_region, gray_1ch, cv::COLOR_BGR2GRAY); threshold(gray_1ch, gray_1ch, atoi(argv[1]), 255, cv::THRESH_BINARY_INV); cv::resize(gray_1ch, frame_resize, cv::Size(28, 28), 0, 0, cv::INTER_AREA); ncnn推理 ncnn::Mat in = ncnn::Mat::from_pixels(frame_resize.data, ncnn::Mat::PIXEL_GRAY, frame_resize.cols, frame_resize.rows); // PIXEL_BGR2GRAY ncnn::Mat out; double total_latency = 0; ncnn::Extractor ex = net.create_extractor(); ex.input("flatten_input", in); ex.extract("dense_2", out); const float *ptr = out.channel(0); int gussed = -1; float guss_exp = -10000000; for (int i = 0; i < out.w * out.h; i++) { printf("%d: %.2f\n", i, ptr[i]); if (guss_exp < ptr[i]) { gussed = i; guss_exp = ptr[i]; } } printf("I think it is number %d!\n", gussed); 在图像上显示预测结果 cv::rectangle(frame, digit_rect, cv::Scalar(0, 255, 0), 2); sprintf(fps_text, "number:%d", gussed); cv::putText(frame, fps_text,cv::Point(40, 40), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(0, 255, 0), 2); 最后memcpy到rtsp帧中 memcpy(data, frame.data, width * height * 3); 编译 mkdir build cd build cmake .. make && make install 生成可执行文件在luckfox_pico_rtsp_opencv-ncnn-mnist文件夹中 ./luckfox_pico_rtsp_opencv-ncnn-mnist/luckfox_pico_rtsp_opencv-ncnn-mnist 5.识别效果 6.附工程开源链接 https://github.com/Ainit-NNTK/luckfox-pico-opencv-ncnn 7.总结 opencv-mobile + ncnn推理速度还行，不过截取数字时框选有误差导致识别出错后续打算优化数字截取框算法和更换为rknn推理框架并对比推理效果 8.附实时推理视频 [localvideo]f22b1dbb88b44173f4999d4684209151[/localvideo]
2024-05-09
回复了主题帖： #AI挑战营第一站# 利用pytorch进行手写数字模型训练的训练和转换

测试模型 def test(model, device, test_loader): model.eval() test_loss = 0 correct = 0 with torch.no_grad(): for data, target in test_loader: data, target = data.to(device), target.to(device) output = model(data) test_loss += F.nll_loss(output, target, reduction='sum').item() # sum up batch loss pred = output.argmax(dim=1, keepdim=True) # get the index of the max log-probability correct += pred.eq(target.view_as(pred)).sum().item() test_loss /= len(test_loader.dataset) print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format( test_loss, correct, len(test_loader.dataset), 100. * correct / len(test_loader.dataset))) 训练模型测试结果：
回复了主题帖：入围名单公布：嵌入式工程师AI挑战营（初阶），获RV1106 Linux 板+摄像头的名单

个人信息已确认，领取板卡，可继续完成&分享挑战营第二站和第三站任务
2024-04-14
回复了主题帖：免费申请：幸狐 RV1106 Linux 开发板（带摄像头），助力AI挑战营应用落地

本帖最后由 NNTK_NLY 于 2024-4-14 19:08 编辑 #AI挑战营第一站# 利用pytorch进行手写数字模型训练的训练和转换预期应用：使用摄像头获取手写图像，对图像进行处理然后进行NPU推理检测图片所写数字；完成摄像头车牌识别功能。
回复了主题帖：【AI挑战营第一站】模型训练：在PC上完成手写数字模型训练，免费申请RV1106开发板

本帖最后由 NNTK_NLY 于 2024-4-14 19:10 编辑模型训练的本质：使用训练数据集来训练构建模型，通过调整参数不断迭代优化模型，使模型训练出期望能力，能够对新数据进行预测或决策。训练的最终结果：生成一个能够对输入新数据进行准确预测或决策的模型。 PyTorch是什么：一种神经网络学习框架，可用于训练深度学习模型 PyTorch目前都支持哪些系统和计算平台：支持系统：linux mac windows ,支持语言：python、C/C++,支持平台：CUDA、CPU #AI挑战营第一站# 利用pytorch进行手写数字模型训练的训练和转换
发表了主题帖： #AI挑战营第一站# 利用pytorch进行手写数字模型训练的训练和转换

安装pytorch https://pytorch.org/get-started/locally/ 加载程序库 import torch import os import numpy as np import torch.nn as nn import torch.optim as optim from torch.utils.data import DataLoader, random_split from torchvision import datasets, transforms 定义超参数 input_size = 1 * 28 * 28 num_classes = 10 num_epochs = 20 batch_size = 256 learning_rate = 0.0001 定义CNN模型 class CNN(nn.Module): def __init__(self): super(CNN, self).__init__() self.layer1 = nn.Sequential( nn.Conv2d(1, 32, kernel_size=5, stride=1, padding=2), nn.ReLU(), nn.MaxPool2d(kernel_size=2, stride=2)) self.layer2 = nn.Sequential( nn.Conv2d(32, 64, kernel_size=5, stride=1, padding=2), nn.ReLU(), nn.MaxPool2d(kernel_size=2, stride=2)) self.drop_out = nn.Dropout() self.fc1 = nn.Linear(7 * 7 * 64, 1000) self.fc2 = nn.Linear(1000, num_classes) def forward(self, x): out = self.layer1(x) out = self.layer2(out) out = out.reshape(out.size(0), -1) out = self.drop_out(out) out = self.fc1(out) out = self.fc2(out) return out 训练模型 # 训练循环 for epoch in range(num_epochs): model.train() # 设置模型为训练模式 running_loss = 0.0 corrects = 0 for inputs, labels in train_loader: optimizer.zero_grad() # 清空之前的梯度 outputs = model(inputs) # 获取模型输出 loss = criterion(outputs, labels) # 计算损失 loss.backward() # 反向传播计算梯度 optimizer.step() # 使用优化器更新权重 _, preds = torch.max(outputs, 1) # 获取预测结果 corrects += torch.sum(preds == labels.data) # 计算正确预测的数量 running_loss += loss.item() # 累加损失 epoch_loss = running_loss / len(train_loader) epoch_acc = corrects.double() / train_total # 验证模型性能 model.eval() # 设置模型为评估模式 val_loss = 0.0 val_corrects = 0 with torch.no_grad(): # 不计算梯度 for inputs, labels in val_loader: outputs = model(inputs) loss = criterion(outputs, labels) val_loss += loss.item() _, preds = torch.max(outputs, 1) val_corrects += torch.sum(preds == labels.data) val_loss = val_loss / len(val_loader) val_acc = val_corrects.double() / val_total # 打印训练结果和验证结果 print( f'Epoch {epoch + 1}/{num_epochs}, Loss: {epoch_loss:.4f}, Train Acc: {epoch_acc:.4f}, Val Loss: {val_loss:.4f}, Val Acc: {val_acc:.4f}') # 如果当前验证准确率比之前的最佳准确率要好，保存当前权重 if val_acc > best_acc: best_acc = val_acc best_model_wts = model.state_dict() # 获取模型权重测试模型 def test(): network.eval() test_loss = 0 correct = 0 with torch.no_grad(): for data, target in test_loader: output = network(data) pred = output.data.max(1, keepdim=True)[1]

发布的帖子

回复过的帖子

NNTK_NLY

#AI挑战营第二站#RV1106的rknn模型转换、仿真推理

先打卡一下，后续优化https://bbs.eeworld.com.cn/forum.php?mod=viewthread&tid=1283213&page=1&extra=#pid3333154

个人信息已确认，领取板卡，可继续完成&分享挑战营第二站和第三站任务

本帖最后由 NNTK_NLY 于 2024-4-14 19:08 编辑 #AI挑战营第一站# 利用pytorch进行手写数字模型训练的训练和转换预期应用：使用摄像头获取手写图像，对图像进行处理然后进行NPU推理检测图片所写数字；完成摄像头车牌识别功能。

最近访客

统计信息

留言

NNTK_NLY

#AI挑战营第二站#RV1106的rknn模型转换、仿真推理

先打卡一下，后续优化https://bbs.eeworld.com.cn/forum.php?mod=viewthread&tid=1283213&page=1&extra=#pid3333154

个人信息已确认，领取板卡，可继续完成&分享挑战营第二站和第三站任务

本帖最后由 NNTK_NLY 于 2024-4-14 19:08 编辑 #AI挑战营第一站# 利用pytorch进行手写数字模型训练的训练和转换 预期应用：使用摄像头获取手写图像，对图像进行处理然后进行NPU推理检测图片所写数字；完成摄像头车牌识别功能。

最近访客

统计信息

留言

推荐博文

本帖最后由 NNTK_NLY 于 2024-4-14 19:08 编辑 #AI挑战营第一站# 利用pytorch进行手写数字模型训练的训练和转换预期应用：使用摄像头获取手写图像，对图像进行处理然后进行NPU推理检测图片所写数字；完成摄像头车牌识别功能。