HonestQiao

  • 2024-06-29
  • 加入了学习《直播回放: FollowMe 4 W5500-EVB-Pico 使用入门》,观看 W5500-EVB-Pico 使用入门

  • 2024-06-04
  • 发表了主题帖: 【拥抱AIGC 应用ChatGPT和OpenAI API】(一二)了解生成式人工智能与OpenAI和ChatGPT

    AIGC的发展,在2024年尤为迅猛,虽然很多人可能没有明确感受到AIGC带来而变化,但实际上,AIGC已经在方方面面开始改变我们的生活了。   《拥抱AIGC》这本书是在今年2月出版的,而OpenAI在5月份发布了多模态的ChatGPT-4o,已经发生了不小的变化,但是这本书依然值得好好阅读,以便打好基础,跟上AIGC的步伐。   在第一部分中,主要是了解生成式人工智能与OpenAI和ChatGPT,目录如下:     要了解AIGC,也就是生成式人工智能,首先就要了解机器学习,他们的关系如下:   可以说,生成式人工智能,是机器学习多年发展不断改进的必然结果之一。 目前的生成式AI,已经能够做到文本生成、图像生成,以及音乐生成,而视频生成也在不断演进中。   在当前的AI世界中,OpenAI可以说是当之无愧的王者,手握ChatGPT神兵利器,所向披靡。 实际上,OpenAI也是多年磨一剑,才修得神功大成。它从2015年的一个研究组织,发展成为到今天的AI领头羊。   OpenAI研究的核心为Transformer,其起点为论文《Attention is all you need》。 论文原文:[1706.03762] Attention Is All You Need (arxiv.org) 这里贴上两张核心的结构图:     这篇论文,对于想要了解的同学来说,非常值得细读。  

  • 2024-05-21
  • 回复了主题帖: 【Beetle ESP32 C6迷你开发板】低功耗蓝牙(BLE)功能初步使用

    Jacktang 发表于 2024-5-21 07:29 第三部分,BLE UART功能使用 有个图片不显示,   多谢提醒,修改了,再看看

  • 回复了主题帖: 【Beetle ESP32 C6迷你开发板】基于ESP32-C6 BLE的温度环境传感器

    Maker_kun 发表于 2024-5-21 08:41 可玩性太强了,回头我也试一试 别回头了,今晚吧

  • 回复了主题帖: 【2023 DigiKey大赛参与奖】开箱帖:NeoPixel RGB、RGBW-N、RGBW-C、RGBW-W灯环

    Maker_kun 发表于 2024-5-21 11:49 显示效果确实不错,一致性强 那必须的

  • 回复了主题帖: 【2023 DigiKey大赛参与奖】开箱帖:NeoPixel RGB、RGBW-N、RGBW-C、RGBW-W灯环

    wangerxian 发表于 2024-5-21 14:32 这个灯上电效果就一直循环了? 谁亮,谁不亮,怎么亮,什么颜色每一颗都可以单独控制!!!

  • 发表了主题帖: 【2023 DigiKey大赛参与奖】开箱帖:NeoPixel RGB、RGBW-N、RGBW-C、RGBW-W灯环

    最近有点忙,被EEWORLD邮件提醒了两次,终于得捷下单了。 这次的参与奖,没有选择大物件,就选了几个小物件,都是NeoPixel灯环。   等了两周,终于等到得捷的快递了,美国UPS转国内顺丰,顺利收到:   即使是小小的物件,得捷依然包装严密,一层层的袋子,再加环保纸质填充材料。   打开后,拆开包装,最后就是4个小小的灯环:     上面四个灯环,正面,灯珠上没有小黄点的,是普通的NeoPixel灯环,RGB三色搭配。 而有黄点的,则是RGBW灯环,RGBW四色搭配的,分别是: RGBW Nature White: 自然白RGBW RGBW Cool White:冷白RGBW RGBW:暖白RGBW   自然白RGBW效果如下: [localvideo]3c600e10b161b2f83bcf0c5ee005c2ee[/localvideo]   冷白RGBW效果如下: [localvideo]8c7be6bdb33357b8673ca9e0b2b0d5ee[/localvideo]   暖白RGBW效果如下: [localvideo]f48e3ff761e768cce3d7706eb668a3ac[/localvideo]    

  • 2024-05-20
  • 回复了主题帖: 【Beetle ESP32 C6迷你开发板】双机通讯并实现蓝牙开关

    wangerxian 发表于 2024-5-20 14:33 上电自动搜索连接的吗? 那必须的

  • 回复了主题帖: 共读入围:《拥抱AIGC 应用ChatGPT和OpenAI API》

    个人信息无误,确认可以完成评测计划。

  • 发表了主题帖: 【Beetle ESP32 C6迷你开发板】基于ESP32-C6 BLE的温度环境传感器

    在这篇分享中,我使用 Beetle ESP32 C6迷你开发板的ESP32-C6核心自带的温度传感器,结合ESP32-C6的BLE,实现标准的温度环境传感器功能。 当然,如果接上一个SHT30的话,那就可以进行实际的环境温度测量了。   一、BLE环境传感器了解 BLE环境传感器是一种利用蓝牙低功耗(Bluetooth Low Energy, BLE)技术来传输数据的传感器,它能够监测和记录环境参数,并通过无线方式将数据发送到接收设备,如智能手机、平板电脑或专用的监控系统。BLE环境传感器因其低功耗、易于集成和使用方便等特点,在智能家居、工业监测、环境研究等领域得到了广泛的应用。   在BLE标准的GATT定义中,有如下的部分:   其中,定义了环境传感器服务的UUID是 0x181A。环境传感器中温度特性对应的的UUID是0x2A6E,表示以0.01°C为单位的整型温度值。   了解了上面的两个定义,后面在实际创建BLE服务的时候,就需要用到。   二、ESP32-C6核心自带的温度传感器 ESP32-C6 内置1个温度传感器,用于测量芯片内部的温度。该温度传感器模组包含一个 8 位 Sigma-Delta 模拟-数字转换器 (ADC) 和一个数字-模拟转换器 (DAC),可以补偿测量结果,减少温度测量的误差。该温度传感器主要用于测量芯片内部的温度变化。芯片内部温度通常高于环境温度,并且受到微控制器的时钟频率或 I/O 负载、外部散热环境等因素影响。 详细的信息,可以从乐鑫文档了解:https://docs.espressif.com/projects/esp-idf/zh_CN/v5.1.2/esp32c6/api-reference/peripherals/temp_sensor.html   要在Arduino中使用该温度传感器,需要先添加对应的驱动库: #include "driver/temperature_sensor.h" 然后启用温度传感器: // 定义温度数据读取变量 temperature_sensor_handle_t temp_sensor = NULL; temperature_sensor_config_t temp_sensor_config = TEMPERATURE_SENSOR_CONFIG_DEFAULT(10, 50); // 启用内置温度传感器 ESP_ERROR_CHECK(temperature_sensor_install(&temp_sensor_config, &temp_sensor)); ESP_ERROR_CHECK(temperature_sensor_enable(temp_sensor)); 启用后,就可以实际用于测量的,对应的代码如下: // 获取温度值 ESP_ERROR_CHECK(temperature_sensor_get_celsius(temp_sensor, &tsens_value)); 实际的温度值,将会读取到 tsens_value 变量中,直接为十进制的摄氏度浮点温度值。   如果需要对外输出华氏温度值,可以用下面的转换函数: // 转换摄氏度到华氏度的函数 float celsiusToFahrenheit(float celsius) { return (celsius * 9.0 / 5.0) + 32.0; }   三、建立BLE 温度环境传感器服务 参考之前分享的使用BLE UATT服务,建立BLE 温度环境传感器服务步骤类似: 首先定义对应的UUID: #define SERVICE_UUID (BLEUUID((uint16_t)0x181A)) #define CHARACTERISTIC_UUID_Temperature_Measurement (BLEUUID((uint16_t)0x2A6E)) 然后定义对应的Characteristic和Descriptor: // 定义温度的Characteristic和Descriptor BLECharacteristic temperatureCharacteristics(CHARACTERISTIC_UUID_Temperature_Measurement, BLECharacteristic::PROPERTY_NOTIFY); BLEDescriptor temperatureDescriptor(BLEUUID((uint16_t)0x2902)); 定义好了以后,就可以依次进行: 创建服务端并设置回调: // Create the BLE Server pServer = BLEDevice::createServer(); pServer->setCallbacks(new MyServerCallbacks()); 创建服务: // Create the BLE Service BLEService *pService = pServer->createService(SERVICE_UUID); 添加Characteristic,设置Descriptor: // Create a BLE Characteristic pService->addCharacteristic(&temperatureCharacteristics); temperatureCharacteristics.addDescriptor(&temperatureDescriptor); 最后启动服务和广播: // Start the service pService->start(); // Start advertising pServer->getAdvertising()->start();   在loop中,当有蓝牙客户端连接后,则发送温度信息: uint16_t temperatureTemp = (uint16_t)(tsens_value * 100); temperatureCharacteristics.setValue(temperatureTemp); temperatureCharacteristics.notify();   最终,完整的代码如下: #include <BLEDevice.h> #include <BLEServer.h> #include <BLEUtils.h> #include <BLE2902.h> #include "driver/temperature_sensor.h" BLEServer *pServer = NULL; BLECharacteristic *pTxCharacteristic; bool deviceConnected = false; // 定义UUID #define SERVICE_UUID (BLEUUID((uint16_t)0x181A)) #define CHARACTERISTIC_UUID_Temperature_Measurement (BLEUUID((uint16_t)0x2A6E)) // 设置设备名称 #define BLE_SERVER_NAME "ESP32-C6 Temperature Sensor" // 设置温度类型 #define UNIT_C // C - 摄氏度 F - 华氏度 // 定义温度的Characteristic和Descriptor BLECharacteristic temperatureCharacteristics(CHARACTERISTIC_UUID_Temperature_Measurement, BLECharacteristic::PROPERTY_NOTIFY); BLEDescriptor temperatureDescriptor(BLEUUID((uint16_t)0x2902)); // 蓝牙连接/断开处理。当有连接/断开事件发生时自动触发 class MyServerCallbacks : public BLEServerCallbacks { void onConnect(BLEServer *pServer) { //当蓝牙连接时会执行该函数 Serial.println("蓝牙已连接"); deviceConnected = true; }; void onDisconnect(BLEServer *pServer) { //当蓝牙断开连接时会执行该函数 Serial.println("蓝牙已断开"); deviceConnected = false; delay(500); // give the bluetooth stack the chance to get things ready pServer->startAdvertising(); // restart advertising } }; // 定义温度数据读取变量 temperature_sensor_handle_t temp_sensor = NULL; temperature_sensor_config_t temp_sensor_config = TEMPERATURE_SENSOR_CONFIG_DEFAULT(10, 50); // 转换摄氏度到华氏度的函数 float celsiusToFahrenheit(float celsius) { return (celsius * 9.0 / 5.0) + 32.0; } void setup() { Serial.begin(115200); BLEBegin(); //初始化蓝牙 // 启用内置温度传感器 ESP_ERROR_CHECK(temperature_sensor_install(&temp_sensor_config, &temp_sensor)); ESP_ERROR_CHECK(temperature_sensor_enable(temp_sensor)); } void loop() { /****************数据发送部分*************/ /****************************************/ if (deviceConnected) { //如果有蓝牙连接,就发送数据 float tsens_value; // 获取温度值 ESP_ERROR_CHECK(temperature_sensor_get_celsius(temp_sensor, &tsens_value)); #ifdef UNIT_C Serial.print("Temperature Celsius value "); Serial.print(tsens_value, 2); Serial.println(" ℃"); #else // 转换为华氏温度值 tsens_value = celsiusToFahrenheit(tsens_value); Serial.print("Temperature Fahrenheit value "); Serial.print(tsens_value, 2); Serial.println(" °F"); #endif uint16_t temperatureTemp = (uint16_t)(tsens_value * 100); temperatureCharacteristics.setValue(temperatureTemp); temperatureCharacteristics.notify(); delay(1000); } /****************************************/ /****************************************/ } void BLEBegin() { // Create the BLE Device BLEDevice::init(/*BLE名称*/ BLE_SERVER_NAME); // Create the BLE Server pServer = BLEDevice::createServer(); pServer->setCallbacks(new MyServerCallbacks()); // Create the BLE Service BLEService *pService = pServer->createService(SERVICE_UUID); // Create a BLE Characteristic pService->addCharacteristic(&temperatureCharacteristics); temperatureCharacteristics.addDescriptor(&temperatureDescriptor); // Start the service pService->start(); // Start advertising pServer->getAdvertising()->start(); Serial.print("BLE Address: "); Serial.println(BLEDevice::getAddress().toString().c_str()); Serial.println("Waiting a client connection to notify..."); }   编译烧录到Beetle ESP32 C6迷你开发板后,看到串口输出:     此时,就可以用手机客户端工具连接了。   四、手机BLE工具连接: 在手机上,使用ERF Connect,就能找到ESP32-C6 BLE设备:     点进连接,然后就能查看温度信息了:     串口监控也会输出对应的信息:     五、总结 上面的分享,就实现了一个基础的BLE环境传感器功能。 如果感兴趣的话,还可以多多了解BLE服务的知识,把温湿度传感器、燃气传感器、环境光传感器、压力传感器等等都可以实现出来。

  • 2024-05-19
  • 发表了主题帖: 【Beetle ESP32 C6迷你开发板】双机通讯并实现蓝牙开关

    在之前的文章 【Beetle ESP32 C6迷你开发板】低功耗蓝牙(BLE)功能初步使用 中,已经实现了UART Service服务,在此基础上,使用两块Beetle ESP32 C6迷你开发板,就可以实现基于UART的双机通讯,并在此基础上,实现蓝牙开关的功能。   一、双机通讯 双机通讯,原则上,需要一个Server,一个Client。 其中,Server部分,直接使用之前文章中的即可。或者参考DFRobot Beetle ESP32 C6迷你开发板的WiKi文章:https://wiki.dfrobot.com.cn/_SKU_DFR1075_FireBeetle_2_Board_ESP32_C6_Advanced_Tutorial#target_2 Client部分,自动扫描周边的BLE设备,然后检查是否有符合要求的Service设备,如果有,则进行连接,然后进行后续的处理。   Server部分的代码如下: #include <BLEDevice.h> #include <BLEUtils.h> #include <BLEServer.h> #include <BLE2902.h> // See the following for generating UUIDs: // https://www.uuidgenerator.net/ #define SERVICE_UUID "6E400001-B5A3-F393-E0A9-E50E24DCCA9E" // UART service UUID #define CHARACTERISTIC_UUID_RX "6E400002-B5A3-F393-E0A9-E50E24DCCA9E" #define CHARACTERISTIC_UUID_TX "6E400003-B5A3-F393-E0A9-E50E24DCCA9E" uint8_t txValue = 0; bool deviceConnected = false; BLECharacteristic *pTxCharacteristic; //蓝牙连接/断开处理。当有连接/断开事件发生时自动触发 class MyServerCallbacks: public BLEServerCallbacks { void onConnect(BLEServer* pServer) { //当蓝牙连接时会执行该函数 Serial.println("蓝牙已连接"); deviceConnected = true; }; void onDisconnect(BLEServer* pServer) { //当蓝牙断开连接时会执行该函数 Serial.println("蓝牙已断开"); deviceConnected = false; delay(500); // give the bluetooth stack the chance to get things ready BLEDevice::startAdvertising(); // restart advertising } }; /****************数据接收部分*************/ /****************************************/ //蓝牙接收数据处理。当收到数据时自动触发 class MyCallbacks: public BLECharacteristicCallbacks { void onWrite(BLECharacteristic *pCharacteristic) { std::string rxValue = pCharacteristic->getValue(); //使用rxValue接收数据 if (rxValue.length() > 0) { Serial.println("*********"); Serial.print("Received Value: "); for (int i = 0; i < rxValue.length(); i++) Serial.print(rxValue[i]); //将接收的数据打印出来 Serial.println(); Serial.println("*********"); } } }; /****************************************/ /****************************************/ void setup() { Serial.begin(115200); Serial.println("Starting BLE work!"); bleBegin(); } /****************数据发送部分*************/ /****************************************/ void loop() { if(deviceConnected){ //当有设备连接时发送数据 pTxCharacteristic->setValue("我是从机"); pTxCharacteristic->notify(); } /****************************************/ /****************************************/ delay(1000); } void bleBegin() { BLEDevice::init(/*BLE名称*/"ESP32-C6 BLE Server"); BLEServer *pServer = BLEDevice::createServer(); pServer->setCallbacks(new MyServerCallbacks()); BLEService *pService = pServer->createService(SERVICE_UUID); BLECharacteristic *pRxCharacteristic = pService->createCharacteristic( CHARACTERISTIC_UUID_RX, BLECharacteristic::PROPERTY_WRITE ); pRxCharacteristic->setCallbacks(new MyCallbacks()); pTxCharacteristic = pService->createCharacteristic( CHARACTERISTIC_UUID_TX, BLECharacteristic::PROPERTY_NOTIFY ); pTxCharacteristic->addDescriptor(new BLE2902()); pService->start(); // BLEAdvertising *pAdvertising = pServer->getAdvertising(); // this still is working for backward compatibility BLEAdvertising *pAdvertising = BLEDevice::getAdvertising(); pAdvertising->addServiceUUID(SERVICE_UUID); pAdvertising->setScanResponse(true); pAdvertising->setMinPreferred(0x06); // functions that help with iPhone connections issue pAdvertising->setMinPreferred(0x12); BLEDevice::startAdvertising(); }   Client部分的代码如下: #include "BLEDevice.h" #define SERVICE_UUID "6E400001-B5A3-F393-E0A9-E50E24DCCA9E" // UART service UUID #define CHARACTERISTIC_UUID_RX "6E400002-B5A3-F393-E0A9-E50E24DCCA9E" #define CHARACTERISTIC_UUID_TX "6E400003-B5A3-F393-E0A9-E50E24DCCA9E" static BLEUUID serviceUUID(SERVICE_UUID); static BLEUUID charTXUUID(CHARACTERISTIC_UUID_RX); static BLEUUID charRXUUID(CHARACTERISTIC_UUID_TX); static boolean doConnect = false; static boolean connected = false; static boolean doScan = false; static BLERemoteCharacteristic* pTXRemoteCharacteristic; static BLERemoteCharacteristic* pRXRemoteCharacteristic; static BLEAdvertisedDevice* myDevice; /****************数据接收部分*************/ /****************************************/ //蓝牙接收数据处理,当收到数据时自动触发 static void notifyCallback(BLERemoteCharacteristic* pBLERemoteCharacteristic, uint8_t* pData, size_t length, bool isNotify) { //传入uint8_t* pData用于存放数据 String BLEData = ""; for(int i = 0; i < length; i++) // BLEData += (char)pData[i]; Serial.println("*********"); Serial.print("Received Value: "); Serial.println(BLEData); Serial.println("*********"); } /****************************************/ /****************************************/ //蓝牙连接/断开处理。当有连接/断开事件发生时自动触发 class MyClientCallback : public BLEClientCallbacks { void onConnect(BLEClient* pclient) { } void onDisconnect(BLEClient* pclient) { connected = false; Serial.println("onDisconnect"); } }; /** * Scan for BLE servers and find the first one that advertises the service we are looking for. */ //蓝牙扫描处理事件。当开启扫描时自动触发 class MyAdvertisedDeviceCallbacks: public BLEAdvertisedDeviceCallbacks { /** * Called for each advertising BLE server. */ void onResult(BLEAdvertisedDevice advertisedDevice) { //Serial.print("BLE Advertised Device found: "); //Serial.println(advertisedDevice.toString().c_str()); // We have found a device, let us now see if it contains the service we are looking for. if (advertisedDevice.haveServiceUUID() && advertisedDevice.isAdvertisingService(serviceUUID)) { BLEDevice::getScan()->stop(); myDevice = new BLEAdvertisedDevice(advertisedDevice); doConnect = true; doScan = true; } // Found our server } // onResult }; // MyAdvertisedDeviceCallbacks void setup() { Serial.begin(115200); Serial.println("Starting Arduino BLE Client application..."); bleBegin(); } void loop() { // If the flag "doConnect" is true then we have scanned for and found the desired // BLE Server with which we wish to connect. Now we connect to it. Once we are // connected we set the connected flag to be true. if (doConnect == true) { if (connectToServer()) { Serial.println("We are now connected to the BLE Server."); } else { Serial.println("We have failed to connect to the server; there is nothin more we will do."); } doConnect = false; } /****************数据发送部分*************/ /****************************************/ if (connected) { //当连接到蓝牙从机时发送数据 pTXRemoteCharacteristic->writeValue("我是主机"); } if(!connected){ //当没有连接到蓝牙从机时重新扫描 BLEDevice::getScan()->start(5,false); // this is just example to start scan after disconnect, most likely there is better way to do it in arduino } /****************************************/ /****************************************/ delay(1000); } void bleBegin() { BLEDevice::init(""); // Retrieve a Scanner and set the callback we want to use to be informed when we // have detected a new device. Specify that we want active scanning and start the // scan to run for 5 seconds. BLEScan* pBLEScan = BLEDevice::getScan(); pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks());//扫描处理函数 pBLEScan->setInterval(1349);//设置扫描间隔时间 pBLEScan->setWindow(449);//主动扫描时间 pBLEScan->setActiveScan(true); pBLEScan->start(5, false);//扫描时间,单位秒 } //蓝牙连接处理 bool connectToServer() { Serial.print("Forming a connection to "); Serial.println(myDevice->getAddress().toString().c_str()); BLEClient* pClient = BLEDevice::createClient(); Serial.println(" - Created client"); pClient->setClientCallbacks(new MyClientCallback()); // Connect to the remove BLE Server. pClient->connect(myDevice); // if you pass BLEAdvertisedDevice instead of address, it will be recognized type of peer device address (public or private) Serial.println(" - Connected to server"); pClient->setMTU(517); //set client to request maximum MTU from server (default is 23 otherwise) // Obtain a reference to the service we are after in the remote BLE server. BLERemoteService* pRemoteService = pClient->getService(serviceUUID); if (pRemoteService == nullptr) { Serial.print("Failed to find our service UUID: "); Serial.println(serviceUUID.toString().c_str()); pClient->disconnect(); return false; } Serial.println(" - Found our service"); // Obtain a reference to the characteristic in the service of the remote BLE server. pTXRemoteCharacteristic = pRemoteService->getCharacteristic(charTXUUID); if (pTXRemoteCharacteristic == nullptr) { Serial.print("Failed to find our characteristic UUID: "); Serial.println(charTXUUID.toString().c_str()); pClient->disconnect(); return false; } pRXRemoteCharacteristic = pRemoteService->getCharacteristic(charRXUUID); if (pRXRemoteCharacteristic == nullptr) { Serial.print("Failed to find our characteristic UUID: "); Serial.println(charRXUUID.toString().c_str()); pClient->disconnect(); return false; } Serial.println(" - Found our characteristic"); if(pRXRemoteCharacteristic->canNotify()) pRXRemoteCharacteristic->registerForNotify(notifyCallback); connected = true; return true; } 在Client部分的connectToServer()中,连接到 SERVICE_UUID 设定的BLE服务上,然后在通过RX、TX对应的Characteristic,来收发消息。   将上述代码,分别烧录到两个ESP32-C6开发板以后,从串口监听可以收到如下的消息:   二、基于双机通讯的蓝牙开关 在双机通讯的基础上,在要做为开关的一端,添加按键的处理,然后发送控制LED的信息到受控的一端,而受控一端收到消息后,根据消息的具体情况来控制LED的亮灭即可。   处理按键部分,可以使用中断来实现,涉及到的代码如下: #define BUTTON 9 //设置引脚9为按键的引脚 pinMode(BUTTON, INPUT_PULLUP); //设置BUTTON引脚为外部中断引脚 attachInterrupt(BUTTON, PinIntEvent, RISING); void PinIntEvent() { Serial.printf("PinInt Event.\r\n"); } 在上,有一个按键D9/IO9,可以用作BOOT,也可以在启动后,做为用户按键:   在上面的按键中断代码中,就使用了这个按键,一旦按键后,就会调用设置好的中断PinIntEvent(),在PinIntEvent()中,可以添加对应的调用代码: void PinIntEvent() { Serial.printf("PinInt Event.\r\n"); if(millis() - lastInterrupt > 300) // we set a 10ms no-interrupts window { if(deviceConnected){ //当有设备连接时发送数据 ledStatus = !ledStatus; lastInterrupt = millis(); for (;;) { if (shared_var_mutex != NULL) { if (xSemaphoreTake(shared_var_mutex, portMAX_DELAY) == pdTRUE) { if(ledStatus) { Serial.println("控制开灯"); digitalWrite(LED, HIGH); } else { Serial.println("控制关灯"); digitalWrite(LED, LOW); } hasMsg = true; xSemaphoreGive(shared_var_mutex); break; } } } } } } 在主循环,添加发送消息的代码即可: void loop() { if(deviceConnected){ //当有设备连接时发送数据 if(hasMsg) { hasMsg = false; pTxCharacteristic->setValue("我是主机"); pTxCharacteristic->notify(); if(ledStatus) { pTxCharacteristic->setValue("ON"); pTxCharacteristic->notify(); } else { pTxCharacteristic->setValue("OFF"); pTxCharacteristic->notify(); } } } /****************************************/ /****************************************/ // delay(1000); }   在受控端,收到消息后,根据消息的具体内容,控制LED,对应代码如下: //蓝牙接收数据处理,当收到数据时自动触发 static void notifyCallback(BLERemoteCharacteristic* pBLERemoteCharacteristic, uint8_t* pData, size_t length, bool isNotify) { //传入uint8_t* pData用于存放数据 String BLEData = ""; for(int i = 0; i < length; i++) // BLEData += (char)pData[i]; Serial.println("*********"); Serial.print("Received Value: "); Serial.println(BLEData); Serial.println("*********"); if(BLEData == "ON"){ Serial.println("开灯"); digitalWrite(LED, HIGH); hasMsg = true; } //判断接收的字符是否为"ON" if(BLEData == "OFF"){ Serial.println("关灯"); digitalWrite(LED, LOW); hasMsg = true; } //判断接收的字符是否为"OFF" }   通过以上的处理后,控制端和受控端,就能够通过BLE UART服务来发送控制信息,控制LED了。   最终,具体的代码如下: Server代码: /* Based on Neil Kolban example for IDF: https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLE%20Tests/SampleServer.cpp Ported to Arduino ESP32 by Evandro Copercini updates by chegewara */ #include <Arduino.h> #include <BLEDevice.h> #include <BLEUtils.h> #include <BLEServer.h> #include <BLE2902.h> // See the following for generating UUIDs: // https://www.uuidgenerator.net/ #define SERVICE_UUID "6E400001-B5A3-F393-E0A9-E50E24DCCA9E" // UART service UUID #define CHARACTERISTIC_UUID_RX "6E400002-B5A3-F393-E0A9-E50E24DCCA9E" #define CHARACTERISTIC_UUID_TX "6E400003-B5A3-F393-E0A9-E50E24DCCA9E" #define LED 15 //设置引脚15为LED引脚 #define BUTTON 9 //设置引脚9为按键的引脚 uint8_t txValue = 0; bool deviceConnected = false; BLECharacteristic *pTxCharacteristic; SemaphoreHandle_t shared_var_mutex = NULL; bool ledStatus = false; unsigned long lastInterrupt = 0; bool hasMsg = false; // 定义外部中断的Mode // 0: 无中断,读取Touch值 // 1:Touch中断,执行 TouchEvent() // 2: 外部IO的中断 #define EXT_ISR_MODE 2 void TouchEvent() { Serial.printf("Touch Event.\r\n"); } void PinIntEvent() { Serial.printf("PinInt Event.\r\n"); if(millis() - lastInterrupt > 300) // we set a 10ms no-interrupts window { if(deviceConnected){ //当有设备连接时发送数据 ledStatus = !ledStatus; lastInterrupt = millis(); for (;;) { if (shared_var_mutex != NULL) { if (xSemaphoreTake(shared_var_mutex, portMAX_DELAY) == pdTRUE) { if(ledStatus) { Serial.println("控制开灯"); digitalWrite(LED, HIGH); } else { Serial.println("控制关灯"); digitalWrite(LED, LOW); } hasMsg = true; xSemaphoreGive(shared_var_mutex); break; } } } } } } //蓝牙连接/断开处理。当有连接/断开事件发生时自动触发 class MyServerCallbacks: public BLEServerCallbacks { void onConnect(BLEServer* pServer) { //当蓝牙连接时会执行该函数 Serial.println("蓝牙已连接"); deviceConnected = true; }; void onDisconnect(BLEServer* pServer) { //当蓝牙断开连接时会执行该函数 Serial.println("蓝牙已断开"); deviceConnected = false; delay(500); // give the bluetooth stack the chance to get things ready BLEDevice::startAdvertising(); // restart advertising } }; /****************数据接收部分*************/ /****************************************/ //蓝牙接收数据处理。当收到数据时自动触发 class MyCallbacks: public BLECharacteristicCallbacks { void onWrite(BLECharacteristic *pCharacteristic) { String rxValue = pCharacteristic->getValue(); //使用rxValue接收数据 //if(rxValue == "ON"){Serial.println("开灯");} //判断接收的字符是否为"ON" if (rxValue.length() > 0) { Serial.println("*********"); Serial.print("Received Value: "); for (int i = 0; i < rxValue.length(); i++) Serial.print(rxValue); //将接收的数据打印出来 Serial.println(); Serial.println("*********"); } } }; /****************************************/ /****************************************/ void setup() { Serial.begin(115200); Serial.println("Starting BLE work!"); bleBegin(); pinMode(LED, OUTPUT); digitalWrite(LED, LOW); shared_var_mutex = xSemaphoreCreateMutex(); // Create the mutex #if 1 == EXT_ISR_MODE // 触摸中断 // Pin: T0(GPIO4), 函数指针:TouchEvent, 阈值: 40 // touchAttachInterrupt(7, TouchEvent, 40); #elif 2 == EXT_ISR_MODE // 下降沿触发 pinMode(BUTTON, INPUT_PULLUP); //设置BUTTON引脚为外部中断引脚 attachInterrupt(BUTTON, PinIntEvent, RISING); #endif } /****************数据发送部分*************/ /****************************************/ void loop() { if(deviceConnected){ //当有设备连接时发送数据 // pTxCharacteristic->setValue("我是从机"); // pTxCharacteristic->notify(); // pTxCharacteristic->setValue("Hello Sever"); // pTxCharacteristic->notify(); // for (;;) { // if (shared_var_mutex != NULL) { // if (xSemaphoreTake(shared_var_mutex, portMAX_DELAY) == pdTRUE) { // pTxCharacteristic->setValue("我是从机"); // pTxCharacteristic->notify(); // xSemaphoreGive(shared_var_mutex); // break; // } // } // } // hasMsg = true; if(hasMsg) { hasMsg = false; pTxCharacteristic->setValue("我是主机"); pTxCharacteristic->notify(); if(ledStatus) { pTxCharacteristic->setValue("ON"); pTxCharacteristic->notify(); } else { pTxCharacteristic->setValue("OFF"); pTxCharacteristic->notify(); } } } /****************************************/ /****************************************/ // delay(1000); } void bleBegin() { BLEDevice::init(/*BLE名称*/"Long name works now"); BLEServer *pServer = BLEDevice::createServer(); pServer->setCallbacks(new MyServerCallbacks()); BLEService *pService = pServer->createService(SERVICE_UUID); BLECharacteristic *pRxCharacteristic = pService->createCharacteristic( CHARACTERISTIC_UUID_RX, BLECharacteristic::PROPERTY_WRITE ); pRxCharacteristic->setCallbacks(new MyCallbacks()); pTxCharacteristic = pService->createCharacteristic( CHARACTERISTIC_UUID_TX, BLECharacteristic::PROPERTY_NOTIFY ); pTxCharacteristic->addDescriptor(new BLE2902()); pService->start(); // BLEAdvertising *pAdvertising = pServer->getAdvertising(); // this still is working for backward compatibility BLEAdvertising *pAdvertising = BLEDevice::getAdvertising(); pAdvertising->addServiceUUID(SERVICE_UUID); pAdvertising->setScanResponse(true); pAdvertising->setMinPreferred(0x06); // functions that help with iPhone connections issue pAdvertising->setMinPreferred(0x12); BLEDevice::startAdvertising(); }   Client代码: /** * A BLE client example that is rich in capabilities. * There is a lot new capabilities implemented. * author unknown * updated by chegewara */ #include "BLEDevice.h" //#include "BLEScan.h" #define LED 15 //设置引脚13为LED引脚 #define SERVICE_UUID "6E400001-B5A3-F393-E0A9-E50E24DCCA9E" // UART service UUID #define CHARACTERISTIC_UUID_RX "6E400002-B5A3-F393-E0A9-E50E24DCCA9E" #define CHARACTERISTIC_UUID_TX "6E400003-B5A3-F393-E0A9-E50E24DCCA9E" // The remote service we wish to connect to. static BLEUUID serviceUUID(SERVICE_UUID); // The characteristic of the remote service we are interested in. static BLEUUID charTXUUID(CHARACTERISTIC_UUID_RX); static BLEUUID charRXUUID(CHARACTERISTIC_UUID_TX); static boolean doConnect = false; static boolean connected = false; static boolean doScan = false; static BLERemoteCharacteristic* pTXRemoteCharacteristic; static BLERemoteCharacteristic* pRXRemoteCharacteristic; static BLEAdvertisedDevice* myDevice; bool hasMsg = false; /****************数据接收部分*************/ /****************************************/ //蓝牙接收数据处理,当收到数据时自动触发 static void notifyCallback(BLERemoteCharacteristic* pBLERemoteCharacteristic, uint8_t* pData, size_t length, bool isNotify) { //传入uint8_t* pData用于存放数据 String BLEData = ""; for(int i = 0; i < length; i++) // BLEData += (char)pData[i]; Serial.println("*********"); Serial.print("Received Value: "); Serial.println(BLEData); Serial.println("*********"); if(BLEData == "ON"){ Serial.println("开灯"); digitalWrite(LED, HIGH); hasMsg = true; } //判断接收的字符是否为"ON" if(BLEData == "OFF"){ Serial.println("关灯"); digitalWrite(LED, LOW); hasMsg = true; } //判断接收的字符是否为"OFF" //Serial.print("Notify callback for characteristic "); //Serial.print(pBLERemoteCharacteristic->getUUID().toString().c_str()); //Serial.print(" of data length "); //Serial.println(length); } /****************************************/ /****************************************/ //蓝牙连接/断开处理。当有连接/断开事件发生时自动触发 class MyClientCallback : public BLEClientCallbacks { void onConnect(BLEClient* pclient) { } void onDisconnect(BLEClient* pclient) { connected = false; Serial.println("onDisconnect"); } }; /** * Scan for BLE servers and find the first one that advertises the service we are looking for. */ //蓝牙扫描处理事件。当开启扫描时自动触发 class MyAdvertisedDeviceCallbacks: public BLEAdvertisedDeviceCallbacks { /** * Called for each advertising BLE server. */ void onResult(BLEAdvertisedDevice advertisedDevice) { //Serial.print("BLE Advertised Device found: "); //Serial.println(advertisedDevice.toString().c_str()); // We have found a device, let us now see if it contains the service we are looking for. if (advertisedDevice.haveServiceUUID() && advertisedDevice.isAdvertisingService(serviceUUID)) { BLEDevice::getScan()->stop(); myDevice = new BLEAdvertisedDevice(advertisedDevice); doConnect = true; doScan = true; } // Found our server } // onResult }; // MyAdvertisedDeviceCallbacks void setup() { Serial.begin(115200); Serial.println("Starting Arduino BLE Client application..."); bleBegin(); pinMode(LED, OUTPUT); digitalWrite(LED, LOW); } void loop() { // If the flag "doConnect" is true then we have scanned for and found the desired // BLE Server with which we wish to connect. Now we connect to it. Once we are // connected we set the connected flag to be true. if (doConnect == true) { if (connectToServer()) { Serial.println("We are now connected to the BLE Server."); } else { Serial.println("We have failed to connect to the server; there is nothin more we will do."); } doConnect = false; } /****************数据发送部分*************/ /****************************************/ if (connected) { //当连接到蓝牙从机时发送数据 // pTXRemoteCharacteristic->writeValue("我是从机"); // hasMsg = true; if(hasMsg) { hasMsg = false; pTXRemoteCharacteristic->writeValue("我是从机"); pTXRemoteCharacteristic->writeValue("OK"); } } if(!connected){ //当没有连接到蓝牙从机时重新扫描 BLEDevice::getScan()->start(5,false); // this is just example to start scan after disconnect, most likely there is better way to do it in arduino delay(1000); } delay(1000); /****************************************/ /****************************************/ } void bleBegin() { BLEDevice::init(""); // Retrieve a Scanner and set the callback we want to use to be informed when we // have detected a new device. Specify that we want active scanning and start the // scan to run for 5 seconds. BLEScan* pBLEScan = BLEDevice::getScan(); pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks());//扫描处理函数 pBLEScan->setInterval(1349);//设置扫描间隔时间 pBLEScan->setWindow(449);//主动扫描时间 pBLEScan->setActiveScan(true); pBLEScan->start(5, false);//扫描时间,单位秒 } //蓝牙连接处理 bool connectToServer() { Serial.print("Forming a connection to "); Serial.println(myDevice->getAddress().toString().c_str()); BLEClient* pClient = BLEDevice::createClient(); Serial.println(" - Created client"); pClient->setClientCallbacks(new MyClientCallback()); // Connect to the remove BLE Server. pClient->connect(myDevice); // if you pass BLEAdvertisedDevice instead of address, it will be recognized type of peer device address (public or private) Serial.println(" - Connected to server"); pClient->setMTU(517); //set client to request maximum MTU from server (default is 23 otherwise) // Obtain a reference to the service we are after in the remote BLE server. BLERemoteService* pRemoteService = pClient->getService(serviceUUID); if (pRemoteService == nullptr) { Serial.print("Failed to find our service UUID: "); Serial.println(serviceUUID.toString().c_str()); pClient->disconnect(); return false; } Serial.println(" - Found our service"); // Obtain a reference to the characteristic in the service of the remote BLE server. pTXRemoteCharacteristic = pRemoteService->getCharacteristic(charTXUUID); if (pTXRemoteCharacteristic == nullptr) { Serial.print("Failed to find our characteristic UUID: "); Serial.println(charTXUUID.toString().c_str()); pClient->disconnect(); return false; } pRXRemoteCharacteristic = pRemoteService->getCharacteristic(charRXUUID); if (pRXRemoteCharacteristic == nullptr) { Serial.print("Failed to find our characteristic UUID: "); Serial.println(charRXUUID.toString().c_str()); pClient->disconnect(); return false; } Serial.println(" - Found our characteristic"); if(pRXRemoteCharacteristic->canNotify()) pRXRemoteCharacteristic->registerForNotify(notifyCallback); connected = true; return true; } 将上述代码烧录到两块开发板,通过串口监听,可以查看对应的输出信息:  在Server板上按BOOT按键 ,就会发送消息到Client板,进行LED的控制了。      

  • 2024-05-18
  • 发表了主题帖: 【Beetle ESP32 C6迷你开发板】低功耗蓝牙(BLE)功能初步使用

    本帖最后由 HonestQiao 于 2024-5-21 18:50 编辑 一、BLE功能了解 ESP32-C6搭载160MHz的高性能RISC-V 32位处理器,支持Wi-Fi 6、Bluetooth 5、Zigbee 3.0、Thread 1.3通讯协议,可接入多种通讯协议的物联网网络。 其中的Bluetooth 5,是蓝牙标准第5代,和Bluetooth 4对比如下:   从上面的对比可以看到,Bluetooth 5比Bluetooth 4提升了一大截。   乐鑫的esp-idf,也为ESP32-C6提供了完善的低功耗蓝牙(BLE)支持能力。   二、BLE扫描功能使用 为了方便分享,本篇分享的实例,使用了友好的Arduino开发平台。 在Arduino的ESP32支持库Arduino-ESP32中,有BLE能力的支持库:https://github.com/espressif/arduino-esp32/tree/master/libraries/BLE BLE的扫描功能,主要用于设备发现,通过周边BLE设备的广播信息,获取BLE设备的基础信息。   要使用设备发现功能,首先需要调用几个关键的库文件: #include <BLEDevice.h> #include <BLEUtils.h> #include <BLEScan.h> #include <BLEAdvertisedDevice.h> 上面的库文件,提供了BLE基本功能支持,扫描功能支持,和设备广播信息支持。   然后实例化一个BLE设备对象: BLEScan *pBLEScan; 在开启扫描功能: pBLEScan = BLEDevice::getScan(); //create new scan pBLEScan->setActiveScan(true); //active scan uses more power, but get results faster pBLEScan->setInterval(100); pBLEScan->setWindow(99); // less or equal setInterval value 同时,还要设置一个广播信息获取后的回调,以便把扫描到的BLE谁被信息输出,具体如下: class MyAdvertisedDeviceCallbacks : public BLEAdvertisedDeviceCallbacks { void onResult(BLEAdvertisedDevice advertisedDevice) { Serial.printf("Advertised Device: %s \n", advertisedDevice.toString().c_str()); } }; pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks()); 上面代码中,定一个基类BLEAdvertisedDeviceCallbacks的类MyAdvertisedDeviceCallbacks,其中的onResult用于接收扫描到的设备信息。   然后,启动扫描功能即可进行扫描和输出信息: BLEScanResults *foundDevices = pBLEScan->start(5, false); Serial.println(foundDevices->getCount()); // 获取扫描到的BLE设备数量 pBLEScan->clearResults(); // 清理扫描结果   完整的代码如下: /* Based on Neil Kolban example for IDF: https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLE%20Tests/SampleScan.cpp Ported to Arduino ESP32 by Evandro Copercini */ #include <BLEDevice.h> #include <BLEUtils.h> #include <BLEScan.h> #include <BLEAdvertisedDevice.h> int scanTime = 5; //In seconds BLEScan *pBLEScan; class MyAdvertisedDeviceCallbacks : public BLEAdvertisedDeviceCallbacks { void onResult(BLEAdvertisedDevice advertisedDevice) { Serial.printf("Advertised Device: %s \n", advertisedDevice.toString().c_str()); } }; void setup() { Serial.begin(115200); Serial.println("Scanning..."); BLEDevice::init(""); pBLEScan = BLEDevice::getScan(); //create new scan pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks()); pBLEScan->setActiveScan(true); //active scan uses more power, but get results faster pBLEScan->setInterval(100); pBLEScan->setWindow(99); // less or equal setInterval value } void loop() { // put your main code here, to run repeatedly: BLEScanResults *foundDevices = pBLEScan->start(scanTime, false); Serial.print("Devices found: "); Serial.println(foundDevices->getCount()); Serial.println("Scan done!"); pBLEScan->clearResults(); // delete results fromBLEScan buffer to release memory delay(2000); }   在Arduino中,按照下面的设置,进行编译烧录:   烧录后,打开串口监控,可以看到扫描输出信息:   上面输出中的L4260,就是我家打印机了。   三、BLE UART功能使用 BLE设备可以通过GATT的GAP定义自身的角色,例如:外围设备(Peripheral)和中心设备(Central)。 GATT 是一个在蓝牙连接之上的发送和接收很短的数据段的通用规范。 GAP定义了设备如何彼此发现、建立连接以及如何实现绑定,同时描述了设备如何成为广播者和观察者,并且实现无需连接的传输。GAP 使你的设备被其他设备可见,并决定了你的设备是否可以或者怎样与合同设备进行交互。 当做为外围设备(Peripheral),可以对外发布自身能够提供的服务。其中,最常见的一项服务是UART服务,可以通过低功耗蓝牙,实现乐思串口通讯的能力。UART服务最早是Nordic Semiconductor 的 BLE 模组提供的自定义服务Nordic UART Service,现在只要遵循其标准定义,那么都可以提供该服务。   通过BLE对外提供服务,主要需要定义服务(Service) 和 标签(Characteristic),以便告知其他设备,自己能够提供的服务和具体的功能:     而在UART服务中,Service包含两个Characteristic,一个被配置只读的通道(RX),另一个配置为只写的通道(TX)。 通常情况下,Service和Characteristic都是一个UUID,UART对应的如下: UART Service:6E400001-B5A3-F393-E0A9-E50E24DCCA9E RX标签:6E400002-B5A3-F393-E0A9-E50E24DCCA9E,表示可以从该标签接收到信息 TX标签:6E400003-B5A3-F393-E0A9-E50E24DCCA9E ,表示可以向改标签写入信息   要在ESP32-C6上提供UART 服务,需要按照以下的步骤进行:    创建BLE服务端    创建BLE服务    创建BLE服务对应的标签    设置标签对应的描述信息    启动服务    启动广播 参考官方的实例,测试代码如下: #include <BLEDevice.h> #include <BLEServer.h> #include <BLEUtils.h> #include <BLE2902.h> BLEServer *pServer = NULL; BLECharacteristic *pTxCharacteristic; bool deviceConnected = false; bool oldDeviceConnected = false; uint8_t txValue = 0; // See the following for generating UUIDs: // https://www.uuidgenerator.net/ #define SERVICE_UUID "6E400001-B5A3-F393-E0A9-E50E24DCCA9E" // UART service UUID #define CHARACTERISTIC_UUID_RX "6E400002-B5A3-F393-E0A9-E50E24DCCA9E" #define CHARACTERISTIC_UUID_TX "6E400003-B5A3-F393-E0A9-E50E24DCCA9E" class MyServerCallbacks : public BLEServerCallbacks { void onConnect(BLEServer *pServer) { deviceConnected = true; }; void onDisconnect(BLEServer *pServer) { deviceConnected = false; } }; class MyCallbacks : public BLECharacteristicCallbacks { void onWrite(BLECharacteristic *pCharacteristic) { String rxValue = pCharacteristic->getValue(); if (rxValue.length() > 0) { Serial.println("*********"); Serial.print("Received Value: "); for (int i = 0; i < rxValue.length(); i++) { Serial.print(rxValue[i]); } Serial.println(); Serial.println("*********"); } } }; void setup() { Serial.begin(115200); // Create the BLE Device BLEDevice::init("ESP32-C6 UART Service"); // Create the BLE Server pServer = BLEDevice::createServer(); pServer->setCallbacks(new MyServerCallbacks()); // Create the BLE Service BLEService *pService = pServer->createService(SERVICE_UUID); // Create a BLE Characteristic pTxCharacteristic = pService->createCharacteristic(CHARACTERISTIC_UUID_TX, BLECharacteristic::PROPERTY_NOTIFY); pTxCharacteristic->addDescriptor(new BLE2902()); BLECharacteristic *pRxCharacteristic = pService->createCharacteristic(CHARACTERISTIC_UUID_RX, BLECharacteristic::PROPERTY_WRITE); pRxCharacteristic->setCallbacks(new MyCallbacks()); // Start the service pService->start(); // Start advertising pServer->getAdvertising()->start(); Serial.println("Waiting a client connection to notify..."); } void loop() { if (deviceConnected) { pTxCharacteristic->setValue(&txValue, 1); pTxCharacteristic->notify(); txValue++; delay(1000); // bluetooth stack will go into congestion, if too many packets are sent pTxCharacteristic->setValue("\nESP32-C6\n"); pTxCharacteristic->notify(); delay(1000); // bluetooth stack will go into congestion, if too many packets are sent } // disconnecting if (!deviceConnected && oldDeviceConnected) { delay(500); // give the bluetooth stack the chance to get things ready pServer->startAdvertising(); // restart advertising Serial.println("start advertising"); oldDeviceConnected = deviceConnected; } // connecting if (deviceConnected && !oldDeviceConnected) { // do stuff here on connecting oldDeviceConnected = deviceConnected; } }   在上述代码中的setup()部分,按照以上步骤,进行了对应的服务的创建,标签的创建,以及服务和广播的启动。 另外,还定义了两个回调类,分别是MyServerCallbacks用于处理蓝牙连接状态,MyCallbacks用于处理收到信息时的处理。   在loop()循环调用中,则进行了信息发送的处理,以及根据连接情况决定是否启动广播的处理。   在Arduino中编译烧录后,串口监听输出如下:  表示ESP32-C6启动了广播,可以用手机或者电脑连接获取服务了。    此时,用前面的扫描程序,可以扫描到:   从上图中可以看到,找到了 ESP32-C6 UART Service 了。   此时,用BLE工具,可以进行连接,就能够收到对应的信息了:   在上面代码的loop()中,有两个设置发送信息的处理: pTxCharacteristic->setValue(&txValue, 1); pTxCharacteristic->notify(); pTxCharacteristic->setValue("\nESP32-C6\n"); pTxCharacteristic->notify(); 第一个表示发送数值,大小为1字节;第二个表示发送一个字符串。   把BLE工具的接收HEX打开,就可以看到发送的数值信息了:       然后,在BLE工具中,给ESP32-C6发送信息:   串口监控就会收到发送的信息了:   四、总结 刚开始用BLE功能时,觉得基于GATT的处理过程很麻烦,但是一旦用上了后,会非常的好用,因为已经有很多标准化的处理了,符合标准定义的设备,可以轻松的进行互联和通信。 如果你的设备,按照标准定义来广播Service和通过Characteristic提供对应的实际功能,那么其他符合这个标准的设备,就能够快速连接获取数据或者发送数据。 现在有很多BLE低功耗设备,如心率传感器、手环、温度传感器、光纤传感器等采用,你不用了解它具体的工作细节,仅需要通过BLE扫描发现对应的BLE设备和服务,然后就可以通过标准定义的Characteristic来获取数据了,非常的方便。

  • 2024-05-15
  • 回复了主题帖: 《深度学习与医学图像处理》【学习分享7】医学图像的语义分割和模型训练实践

    秦天qintian0303 发表于 2024-5-14 08:26 医学图像的细节太多了,有很多隐藏的信息,这直接AI模型都跑了,厉害 看了书,就得练习练习实战一下,更加能够了解了

  • 2024-05-13
  • 发表了主题帖: 《深度学习与医学图像处理》【学习分享7】医学图像的语义分割和模型训练实践

    《深度学习与医学图像处理》第6章的内容,讲的是医学图像的语义分割。 医学图像的语义分割是一种计算机视觉处理技术,用于将医学图像中的数据分配到特定的类别,从而区分出图像中的不同结构和组织。语义分割对于医学图像分析尤为重要,可以帮助医生更准确地识别和理解图像中的解剖结构,从而提高疾病的诊断、治疗规划和手术导航的准确性。   语义分割涉及到不少关键性的步骤,包括: 图像预处理:对原始图像进行去噪、增强等处理,以提高图像质量。 特征提取:使用各种算法(如卷积神经网络CNN)提取图像特征。 分割网络:设计和训练一个神经网络模型,该网络能够识别图像中的不同区域,并将其分割成不同的类别。 优化处理:对分割结果进行优化,如形态学操作、平滑等,以提高分割的准确性。 测试评估:使用各种指标(如交并比IoU、Dice系数等)评估分割结果的质量。 在本章的内容中,重点讲解了以上步骤中所涉及到的损失函数、评价指标、分割模型,并提供了实战的指导。   一、内容思维导图 本章内容部分的思维导图如下:     了解了上图中的相关内容后,就可以按照书中的指导,进行实战了。   二、环境准备 在 【学习分享6】医学图像分类处理和模型训练实践 的环境基础上,可以进行本章的实战,不过还需要安装一下如下的python库: pip install SimpleITK==2.0.2   因为本章涉及到数据的语义分割,会进行图片的展示,而我是在远程服务器上进行训练的,所以需要做一些预备工作,以便通过vnc来进行图像数据的显示。 # 安装xvfb sudo apt install -y xvfb # 开启xvfb Xvfb :0 -screen 0 1280x960x24 -listen tcp -ac +extension GLX +extension RENDER & # 设置默认显示 export DISPLAY=:0 # 安装x11vnc sudo apt install -y x11vnc # 启动x11vnc sudo x11vnc -display :0 -forever -shared -rfbport 端口 -passwd 密码 & 通过上面的步骤,开启vnc,然后在本地电脑,使用vnc客户端工具连接。 然后使用python测试画图: import matplotlib.pyplot as plt import numpy as np # 生成数据 x = np.arange(0, 10, 0.1) # 横坐标数据为从0到10之间,步长为0.1的等差数组 y = np.sin(x) # 纵坐标数据为 x 对应的 sin(x) 值 # 生成图形 plt.plot(x, y) # 显示图形 plt.show()   能够正常显示图片,就可以后续处理了:     三、测试数据集下载 本章的测试数据集,使用的而是MRI公开数据集,从 Medical Segmentation Decathlon 下载,然后解压。 tar xvf Task01_BrainTumour.tar ls -l Task01_BrainTumour   解压后的文件如下:   四、数据预处理 首先,查看一下数据对应的图像,对应的代码如下: import os import SimpleITK as sitk import matplotlib.pyplot as plt import numpy as np import json from skimage import measure import random # import tensorflow as tf import tensorflow.compat.v1 as tf tf.disable_v2_behavior() tf.logging.set_verbosity(tf.logging.ERROR) from tensorflow.keras.layers import Layer from tensorflow import keras from tensorflow.keras.layers import Conv3D, Conv3DTranspose, ReLU, UpSampling3D, Input, MaxPool3D, Concatenate from tensorflow.keras import Model def z_score_norm(img, mean=None, std=None): if mean is None: mean = np.mean(img) if std is None: std = np.std(img) return (img - mean) / (std + 1e-6) def show_imgs(images, cols=5): rows = len(images) titles = ['FLAIR', 'T1w', 'T1gd', 'T2w', 'Label'] f, axes = plt.subplots(max(1, rows), cols) axes_ravel = axes.ravel() for i, (image, label) in enumerate(images): ds = np.where(label != 0)[0] ds.sort() slice = ds[len(ds) // 2] for j in range(cols-1): axes_ravel[i*cols+j].set_axis_off() axes_ravel[i*cols+j].imshow(image[slice, :, :, j], cmap='Greys_r') axes_ravel[i*cols+j].set_title(titles[j]) axes_ravel[(i+1)*cols-1].set_axis_off() axes_ravel[(i+1)*cols-1].imshow(label[slice, :, :], cmap='Greys_r') axes_ravel[(i+1)*cols-1].set_title(titles[-1]) f.tight_layout() plt.subplots_adjust(wspace=0.01, hspace=0) plt.show() def read_img(img_path, label_path=None): img_itk = sitk.ReadImage(img_path) img_np = sitk.GetArrayFromImage(img_itk) img_np = np.moveaxis(img_np, 0, 3) if label_path is not None: label_itk = sitk.ReadImage(label_path) label_np = sitk.GetArrayFromImage(label_itk) return img_np, label_np return img_np img_path = '../data/Task01_BrainTumour/imagesTr/BRATS_001.nii.gz' label_path = '../data/Task01_BrainTumour/labelsTr/BRATS_001.nii.gz' img_np, label_np = read_img(img_path, label_path) show_imgs([[img_np, label_np]])   运行后,会显示上面代码中对应的数据的图像:   因为是测试数据集,所以包含了实际的病灶部位的图像。   然后,对图像进行预处理,对应的代码如下: import os import SimpleITK as sitk import matplotlib.pyplot as plt import numpy as np import json from skimage import measure import random # import tensorflow as tf import tensorflow.compat.v1 as tf tf.disable_v2_behavior() tf.logging.set_verbosity(tf.logging.ERROR) from tensorflow.keras.layers import Layer from tensorflow import keras from tensorflow.keras.layers import Conv3D, Conv3DTranspose, ReLU, UpSampling3D, Input, MaxPool3D, Concatenate from tensorflow.keras import Model def z_score_norm(img, mean=None, std=None): if mean is None: mean = np.mean(img) if std is None: std = np.std(img) return (img - mean) / (std + 1e-6) def show_imgs(images, cols=5): rows = len(images) titles = ['FLAIR', 'T1w', 'T1gd', 'T2w', 'Label'] f, axes = plt.subplots(max(1, rows), cols) axes_ravel = axes.ravel() for i, (image, label) in enumerate(images): ds = np.where(label != 0)[0] ds.sort() slice = ds[len(ds) // 2] for j in range(cols-1): axes_ravel[i*cols+j].set_axis_off() axes_ravel[i*cols+j].imshow(image[slice, :, :, j], cmap='Greys_r') axes_ravel[i*cols+j].set_title(titles[j]) axes_ravel[(i+1)*cols-1].set_axis_off() axes_ravel[(i+1)*cols-1].imshow(label[slice, :, :], cmap='Greys_r') axes_ravel[(i+1)*cols-1].set_title(titles[-1]) f.tight_layout() plt.subplots_adjust(wspace=0.01, hspace=0) plt.show() def read_img(img_path, label_path=None): img_itk = sitk.ReadImage(img_path) img_np = sitk.GetArrayFromImage(img_itk) img_np = np.moveaxis(img_np, 0, 3) if label_path is not None: label_itk = sitk.ReadImage(label_path) label_np = sitk.GetArrayFromImage(label_itk) return img_np, label_np return img_np def extract_ordered_overlap_patches(img, label, patch_size, s=16): img = img[:, 20:-20, 20:-20, :] d, h, w, _ = img.shape patch_d, patch_h, patch_w = patch_size sd = d - patch_d sh = h - patch_h sw = w - patch_w std = s sth = s*2 stw = s*2 patch_list = [] pos_list = [] if label is not None: label = label[:, 20:-20, 20:-20] for i in range(sd // std + 1): for j in range(sh // sth + 1): for k in range(sw//stw + 1): patch_img = img[i*std:min(d, i*std+patch_d), j*sth:min(h, j*sth+patch_h), k*stw:min(w, k*stw+patch_w), :] if label is not None: patch_label = label[i*std:min(d, i*std+patch_d), j*sth:min(h, j*sth+patch_h), k*stw:min(w, k*stw+patch_w)] if patch_label.shape != tuple(patch_size): continue if np.count_nonzero(patch_label)/np.count_nonzero(label) >= 0.2: patch_list.append((patch_img, patch_label)) pos_list.append((i, j, k)) else: patch_list.append(patch_img) pos_list.append((i, j, k)) return patch_list, pos_list # pre-process images def preprocess(image): # z-score normalization in each slice and each channel for i in range(image.shape[3]): for z in range(image.shape[0]): img_slice = image[z, :, :, i] image[z, :, :, i] = z_score_norm(img_slice) return image img_path = '../data/Task01_BrainTumour/imagesTr/BRATS_001.nii.gz' label_path = '../data/Task01_BrainTumour/labelsTr/BRATS_001.nii.gz' img_np, label_np = read_img(img_path, label_path) show_imgs([[img_np, label_np]]) patch_size = (32, 160, 160) patch_list, _ = extract_ordered_overlap_patches(img_np, label_np, patch_size) show_imgs(patch_list[:1])   运行后,结果如下:   紧接着,就是数据生成器部分了,对应的代码如下: import os import SimpleITK as sitk import matplotlib.pyplot as plt import numpy as np import json from skimage import measure import random # import tensorflow as tf import tensorflow.compat.v1 as tf tf.disable_v2_behavior() tf.logging.set_verbosity(tf.logging.ERROR) from tensorflow.keras.layers import Layer from tensorflow import keras from tensorflow.keras.layers import Conv3D, Conv3DTranspose, ReLU, UpSampling3D, Input, MaxPool3D, Concatenate from tensorflow.keras import Model def z_score_norm(img, mean=None, std=None): if mean is None: mean = np.mean(img) if std is None: std = np.std(img) return (img - mean) / (std + 1e-6) def show_imgs(images, cols=5): rows = len(images) titles = ['FLAIR', 'T1w', 'T1gd', 'T2w', 'Label'] f, axes = plt.subplots(max(1, rows), cols) axes_ravel = axes.ravel() for i, (image, label) in enumerate(images): ds = np.where(label != 0)[0] ds.sort() slice = ds[len(ds) // 2] for j in range(cols-1): axes_ravel[i*cols+j].set_axis_off() axes_ravel[i*cols+j].imshow(image[slice, :, :, j], cmap='Greys_r') axes_ravel[i*cols+j].set_title(titles[j]) axes_ravel[(i+1)*cols-1].set_axis_off() axes_ravel[(i+1)*cols-1].imshow(label[slice, :, :], cmap='Greys_r') axes_ravel[(i+1)*cols-1].set_title(titles[-1]) f.tight_layout() plt.subplots_adjust(wspace=0.01, hspace=0) plt.show() def read_img(img_path, label_path=None): img_itk = sitk.ReadImage(img_path) img_np = sitk.GetArrayFromImage(img_itk) img_np = np.moveaxis(img_np, 0, 3) if label_path is not None: label_itk = sitk.ReadImage(label_path) label_np = sitk.GetArrayFromImage(label_itk) return img_np, label_np return img_np def extract_ordered_overlap_patches(img, label, patch_size, s=16): img = img[:, 20:-20, 20:-20, :] d, h, w, _ = img.shape patch_d, patch_h, patch_w = patch_size sd = d - patch_d sh = h - patch_h sw = w - patch_w std = s sth = s*2 stw = s*2 patch_list = [] pos_list = [] if label is not None: label = label[:, 20:-20, 20:-20] for i in range(sd // std + 1): for j in range(sh // sth + 1): for k in range(sw//stw + 1): patch_img = img[i*std:min(d, i*std+patch_d), j*sth:min(h, j*sth+patch_h), k*stw:min(w, k*stw+patch_w), :] if label is not None: patch_label = label[i*std:min(d, i*std+patch_d), j*sth:min(h, j*sth+patch_h), k*stw:min(w, k*stw+patch_w)] if patch_label.shape != tuple(patch_size): continue if np.count_nonzero(patch_label)/np.count_nonzero(label) >= 0.2: patch_list.append((patch_img, patch_label)) pos_list.append((i, j, k)) else: patch_list.append(patch_img) pos_list.append((i, j, k)) return patch_list, pos_list # pre-process images def preprocess(image): # z-score normalization in each slice and each channel for i in range(image.shape[3]): for z in range(image.shape[0]): img_slice = image[z, :, :, i] image[z, :, :, i] = z_score_norm(img_slice) return image # data loader def data_generator(data_dir, path_list, target_shape, batch_size, is_training, buffer_size=8): if not is_training: buffer_size = 1 else: random.shuffle(path_list) buffer_size = min(len(path_list), buffer_size) k = len(path_list) // buffer_size for i in range(k): data_list = [] for j in range(i*buffer_size, (i+1)*buffer_size): img_path = path_list[j]['image'].replace('./', data_dir) label_path = path_list[j]['label'].replace('./', data_dir) if not os.path.exists(img_path) or not os.path.exists(label_path): continue img, label = read_img(img_path, label_path) img = preprocess(img) patch_list, _ = extract_ordered_overlap_patches(img, label, target_shape) data_list += patch_list X = np.array([it[0] for it in data_list]) Y = np.array([it[1] for it in data_list]) if is_training: index = np.random.permutation(len(data_list)) X = X[index, ...] Y = Y[index, ...] for step in range(X.shape[0]//batch_size-1): x = X[step * batch_size:(step + 1) * batch_size, ...] y = Y[step * batch_size:(step + 1) * batch_size, ...] yield x, y img_path = '../data/Task01_BrainTumour/imagesTr/BRATS_001.nii.gz' label_path = '../data/Task01_BrainTumour/labelsTr/BRATS_001.nii.gz' img_np, label_np = read_img(img_path, label_path) show_imgs([[img_np, label_np]]) patch_size = (32, 160, 160) patch_list, _ = extract_ordered_overlap_patches(img_np, label_np, patch_size) show_imgs(patch_list[:1]) data_dir = '../data/Task01_BrainTumour/' with open(os.path.join(data_dir, 'dataset.json'), 'r') as f: data = json.load(f) train_path_list = data['training'] # patch_size = (32, 160, 160) test_data = data_generator(data_dir, train_path_list[:3], patch_size, 1, False, 1) for x,y in test_data: print(x.shape, y.shape)   运行后,输出结果如下:   有了这个输出,说明准备工作都做好,可以开始实际的训练了。   五、模型训练 模型训练部分的代码如下: import os import SimpleITK as sitk import matplotlib.pyplot as plt import numpy as np import json from skimage import measure import random # import tensorflow as tf import tensorflow.compat.v1 as tf tf.disable_v2_behavior() tf.logging.set_verbosity(tf.logging.ERROR) from tensorflow.keras.layers import Layer from tensorflow import keras from tensorflow.keras.layers import Conv3D, Conv3DTranspose, ReLU, UpSampling3D, Input, MaxPool3D, Concatenate from tensorflow.keras import Model os.environ["CUDA_VISIBLE_DEVICES"] = "0,1,2,3" def z_score_norm(img, mean=None, std=None): if mean is None: mean = np.mean(img) if std is None: std = np.std(img) return (img - mean) / (std + 1e-6) def show_imgs(images, cols=5): rows = len(images) titles = ['FLAIR', 'T1w', 'T1gd', 'T2w', 'Label'] f, axes = plt.subplots(max(1, rows), cols) axes_ravel = axes.ravel() for i, (image, label) in enumerate(images): ds = np.where(label != 0)[0] ds.sort() slice = ds[len(ds) // 2] for j in range(cols-1): axes_ravel[i*cols+j].set_axis_off() axes_ravel[i*cols+j].imshow(image[slice, :, :, j], cmap='Greys_r') axes_ravel[i*cols+j].set_title(titles[j]) axes_ravel[(i+1)*cols-1].set_axis_off() axes_ravel[(i+1)*cols-1].imshow(label[slice, :, :], cmap='Greys_r') axes_ravel[(i+1)*cols-1].set_title(titles[-1]) f.tight_layout() plt.subplots_adjust(wspace=0.01, hspace=0) plt.show() def read_img(img_path, label_path=None): img_itk = sitk.ReadImage(img_path) img_np = sitk.GetArrayFromImage(img_itk) img_np = np.moveaxis(img_np, 0, 3) if label_path is not None: label_itk = sitk.ReadImage(label_path) label_np = sitk.GetArrayFromImage(label_itk) return img_np, label_np return img_np def extract_ordered_overlap_patches(img, label, patch_size, s=16): img = img[:, 20:-20, 20:-20, :] d, h, w, _ = img.shape patch_d, patch_h, patch_w = patch_size sd = d - patch_d sh = h - patch_h sw = w - patch_w std = s sth = s*2 stw = s*2 patch_list = [] pos_list = [] if label is not None: label = label[:, 20:-20, 20:-20] for i in range(sd // std + 1): for j in range(sh // sth + 1): for k in range(sw//stw + 1): patch_img = img[i*std:min(d, i*std+patch_d), j*sth:min(h, j*sth+patch_h), k*stw:min(w, k*stw+patch_w), :] if label is not None: patch_label = label[i*std:min(d, i*std+patch_d), j*sth:min(h, j*sth+patch_h), k*stw:min(w, k*stw+patch_w)] if patch_label.shape != tuple(patch_size): continue if np.count_nonzero(patch_label)/np.count_nonzero(label) >= 0.2: patch_list.append((patch_img, patch_label)) pos_list.append((i, j, k)) else: patch_list.append(patch_img) pos_list.append((i, j, k)) return patch_list, pos_list # pre-process images def preprocess(image): # z-score normalization in each slice and each channel for i in range(image.shape[3]): for z in range(image.shape[0]): img_slice = image[z, :, :, i] image[z, :, :, i] = z_score_norm(img_slice) return image # data loader def data_generator(data_dir, path_list, target_shape, batch_size, is_training, buffer_size=8): if not is_training: buffer_size = 1 else: random.shuffle(path_list) buffer_size = min(len(path_list), buffer_size) k = len(path_list) // buffer_size for i in range(k): data_list = [] for j in range(i*buffer_size, (i+1)*buffer_size): img_path = path_list[j]['image'].replace('./', data_dir) label_path = path_list[j]['label'].replace('./', data_dir) if not os.path.exists(img_path) or not os.path.exists(label_path): continue img, label = read_img(img_path, label_path) img = preprocess(img) patch_list, _ = extract_ordered_overlap_patches(img, label, target_shape) data_list += patch_list X = np.array([it[0] for it in data_list]) Y = np.array([it[1] for it in data_list]) if is_training: index = np.random.permutation(len(data_list)) X = X[index, ...] Y = Y[index, ...] for step in range(X.shape[0]//batch_size-1): x = X[step * batch_size:(step + 1) * batch_size, ...] y = Y[step * batch_size:(step + 1) * batch_size, ...] yield x, y class GroupNorm(Layer): def __init__(self, groups=4): super(GroupNorm, self).__init__() self.G = groups self.eps = 1e-5 def build(self, input_shape): self.group = self.G if input_shape[-1] % self.G == 0 else 1 self.channel = input_shape[-1] self.group = min(self.channel, self.group) self.split = self.channel // self.group self.gamma = self.add_weight(name='gamma_gn', shape=(1, 1, 1, 1, input_shape[-1]), initializer='ones', trainable=True) self.beta = self.add_weight(name='beta_gn', shape=(1, 1, 1, 1, input_shape[-1]), initializer='zeros', trainable=True) def call(self, inputs): N, D, H, W, C = tf.keras.backend.int_shape(inputs) inputs = tf.reshape(inputs, [-1, D, H, W, self.group, self.split]) mean, var = tf.nn.moments(inputs, [1, 2, 3, 5]) mean = tf.reshape(mean, [-1, 1, 1, 1, self.group, 1]) var = tf.reshape(var, [-1, 1, 1, 1, self.group, 1]) ipt = (inputs - mean) / tf.sqrt(var + self.eps) output = tf.reshape(inputs, [-1, D, H, W, C]) * self.gamma + self.beta return output def conv_res_block(x, filters, activation=ReLU(), kernel_size=3, strides=1, padding='same', num_layers=1): if num_layers == 1: x = Conv3D(filters, kernel_size, strides=strides, padding=padding)(x) x = GroupNorm()(x) x = activation(x) return x shortcut = Conv3D(filters, 1, strides=strides, padding=padding)(x) shortcut = GroupNorm()(shortcut) shortcut = activation(shortcut) for i in range(num_layers): x = Conv3D(filters, kernel_size, strides=strides, padding=padding)(x) x = activation(x) x = x + shortcut return x def upsample_block(x, filters, activation=ReLU(), kernel_size=3, strides=1, padding='same', deconv=False): if deconv: x = Conv3DTranspose(filters, 2, strides=2, padding=padding)(x) else: x = UpSampling3D(size=2)(x) x = Conv3D(filters, kernel_size, strides=strides, padding=padding)(x) x = GroupNorm()(x) x = activation(x) return x # unet3d model def Unet3d(img_shape, n_filters, n_class): inputs = Input(shape=img_shape, name='input') l1 = conv_res_block(inputs, n_filters, num_layers=1) m1 = MaxPool3D()(l1) l2 = conv_res_block(m1, n_filters * 2, num_layers=2) m2 = MaxPool3D()(l2) l3 = conv_res_block(m2, n_filters * 4, num_layers=3) m3 = MaxPool3D()(l3) l4 = conv_res_block(m3, n_filters * 8, num_layers=3) m4 = MaxPool3D()(l4) l5 = conv_res_block(m4, n_filters * 16, num_layers=3) up6 = upsample_block(l5, n_filters * 8) l6 = conv_res_block(Concatenate()([up6, l4]), n_filters * 8, num_layers=3) up7 = upsample_block(l6, n_filters * 4) l7 = conv_res_block(Concatenate()([up7, l3]), n_filters * 4, num_layers=3) up8 = upsample_block(l7, n_filters * 2) l8 = conv_res_block(Concatenate()([up8, l2]), n_filters * 2, num_layers=2) up9 = upsample_block(l8, n_filters) l9 = conv_res_block(Concatenate()([up9, l1]), n_filters, num_layers=1) out = Conv3D(n_class, 1, padding='same', activation=keras.activations.softmax)(l9) model = Model(inputs=inputs, outputs=out, name='output') return model # loss function def explog_loss(y_true, y_pred, n_class, weights=1., w_d=0.8, w_c=0.2, g_d=0.3, g_c=0.3, eps=1e-5): """ Compute exp-log loss Args: y_true: ground truth with dimension of (batch, depth, height, width) y_pred: prediction with dimension of (batch, depth, height, width, n_class) n_class: classes weights: weights of n classes, a float number or vector with dimension of (n,) w_d: weight of dice loss w_c: weight of cross entropy loss g_d: exponent of dice loss g_c: exponent of cross entropy loss Returns: score: exp-log loss """ y_pred = tf.cast(y_pred, tf.float32) y_true = tf.cast(tf.one_hot(y_true, n_class), tf.float32) y_true = tf.reshape(y_true, [-1, n_class]) y_pred = tf.reshape(y_pred, [-1, n_class]) y_pred = tf.clip_by_value(y_pred, eps, 1.0-eps) intersection = tf.reduce_sum(y_true * y_pred, axis=0) union = tf.reduce_sum(y_true, axis=0) + tf.reduce_sum(y_pred, axis=0) dice = (2 * intersection + eps) / (union + eps) dice = tf.clip_by_value(dice, eps, 1.0-eps) dice_log_loss = -tf.math.log(dice) Ld = tf.reduce_mean(tf.pow(dice_log_loss, g_d)) wce = weights * y_true * tf.pow(-tf.math.log(y_pred), g_c) Lc = tf.reduce_mean(wce) score = w_d * Ld + w_c * Lc return score # metrics def dice_score(y_true, y_pred, n_class, exp=1e-5): """ Compute dice score with ground truth and prediction without argmax Args: y_true: ground truth with dimension of (batch, depth, height, width) y_pred: prediction with dimension of (batch, depth, height, width, n_class) n_class: classes number Returns: score: average dice score in n classes """ dices = [] y_pred = np.argmax(y_pred, axis=-1) for i in range(1, n_class): pred = y_pred == i label = y_true == i intersection = 2 * np.sum(label * pred, axis=(1, 2, 3)) + exp union = np.sum(label, axis=(1, 2, 3)) + np.sum(pred, axis=(1, 2, 3)) + exp dice = intersection / union dices.append(dice) score = np.mean(dices) return score # training process def train(display_step=10): batch_size = 1 epochs = 500 input_size = [32, 160, 160, 4] n_class = 4 first_channels = 8 lr = 0.001 save_model_dir = '../saved_models/' data_dir = '../data/Task01_BrainTumour/' with open(os.path.join(data_dir, 'dataset.json'), 'r') as f: data_info = json.load(f) path_list = data_info['training'] n_sample = len(path_list) train_path_list = path_list[:int(n_sample*0.8)] val_path_list = path_list[int(n_sample * 0.8):] model = Unet3d(input_size, first_channels, n_class) input = model.input pred = model.output label = tf.placeholder(tf.int32, shape=[None] + input_size[:3]) loss_tf = explog_loss(label, pred, n_class, weights=[1, 10, 20, 20]) global_step = tf.Variable(0, name='global_step', trainable=False) lr_schedule = tf.train.exponential_decay( lr, global_step, decay_steps=5000, decay_rate=0.98) optimizer = tf.train.AdamOptimizer(learning_rate=lr_schedule) train_opt = optimizer.minimize(loss_tf, global_step=global_step) init_op = tf.global_variables_initializer() saver = tf.train.Saver() print("start optimize") config = tf.compat.v1.ConfigProto() config.gpu_options.allow_growth=True with tf.Session(config=config) as sess: sess.run(init_op) for epoch in range(epochs): # training print('*'*20, 'Train Epoch %d'%epoch, '*'*20) steps = 0 train_loss_avg = 0 train_dice_avg = 0 train_dataset = data_generator(data_dir, train_path_list, input_size[:3], batch_size, True) for x, y in train_dataset: _, loss, pred_logits = sess.run([train_opt, loss_tf, pred], feed_dict={input: x, label: y}) dice = dice_score(y, pred_logits, n_class) train_dice_avg += dice train_loss_avg += loss steps += 1 if steps % display_step==0: print('epoch %d, steps %d, train loss: %.4f, train dice: %.4f' % ( epoch, steps, train_loss_avg / steps, train_dice_avg / steps)) train_loss_avg /= steps train_dice_avg /= steps # validation print('*'*20, 'Valid Epoch %d'%epoch, '*'*20) steps = 0 val_loss_avg = 0 val_dice_avg = 0 val_dataset = data_generator(data_dir, val_path_list, input_size[:3], batch_size, False) for x, y in val_dataset: val_loss, pred_logits = sess.run([loss_tf, pred], feed_dict={input: x, label: y}) dice = dice_score(y, pred_logits, n_class) val_dice_avg += dice steps += 1 val_loss_avg += val_loss if steps % display_step==0: print('Epoch {:}, valid steps {:}, loss={:.4f}'.format(epoch, steps, val_loss)) val_loss_avg /= steps val_dice_avg /= steps print( 'epoch %d, steps %d, validation loss: %.4f, val dice: %4f' % (epoch, steps, val_loss_avg, val_dice_avg)) print('*'*20, 'Valid Epoch %d'%epoch, '*'*20) # save model saver.save(sess, os.path.join(save_model_dir, "epoch_%d_%.4f_model" % (epoch, val_dice_avg)), write_meta_graph=False) def save_img(img_np, save_path): img_itk = sitk.GetImageFromArray(img_np) sitk.WriteImage(img_itk, save_path) def recover_img(patch_preds, pos_list, strides, ori_shape): sd, sh, sw = strides patch_shape = patch_preds[0].shape pd, ph, pw = patch_shape img = np.zeros(ori_shape) for patch, pos in zip(patch_preds, pos_list): i, j, k = pos img[i*sd:i*sd+pd, j*sh+20:j*sh+20+ph, k*sw+20:k*sw+20+pw] = patch return img def predict(model_path, patch_list, input_size, first_channels, n_class): input_shape = (1,) + tuple(input_size) inputs = tf.placeholder(tf.float32, shape=input_shape) model = Unet3d(input_size, first_channels, n_class) prediction = model(inputs) saver = tf.train.Saver() init_op = tf.global_variables_initializer() preds = [] with tf.Session() as sess: sess.run(init_op) saver.restore(sess, model_path) for i in range(len(patch_list)): pred = sess.run(prediction, feed_dict={inputs: np.expand_dims(patch_list[i], 0)}) pred = np.squeeze(np.argmax(pred, -1)) preds.append(pred) return preds def test(model_path, img_path, save_dir): patch_size = [32, 160, 160, 4] n_class = 4 first_channels = 8 strides = (16, 32, 32) if not os.path.exists(save_dir): os.mkdir(save_dir) save_path = os.path.join(save_dir, img_path.split('/')[-1]) img0 = read_img(img_path) img1 = preprocess(img0) patch_list, pos_list = extract_ordered_overlap_patches(img1, None, patch_size[:3]) preds = predict(model_path, patch_list, patch_size, first_channels, n_class) pred = recover_img(preds, pos_list, strides, img0.shape[:3]) save_img(pred, save_path) return img0, pred if __name__ == '__main__': if True: train() if False: model_path = '../saved_models/epoch_20_0.6411_model' img_path = '../data/Task01_BrainTumour/imagesTs/BRATS_485.nii.gz' save_dir = '../data/Task01_BrainTumour/imagesPre/' img, pred = test(model_path, img_path, save_dir) show_imgs([(img, pred)])   模型训练过程中,会输出当前的进度,具体如下:   训练的时间较长,需要耐心等待。   训练过过程中,可以查看生成的文件:       六、模型的测试 模型测试部分的代码如下: import os import SimpleITK as sitk import matplotlib.pyplot as plt import numpy as np import json from skimage import measure import random # import tensorflow as tf import tensorflow.compat.v1 as tf tf.disable_v2_behavior() tf.logging.set_verbosity(tf.logging.ERROR) from tensorflow.keras.layers import Layer from tensorflow import keras from tensorflow.keras.layers import Conv3D, Conv3DTranspose, ReLU, UpSampling3D, Input, MaxPool3D, Concatenate from tensorflow.keras import Model os.environ["CUDA_VISIBLE_DEVICES"]="3" def z_score_norm(img, mean=None, std=None): if mean is None: mean = np.mean(img) if std is None: std = np.std(img) return (img - mean) / (std + 1e-6) def show_imgs(images, cols=5): rows = len(images) titles = ['FLAIR', 'T1w', 'T1gd', 'T2w', 'Label'] f, axes = plt.subplots(max(1, rows), cols) axes_ravel = axes.ravel() for i, (image, label) in enumerate(images): ds = np.where(label != 0)[0] ds.sort() slice = ds[len(ds) // 2] for j in range(cols-1): axes_ravel[i*cols+j].set_axis_off() axes_ravel[i*cols+j].imshow(image[slice, :, :, j], cmap='Greys_r') axes_ravel[i*cols+j].set_title(titles[j]) axes_ravel[(i+1)*cols-1].set_axis_off() axes_ravel[(i+1)*cols-1].imshow(label[slice, :, :], cmap='Greys_r') axes_ravel[(i+1)*cols-1].set_title(titles[-1]) f.tight_layout() plt.subplots_adjust(wspace=0.01, hspace=0) plt.show() def read_img(img_path, label_path=None): img_itk = sitk.ReadImage(img_path) img_np = sitk.GetArrayFromImage(img_itk) img_np = np.moveaxis(img_np, 0, 3) if label_path is not None: label_itk = sitk.ReadImage(label_path) label_np = sitk.GetArrayFromImage(label_itk) return img_np, label_np return img_np def extract_ordered_overlap_patches(img, label, patch_size, s=16): img = img[:, 20:-20, 20:-20, :] d, h, w, _ = img.shape patch_d, patch_h, patch_w = patch_size sd = d - patch_d sh = h - patch_h sw = w - patch_w std = s sth = s*2 stw = s*2 patch_list = [] pos_list = [] if label is not None: label = label[:, 20:-20, 20:-20] for i in range(sd // std + 1): for j in range(sh // sth + 1): for k in range(sw//stw + 1): patch_img = img[i*std:min(d, i*std+patch_d), j*sth:min(h, j*sth+patch_h), k*stw:min(w, k*stw+patch_w), :] if label is not None: patch_label = label[i*std:min(d, i*std+patch_d), j*sth:min(h, j*sth+patch_h), k*stw:min(w, k*stw+patch_w)] if patch_label.shape != tuple(patch_size): continue if np.count_nonzero(patch_label)/np.count_nonzero(label) >= 0.2: patch_list.append((patch_img, patch_label)) pos_list.append((i, j, k)) else: patch_list.append(patch_img) pos_list.append((i, j, k)) return patch_list, pos_list # pre-process images def preprocess(image): # z-score normalization in each slice and each channel for i in range(image.shape[3]): for z in range(image.shape[0]): img_slice = image[z, :, :, i] image[z, :, :, i] = z_score_norm(img_slice) return image # data loader def data_generator(data_dir, path_list, target_shape, batch_size, is_training, buffer_size=8): if not is_training: buffer_size = 1 else: random.shuffle(path_list) buffer_size = min(len(path_list), buffer_size) k = len(path_list) // buffer_size for i in range(k): data_list = [] for j in range(i*buffer_size, (i+1)*buffer_size): img_path = path_list[j]['image'].replace('./', data_dir) label_path = path_list[j]['label'].replace('./', data_dir) if not os.path.exists(img_path) or not os.path.exists(label_path): continue img, label = read_img(img_path, label_path) img = preprocess(img) patch_list, _ = extract_ordered_overlap_patches(img, label, target_shape) data_list += patch_list X = np.array([it[0] for it in data_list]) Y = np.array([it[1] for it in data_list]) if is_training: index = np.random.permutation(len(data_list)) X = X[index, ...] Y = Y[index, ...] for step in range(X.shape[0]//batch_size-1): x = X[step * batch_size:(step + 1) * batch_size, ...] y = Y[step * batch_size:(step + 1) * batch_size, ...] yield x, y class GroupNorm(Layer): def __init__(self, groups=4): super(GroupNorm, self).__init__() self.G = groups self.eps = 1e-5 def build(self, input_shape): self.group = self.G if input_shape[-1] % self.G == 0 else 1 self.channel = input_shape[-1] self.group = min(self.channel, self.group) self.split = self.channel // self.group self.gamma = self.add_weight(name='gamma_gn', shape=(1, 1, 1, 1, input_shape[-1]), initializer='ones', trainable=True) self.beta = self.add_weight(name='beta_gn', shape=(1, 1, 1, 1, input_shape[-1]), initializer='zeros', trainable=True) def call(self, inputs): N, D, H, W, C = tf.keras.backend.int_shape(inputs) inputs = tf.reshape(inputs, [-1, D, H, W, self.group, self.split]) mean, var = tf.nn.moments(inputs, [1, 2, 3, 5]) mean = tf.reshape(mean, [-1, 1, 1, 1, self.group, 1]) var = tf.reshape(var, [-1, 1, 1, 1, self.group, 1]) ipt = (inputs - mean) / tf.sqrt(var + self.eps) output = tf.reshape(inputs, [-1, D, H, W, C]) * self.gamma + self.beta return output def conv_res_block(x, filters, activation=ReLU(), kernel_size=3, strides=1, padding='same', num_layers=1): if num_layers == 1: x = Conv3D(filters, kernel_size, strides=strides, padding=padding)(x) x = GroupNorm()(x) x = activation(x) return x shortcut = Conv3D(filters, 1, strides=strides, padding=padding)(x) shortcut = GroupNorm()(shortcut) shortcut = activation(shortcut) for i in range(num_layers): x = Conv3D(filters, kernel_size, strides=strides, padding=padding)(x) x = activation(x) x = x + shortcut return x def upsample_block(x, filters, activation=ReLU(), kernel_size=3, strides=1, padding='same', deconv=False): if deconv: x = Conv3DTranspose(filters, 2, strides=2, padding=padding)(x) else: x = UpSampling3D(size=2)(x) x = Conv3D(filters, kernel_size, strides=strides, padding=padding)(x) x = GroupNorm()(x) x = activation(x) return x # unet3d model def Unet3d(img_shape, n_filters, n_class): inputs = Input(shape=img_shape, name='input') l1 = conv_res_block(inputs, n_filters, num_layers=1) m1 = MaxPool3D()(l1) l2 = conv_res_block(m1, n_filters * 2, num_layers=2) m2 = MaxPool3D()(l2) l3 = conv_res_block(m2, n_filters * 4, num_layers=3) m3 = MaxPool3D()(l3) l4 = conv_res_block(m3, n_filters * 8, num_layers=3) m4 = MaxPool3D()(l4) l5 = conv_res_block(m4, n_filters * 16, num_layers=3) up6 = upsample_block(l5, n_filters * 8) l6 = conv_res_block(Concatenate()([up6, l4]), n_filters * 8, num_layers=3) up7 = upsample_block(l6, n_filters * 4) l7 = conv_res_block(Concatenate()([up7, l3]), n_filters * 4, num_layers=3) up8 = upsample_block(l7, n_filters * 2) l8 = conv_res_block(Concatenate()([up8, l2]), n_filters * 2, num_layers=2) up9 = upsample_block(l8, n_filters) l9 = conv_res_block(Concatenate()([up9, l1]), n_filters, num_layers=1) out = Conv3D(n_class, 1, padding='same', activation=keras.activations.softmax)(l9) model = Model(inputs=inputs, outputs=out, name='output') return model # loss function def explog_loss(y_true, y_pred, n_class, weights=1., w_d=0.8, w_c=0.2, g_d=0.3, g_c=0.3, eps=1e-5): """ Compute exp-log loss Args: y_true: ground truth with dimension of (batch, depth, height, width) y_pred: prediction with dimension of (batch, depth, height, width, n_class) n_class: classes weights: weights of n classes, a float number or vector with dimension of (n,) w_d: weight of dice loss w_c: weight of cross entropy loss g_d: exponent of dice loss g_c: exponent of cross entropy loss Returns: score: exp-log loss """ y_pred = tf.cast(y_pred, tf.float32) y_true = tf.cast(tf.one_hot(y_true, n_class), tf.float32) y_true = tf.reshape(y_true, [-1, n_class]) y_pred = tf.reshape(y_pred, [-1, n_class]) y_pred = tf.clip_by_value(y_pred, eps, 1.0-eps) intersection = tf.reduce_sum(y_true * y_pred, axis=0) union = tf.reduce_sum(y_true, axis=0) + tf.reduce_sum(y_pred, axis=0) dice = (2 * intersection + eps) / (union + eps) dice = tf.clip_by_value(dice, eps, 1.0-eps) dice_log_loss = -tf.math.log(dice) Ld = tf.reduce_mean(tf.pow(dice_log_loss, g_d)) wce = weights * y_true * tf.pow(-tf.math.log(y_pred), g_c) Lc = tf.reduce_mean(wce) score = w_d * Ld + w_c * Lc return score # metrics def dice_score(y_true, y_pred, n_class, exp=1e-5): """ Compute dice score with ground truth and prediction without argmax Args: y_true: ground truth with dimension of (batch, depth, height, width) y_pred: prediction with dimension of (batch, depth, height, width, n_class) n_class: classes number Returns: score: average dice score in n classes """ dices = [] y_pred = np.argmax(y_pred, axis=-1) for i in range(1, n_class): pred = y_pred == i label = y_true == i intersection = 2 * np.sum(label * pred, axis=(1, 2, 3)) + exp union = np.sum(label, axis=(1, 2, 3)) + np.sum(pred, axis=(1, 2, 3)) + exp dice = intersection / union dices.append(dice) score = np.mean(dices) return score # training process def train(): batch_size = 1 epochs = 500 input_size = [32, 160, 160, 4] n_class = 4 first_channels = 8 lr = 0.001 save_model_dir = '../saved_models/' data_dir = '../data/Task01_BrainTumour/' with open(os.path.join(data_dir, 'dataset.json'), 'r') as f: data_info = json.load(f) path_list = data_info['training'] n_sample = len(path_list) train_path_list = path_list[:int(n_sample*0.8)] val_path_list = path_list[int(n_sample * 0.8):] model = Unet3d(input_size, first_channels, n_class) input = model.input pred = model.output label = tf.placeholder(tf.int32, shape=[None] + input_size[:3]) loss_tf = explog_loss(label, pred, n_class, weights=[1, 10, 20, 20]) global_step = tf.Variable(0, name='global_step', trainable=False) lr_schedule = tf.train.exponential_decay( lr, global_step, decay_steps=5000, decay_rate=0.98) optimizer = tf.train.AdamOptimizer(learning_rate=lr_schedule) train_opt = optimizer.minimize(loss_tf, global_step=global_step) init_op = tf.global_variables_initializer() saver = tf.train.Saver() with tf.Session() as sess: sess.run(init_op) for epoch in range(epochs): # training steps = 0 train_loss_avg = 0 train_dice_avg = 0 train_dataset = data_generator(data_dir, train_path_list, input_size[:3], batch_size, True) for x, y in train_dataset: _, loss, pred_logits = sess.run([train_opt, loss_tf, pred], feed_dict={input: x, label: y}) dice = dice_score(y, pred_logits, n_class) train_dice_avg += dice train_loss_avg += loss steps += 1 print('epoch %d, steps %d, train loss: %.4f, train dice: %.4f' % ( epoch, steps, train_loss_avg / steps, train_dice_avg / steps)) train_loss_avg /= steps train_dice_avg /= steps # validation steps = 0 val_loss_avg = 0 val_dice_avg = 0 val_dataset = data_generator(data_dir, val_path_list, input_size[:3], batch_size, False) for x, y in val_dataset: val_loss, pred_logits = sess.run([loss_tf, pred], feed_dict={input: x, label: y}) dice = dice_score(y, pred_logits, n_class) val_dice_avg += dice steps += 1 val_loss_avg += val_loss val_loss_avg /= steps val_dice_avg /= steps print( 'epoch %d, steps %d, validation loss: %.4f, val dice: %4f' % (epoch, steps, val_loss_avg, val_dice_avg)) # save model saver.save(sess, os.path.join(save_model_dir, "epoch_%d_%.4f_model" % (epoch, val_dice_avg)), write_meta_graph=False) def save_img(img_np, save_path): img_itk = sitk.GetImageFromArray(img_np) sitk.WriteImage(img_itk, save_path) def recover_img(patch_preds, pos_list, strides, ori_shape): sd, sh, sw = strides patch_shape = patch_preds[0].shape pd, ph, pw = patch_shape img = np.zeros(ori_shape) for patch, pos in zip(patch_preds, pos_list): i, j, k = pos img[i*sd:i*sd+pd, j*sh+20:j*sh+20+ph, k*sw+20:k*sw+20+pw] = patch return img def predict(model_path, patch_list, input_size, first_channels, n_class): input_shape = (1,) + tuple(input_size) inputs = tf.placeholder(tf.float32, shape=input_shape) model = Unet3d(input_size, first_channels, n_class) prediction = model(inputs) saver = tf.train.Saver() init_op = tf.global_variables_initializer() preds = [] with tf.Session() as sess: sess.run(init_op) saver.restore(sess, model_path) for i in range(len(patch_list)): pred = sess.run(prediction, feed_dict={inputs: np.expand_dims(patch_list[i], 0)}) pred = np.squeeze(np.argmax(pred, -1)) preds.append(pred) return preds def test(model_path, img_path, save_dir): patch_size = [32, 160, 160, 4] n_class = 4 first_channels = 8 strides = (16, 32, 32) if not os.path.exists(save_dir): os.mkdir(save_dir) save_path = os.path.join(save_dir, img_path.split('/')[-1]) img0 = read_img(img_path) img1 = preprocess(img0) patch_list, pos_list = extract_ordered_overlap_patches(img1, None, patch_size[:3]) preds = predict(model_path, patch_list, patch_size, first_channels, n_class) pred = recover_img(preds, pos_list, strides, img0.shape[:3]) save_img(pred, save_path) return img0, pred if __name__ == '__main__': if False: train() if True: model_path = '../saved_models/epoch_271_0.6776_model' img_path = '../data/Task01_BrainTumour/imagesTs/BRATS_485.nii.gz' save_dir = '../data/Task01_BrainTumour/imagesPre/' img, pred = test(model_path, img_path, save_dir) show_imgs([(img, pred)])   需要注意的是,model_path,需要根据模型训练部分最终的结果,进行填写,才能进行实际的训练测试。 在上述代码中,是BRATS_485.nii.gz做为输入数据,针对模型进行测试。   代码运行后,结果如下:     从上图中可以看到,与肿瘤相关的部分,被准确识别并切割出来了。   七、总结 通过5、6两章的学习和实战,才算扣了医学图像处理的深度学习处理的门了,了解到在医学图像处理方面,模型从数据准备到训练测试的完成步骤,为后续的学习打下基础。

  • 2024-05-09
  • 发表了主题帖: 《深度学习与医学图像处理》【学习分享6】医学图像分类处理和模型训练实践

    本帖最后由 HonestQiao 于 2024-5-10 08:01 编辑 《深度学习与医学图像处理》的第五章,重点讲述了医学图像的分类处理。 这里的分类,是指通过深度处理工具和算法,对医学图像进行分类训练,最终对图片数据进行预测。   一、内容思维导图 本章的思维导图如下:   上图中,涉及到的概念非常多,虽然都配备了对应的代码,但是如果没有一定的基础,读起来比较费力,而且代码也可能运行不起来。   经过反复的阅读理解,以及尝试,最终成功进行了实战。   二、环境准备 首先建立专门的python环境 conda create -n doctor python==3.10 conda activate doctor 然后,安装需要的扩展: pip install pydicom pip install numpy pip install scikit-image pip install sklearn pip install tensorflow==2.8.0 pip install kaggle 准备好环境以后,需要检测 tensorflow是否能够使用GPU,否则训练速度会大打折扣。 import tensorflow as tf tf.config.list_physical_devices('GPU') 输出结果如下,说明tensorflow可以正常使用gpu   如果最后一步的输出如下:   说明不能用到GPU,只能用CPU运算,速度就会慢很多的。   三、测试数据集下载 是占用的数据集,是 RSNA Intracranial Hemorrhage Detection Challenge (2019) 这个竞赛对应的数据集,可以试用kaggle直接下载: kaggle competitions download -c rsna-intracranial-hemorrhage-detection 下载前,先要去 RSNA Intracranial Hemorrhage Detection 注册账户,并且统一许可,才能下载。   下载的数据集为一个182G的压缩包:   解压后的大小有431G:   需要准备足够的磁盘空间来跑这次的实战。   测试数据集准备好以后,就可以参考书上的步骤,进行数据预处理、训练和测试了。   四、数据预处理 训练对应的代码如下: import os import csv import numpy as np import pydicom as dicom from skimage.transform import resize from sklearn.metrics import roc_auc_score, precision_score, recall_score, f1_score # import tensorflow as tf import tensorflow.compat.v1 as tf tf.disable_v2_behavior() def z_score_norm(img, mean=None, std=None): if mean is None: mean = np.mean(img) if std is None: std = np.std(img) return (img - mean) / (std + 1e-6) #=============Process original data & Save train and test npy files def get_random_data(train_label_csv, pick_num): kaggle_list = { 'epidural': 0, 'intraparenchymal': 1, 'intraventricular': 2, 'subarachnoid': 3, 'subdural': 4, 'any': 5, } train_split_list = { 'any': [], 'epidural': [], 'intraparenchymal': [], 'intraventricular': [], 'subarachnoid': [], 'subdural': [], } f = csv.reader(open(train_label_csv)) train_dict = [0,0,0,0,0] for ID_info, label in list(f)[1:]: tok = ID_info.split('_') ID = '_'.join(tok[:2]) ICH_name = tok[2] if int(label) == 1 and ICH_name != 'any': train_dict[kaggle_list[ICH_name]] = int(label) if ICH_name == 'any': key_candidate = np.where(np.array(train_dict)>0)[0] if len(key_candidate) == 0: key = 'any' else: key = list(kaggle_list.keys())[np.random.choice(key_candidate)] train_split_list[key].append([ID, train_dict]) train_dict = [0,0,0,0,0] res_list = [] for i, key in enumerate(train_split_list): np.random.shuffle(train_split_list[key]) res_list.append(train_split_list[key][:pick_num[i]]) return res_list def dicom_2_npy(path_list): npy_list = [] for path in path_list: try: ds = dicom.read_file(path) except: print('bad', path) continue pix = ds.pixel_array Intercept = ds.RescaleIntercept Slope = ds.RescaleSlope pix= pix*Slope + Intercept if pix.shape != (512,512): pix = resize(pix, (512,512)) npy_list.append(np.reshape(pix, (1,512,512))) dicom_npy = np.concatenate(npy_list, axis=0) return dicom_npy def save_train_test_npy(res_list, data_path, output_path, data_set_dict, read_count): for label, (train_num, test_num) in data_set_dict.items(): res_info = res_list[label] path_list = [] label_list = [] for i in range(train_num): ID, la = res_info[read_count[label] + i] dcm_name = ID + '.dcm' path_list.append(os.path.join(data_path, dcm_name)) label_list.append(la) read_count[label] += train_num dicom_npy = dicom_2_npy(path_list) np.save(os.path.join(output_path, 'trainval_img_%d_%d.npy'%(train_num, label)), dicom_npy) np.save(os.path.join(output_path, 'trainval_label_%d_%d.npy'%(train_num, label)), label_list) path_list = [] label_list = [] for i in range(test_num): ID, la = res_info[read_count[label] + i] dcm_name = ID + '.dcm' path_list.append(os.path.join(data_path, dcm_name)) label_list.append(la) read_count[label] += test_num dicom_npy = dicom_2_npy(path_list) np.save(os.path.join(output_path, 'test_img_%d_%d.npy'%(train_num, label)), dicom_npy) np.save(os.path.join(output_path, 'test_label_%d_%d.npy'%(test_num, label)), label_list) print('save %d done'%label) return # ============= # data_root = './data/' data_root = '/workspace/rsna-intracranial-hemorrhage-detection' train_img_path = os.path.join(data_root, 'stage_2_train') train_label = os.path.join(data_root, 'stage_2_train.csv') npy_path = os.path.join(data_root, 'stage_2_train_npy') if not os.path.exists(npy_path): os.mkdir(npy_path) res_list = get_random_data(train_label, pick_num=[8000,2000,2000,2000,2000,2000]) read_count = {0:0, 1:0, 2:0, 3:0, 4:0, 5:0} data_set = { 0:[5000, 1000], 1:[1000, 200], 2:[1000, 200], 3:[1000, 200], 4:[1000, 200], 5:[1000, 200], } save_train_test_npy(res_list, train_img_path, npy_path, data_set, read_count)   直接使用python运行,输出如下:     最终会得到原始数据的.npy文件:   五、模型训练 参考书上的实例,模型运行的代码如下: import os import csv import numpy as np import pydicom as dicom from skimage.transform import resize from sklearn.metrics import roc_auc_score, precision_score, recall_score, f1_score # import tensorflow as tf import tensorflow.compat.v1 as tf tf.disable_v2_behavior() tf.logging.set_verbosity(tf.logging.ERROR) def z_score_norm(img, mean=None, std=None): if mean is None: mean = np.mean(img) if std is None: std = np.std(img) return (img - mean) / (std + 1e-6) #=============Process original data & Save train and test npy files def get_random_data(train_label_csv, pick_num): kaggle_list = { 'epidural': 0, 'intraparenchymal': 1, 'intraventricular': 2, 'subarachnoid': 3, 'subdural': 4, 'any': 5, } train_split_list = { 'any': [], 'epidural': [], 'intraparenchymal': [], 'intraventricular': [], 'subarachnoid': [], 'subdural': [], } f = csv.reader(open(train_label_csv)) train_dict = [0,0,0,0,0] for ID_info, label in list(f)[1:]: tok = ID_info.split('_') ID = '_'.join(tok[:2]) ICH_name = tok[2] if int(label) == 1 and ICH_name != 'any': train_dict[kaggle_list[ICH_name]] = int(label) if ICH_name == 'any': key_candidate = np.where(np.array(train_dict)>0)[0] if len(key_candidate) == 0: key = 'any' else: key = list(kaggle_list.keys())[np.random.choice(key_candidate)] train_split_list[key].append([ID, train_dict]) train_dict = [0,0,0,0,0] res_list = [] for i, key in enumerate(train_split_list): np.random.shuffle(train_split_list[key]) res_list.append(train_split_list[key][:pick_num[i]]) return res_list def dicom_2_npy(path_list): npy_list = [] for path in path_list: try: ds = dicom.read_file(path) except: print('bad', path) continue pix = ds.pixel_array Intercept = ds.RescaleIntercept Slope = ds.RescaleSlope pix= pix*Slope + Intercept if pix.shape != (512,512): pix = resize(pix, (512,512)) npy_list.append(np.reshape(pix, (1,512,512))) dicom_npy = np.concatenate(npy_list, axis=0) return dicom_npy def save_train_test_npy(res_list, data_path, output_path, data_set_dict, read_count): for label, (train_num, test_num) in data_set_dict.items(): res_info = res_list[label] path_list = [] label_list = [] for i in range(train_num): ID, la = res_info[read_count[label] + i] dcm_name = ID + '.dcm' path_list.append(os.path.join(data_path, dcm_name)) label_list.append(la) read_count[label] += train_num dicom_npy = dicom_2_npy(path_list) np.save(os.path.join(output_path, 'trainval_img_%d_%d.npy'%(train_num, label)), dicom_npy) np.save(os.path.join(output_path, 'trainval_label_%d_%d.npy'%(train_num, label)), label_list) path_list = [] label_list = [] for i in range(test_num): ID, la = res_info[read_count[label] + i] dcm_name = ID + '.dcm' path_list.append(os.path.join(data_path, dcm_name)) label_list.append(la) read_count[label] += test_num dicom_npy = dicom_2_npy(path_list) np.save(os.path.join(output_path, 'test_img_%d_%d.npy'%(train_num, label)), dicom_npy) np.save(os.path.join(output_path, 'test_label_%d_%d.npy'%(test_num, label)), label_list) print('save %d done'%label) return #============= # data_root = './data/' # train_img_path = os.path.join(data_root, 'stage_2_train') # train_label = os.path.join(data_root, 'stage_2_train.csv') # npy_path = os.path.join(data_root, 'stage_2_train_npy') # if not os.path.exists(npy_path): # os.mkdir(npy_path) # res_list = get_random_data(train_label, pick_num=[8000,2000,2000,2000,2000,2000]) # read_count = {0:0, 1:0, 2:0, 3:0, 4:0, 5:0} # data_set = { # 0:[5000, 1000], # 1:[1000, 200], # 2:[1000, 200], # 3:[1000, 200], # 4:[1000, 200], # 5:[1000, 200], # } # save_train_test_npy(res_list, train_img_path, npy_path, data_set, read_count) def data_generator(data, label, batch_size, is_training): data = np.array(data) label = np.array(label) n_step = len(data) // batch_size while True: if is_training: np.random.seed(123) np.random.shuffle(data) np.random.seed(123) np.random.shuffle(label) for step in range(n_step): x = data[step*batch_size:(step+1)*batch_size] x = np.array(x) x = np.expand_dims(x, -1) y = label[step*batch_size:(step+1)*batch_size] y = np.array(y) yield x,y def _conv_bn_relu(x, filters, kernel_size=3, strides=1, padding='same'): x = tf.layers.conv2d(x, filters=filters, kernel_size=kernel_size, strides=strides, padding=padding) x = tf.layers.batch_normalization(x) x = tf.nn.relu(x) return x def conv_res_block(x, filters, padding='same', num_layers=1): if num_layers == 1: x = _conv_bn_relu(x, filters, kernel_size=7, strides=2,) return x for _ in range(num_layers): input_shape = x.get_shape() if input_shape[-1] != filters: shortcut = _conv_bn_relu(x, filters, kernel_size=1, strides=2, padding=padding) x = _conv_bn_relu(x, filters//4, kernel_size=1, strides=2, padding=padding) else: shortcut = x x = _conv_bn_relu(x, filters//4, kernel_size=1, padding=padding) x = _conv_bn_relu(x, filters//4, kernel_size=3, padding=padding) x = _conv_bn_relu(x, filters, kernel_size=1, padding=padding) x = x + shortcut return x def build_res_network(inputs, n_filters, n_class, rate=.2, is_training=True): l1 = conv_res_block(inputs, n_filters, num_layers=1) l2 = conv_res_block(l1, n_filters*4, num_layers=3) l3 = conv_res_block(l2, n_filters*8, num_layers=4) l4 = conv_res_block(l3, n_filters*16, num_layers=23) l5 = conv_res_block(l4, n_filters*32, num_layers=3) block_shape = l5.get_shape() pool2 = tf.layers.average_pooling2d(l5, pool_size=(block_shape[1], block_shape[2]), strides=(1,1)) fc = tf.layers.flatten(pool2) fc = tf.layers.dense(fc, units=512) fc = tf.layers.dropout(fc, rate=rate, training=is_training) fc = tf.layers.dense(fc, units=128) fc = tf.layers.dropout(fc, rate=rate, training=is_training) out = tf.layers.dense(fc, units=n_class) out = tf.nn.softmax(out) return out #=============compute mean&std def compute_mean_std(npy_path): img_path = [] for f in os.listdir(npy_path): if ".npy" in f and "img" in f: img_path.append(f) data = [] for img_file in img_path: images = np.load(os.path.join(npy_path, img_file)) data.extend(images) mean = np.mean(data) std = np.std(data) np.save('./mean_std.npy',[mean, std]) # compute_mean_std(npy_path) def load_split_data(img_path, label_path, thre=.8): train_data = [] valid_data = [] train_label = [] valid_label = [] for img_file, label_file in zip(img_path, label_path): images = np.load(img_file) labels = np.load(label_file) images = images[:,64:448,64:448] labels = np.clip(labels.sum(axis=-1), 0, 1) labels = np.expand_dims(labels, -1) labels = np.concatenate([1-labels, labels], axis=-1) split_num = int(len(images)*thre) train_data.extend(images[:split_num]) valid_data.extend(images[split_num:]) train_label.extend(labels[:split_num]) valid_label.extend(labels[split_num:]) (mean, std) = np.load('mean_std.npy') train_data = z_score_norm(train_data, mean, std) valid_data = z_score_norm(valid_data, mean, std) return train_data, train_label, valid_data, valid_label def cross_entropy(labels, logits): return -tf.reduce_mean(labels*tf.math.log(tf.clip_by_value(logits,1e-10, 1.0-1e-7)), name='cross_entropy') def train(train_img_npy, train_label_npy, save_model_dir, batch_size=32, epochs=100, \ input_size=[384, 384, 1], n_class=2, first_channels=64, lr=0.0001, display_step=100): print("start train") train_data, train_label, valid_data, valid_label = load_split_data(train_img_npy, train_label_npy, thre=0.9) train_dataset = data_generator(train_data, train_label, batch_size, True) val_dataset = data_generator(valid_data, valid_label, 1, False) step_train = len(train_data)//batch_size # step_train = 500 step_valid = len(valid_data) is_training = tf.placeholder(tf.bool) input = tf.placeholder(dtype=tf.float32, shape=[None, input_size[0], input_size[1], input_size[2]]) print('build network') logit = build_res_network(input, first_channels, n_class, rate=.2, is_training=is_training) label = tf.placeholder(dtype=tf.float32, shape=[None, n_class]) loss_tf = cross_entropy(labels=label, logits=logit) global_step = tf.Variable(0, name='global_step', trainable=False) optimizer = tf.train.AdamOptimizer(learning_rate=lr) train_opt = optimizer.minimize(loss_tf, global_step=global_step) init_op = tf.global_variables_initializer() saver = tf.train.Saver(max_to_keep=epochs) print("start optimize") config = tf.compat.v1.ConfigProto() config.gpu_options.allow_growth=True with tf.compat.v1.Session(config=config) as sess: sess.run(init_op) min_loss = 10.0 for epoch in range(epochs): total_loss = [] print('*'*20, 'Train Epoch %d'%epoch, '*'*20) for step in range(step_train): x,y = next(train_dataset) _, loss, pred_logits = sess.run([train_opt, loss_tf, logit], feed_dict={input:x, label:y, is_training:True}) total_loss.append(loss) if step % display_step==0: print('Epoch {:}, train steps {:}, loss={:.4f}'.format(epoch, step, loss), flush=True) print('Epoch {:}, train Avg loss={:.4f}, lr={:.4f}'.format(epoch, np.mean(total_loss), lr)) print('*'*20, 'Valid Epoch %d'%epoch, '*'*20) total_loss=[] all_num = step_valid TP = 0 for step in range(step_valid): x,y = next(val_dataset) val_loss, pred_logits = sess.run([loss_tf, logit], feed_dict={input:x, label:y, is_training:False}) y_pred = np.argmax(pred_logits, axis=-1) y = np.argmax(y, axis=-1) total_loss.append(val_loss) if y[0] == y_pred[0]: TP += 1 if step % display_step==0: print('Epoch {:}, valid steps {:}, loss={:.4f}'.format(epoch, step, val_loss)) val_loss_avg = np.mean(total_loss) print('Epoch {:}, valid Avg loss={:.4f}, acc={:.4f}'.format(epoch, val_loss_avg, TP*1.0/all_num)) print('*'*20, 'Valid Epoch %d'%epoch, '*'*20) saver.save(sess, os.path.join(save_model_dir, 'epoch_%03d_%0.4f_model' % (epoch, val_loss_avg)), write_meta_graph=False) saver.save(sess, os.path.join(save_model_dir, 'last_weight_model'), write_meta_graph=False) if min_loss > val_loss_avg: min_loss = val_loss_avg saver.save(sess, os.path.join(save_model_dir, 'best_weight_model'), write_meta_graph=False) #============test def test(test_img_npy, test_label_npy, save_model, batch_size=1, input_size=[384, 384, 1], n_class=2, first_channels=64, display_step=100): test_data, test_label, _, _ = load_split_data(test_img_npy, test_label_npy, thre=1) test_dataset = data_generator(test_data, test_label, batch_size, False) input = tf.placeholder(dtype=tf.float32, shape=[None, input_size[0], input_size[1], input_size[2]]) logit = build_res_network(input, first_channels, n_class, rate=.2, is_training=False) label = tf.placeholder(dtype=tf.float32, shape=[None, n_class]) saver = tf.train.Saver() with tf.Session() as sess: saver.restore(sess, save_model) pred_label = [] cls_label = [] y_prob = [] all_num = len(test_data) TP = 0 for step in range(all_num): x, y = next(test_dataset) pred_logits = sess.run(logit, feed_dict={input:x, label:y}) pred_y = np.argmax(pred_logits,1).astype(np.int32) label_y = np.argmax(y,1).astype(np.int32) pred_label.append(pred_y[0]) cls_label.append(label_y[0]) y_prob.append(pred_logits[0][1]) if label_y[0]==pred_y[0]: TP += 1 if step%display_step == 0: print('Test steps {:} y true {:} y pred{:}'.format(step, label_y, pred_y)) auc = roc_auc_score(cls_label, y_prob) precision = precision_score(cls_label, pred_label) recall = recall_score(cls_label, pred_label) f1 = f1_score(cls_label, pred_label) print('Tesy AUC={:4f}, Avg acc={:4f}, Precision={:4f}, Recall={:4f}, F1={:4f}'.format(auc, TP*1.0/all_num, precision, recall, f1)) # npy_path = '/Users/kai/Downloads/Chapter5/' npy_path = '/workspace/rsna-intracranial-hemorrhage-detection/stage_2_train_npy/' print("compute mean std") compute_mean_std(npy_path) train_img_npy = [ os.path.join(npy_path, 'trainval_img_5000_0.npy'), os.path.join(npy_path, 'trainval_img_1000_1.npy'), os.path.join(npy_path, 'trainval_img_1000_2.npy'), os.path.join(npy_path, 'trainval_img_1000_3.npy'), os.path.join(npy_path, 'trainval_img_1000_4.npy'), os.path.join(npy_path, 'trainval_img_1000_5.npy'), ] train_label_npy = [ os.path.join(npy_path, 'trainval_label_5000_0.npy'), os.path.join(npy_path, 'trainval_label_1000_1.npy'), os.path.join(npy_path, 'trainval_label_1000_2.npy'), os.path.join(npy_path, 'trainval_label_1000_3.npy'), os.path.join(npy_path, 'trainval_label_1000_4.npy'), os.path.join(npy_path, 'trainval_label_1000_5.npy'), ] test_img_npy = [ os.path.join(npy_path, 'test_img_1000_0.npy'), os.path.join(npy_path, 'test_img_200_1.npy'), os.path.join(npy_path, 'test_img_200_2.npy'), os.path.join(npy_path, 'test_img_200_3.npy'), os.path.join(npy_path, 'test_img_200_4.npy'), os.path.join(npy_path, 'test_img_200_5.npy'), ] test_label_npy = [ os.path.join(npy_path, 'test_label_1000_0.npy'), os.path.join(npy_path, 'test_label_200_1.npy'), os.path.join(npy_path, 'test_label_200_2.npy'), os.path.join(npy_path, 'test_label_200_3.npy'), os.path.join(npy_path, 'test_label_200_4.npy'), os.path.join(npy_path, 'test_label_200_5.npy'), ] ''' save_model_dir = './temp/saved_model' if not os.path.exists(save_model_dir): os.makedirs(save_model_dir) train(train_img_npy, train_label_npy, save_model_dir, display_step=1) ''' if __name__ == "__main__": # 训练 if True: save_model_dir = './saved_model' train(train_img_npy, train_label_npy, save_model_dir, display_step=10) # 测试 if False: save_model = './saved_model/epoch_036_0.5757_model' test(test_img_npy, test_label_npy, save_model)   运行上述代码后,输出如下:   热身工作完成后,训练过程正式开始: 这个过程,较为漫长,建议睡觉前开启,早上醒来再看进度吧。数据需要迭代100次,才最终完成。     训练完成后,生成的模型如下:             六、模型测试 现在模型训练完成了,就可以用提供的数据进行测试,检验模型的具体应用能力了。 在前面数据预处理的时候,就已经生成了用于测试的数据,占比为20%:   对应测试的代码如下: import os import csv import numpy as np import pydicom as dicom from skimage.transform import resize from sklearn.metrics import roc_auc_score, precision_score, recall_score, f1_score # import tensorflow as tf import tensorflow.compat.v1 as tf tf.disable_v2_behavior() tf.logging.set_verbosity(tf.logging.ERROR) def z_score_norm(img, mean=None, std=None): if mean is None: mean = np.mean(img) if std is None: std = np.std(img) return (img - mean) / (std + 1e-6) #=============Process original data & Save train and test npy files def get_random_data(train_label_csv, pick_num): kaggle_list = { 'epidural': 0, 'intraparenchymal': 1, 'intraventricular': 2, 'subarachnoid': 3, 'subdural': 4, 'any': 5, } train_split_list = { 'any': [], 'epidural': [], 'intraparenchymal': [], 'intraventricular': [], 'subarachnoid': [], 'subdural': [], } f = csv.reader(open(train_label_csv)) train_dict = [0,0,0,0,0] for ID_info, label in list(f)[1:]: tok = ID_info.split('_') ID = '_'.join(tok[:2]) ICH_name = tok[2] if int(label) == 1 and ICH_name != 'any': train_dict[kaggle_list[ICH_name]] = int(label) if ICH_name == 'any': key_candidate = np.where(np.array(train_dict)>0)[0] if len(key_candidate) == 0: key = 'any' else: key = list(kaggle_list.keys())[np.random.choice(key_candidate)] train_split_list[key].append([ID, train_dict]) train_dict = [0,0,0,0,0] res_list = [] for i, key in enumerate(train_split_list): np.random.shuffle(train_split_list[key]) res_list.append(train_split_list[key][:pick_num[i]]) return res_list def dicom_2_npy(path_list): npy_list = [] for path in path_list: try: ds = dicom.read_file(path) except: print('bad', path) continue pix = ds.pixel_array Intercept = ds.RescaleIntercept Slope = ds.RescaleSlope pix= pix*Slope + Intercept if pix.shape != (512,512): pix = resize(pix, (512,512)) npy_list.append(np.reshape(pix, (1,512,512))) dicom_npy = np.concatenate(npy_list, axis=0) return dicom_npy def save_train_test_npy(res_list, data_path, output_path, data_set_dict, read_count): for label, (train_num, test_num) in data_set_dict.items(): res_info = res_list[label] path_list = [] label_list = [] for i in range(train_num): ID, la = res_info[read_count[label] + i] dcm_name = ID + '.dcm' path_list.append(os.path.join(data_path, dcm_name)) label_list.append(la) read_count[label] += train_num dicom_npy = dicom_2_npy(path_list) np.save(os.path.join(output_path, 'trainval_img_%d_%d.npy'%(train_num, label)), dicom_npy) np.save(os.path.join(output_path, 'trainval_label_%d_%d.npy'%(train_num, label)), label_list) path_list = [] label_list = [] for i in range(test_num): ID, la = res_info[read_count[label] + i] dcm_name = ID + '.dcm' path_list.append(os.path.join(data_path, dcm_name)) label_list.append(la) read_count[label] += test_num dicom_npy = dicom_2_npy(path_list) np.save(os.path.join(output_path, 'test_img_%d_%d.npy'%(train_num, label)), dicom_npy) np.save(os.path.join(output_path, 'test_label_%d_%d.npy'%(test_num, label)), label_list) print('save %d done'%label) return #============= # data_root = './data/' # train_img_path = os.path.join(data_root, 'stage_2_train') # train_label = os.path.join(data_root, 'stage_2_train.csv') # npy_path = os.path.join(data_root, 'stage_2_train_npy') # if not os.path.exists(npy_path): # os.mkdir(npy_path) # res_list = get_random_data(train_label, pick_num=[8000,2000,2000,2000,2000,2000]) # read_count = {0:0, 1:0, 2:0, 3:0, 4:0, 5:0} # data_set = { # 0:[5000, 1000], # 1:[1000, 200], # 2:[1000, 200], # 3:[1000, 200], # 4:[1000, 200], # 5:[1000, 200], # } # save_train_test_npy(res_list, train_img_path, npy_path, data_set, read_count) def data_generator(data, label, batch_size, is_training): data = np.array(data) label = np.array(label) n_step = len(data) // batch_size while True: if is_training: np.random.seed(123) np.random.shuffle(data) np.random.seed(123) np.random.shuffle(label) for step in range(n_step): x = data[step*batch_size:(step+1)*batch_size] x = np.array(x) x = np.expand_dims(x, -1) y = label[step*batch_size:(step+1)*batch_size] y = np.array(y) yield x,y def _conv_bn_relu(x, filters, kernel_size=3, strides=1, padding='same'): x = tf.layers.conv2d(x, filters=filters, kernel_size=kernel_size, strides=strides, padding=padding) x = tf.layers.batch_normalization(x) x = tf.nn.relu(x) return x def conv_res_block(x, filters, padding='same', num_layers=1): if num_layers == 1: x = _conv_bn_relu(x, filters, kernel_size=7, strides=2,) return x for _ in range(num_layers): input_shape = x.get_shape() if input_shape[-1] != filters: shortcut = _conv_bn_relu(x, filters, kernel_size=1, strides=2, padding=padding) x = _conv_bn_relu(x, filters//4, kernel_size=1, strides=2, padding=padding) else: shortcut = x x = _conv_bn_relu(x, filters//4, kernel_size=1, padding=padding) x = _conv_bn_relu(x, filters//4, kernel_size=3, padding=padding) x = _conv_bn_relu(x, filters, kernel_size=1, padding=padding) x = x + shortcut return x def build_res_network(inputs, n_filters, n_class, rate=.2, is_training=True): l1 = conv_res_block(inputs, n_filters, num_layers=1) l2 = conv_res_block(l1, n_filters*4, num_layers=3) l3 = conv_res_block(l2, n_filters*8, num_layers=4) l4 = conv_res_block(l3, n_filters*16, num_layers=23) l5 = conv_res_block(l4, n_filters*32, num_layers=3) block_shape = l5.get_shape() pool2 = tf.layers.average_pooling2d(l5, pool_size=(block_shape[1], block_shape[2]), strides=(1,1)) fc = tf.layers.flatten(pool2) fc = tf.layers.dense(fc, units=512) fc = tf.layers.dropout(fc, rate=rate, training=is_training) fc = tf.layers.dense(fc, units=128) fc = tf.layers.dropout(fc, rate=rate, training=is_training) out = tf.layers.dense(fc, units=n_class) out = tf.nn.softmax(out) return out #=============compute mean&std def compute_mean_std(npy_path): img_path = [] for f in os.listdir(npy_path): if ".npy" in f and "img" in f: img_path.append(f) data = [] for img_file in img_path: images = np.load(os.path.join(npy_path, img_file)) data.extend(images) mean = np.mean(data) std = np.std(data) np.save('./mean_std.npy',[mean, std]) # compute_mean_std(npy_path) def load_split_data(img_path, label_path, thre=.8): train_data = [] valid_data = [] train_label = [] valid_label = [] for img_file, label_file in zip(img_path, label_path): images = np.load(img_file) labels = np.load(label_file) images = images[:,64:448,64:448] labels = np.clip(labels.sum(axis=-1), 0, 1) labels = np.expand_dims(labels, -1) labels = np.concatenate([1-labels, labels], axis=-1) split_num = int(len(images)*thre) train_data.extend(images[:split_num]) valid_data.extend(images[split_num:]) train_label.extend(labels[:split_num]) valid_label.extend(labels[split_num:]) (mean, std) = np.load('mean_std.npy') train_data = z_score_norm(train_data, mean, std) valid_data = z_score_norm(valid_data, mean, std) return train_data, train_label, valid_data, valid_label def cross_entropy(labels, logits): return -tf.reduce_mean(labels*tf.math.log(tf.clip_by_value(logits,1e-10, 1.0-1e-7)), name='cross_entropy') def train(train_img_npy, train_label_npy, save_model_dir, batch_size=32, epochs=100, \ input_size=[384, 384, 1], n_class=2, first_channels=64, lr=0.0001, display_step=100): print("start train") train_data, train_label, valid_data, valid_label = load_split_data(train_img_npy, train_label_npy, thre=0.9) train_dataset = data_generator(train_data, train_label, batch_size, True) val_dataset = data_generator(valid_data, valid_label, 1, False) step_train = len(train_data)//batch_size # step_train = 500 step_valid = len(valid_data) is_training = tf.placeholder(tf.bool) input = tf.placeholder(dtype=tf.float32, shape=[None, input_size[0], input_size[1], input_size[2]]) print('build network') logit = build_res_network(input, first_channels, n_class, rate=.2, is_training=is_training) label = tf.placeholder(dtype=tf.float32, shape=[None, n_class]) loss_tf = cross_entropy(labels=label, logits=logit) global_step = tf.Variable(0, name='global_step', trainable=False) optimizer = tf.train.AdamOptimizer(learning_rate=lr) train_opt = optimizer.minimize(loss_tf, global_step=global_step) init_op = tf.global_variables_initializer() saver = tf.train.Saver(max_to_keep=epochs) print("start optimize") config = tf.compat.v1.ConfigProto() config.gpu_options.allow_growth=True with tf.compat.v1.Session(config=config) as sess: sess.run(init_op) min_loss = 10.0 for epoch in range(epochs): total_loss = [] print('*'*20, 'Train Epoch %d'%epoch, '*'*20) for step in range(step_train): x,y = next(train_dataset) _, loss, pred_logits = sess.run([train_opt, loss_tf, logit], feed_dict={input:x, label:y, is_training:True}) total_loss.append(loss) if step % display_step==0: print('Epoch {:}, train steps {:}, loss={:.4f}'.format(epoch, step, loss), flush=True) print('Epoch {:}, train Avg loss={:.4f}, lr={:.4f}'.format(epoch, np.mean(total_loss), lr)) print('*'*20, 'Valid Epoch %d'%epoch, '*'*20) total_loss=[] all_num = step_valid TP = 0 for step in range(step_valid): x,y = next(val_dataset) val_loss, pred_logits = sess.run([loss_tf, logit], feed_dict={input:x, label:y, is_training:False}) y_pred = np.argmax(pred_logits, axis=-1) y = np.argmax(y, axis=-1) total_loss.append(val_loss) if y[0] == y_pred[0]: TP += 1 if step % display_step==0: print('Epoch {:}, valid steps {:}, loss={:.4f}'.format(epoch, step, val_loss)) val_loss_avg = np.mean(total_loss) print('Epoch {:}, valid Avg loss={:.4f}, acc={:.4f}'.format(epoch, val_loss_avg, TP*1.0/all_num)) print('*'*20, 'Valid Epoch %d'%epoch, '*'*20) saver.save(sess, os.path.join(save_model_dir, 'epoch_%03d_%0.4f_model' % (epoch, val_loss_avg)), write_meta_graph=False) saver.save(sess, os.path.join(save_model_dir, 'last_weight_model'), write_meta_graph=False) if min_loss > val_loss_avg: min_loss = val_loss_avg saver.save(sess, os.path.join(save_model_dir, 'best_weight_model'), write_meta_graph=False) #============test def test(test_img_npy, test_label_npy, save_model, batch_size=1, input_size=[384, 384, 1], n_class=2, first_channels=64, display_step=100): test_data, test_label, _, _ = load_split_data(test_img_npy, test_label_npy, thre=1) test_dataset = data_generator(test_data, test_label, batch_size, False) input = tf.placeholder(dtype=tf.float32, shape=[None, input_size[0], input_size[1], input_size[2]]) logit = build_res_network(input, first_channels, n_class, rate=.2, is_training=False) label = tf.placeholder(dtype=tf.float32, shape=[None, n_class]) saver = tf.train.Saver() with tf.Session() as sess: saver.restore(sess, save_model) pred_label = [] cls_label = [] y_prob = [] all_num = len(test_data) TP = 0 for step in range(all_num): x, y = next(test_dataset) pred_logits = sess.run(logit, feed_dict={input:x, label:y}) pred_y = np.argmax(pred_logits,1).astype(np.int32) label_y = np.argmax(y,1).astype(np.int32) pred_label.append(pred_y[0]) cls_label.append(label_y[0]) y_prob.append(pred_logits[0][1]) if label_y[0]==pred_y[0]: TP += 1 if step%display_step == 0: print('Test steps {:} y true {:} y pred{:}'.format(step, label_y, pred_y)) auc = roc_auc_score(cls_label, y_prob) precision = precision_score(cls_label, pred_label) recall = recall_score(cls_label, pred_label) f1 = f1_score(cls_label, pred_label) print('Tesy AUC={:4f}, Avg acc={:4f}, Precision={:4f}, Recall={:4f}, F1={:4f}'.format(auc, TP*1.0/all_num, precision, recall, f1)) # npy_path = '/Users/kai/Downloads/Chapter5/' # npy_path = '/data/projects/kaggle/rsna-intracranial-hemorrhage-detection/stage_2_train_npy/' npy_path = '/workspace/rsna-intracranial-hemorrhage-detection/stage_2_train_npy/' # print("compute mean std") # compute_mean_std(npy_path) train_img_npy = [ os.path.join(npy_path, 'trainval_img_5000_0.npy'), os.path.join(npy_path, 'trainval_img_1000_1.npy'), os.path.join(npy_path, 'trainval_img_1000_2.npy'), os.path.join(npy_path, 'trainval_img_1000_3.npy'), os.path.join(npy_path, 'trainval_img_1000_4.npy'), os.path.join(npy_path, 'trainval_img_1000_5.npy'), ] train_label_npy = [ os.path.join(npy_path, 'trainval_label_5000_0.npy'), os.path.join(npy_path, 'trainval_label_1000_1.npy'), os.path.join(npy_path, 'trainval_label_1000_2.npy'), os.path.join(npy_path, 'trainval_label_1000_3.npy'), os.path.join(npy_path, 'trainval_label_1000_4.npy'), os.path.join(npy_path, 'trainval_label_1000_5.npy'), ] test_img_npy = [ os.path.join(npy_path, 'test_img_1000_0.npy'), os.path.join(npy_path, 'test_img_200_1.npy'), os.path.join(npy_path, 'test_img_200_2.npy'), os.path.join(npy_path, 'test_img_200_3.npy'), os.path.join(npy_path, 'test_img_200_4.npy'), os.path.join(npy_path, 'test_img_200_5.npy'), ] test_label_npy = [ os.path.join(npy_path, 'test_label_1000_0.npy'), os.path.join(npy_path, 'test_label_200_1.npy'), os.path.join(npy_path, 'test_label_200_2.npy'), os.path.join(npy_path, 'test_label_200_3.npy'), os.path.join(npy_path, 'test_label_200_4.npy'), os.path.join(npy_path, 'test_label_200_5.npy'), ] ''' save_model_dir = './temp/saved_model' if not os.path.exists(save_model_dir): os.makedirs(save_model_dir) train(train_img_npy, train_label_npy, save_model_dir, display_step=1) ''' if __name__ == "__main__": # 训练 if False: save_model_dir = './saved_model' train(train_img_npy, train_label_npy, save_model_dir, display_step=10) # 测试 if True: save_model = './saved_model/epoch_036_0.6214_model' test(test_img_npy, test_label_npy, save_model) 上述save_model,根据前面训练出来的模型选择一个即可: 选择epoch_036运行后,结果如下: 选择epoch_099后,运行结果如下:    在上面的输出中,各项具体含义如下: AUC:AUC值为ROC曲线所覆盖的区域面积,AUC越大,分类效果越好。 Avg acc:在所有预测出来的正例中有多少是真的正例。 Precision:表示模型对所有正预测中正确预测正数的能力。 Recall:表示模型从实际阳性样本中正确预测阳性的能力。召回率得分越高,机器学习模型在识别正面和负面样本能力就越好 F1:是精度和召回率分数的谐波平均值,在选择精度或召回率分数可能导致模型分别给出高误报和漏报的情况下用作指标。   七、总结 老实说,顺利的跑完模型训练测试,验证这章学习的内容,不是件容易的事情,需要好好研究学习实践。 不过,通过测试数据集合完整的模型训练过程,得以对相关的概念有了更具象化的了解,受益匪浅。

  • 2024-05-07
  • 发表了主题帖: 《深度学习与医学图像处理》【学习分享5】医学数字图像的数据预处理

    在实际应用场景中,获取的原始数据可能会存在一些问题,例如图片尺寸、对比度不统一等,需要进行一些预处理,才能用于模型的训练。   一、扩展库安装 通过本书第4章的学习,了解到,通常会使用SimpleTK、NumPy、sickit-image、TensorFlow等库进行处理。 要使用这些库进行处理,需要先使用pip进行安装: pip install SimpleITK pip install scikit-image pip install elasticdeform pip install tensorflow 另外,还有些安装一些图形处理库,方便可视化: pip install pillow pip instal matplotlib pip install python-opencv   二、导入需要使用的库 然后就可以在代码中,引入响应的库,以便后续代码调用: import math import SimpleITK as sitk import matplotlib.pyplot as plt import numpy as np import elasticdeform as edf from skimage import measure, color, morphology, transform, exposure, data, io import tensorflow as tf from PIL import Image import cv2   三、数据预处理 1. 插值: 插值法是很常见的图像处理算法,通常使用最邻近插值法和双线性插值法。   最邻近插值法是最简单的插值法,基本原理如下图:   双线性插值法的原理如下图:     其原理效果效果如下:       两重插值法对应的代码如下: # 采用最近插值法或线性插值法 def Linear_interpolation(img, target_shape, interpolation_method): if interpolation_method == 'nearest_neighbor': return transform.resize(img, target_shape, order=0, preserve_range=True) elif interpolation_method == 'bilinear': return transform.resize(img, target_shape, order=1, preserve_range=True) else: raise NameError('Undefined Method.')   我是用了一张颅内CT检查的DICOM样本数据来进行测试,结合之之前的代码,进一步进行处理: # 图片转换为SimpleITK图片格式 skit_img = img_as_float(np.asarray(final_image)) # 调用最近邻插值法处理 nearest_neighbor_img = Linear_interpolation(skit_img, (512, 512), 'nearest_neighbor') # 调用双线性插值法处理 biliner_img = Linear_interpolation(skit_img, (512, 512), 'bilinear') # 存储图片 cv_image1 = img_as_ubyte(nearest_neighbor_img) cv2.imwrite("1.jpg", cv_image1) cv_image2 = img_as_ubyte(biliner_img) cv2.imwrite("2.jpg", cv_image1) # 显示图片 plt.figure() plt.subplot(2,2,1) plt.imshow(np.asarray(final_image),cmap='gray') plt.subplot(2,2,3) plt.imshow(nearest_neighbor_img,cmap='gray') plt.subplot(2,2,4) plt.imshow(biliner_img,cmap='gray') plt.show()   不过上面的图片中,可能看不到较为显著的差别,需要进一步放大,才能看到差异。   2. 重采样 根据采样设备的不同,获取的数医学图像的像素间距可能存在差异,这就需要进行预处理,重采样为统一标准的图像,以便进行训练。 重采样的时候,还需要考虑使用合理的插值算法:真实尺寸 = 像素个数 * 像素间距 医学数据重采样常见的算法如下: # 重采样 def Resample(sitkimg, spacing = None, resolution=None, interpolation = 0): spacing_ori = np.array(sitkimg.GetSpacing()) resolution_ori = np.array(sitkimg.GetSize()) inte = sitk.sitkNearestNeighbor if interpolation == 0 else sitk.sitkLinear if spacing: f = spacing_ori / spacing resolution = np.int32(np.round(resolution_ori * f)) elif resolution: resolution = np.array(resolution) f = resolution / resolution_ori spacing = spacing_ori / f else: return sitkimg origin = np.array(0.5 * (spacing - spacing_ori) / spacing_ori) origin = sitkimg.TransformContinuousIndexToPhysicalPoint(origin) rs = sitk.ResampleImageFilter() rs.SetOutputSpacing(spacing) rs.SetOutputOrigin(origin) rs.SetSize(resolution.tolist()) rs.SetOutputDirection(sitkimg.GetDirection()) rs.SetOutputPixelType(sitkimg.GetPixelID()) rs.SetInterpolator(inte) rs_sitkimg = rs.Execute(sitkimg) return rs_sitkimg   继续使用前面的原始图像数据,将原始图片,转换为像素间距为(0.5, 0.5)的数据,对应的调用代码如下: origin = np.array(final_image) skitimg_origin = sitk.GetImageFromArray(origin) plt.figure() plt.subplot(1,2,1) plt.imshow(sitk.GetArrayFromImage(skitimg_origin),cmap='gray') # plt.show() spacing_ori = np.array(skitimg_origin.GetSpacing()) resolution_ori = np.array(skitimg_origin.GetSize()) print("spacing_ori: ", spacing_ori) print("resolution_ori: ", resolution_ori) skit_img_resample = Resample(skitimg_origin, spacing = (0.5, 0.5), resolution = None, interpolation = 0) plt.subplot(1,2,2) plt.imshow(sitk.GetArrayFromImage(skit_img_resample),cmap='gray') plt.show()   实际处理后的效果如下:   上面的结果中,两个图看起来一样,但是从图像的刻度可以看到,像素间距已经不同了。   3. 信号强度直方图的分析和均衡化 根据采样设备的不同,获取的数医学图像的信号强度也会存在差异,导致强度分布不统一。 这就需要使用到信号强度分析和处理,可以试用Matplotlib和Skimage进行信号强度的处理,对应的代码如下: # 信号强度直方图的均衡化 def Histogram_equalization(img): # 显示原图像 plt.subplot(2,2,1) plt.imshow(img, cmap='gray') plt.gca().invert_yaxis() # plt.show() # 显示原图像信号强度 plt.subplot(2,2,2) n, bins, patches = plt.hist(img, bins=5, facecolor='red', alpha=0.75) # plt.show() # 信号强度直方图的均衡化和可视化 img_res = exposure.equalize_hist(img) plt.subplot(2,2,3) plt.imshow(img_res, cmap='gray') plt.gca().invert_yaxis() # plt.show() # 显示均衡化后的信号强度直方图 plt.subplot(2,2,4) n, bins, patches = plt.hist(img_res, bins=5, facecolor='red', alpha=0.75) # plt.show() return img_res   对应的调用代码如下: plt.figure() skitimg_res = Histogram_equalization(sitk.GetArrayFromImage(skitimg_origin)) plt.show() 最终的结果如下:   从上图可以看到,原有图像较暗,处理后的图像对比度提高,能够更好的进行分析处理。     医学图像数据处理中,还包括数据归一化、连通域分析、形态学方法处理等,这里就不一一展示了。 经过合适的数据处理,就能够让原始的数据变得统一规范,方便后续的训练操作了。

  • 2024-05-05
  • 回复了主题帖: 《深度学习与医学图像处理》【学习分享4】DICOM和Nifti医学影像格式读取和可视化

    Jacktang 发表于 2024-5-5 07:35 CT成像就是这样的原理么 这个不是原理。   是CT扫描后的数据,会采用这些方式保存。

  • 2024-05-03
  • 发表了主题帖: 《深度学习与医学图像处理》【学习分享4】DICOM和Nifti医学影像格式读取和可视化

    ## 一、常用医学影像格式 医学检查后获取的数据需要进行保存,以便后续进行查阅和使用。检查的数据,通常是无法直接保存为图片数据的因为那样会丢失很多关键信息,因此需要保存为专用的医学影像格式。 在实际使用中,最为常用的医学影像格式为DICOM和Nifti。DICOM和NIfTI是两种在医疗影像领域广泛使用的图像数据格式,它们各自具有不同的特点和应用场景。 这两种格式的对比如下: ### DICOM格式 1 综合性:DICOM是一种国际标准,用于医疗影像的获取、存储、打印和传输。它不仅包含图像数据,还包含丰富的患者信息、检查信息和设备参数。 2 兼容性:DICOM格式的文件被广泛接受,几乎所有的医学影像设备和图像处理软件都支持DICOM格式。 3 结构性:DICOM文件由一系列二维图像组成,每个文件代表一个单独的切片,可以包含多个切片,形成一系列图像。 4 信息存储:DICOM文件中存储了大量的元数据,包括患者信息、检查参数、图像的像素数据等。 ## NIfTI格式 1. 专用性:NIfTI格式最初是为神经影像学设计的,它特别适合存储三维或四维的大脑成像数据。 2. 简洁性:NIfTI格式通常使用单一文件存储图像数据和相关的元数据,这使得文件管理更为简便。 3. 扩展性:NIfTI格式支持扩展名为.nii的单一文件或.img/.hdr的两个文件,后者保持了与ANALYZE格式的兼容性。 4. 分析友好:NIfTI格式在神经科学研究中非常流行,因为它方便了图像的分析和处理,尤其是在进行三维或更高维度的图像分析时。 ### 对比 1. 存储方式:DICOM通常以多个二维切片存储,而NIfTI通常以单一的三维或四维数据集存储。 2. 信息丰富度:DICOM包含更丰富的患者信息和设备参数,而NIfTI则专注于图像数据本身,并附带必要的空间定位信息。 3. 使用场景:DICOM适用于临床医疗影像的存储和传输,而NIfTI更适合于科研领域,特别是在神经影像学分析中。 4. 软件支持:虽然两种格式都被广泛支持,但DICOM在医疗影像设备和医院信息系统中更为普遍,NIfTI则在科研软件和图像分析工具中更受青睐。 在Python中,通过合适的库,可以很好的提供对DICOM和Nifti影像格式数据的支持。 ## 二、DICOM影像格式 在python中,安装pydicom,即可对该格式提供支持。 ``` pip install pydicom ``` 另外,还需要安装其他的库,以便提供后续学习的支持: ``` pip install numpy pip install pillow pip install pytest ``` 安装该支持库后,默认会提供多种测试数据: 使用下面的代码即可调用: ``` import pydicom from pydicom.data import get_testdata_files # 获取Pydicom内置DICOM文件 filename = get_testdata_files('MR_small.dcm') #读取DICOM文件 ds = pydicom.dcmread(filename[0]) print(ds) ``` 为了实际学习,我下载了 Covid_Scans 的数据集,并拷贝其中的两个文件,用于测试: 参考上述代码,编写读取的程序: ``` import pydicom from pydicom.data import get_testdata_files # 读取下载的测试数据文件 ds = pydicom.dcmread('data/56364397.dcm') # ds = pydicom.dcmread('data/56364823.dcm') print(ds) ``` 运行后,输出如下: 通过书本上的主要参数说明,可以了解数据字段的具体含义: 不过,光看这些数据,还是不太直观,如果能还原为图片进行呈现,就更合适了。 经过了解,使用如下的代码即可: ``` import numpy as np from PIL import Image new_image = ds.pixel_array.astype(float) scaled_image = (np.maximum(new_image, 0) / new_image.max()) * 255.0 scaled_image = np.uint8(scaled_image) final_image = Image.fromarray(scaled_image) final_image.show() ``` 上述代码中,将数据转换为np格式,然后用Pillow读取数据转换为最终的图像,进行呈现。 具体效果如下: 可以看到,这个文件是图像数据检测时候的相关信息。 再用另外一个数据,得到的图像如下: 这个就是实际的肺部检测数据对应的图像了。 ## 三、Nifti影像格式 同样的,先安装几个库提供支持: ``` pip install nibabel pip install opencv-python pip install imageio pip install matplotlib ``` nibabel是提供Nifti对应的.nii文件支持的库,也提供了测试数据集: 调用测试数据的代码如下: ``` import os import nibabel as nib from nibabel.testing import data_path file_path = os.path.join(data_path, 'example4d.nii.gz') img = nib.load(file_path) ``` nibabel.testing.data_path表示上述安装后对应的测试数据集目录。 为了实际学习,我下载了一个测试数据集,使用如下: 然后使用下面的代码读取: ``` import os import nibabel as nib from nibabel.testing import data_path file_path = 'data/nifti/OBJECT_phantom_T2W_TSE_Tra_17_1.nii' img = nib.load(file_path) ``` 数据对应的具体信息,可以使用如下的方式读取: ``` #放射矩阵 affine = img.affine print(affine) # 元数据 header = img.header print(header) # 访问header元数据 print(header['sizeof_hdr']) # 修改 # header['sizeof_hdr'] = 200 # 读取影像数据 data = img.get_fdata() print(data.dtype, data.shape) ``` 输出如下: 同样的,我也进行了学习,将其转换为对应的图片进行呈现,具体代码如下: ``` import cv2 import numpy as np img_fdata=(data-data.min())/(data.max()-data.min())*255 #开始转换图像 (x,y,z) = img.shape for i in range(z):   #是z的图象序列 slice = img_fdata[:, :, i]  #选择哪个方向的切片自己决定 # print(os.path.join(img_f_path, '{}.png'.format(i))) # cv2.imwrite(os.path.join(img_f_path, '{}.png'.format(i)), slice) cv2.imshow("Image", slice) cv2.waitKey (0) cv2.destroyAllWindows() ``` 实际输入如下: 通过了解主要使用的医学图像影像格式,后面就能够对数据集进行实际的操作使用了,如关键信息和数据的提取,以及对数据进行二次加工。

  • 回复了主题帖: 《深度学习与医学图像处理》【学习分享3】常见医学图像数据

    lugl4313820 发表于 2024-5-1 07:02 这些看起来先进的设备,好象只给出平面图像。帮主大佬,来个还原3D图像,这样,对医生的诊断就意义了。 刚学习,还没入门。 还请 @lugl4313820 多多指点!

  • 2024-04-30
  • 回复了主题帖: 测评入围(第三波):Beetle ESP32 C6迷你开发板 基于ESP32-C6芯片

    个人信息无误,确认可以完成评测计划。

统计信息

已有420人来访过

  • 芯积分:1147
  • 好友:4
  • 主题:104
  • 回复:134

留言

你需要登录后才可以留言 登录 | 注册


现在还没有留言