【HZHY-AI300G智能盒试用连载体验】+ 具有 Local AI 功能的工业用照明控制器

有关wifi不能正常开启的问题,我按照技术支持的建议,重新烧录了 网盘中ubuntu 20 的镜像就可以了。

下面测试下网速：

作为发送端，网速大概在 10M附近，作为接收端，网速大概在 50M 左右。相比有线网络还是差一点哈。这里算是解决了一个问题，另一个问题什么时候可以解决呢？敬请期待。

社区的小伙伴们，好几天没有更新了，呜呜呜。今天补充下，有关第一章中的第二个 驱动因为缺少内核树无法本机编译 的问题如何解决的。

因为官方的镜像是 ubuntu 的，因为涉及到驱动层的开发，我个人更倾向那些非发行版的（比如从 buildroot 中编译出来的）。所以，我从对应的 SDK 中编译了 buildroot 的 img 镜像，然后使用工具烧录进去。

▸ ls -l update.img

lrwxrwxrwx. 1 red red 26 Jul 18 13:49 update.img -> ../update/Image/update.img

特别地，在编译 buildroot 的 img 镜像中，会生成一个内核的设备树，方便驱动开发，如下所示：

▸ ls -l linux-headers.tar

-rw-r--r--. 1 red red 81489920 Jul 18 13:49 linux-headers.tar

有了这个内核树之后，就可以开发驱动了，一个简单的 hello_world 驱动：

#include   

#include   

#include   

  

MODULE_LICENSE("GPL");  

MODULE_AUTHOR("Test");  

MODULE_DESCRIPTION("A simple Hello World driver");  

  

static int __init hello_init(void) {  

    printk(KERN_ALERT "Hello, world\\n");  

    return 0;  

}  

  

static void __exit hello_exit(void) {  

    printk(KERN_ALERT "Goodbye, cruel world\\n");  

}  

  

module_init(hello_init);  

module_exit(hello_exit);

对应的 Makefile 参考：

obj-m += hello_world.o  

KDIR=指定源码树的目录

CROSS_TOOLS=指定交叉编译工具链的前缀

  

all:  

        make ARCH=arm64 CROSS_COMPILE=$(CROSS_TOOLS) -C $(KDIR) M=$(PWD) modules  

  

clean:  

        make ARCH=arm64 CROSS_COMPILE=$(CROSS_TOOLS) -C $(KDIR) M=$(PWD) clean

执行 make,生成 hello_world.ko：

▸ ls -l hello_world.ko

-rw-r--r--. 1 red red 136488 Jul 19 06:45 hello_world.ko

然后通过 scp 发送到 AI300G，加载、卸载驱动测试：

按照计划，第一阶段具备内核、驱动和应用的开发能力已经告一段落。下一步就是开发 USB 转 LoRa 模块相关驱动了，敬请期待哦。

前言

搁了这么久，目前我设计的 USB 转 LoRa 模块（起名叫：Red LoRa Module，后续简称 RLM）的驱动开发的差不多了，基本实现了双向的 USB 到 LoRa 之间的转换，今天就发文总结以下，因为后面就要到光照度算法部分开发了，期待哦。

概述

首先看下 RLM 的，结构框图：

再看一下3D图和实物图：

这部分驱动是运行在 STM32F103 上面，通过 CDC ACM 配置，实现在 PC 端枚举出来对应的节点。另一方面通过 SPI 和 LoRa 通信，最终实现 USB 和 LoRa 之间的透传。这部分代码是开源的，我是在 https://github.com/blackmagic-debug/blackmagic.git 开源项目的基础上修改实现的。这部分固件代码我放在我的 gitee 仓库 rlm_hsi，感兴趣的小伙伴可以去看看。

经验总结

经过这几天的 USB 转 LoRa 固件功能的开发，总结出来比较深刻的经验有如下几点：

1. 使用 STM32F103CBT6 的 USB，USB 的时钟要固定在48MHZ，并且官方不建议使用内部晶振提供这个时钟，在选择使用 cubeMX 软件的时候，如果选择 HSI 提供这个始终，也会提示你有冲突，但是不要紧，想用  HSI 还是可以的，不过不建议哦，因为我这个板子的图画的比较早了，当时没有考虑到，所以就硬着头皮往下开发了，好在成功了。


2. 使用 STM32F103CBT6 的 USB 的时候，为了让 PC 可以枚举设备，在电路上需要在 D+ 上设计一个上拉电阻，如果想实现动态热插拔，这个电阻另一端需要连接到一个可以控制的 GPIO 管脚，通过拉高这个 GPIO，实现动态热插拔，如果不需要这个功能，就直接上拉到 VCC（我这里是直接接到了 3V3） 就行了。


3. 在使用 SPI 的时候，因为我想用软件控制 NSS，发现必须要同时配置 SPI 的 CR1 的 SSM 和 SSI 为 1。否则是不能配置到主机模式软件NSS的。


4. 在调试 LoRa 模块的时候，发现 spi 速率太低也会导致通信异常，最后我设定在 SPI 时钟 2 分频可以正常工作。

效果演示

说了这么多，我们看下实际的效果，首先是设备枚举：

[349794.851655] usb 3-4.2: new full-speed USB device number 49 using xhci_hcd

[349795.044759] usb 3-4.2: New USB device found, idVendor=1d51, idProduct=6118, bcdDevice= 0.09

[349795.044777] usb 3-4.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[349795.044783] usb 3-4.2: Product: Red Lora Module v1.10.0-926-g8f2291e7-dirty

[349795.044787] usb 3-4.2: Manufacturer: Red Studio

[349795.044791] usb 3-4.2: SerialNumber: E1C3A7D0

[349795.054870] cdc_acm 3-4.2:1.0: ttyACM2: USB ACM device

接着我们看下双向的数据透传视频演示：

训练图像估计光照度算法模型

前言

这一篇就到了图像估计光照度算法章节，这篇我主要记录如何使用 tensorflow2 训练一个从图片中估计光照度的算法。一般的流程是拍摄多张图片以及用光照度计来检测其光照度值，分别作为输入和输出。但是在本章呢，为了起到演示的作用，数据集我会使用 MIT-Adobe FiveK Dataset 。光照度值呢，我使用图片的 rgb 数值经过算法r*0.2126+g*0.7152+b*0.0722计算亮度。这样就有了一定数量的数据集。也就有基础进行后续的训练和测试了。下面准备进入正文。

数据获取

因为 MIT-Adobe FiveK Dataset 数据集包含了 5000 张原始 dng 图像和 5 和专家A,B,C,D,E进行处理之后的 tiff 图像(一般地，这个数据集是用来训练图像增强相关的模型的，我这里就用来训练光照度估计算法了，嘿嘿)。因为完整的数据压缩包太大了~50GB。受限电脑的容量和速度，我选择了使用脚本逐个下载这些图片（因为这些图片的下载路径有规律，再加上这些图片的名字在官网可以下载下来，所以脚本就读取包含图片名字的文件，然后逐个拼接下载路径，使用 curl 工具完成下载）。这里，我选择了下载原始 dng 图片和专家 C 的 tiff 图片。

下载 dng 原始文件的脚本是：

#!/usr/bin/bash



#改变当前工作路径

CURRENT_PATH="/home/red/Downloads/fivek_dataset/expertc"

#本文件所在路径

cd ${CURRENT_PATH}

#改变当前路径



#存储图像名称的list

img_lst=[]

#读取图片名列表

files_name=`cat filesAdobe.txt`

files_mit_name=`cat filesAdobeMIT.txt`



j=0

for i in ${files_name};do

    # https://data.csail.mit.edu/graphics/fivek/img/dng/a0001-jmac_DSC1459.dng

    URL='https://data.csail.mit.edu/graphics/fivek/img/dng/'${i}'.dng'

    file_cur=${URL##*/}

    echo "Downloading ${URL}@${j}"

    j=$((j+1))

    if [ -f ${file_cur} ];then

        echo "${file_cur} exist"

    else

        # echo "${file_cur} no exist, it's you"

        # break

        curl -O ${URL}

    fi

done

下载专家 C 处理后的文件脚本是：

#!/usr/bin/bash



#改变当前工作路径

CURRENT_PATH="/home/red/Downloads/fivek_dataset/expertc"

#本文件所在路径

cd ${CURRENT_PATH}

#改变当前路径



#存储图像名称的list

img_lst=[]

#读取图片名列表

files_name=`cat filesAdobe.txt`

files_mit_name=`cat filesAdobeMIT.txt`



j=0

for i in ${files_name};do

    #下载由 expert C 所调整的图像(可根据需要下载其它的四类图像)

    URL='https://data.csail.mit.edu/graphics/fivek/img/tiff16_c/'${i}'.tif'

    file_cur=${URL##*/}

    echo "Downloading ${URL}@${j}"

    j=$((j+1))

    if [ -f ${file_cur} ];then

        echo "${file_cur} exist"

    else

        echo "${file_cur} no exist, it's you"

        # break

        curl -O ${URL}

    fi

done

经过了好几天断断续续的下载，最后我一共得到了 1000 张左右图片。有了图片之后，下一步就是计算光照度了，这里使用 python 脚本和 pillow 包完成，为了后续移植到 AI300G 上，我将图片缩放到了统一的 255*255。并且将计算的光照度和图像的名称存储到一个 csv 文件。这部分脚本如下：

#!/bin/env python3



import sys

import csv

import os

import re



from PIL import Image



gs_illumiance_csv_file_fd=0

gs_illumiance_csv_file_name='illumiance_estimate.csv'

gs_illumiance_data_list=[['Name', 'Illuminance']]

DEST_DIR_NAME=r'PNG255'



def illuname_estimate(t):

    r,g,b=t

    return r*0.2126+g*0.7152+b*0.0722





def get_pic_pixels(pic_name):

    with Image.open(pic_name) as pic:

        ans=0

        pic=pic.resize((255,255))

        print(f'raw name:{pic_name}')

        match=re.match(r'\\w+\\/(\\S+)\\.\\w+', pic_name)

        if match:

            basename=match.group(1)

            basename=DEST_DIR_NAME+'/'+basename+'.png'

            print(f'new name:{basename}')

            pic.save(basename)

            #  pic.show()

        width, heigh = pic.size

        for x in range(width):

            for y in range(heigh):

                r, g, b = pic.getpixel((x, y))

                ans=ans+illuname_estimate((r,g,b))



    # 光照度取整

    ans=round(ans)

    print(f'{pic_name}: illuname ans:{ans}')

    return ans



def insert_item(pic_name, illumiance_estimate):

    global gs_illumiance_csv_file_fd

    global gs_illumiance_csv_file_name

    global gs_illumiance_data_list

    item_template=['NONE', -1]

    item_template[0]=pic_name

    item_template[1]=illumiance_estimate

    gs_illumiance_data_list.append(item_template)



def do_with_dir(dir_name):

    for filename in os.listdir(dir_name):

        filepath=os.path.join(dir_name, filename)

        if (os.path.isfile(filepath)):

            print("do input %s" %(filepath))

            ans=get_pic_pixels(filepath)

            insert_item(filename, ans)

            #  return



if len(sys.argv) > 1:

    print("do input dir:%s" %(sys.argv[1]))

    if not os.path.exists(DEST_DIR_NAME):

        os.makedirs(DEST_DIR_NAME)

    do_with_dir(sys.argv[1])

    gs_illumiance_csv_file_fd=open(gs_illumiance_csv_file_name, 'w', newline='')

    csv.writer(gs_illumiance_csv_file_fd).writerows(gs_illumiance_data_list)

else:

    print("Please input pic name")

这样就得到了类似下面的数据集：

▸ head illumiance_estimate.csv

Name,Illuminance

a0351-MB_070908_006_dng.jpeg,3680630

a0100-AlexWed07-9691_dng.jpeg,1258657

a0147-kme_333.jpeg,5168820

a0261-_DSC2228_dng.jpeg,2571498

a0255-_DSC1448.jpeg,8747593

a0054-kme_097.jpeg,5351908

a0393-_DSC0040.jpeg,1783394

a0304-dgw_137_dng.jpeg,3118835

a0437-jmacDSC_0011.jpeg,6140107

至此有了一定数量的数据集（这里我使用了667张照片），接下来就是模型训练了。

模型训练

模型训练的基本思想就是，首先将数据集按比例(4:1)拆分为训练集和测试集，然后使用 tensorflow 建立模型训练参数进行检验。

大概流程是：

1. 首先是根据 csv 文件建立 tensorflow dataset 格式的数据集；


2. 建立模型使用数据集进行模型训练和测试

这部分代码为：

#!/usr/bin/python3.11



TF_ENABLE_ONEDNN_OPTS=0



import numpy as np

import os

import PIL

import PIL.Image

import tensorflow as tf

import pathlib

import csv

import pandas as pd

import tensorflow.data

import sys

import matplotlib.pyplot as plt



AUTOTUNE=tensorflow.data.AUTOTUNE

BATCH_SIZE=32

IMG_WIDTH=255

IMG_HEIGHT=255

ILLUMINACE_FILE=r'illumiance_estimate.csv'

print(tf.__version__)



import tensorflow as tf

import pandas as pd



image_count = len(os.listdir(r'JP'))

print(f'whole img count={image_count}')

# 假设CSV文件有两列：'image_path' 和 'label'

df = pd.read_csv(ILLUMINACE_FILE)



# 将DataFrame转换为TensorFlow可以处理的格式

image_paths = df['Name'].values

labels = df['Illuminance'].values

labels = labels.astype(np.float32)

labels /= 16777215.0



# 创建一个Dataset

gs_dataset = tf.data.Dataset.from_tensor_slices((image_paths, labels))



print(type(gs_dataset))

print(gs_dataset)

print(r'-------------------------------------------')

# 定义一个函数来加载和预处理图像

def load_and_preprocess_image(image_path, label):

    print(image_path)

    image_path='JP/'+image_path

    image = tf.io.read_file(image_path)

    image = tf.image.decode_jpeg(image, channels=3)

    image = tf.image.resize(image, [IMG_WIDTH, IMG_HEIGHT])

    #  image /= 255.0  # 归一化

    return image, label



# 应用这个函数到Dataset上

gs_dataset = gs_dataset.map(load_and_preprocess_image)

# 打乱数据

gs_dataset = gs_dataset.shuffle(image_count, reshuffle_each_iteration=False)



val_size = int(image_count * 0.2)



gs_train_ds = gs_dataset.skip(val_size)

gs_val_ds = gs_dataset.take(val_size)



def configure_for_performance(ds):

    ds = ds.cache()

    ds = ds.shuffle(buffer_size=1000)

    ds = ds.batch(BATCH_SIZE)

    ds = ds.prefetch(buffer_size=AUTOTUNE)

    return ds



gs_train_ds = configure_for_performance(gs_train_ds)

gs_val_ds = configure_for_performance(gs_val_ds)



image_batch, illuminance_batch = next(iter(gs_train_ds))



#  plt.figure(figsize=(10, 10))



#  for i in range(9):

  #  ax = plt.subplot(3, 3, i + 1)

  #  print(image_batch[i])

  #  #  img_data=image_batch[i].numpy()*255.0

  #  #  plt.imshow(img_data.astype("uint8"))

  #  plt.imshow(image_batch[i].numpy().astype("uint8"))

  #  illuminance = illuminance_batch[i]

  #  plt.title(illuminance.numpy())

  #  plt.axis("off")



#  plt.show()



#  sys.exit()



model = tf.keras.Sequential([

  tf.keras.layers.Rescaling(1./255),

  tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(IMG_WIDTH, IMG_HEIGHT, 3)),

  tf.keras.layers.MaxPooling2D(2, 2),

  tf.keras.layers.Conv2D(64, (3, 3), activation='relu'),

  tf.keras.layers.MaxPooling2D(2, 2),

  tf.keras.layers.Conv2D(32, 3, activation='relu'),

  tf.keras.layers.MaxPooling2D(),

  tf.keras.layers.Flatten(),

  tf.keras.layers.Dense(128, activation='relu'),

  tf.keras.layers.Dense(1)

])



model.compile(

  optimizer='adam',

  loss='mean_squared_error')



model.fit(

  gs_train_ds,

  validation_data=gs_val_ds,

  epochs=12

)



model.save("illu_v01")

执行上述代码，可以看到最后的 loss 和 val_loss 为：

▸ ./train_tf2_v2.py

2024-08-08 13:41:48.341117: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.

2024-08-08 13:41:48.342596: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.

2024-08-08 13:41:48.363696: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered

2024-08-08 13:41:48.363729: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered

2024-08-08 13:41:48.364549: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered

2024-08-08 13:41:48.368601: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.

2024-08-08 13:41:48.368762: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.

To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

2024-08-08 13:41:48.801750: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

2.15.0

whole img count=667

2024-08-08 13:41:51.138713: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355

2024-08-08 13:41:51.139135: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2256] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.

Skipping registering GPU devices...

<class 'tensorflow.python.data.ops.from_tensor_slices_op._TensorSliceDataset'>

<_TensorSliceDataset element_spec=(TensorSpec(shape=(), dtype=tf.string, name=None), TensorSpec(shape=(), dtype=tf.float32, name=None))>

-------------------------------------------

Tensor("args_0:0", shape=(), dtype=string)

Epoch 1/12

17/17 [==============================] - 11s 603ms/step - loss: 98.9302 - val_loss: 0.1012

Epoch 2/12

17/17 [==============================] - 8s 495ms/step - loss: 0.0493 - val_loss: 0.0043

Epoch 3/12

17/17 [==============================] - 8s 481ms/step - loss: 0.0078 - val_loss: 0.0043

Epoch 4/12

17/17 [==============================] - 8s 479ms/step - loss: 0.0025 - val_loss: 0.0040

Epoch 5/12

17/17 [==============================] - 8s 477ms/step - loss: 0.0023 - val_loss: 0.0029

Epoch 6/12

17/17 [==============================] - 8s 480ms/step - loss: 0.0021 - val_loss: 0.0028

Epoch 7/12

17/17 [==============================] - 8s 482ms/step - loss: 0.0020 - val_loss: 0.0028

Epoch 8/12

17/17 [==============================] - 8s 482ms/step - loss: 0.0019 - val_loss: 0.0027

Epoch 9/12

17/17 [==============================] - 8s 482ms/step - loss: 0.0018 - val_loss: 0.0026

Epoch 10/12

17/17 [==============================] - 8s 485ms/step - loss: 0.0017 - val_loss: 0.0026

Epoch 11/12

17/17 [==============================] - 8s 485ms/step - loss: 0.0015 - val_loss: 0.0023

Epoch 12/12

17/17 [==============================] - 8s 484ms/step - loss: 0.0011 - val_loss: 0.0020

并且模型也保存在了 illu_v01 目录。

▸ ls illu_v01/

assets  fingerprint.pb  keras_metadata.pb  saved_model.pb  variables

模型测试

现在有可模型，下面就是测试下自己的模型，使用下述 python 代码在 PC 端进行测试：

#!/usr/bin/python3.11



import numpy as np

import os

import PIL

import PIL.Image

import tensorflow as tf

import pathlib

import csv

import pandas as pd

import tensorflow.data

import sys

import matplotlib.pyplot as plt



IMG_WIDTH=255

IMG_HEIGHT=255



reload_model=tf.keras.models.load_model("illu_v01")

image_path=r'./JP/a0001-jmac_DSC1459.jpeg'

if len(sys.argv) < 2:

    print('Please input some pic to predict')

    sys.exit()

else:

    image_path=sys.argv[1]





image = tf.io.read_file(image_path)

image = tf.image.decode_jpeg(image, channels=3)

image = tf.image.resize(image, [IMG_WIDTH, IMG_HEIGHT])

image = tf.reshape(image, [1, IMG_WIDTH, IMG_HEIGHT, 3])



#  sys.exit()



predictions=reload_model.predict(image)

print(f'{image_path} ans={predictions*16777215}')

简单测试下模型：

check_tf2.py JP/a0001-jmac_DSC1459.jpeg

2024-08-08 13:57:08.263506: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.

2024-08-08 13:57:08.264895: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.

2024-08-08 13:57:08.285614: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered

2024-08-08 13:57:08.285646: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered

2024-08-08 13:57:08.286510: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered

2024-08-08 13:57:08.290464: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.

2024-08-08 13:57:08.290608: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.

To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

2024-08-08 13:57:08.725843: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

2024-08-08 13:57:11.051710: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355

2024-08-08 13:57:11.051982: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2256] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.

Skipping registering GPU devices...

1/1 [==============================] - 0s 57ms/step

JP/a0001-jmac_DSC1459.jpeg ans=[[check_tf2.py JP/a0001-jmac_DSC1459.jpeg

2024-08-08 13:57:08.263506: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.

2024-08-08 13:57:08.264895: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.

2024-08-08 13:57:08.285614: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered

2024-08-08 13:57:08.285646: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered

2024-08-08 13:57:08.286510: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered

2024-08-08 13:57:08.290464: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.

2024-08-08 13:57:08.290608: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.

To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

2024-08-08 13:57:08.725843: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

2024-08-08 13:57:11.051710: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355

2024-08-08 13:57:11.051982: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2256] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.

Skipping registering GPU devices...

1/1 [==============================] - 0s 57ms/step

JP/a0001-jmac_DSC1459.jpeg ans=[[5459503.]].]]

发现估计的光照度值是 5459503 和实际的 5363799 对比一下还是有15%左右的误差。但是目前为止，整个模型训练测试流程已经完成，下一步在是PC端模拟拉流使用模型对图像进行实时计算了，期待哦。

在 PC 使用 C++ 加载光照度估计模型

上一篇中，我们介绍了使用 Python 训练和测试模型，这一篇介绍如何使用 C++ 加载我们保存的模型并测试。

要在  PC 上使用 C++ 来加载和调用模型，需要从 tensorflow 的源码编译出对应的库和引用头文件。特别提一点，tensorflow 使用 Bazel 构建。

首先还是配置 ./configure ，然后是执行bazel build //tensorflow:libtensorflow_cc.so /tensorflow:libtensorflow_cc.so //tensorflow:install_headers开始编译，这个过程还是很耗时的，编译成功（要好几个小时）后的截图：

编译生成的动态库和头文件目录在 bazel/tensoflow 在如下目录，但是缺少libtensorflow_framework.so的一个软连接（编译的时候会提示找不到tensorflow_framework库），导致编译不通过，添加就好了。

写一个加载光照度模型并打印预测值的C++代码如下

#include 



using namespace tensorflow;

using namespace std;



int main() {

  SessionOptions options;

  RunOptions run_options;



  SavedModelBundle bundle;

  Status status = LoadSavedModel(options, run_options, "/home/red/Downloads/fivek_dataset/test_mark_illuminance_level/illu_v03", {"serve"}, &bundle);

  if (!status.ok()) {

    std::cerr << "Error loading model: " << status.ToString() << std::endl;

    return 1;

  }



  // Access the session

  Session* session = bundle.session.get();



  // Create input tensor

  Tensor input_tensor(DT_FLOAT, TensorShape({1, 255, 255, 3}));

  // Fill input tensor with data

  auto input_tensor_flat = input_tensor.flat<float>();

  std::cout << "size of input tensor is " << input_tensor_flat.size() << std::endl;

    for (int i = 0; i < input_tensor_flat.size(); ++i) {

        input_tensor_flat(i) = 255.0;

    }



  // Run inference

  std::vector outputs;

  Status run_status = session->Run({{"serving_default_rescaling_input", input_tensor}}, {"StatefulPartitionedCall"}, {}, &outputs);

  if (!run_status.ok()) {

    std::cerr << "Error running model: " << run_status.ToString() << std::endl;

    return 1;

  }



const Eigen::TensorMapfloat, 1, Eigen::RowMajor>, Eigen::Aligned>& prediction = outputs[0].flat<float>();

const long count = prediction.size();

for (int i = 0; i < count; ++i) {

        const float value = prediction(i);

        // value是该张量以一维数组表示时在索引i处的值。

    std::cout << "hey hey " << value << std::endl;

}

  // Process output tensor

  Tensor ans = outputs[0];

  // auto ans_value = ans.tensor();

  auto ans_value = ans.tensor<float, 2>();

  std::cout << ans_value(0,0) << std::endl;

  return 0;

}

上面例程中有两个关键的信息，就是输入输出tensor的名字，这个如果写的不对，模型预测的时候会提示找不到对应的tensor。这个名字我是怎么知道的呢？使用tensorflow自带的saved_model_cli.py工具，具体的命令是：

python3.11 tensorflow/python/tools/saved_model_cli.py show --dir ~/Downloads/fivek_dataset/test_mark_illuminance_level/illu_v02/ --tag_set serve --signature_def serving_default

其中 --dir 后跟随自己的模型目录就可以了。查看一下输入输出的名字：

The given SavedModel SignatureDef contains the following input(s):

  inputs['rescaling_input'] tensor_info:

      dtype: DT_FLOAT

      shape: (-1, 255, 255, 3)

      name: serving_default_rescaling_input:0

The given SavedModel SignatureDef contains the following output(s):

  outputs['dense_1'] tensor_info:

      dtype: DT_FLOAT

      shape: (-1, 1)

      name: StatefulPartitionedCall:0

Method name is: tensorflow/serving/predict

可以看到输入和输出的名字分别是 serving_default_rescaling_input 和 StatefulPartitionedCall。和前面给出的 C++ 程序也可以对应起来了。

然后再看下Makefile文件，需要添加 tensorflow 的头文件和库文件连接路径：

TARGET=tfcpp



CFLAGS:=-I/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/include/

CFLAGS+=-I/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/include/src

CFLAGS+=-I/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/include/_virtual_includes/float8/

CFLAGS+=-I/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/include/_virtual_includes/int4/



LDFLAGS:=-L/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow -ltensorflow_framework

LDFLAGS+=-L/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow -ltensorflow



$(TARGET):$(TARGET).cpp

        g++ $(CFLAGS) $(LDFLAGS) $^ -o $@



clean:

        rm -frv $(TARGET)

编译生成测试程序

▸ make

g++ -I/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/include/ -I/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/include/src -I/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/include/_virtual_includes/float8/ -I/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/include/_virtual_includes/int4/ -L/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow -ltensorflow_framework -L/home/red/.cache/bazel/_bazel_red/81f6b3978d226a63c6d017ab1c0efa9f/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow -ltensorflow tfcpp.cpp -o tfcpp

▸ file tfcpp

tfcpp: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=1a13ba4942e9540b56553e7a3d4ab13e790de6ce, for GNU/Linux 3.2.0, not stripped

让我们看下C++调用我们模型，对一个全白色图片的测试结果：

▸ ./tfcpp

size of input tensor is 195075

heyhey 0.855759

0.855759

然后对比下 python 模型下的结果：

▸ cat check_tf2.py

#!/usr/bin/python3.11



import numpy as np

import os

import PIL

import PIL.Image

import tensorflow as tf

import pathlib

import csv

import pandas as pd

import tensorflow.data

import sys

import matplotlib.pyplot as plt



IMG_WIDTH=255

IMG_HEIGHT=255



reload_model=tf.keras.models.load_model("illu_v03")



image = tf.constant(255, shape=(1, 255, 255, 3), dtype=tf.float32)



predictions=reload_model.predict(image)

print(f'predict={predictions}')



▸ ./check_tf2.py

1/1 [==============================] - 0s 58ms/step

predict=[[0.85575885]]

可以看到还是很一致的。

目前 C++ 调用 Python 训练模型这一条路也走通了，下一步就是要移植到 AI300G 上了，敬请期待哦。