从虹软开放了2.0版本SDK以来,因为具备免费、离线使用的特色,咱们公司在人脸识别门禁应用中使用了虹软SDK,识别效果还不错,所以比较关注虹软SDK的官方动态。近期上线了ArcFace 3.0 SDK版本,确实作了比较大的更新。html
特征比对支持比对模型选择,有生活照比对模型和人证比对模型c++
识别率、防攻击效果显著提高算法
特征值更新,升级后人脸库需从新注册数据结构
人脸检测同时支持全角度及单一角度指针
新增了一种图像数据传入方式code
在V3.0版本接入过程当中,发现使用新的图像数据结构仍是具备必定难度的,本文将从如下几点对该图像数据结构及使用方式进行介绍orm
SDK接口变更htm
图像数据结构blog
步长的做用接口
OpenCV图像数据结构转换为虹软图像数据结构
在接入ArcFace 3.0 SDK时,发现新增了ASFDetectFacesEx、ASFFaceFeatureExtractEx、ASFProcessEx、ASFProcessEx_IR一组接口,该组接口使用LPASF_ImageData
结构体指针的方式传入图像数据,以人脸检测接口为例,具体接口比对以下:
原始接口:
MRESULT ASFDetectFaces( MHandle hEngine, // [in] 引擎handle MInt32 width, // [in] 图片宽度 MInt32 height, // [in] 图片高度 MInt32 format, // [in] 颜色空间格式 MUInt8* imgData, // [in] 图片数据 LPASF_MultiFaceInfo detectedFaces, // [out]检测到的人脸信息 ASF_DetectModel detectModel = ASF_DETECT_MODEL_RGB // [in] 预留字段,当前版本使用默认参数便可 );
新增接口:
MRESULT ASFDetectFacesEx( MHandle hEngine, // [in] 引擎handle LPASF_ImageData imgData, // [in] 图片数据 LPASF_MultiFaceInfo detectedFaces, // [out] 检测到的人脸信息 ASF_DetectModel detectModel = ASF_DETECT_MODEL_RGB // [in] 预留字段,当前版本使用默认参数便可 );
相对于原始接口,新增接口经过传入LPASF_ImageData
图像数据结构指针替代原始接口传入图像数据的方式。
新增的图像数据结构引入了步长pi32Pitch
的概念。
步长定义:图像对齐后一行的字节数。
图像结构定义:
typedef LPASVLOFFSCREEN LPASF_ImageData; typedef struct __tag_ASVL_OFFSCREEN { MUInt32 u32PixelArrayFormat; MInt32 i32Width; MInt32 i32Height; MUInt8* ppu8Plane[4]; MInt32 pi32Pitch[4]; }ASVLOFFSCREEN, *LPASVLOFFSCREEN;
虹软官方文档中对该图像数据结构的介绍:
类型 | 变量名 | 描述 |
---|---|---|
MUInt32 | u32PixelArrayFormat | 颜色格式 |
MInt32 | i32Width | 图像宽度 |
MInt32 | i32Height | 图像高度 |
MUInt8* | ppu8Plane | 图像数据 |
MInt32 | pi32Pitch | 图像步长 |
OpenCV提供了IplImage
和Mat
两种比较经常使用的图像数据结构。
IplImage 图像数据结构
typedef struct _IplImage { int width; /* Image width in pixels. */ int height; /* Image height in pixels. */ char *imageData; /* Pointer to aligned image data. */ int widthStep; /* Size of aligned image row in bytes. */ ... //其余字段这里不作展现,感兴趣的小伙伴能够查看下opencv中的头文件 } IplImage;
Mat 图像数据结构
属性 | 说明 |
---|---|
cols | 矩阵的列数(图像宽度) |
rows | 矩阵的行数(图像高度) |
data | uchar型的指针。Mat类分为了两个部分:矩阵头和指向矩阵数据部分的指针,data就是指向矩阵数据的指针。 |
step | 图像对齐以后一行的字节数 |
经过以上描述咱们看到OpenCV和虹软算法库针对图像数据结构都引入了图像步长的概念,这里咱们了解一下图像步长。
OpenCV 读图会作图像对齐
以下图,一张尺寸为998x520
的图像,使用OpenCV读取图像数据后,图像尺寸仍为998x520
,颜色格式为BGR24
,可是图像步长并非998 * 3
,而是1000 * 3
,右边填充了2个像素,OpenCV对图像作了四字节对齐,虹软SDK内部算法再经过传入的图像宽度去计算步长则会出现误差,图像数据错乱,基本不可能检测到人脸。
如下是对一张大小为1000x554
的图片,以不一样步长进行解析的结果:
以1000为步长解析 | 以996为步长解析 |
---|---|
![]() |
![]() |
能够看到,对于一张图像,若是使用了错误的步长去解析,咱们可能就没法看到正确的图像内容。
结论:经过引入图像步长可以有效的避免高字节对齐的问题。
当前C/C++开发者对图像进行编解码处理通常都会用到OpenCV库,这里咱们介绍一下如何将OpenCV转换为虹软的图像数据结构。虹软官方文档中说明支持七种颜色格式,咱们就列出七种颜色格式的转换方法。
OpenCV 读取过来的图像通常为BGR24
格式,可以使用下述方法进行图像数据结构转换。
若原图为红外图像,需将图像转换为ASVL_PAF_GRAY
格式(官网文档中也有示例),再使用下述方法进行转换。
IplImage 转 ASVLOFFSCREEN
int ColorSpaceConversion(MInt32 format, IplImage* img, ASVLOFFSCREEN& offscreen) { switch (format) //原始图像颜色格式 { case ASVL_PAF_I420: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img->width; offscreen.i32Height = img->height; offscreen.pi32Pitch[0] = img->widthStep; offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0] >> 1; offscreen.pi32Pitch[2] = offscreen.pi32Pitch[0] >> 1; offscreen.ppu8Plane[0] = (MUInt8*)img->imageData; offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0]; offscreen.ppu8Plane[2] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0] * 5 / 4; break; case ASVL_PAF_YUYV: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img->width; offscreen.i32Height = img->height; offscreen.pi32Pitch[0] = img->widthStep; offscreen.ppu8Plane[0] = (MUInt8*)img->imageData; break; case ASVL_PAF_NV12: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img->width; offscreen.i32Height = img->height; offscreen.pi32Pitch[0] = img->widthStep; offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0]; offscreen.ppu8Plane[0] = (MUInt8*)img->imageData; offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height; break; case ASVL_PAF_NV21: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img->width; offscreen.i32Height = img->height; offscreen.pi32Pitch[0] = img->widthStep; offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0]; offscreen.ppu8Plane[0] = (MUInt8*)img->imageData; offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height; break; case ASVL_PAF_RGB24_B8G8R8: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img->width; offscreen.i32Height = img->height; offscreen.pi32Pitch[0] = img->widthStep; offscreen.ppu8Plane[0] = (MUInt8*)img->imageData; break; case ASVL_PAF_DEPTH_U16: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img->width; offscreen.i32Height = img->height; offscreen.pi32Pitch[0] = img->widthStep; offscreen.ppu8Plane[0] = (MUInt8*)img->imageData; break; case ASVL_PAF_GRAY: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img->width; offscreen.i32Height = img->height; offscreen.pi32Pitch[0] = img->widthStep; offscreen.ppu8Plane[0] = (MUInt8*)img->imageData; break; default: return 0; } return 1; }
Mat 转 ASVLOFFSCREEN
int ColorSpaceConversion(MInt32 format, cv::Mat img, ASVLOFFSCREEN& offscreen) { switch (format) //原始图像颜色格式 { case ASVL_PAF_I420: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img.cols; offscreen.i32Height = img.rows; offscreen.pi32Pitch[0] = img.step; offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0] >> 1; offscreen.pi32Pitch[2] = offscreen.pi32Pitch[0] >> 1; offscreen.ppu8Plane[0] = img.data; offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0]; offscreen.ppu8Plane[2] = offscreen.ppu8Plane[0] + offscreen.i32Height * offscreen.pi32Pitch[0] * 5 / 4; break; case ASVL_PAF_YUYV: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img.cols; offscreen.i32Height = img.rows; offscreen.pi32Pitch[0] = img.step; offscreen.ppu8Plane[0] = img.data;; break; case ASVL_PAF_NV12: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img.cols; offscreen.i32Height = img.rows; offscreen.pi32Pitch[0] = img.step; offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0]; offscreen.ppu8Plane[0] = img.data; offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height; break; case ASVL_PAF_NV21: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img.cols; offscreen.i32Height = img.rows; offscreen.pi32Pitch[0] = img.step; offscreen.pi32Pitch[1] = offscreen.pi32Pitch[0]; offscreen.ppu8Plane[0] = img.data; offscreen.ppu8Plane[1] = offscreen.ppu8Plane[0] + offscreen.pi32Pitch[0] * offscreen.i32Height; break; case ASVL_PAF_RGB24_B8G8R8: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img.cols; offscreen.i32Height = img.rows; offscreen.pi32Pitch[0] = img.step; offscreen.ppu8Plane[0] = img.data; break; case ASVL_PAF_DEPTH_U16: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img.cols; offscreen.i32Height = img.rows; offscreen.pi32Pitch[0] = img.step; offscreen.ppu8Plane[0] = img.data; break; case ASVL_PAF_GRAY: offscreen.u32PixelArrayFormat = (unsigned int)format; offscreen.i32Width = img.cols; offscreen.i32Height = img.rows; offscreen.pi32Pitch[0] = img.step; offscreen.ppu8Plane[0] = img.data; break; default: return 0; } return 1; }
举例说明
这里引用了虹软官网文档中的示例,但使用了上述的图像格式转换方法。
//opencv方式裁剪图片 void CutIplImage(IplImage* src, IplImage* dst, int x, int y) { CvSize size = cvSize(dst->width, dst->height);//区域大小 cvSetImageROI(src, cvRect(x, y, size.width, size.height));//设置源图像ROI cvCopy(src, dst); //复制图像 cvResetImageROI(src);//源图像用完后,清空ROI }
IplImage* originalImg = cvLoadImage("1280 x 720.jpg"); //图像裁剪,宽度作四字节对齐,若能保证图像是四字节对齐这步能够不用作 IplImage* img = cvCreateImage(cvSize(originalImg->width - originalImg->width % 4, originalImg->height), IPL_DEPTH_8U, originalImg->nChannels); CutIplImage(originalImg, img, 0, 0); //图像数据以结构体形式传入,对更高精度的图像兼容性更好 ASF_MultiFaceInfo detectedFaces = { 0 }; ASVLOFFSCREEN offscreen = { 0 }; //IplImage 转 ASVLOFFSCREEN ColorSpaceConversion(ASVL_PAF_RGB24_B8G8R8, img, offscreen); if (img) { MRESULT res = ASFDetectFacesEx(handle, &offscreen, &detectedFaces); if (MOK != res) { printf("ASFDetectFacesEx failed: %d\n", res); } else { // 打印人脸检测结果 for (int i = 0; i < detectedFaces.faceNum; i++) { printf("Face Id: %d\n", detectedFaces.faceID[i]); printf("Face Orient: %d\n", detectedFaces.faceOrient[i]); printf("Face Rect: (%d %d %d %d)\n", detectedFaces.faceRect[i].left, detectedFaces.faceRect[i].top, detectedFaces.faceRect[i].right, detectedFaces.faceRect[i].bottom); } } //释放图像内存,这里只是作人脸检测,若还须要作特征提取等处理,图像数据不必释放这么早 cvReleaseImage(&img); } cvReleaseImage(&originalImg);
我的总结 :经过研究发现V3.0 版本SDK使用老接口也是能够正常使用的,新接口对更高字节对齐的图像兼容性更好。
Demo可在虹软人脸识别开放平台下载