# OpenCVJniFaceDetect

**Repository Path**: chenjim/OpenCVJniFaceDetect

## Basic Information

- **Project Name**: OpenCVJniFaceDetect
- **Description**: Android基于OpenCV通过JNI识别并显示人脸位置
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 1
- **Created**: 2020-04-14
- **Last Updated**: 2022-09-30

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README


@[toc]

## Android基于OpenCV通过JNI识别并显示人脸位置
> OpenCV介绍地址：[https://docs.opencv.org/2.4/doc/tutorials/introduction/android_binary_package/O4A_SDK.html](https://docs.opencv.org/2.4/doc/tutorials/introduction/android_binary_package/O4A_SDK.html)  
 Android OpenCV Java Demo地址: [https://github.com/kongqw/OpenCVForAndroid](https://github.com/kongqw/OpenCVForAndroid)，其中人脸比对在[Master分支](https://github.com/kongqw/FaceDetectLibrary/tree/master)  
 本文基于JNI实现，源码地址：[Gitee：OpenCVJniFaceDetect ](https://gitee.com/chenjim/OpenCVJniFaceDetect)



### 设计思路
取Camera API 中onPreviewFrame 回调的YUV数据送到JNI，  
进一步用OpenCV的API识别并画出人脸区域，再通过ANativeWindow显示到surface。

### 代码设计说明
#### 效果如下
<img src="https://pic.chenjim.com/20200414173709.jpg?x-oss-process=style/blog"  width = "200" />

#### 代码结构如下
<img src="https://pic.chenjim.com/20200414170321.png?x-oss-process=style/blog" />

其中  
`app\src\main\cpp\include`  来自 `opencv-3.4.3-android-sdk\sdk\native\jni\include`  
`app\src\main\assets\lbpcascade_frontalface.xml` 来自 `opencv-3.4.3-android-sdk\sdk\etc\lbpcascades`  
`app\src\main\jniLibs\`  来自 `opencv-3.4.3-android-sdk\sdk\native\libs`

#### JNI识别人脸并画区域代码如下
```C++
    // nv21的数据
    jbyte *data = env->GetByteArrayElements(data_, NULL);
    //mat  data->Mat
    //1、高 2、宽
    Mat src(h + h / 2, w, CV_8UC1, data);
    //颜色格式的转换 nv21->RGBA
    //将 nv21的yuv数据转成了rgba
    cvtColor(src, src, COLOR_YUV2RGBA_NV21);
    // 可以将Mat的数据写到存储卡，正在写的过程 退出了，导致文件丢失数据
    //imwrite("/sdcard/src.jpg",src);
    if (cameraId == 1) {
        //前置摄像头，需要逆时针旋转90度
        rotate(src, src, ROTATE_90_COUNTERCLOCKWISE);
        //水平翻转 镜像
        flip(src, src, 1);
    } else {
        //顺时针旋转90度
        rotate(src, src, ROTATE_90_CLOCKWISE);
    }
    Mat gray;
    //灰色
    cvtColor(src, gray, COLOR_RGBA2GRAY);
    //增强对比度 (直方图均衡)
    equalizeHist(gray, gray);

    std::vector<Rect> faces;
    //定位人脸 N个
    tracker->process(gray);
    tracker->getObjects(faces);
    for (Rect face : faces) {
        //画矩形 分别指定 bgra
        rectangle(src, face, Scalar(255, 0, 0));
    }
```
#### 通过ANativeWindow显示RGBA数据到surface代码如下
```C++
        ANativeWindow_setBuffersGeometry(window, src.cols, src.rows, WINDOW_FORMAT_RGBA_8888);
        ANativeWindow_Buffer buffer;
        do {
            //lock失败 直接brek出去
            if (ANativeWindow_lock(window, &buffer, 0)) {
                ANativeWindow_release(window);
                window = 0;
                break;
            }
            //src.data ： rgba的数据
            //把src.data 拷贝到 buffer.bits 里去
            // 一行一行的拷贝
            //一行需要多少像素 * 4(RGBA)，当stride>width时直接memcpy会显示异常
            //memcpy(buffer.bits, src.data, buffer.stride * buffer.height * 4);
            
            UpdateFrameBuffer(&buffer, src.data);

            //提交刷新
            ANativeWindow_unlockAndPost(window);
        } while (0);
```
#### 将RGA数据填充到ANativeWindow_Buffer代码如下
参考自[Github：ndk-samples：webp_view.cpp#UpdateFrameBuffer](https://github.com/android/ndk-samples/blob/master/webp/view/src/main/cpp/webp_view.cpp)
```C++
*
 * UpdateFrameBuffer():
 *     Internal function to perform bits copying onto current frame buffer
 *     src:
 *        - if nullptr, blank it
 *        - otherwise,  copy to given buf
 *     assumption:
 *         src and bug MUST be in the same geometry format & layout
 */
void UpdateFrameBuffer(ANativeWindow_Buffer *buf, uint8_t *src) {
    // src is either null: to blank the screen
    //     or holding exact pixels with the same fmt [stride is the SAME]
    uint8_t *dst = reinterpret_cast<uint8_t *> (buf->bits);
    uint32_t bpp;
    switch (buf->format) {
        case WINDOW_FORMAT_RGB_565:
            bpp = 2;
            break;
        case WINDOW_FORMAT_RGBA_8888:
        case WINDOW_FORMAT_RGBX_8888:
            bpp = 4;
            break;
        default:
            assert(0);
            return;
    }
    uint32_t stride, width;
    stride = buf->stride * bpp;
    width = buf->width * bpp;
    if (src) {
        for (auto height = 0; height < buf->height; ++height) {
            memcpy(dst, src, width);
            dst += stride, src += width;
        }
    } else {
        for (auto height = 0; height < buf->height; ++height) {
            memset(dst, 0, width);
            dst += stride;
        }
    }
}

```

### 注意问题说明
- OpenCV 需要依赖 gnustl_static, NDK r18b中 gnustl_static被移除了，注意选择NDK17及以下版本
- 目前Camera预览数据是640X480，在骁龙820手机设备上单帧需要10ms左右，加大尺寸效率会降低
- OpenCV提供的模型对侧脸、脸部明暗相差大等情况的识别效果不是特别好