jeudi 27 avril 2017

Speed up face-detection by using pthread in C++11/Linux

I am the beginner of C++, and trying to do real-time face-detection by using OpenCV and SIMD Library(support ARM NEON) on raspberry pi 3 and camera. After using SIMD Library, the FPS is about 7. The FPS is OK but there is a little bit delay to the camera.

The original program flow:

  1. Global variables: cv::Mat imageToDoFaceDetection, cv::Point upperLeftPoint, cv::Point lowerRightPoint,
  2. Capture Image From Camera
  3. Detect face in given image and put the output the cv::Point(s)
  4. Draw the rectangle(s) on original image
  5. output to the window
  6. go back to (2) by while loop

Here is the code:

cv::Mat imageToDoFaceDetection;
cv::Point upperLeftPoint;
cv::Point lowerRightPoint;
int frameCount = 0;
int main(){
    while(1){
        imageToDoFaceDetection = captureImageFromCamera();
        if(frameCount%4==0){  //doing face detection on every 4 frames
            faceDetection();
        }
        drawRectangleOnImage(imageToDoFaceDetection,upperLeftPoint,lowerRightPoint);
        showImage(imageToDoFaceDetection);
        frameCount++;
    }
    return 0;
}
void faceDetection(){
    ...
    //Function will read the global var imageToDoFaceDetection,
    //And put the outputs to the upperLeftPoint, lowerRightPoint
    //Assume that the function is work
    return;
}

The FPS is OK but it suffer from delay to the camera. Because of time consumed by faceDetection(), image output to the window will be delayed.

I am trying to solve the delay problem by using pthread. I assumed the face in the images is moving slow, so I would like to create pthread and do faceDetection() on pthread and drawRectangleOnImage() using the location detected on frames before.

That mean image output to the window (showImage()) will not wait the faceDetection(), both threads will do the job on the same time. On one pthread drawRectangleOnImage() and showImage() will use the Points detected which may detected by a little number of frames before. On the other pthread, faceDetection() will update the Points continuously. Because the face is moving slow, so showing the delayed position of rectangle of faceDetection() is OK.

And I have tried fork() but the camera can be used by one process ONLY. The updated code:

cv::Mat imageToDoFaceDetection;
cv::Point upperLeftPoint;
cv::Point lowerRightPoint;
int frameCount = 0;
pthread_t tid;
int main(){
    While(1){
        imageToDoFaceDetection = captureImageFromCamera();
        //only 2 pthreads will be created.
        if(frameCount==0){
             pthread_create(&pid,NULL,pthread1_detection,NULL);
        }
        if(frameCount==2){
             pthread_create(&pid,NULL,pthread2_detection,NULL);
        }
        drawRectangleOnImage(imageToDoFaceDetection,upperLeftPoint,
                             lowerRightPoint);
        showImage(imageToDoFaceDetection);
        frameCount++;
    }
    return 0;
}

void* pthread1_detection(void* data){
    while(1){
         if(frameCount%4==0){
             faceDetection();
         }
    }
    pthread_exit(NULL);
}

void* pthread2_detection(void* data){
    while(1){
         if(frameCount%4==2){
             faceDetection();
         }
    }
    pthread_exit(NULL);
}

void faceDetection(){
    ...
}

After executing the code, it will show Segmentation Fault. I had been struggled on this problem for many hours........!!! (Even longer time for solving face-detection problem by using SIMD library....)

I would like to know the reason and the solution. Thank all and have a nice day.

Aucun commentaire:

Enregistrer un commentaire