SURF in OpenCV


Let us  now see what is SURF. SURF  stands for Speeded Up Robust Features. It is an algorithm which extracts  some unique keypoints and descriptors from an image. More details on the algorithm can be found here and a note on its implementation in OpenCV can be found here

SURF Keypoints of my palm

                    A set of SURF  keypoints and descriptors can be extracted from an image and then used later to detect the same image. SURF uses an intermediate image representation called Integral Image, which is computed from the input image and is used to speed up the calculations in any rectangular area.It is formed by summing up the pixel values of the x,y

co-ordinates from origin to the end of the image. This makes computation time invariant to change in size and is particularly useful while encountering large images. The SURF detector is based on the determinant of the Hessian matrix. The SURF descriptor describes how pixel intensities are distributed within a scale dependent neighborhood of each interest point detected by Fast Hessian

Object detection using SURF is scale and rotation invariant which makes it very powerful. Also it doesn’t require long and tedious training as in case of using cascaded haar classifier based detection. But the detection time of SURF is a little longer than Haar, but it doesn’t make much problem in most situations if the robot takes some tens of millisecond more for detection. Since this method is rotation invariant, it is possible to successfully detect objects in any orientation. This will be particularly useful in mobile robots where it may encounter situations in which it has to recognize objects which may be at different orientations than the trained image, say for example , the robot was trained with the upright image of an object and it has to detect a fallen object. Detection using  haar features fails miserably in this case.  OK, lets now move from theory to practical, the actual way things works.

OpenCV library provides an example of detection called find_obj.cpp. It can be found at the OpenCV-x.x.x/samples/c/ folder of the source tar file, where x.x.x stands for the version number. It loads two images, finds the SURF keypoints and descriptors , compares them and finds a matching if there is any. But this sample code will be a bit tough for beginners. So let us move slowly, step by step. As the first step , we can find the SURF keypoints and descriptors in an frame captured from the webcam. The code for the same is given below:

//*******************surf.cpp******************//
//********** SURF implementation in OpenCV*****//
//**loads video from webcam, grabs frames computes SURF keypoints and descriptors**//  //** and marks them**//

//****author: achu_wilson@rediffmail.com****//

#include
#include
#include
#include

using namespace std;
int main(int argc, char** argv)
{
  CvMemStorage* storage = cvCreateMemStorage(0);
cvNamedWindow("Image", 1);
int key = 0;
static CvScalar red_color[] ={0,0,255};
CvCapture* capture = cvCreateCameraCapture(0);
CvMat* prevgray = 0, *image = 0, *gray =0;
while( key != 'q' )
{
int firstFrame = gray == 0;
IplImage* frame = cvQueryFrame(capture);
if(!frame)
break;
if(!gray)
{
image = cvCreateMat(frame->height, frame->width, CV_8UC1);
}
//Convert the RGB image obtained from camera into Grayscale
cvCvtColor(frame, image, CV_BGR2GRAY);
//Define sequence for storing surf keypoints and descriptors
CvSeq *imageKeypoints = 0, *imageDescriptors = 0;
int i;

//Extract SURF points by initializing parameters
CvSURFParams params = cvSURFParams(500, 1);
cvExtractSURF( image, 0, &imageKeypoints, &imageDescriptors, storage, params );
printf("Image Descriptors: %d\n", imageDescriptors->total);

//draw the keypoints on the captured frame
for( i = 0; i total; i++ )
{
CvSURFPoint* r = (CvSURFPoint*)cvGetSeqElem( imageKeypoints, i );
CvPoint center;
int radius;
center.x = cvRound(r->pt.x);
center.y = cvRound(r->pt.y);
radius = cvRound(r->size*1.2/9.*2);
cvCircle( frame, center, radius, red_color[0], 1, 8, 0 );
}
cvShowImage( "Image", frame );

cvWaitKey(30);
}
cvDestroyWindow("Image");
return 0;

}

        The explanation of the code is straightforward. It captures a frame from the camera, then converts it into  grayscale ( because OpenCV SURF implementation works on grayscale images). The function cvSURFParams sets the various algorithm parameters. The function cvExtractSURF extracts the keypoints and desciptors into the corresponding sequences. Now circles are drawn with these keypoints as center and the strength of descriptors as the radius. Below are some images showing the keypoints captured.
SURF keypoints of my mobile phone
SURF keypoints of my mobile phone
The above picture shows the SURF keypoints captured by myself holding a mobile phone. The background wall has no strong intensity variations and hence no keypoints exist. An average of about 125 keypoints are detected in the above image and is shown in th terminal.  Below are some more images.

SURF Keypoints of my palm

From the above set of images, it can be clearly seen that SURF keypoints are those pixels whose average intensity values differ much greatly from the immediate neighbors, and by using the descriptor, a relation between the reliable keypoints and the neighboring pixels are obtained. Once the SURF descriptors and keypoints of two images are calculated, then they can be compared using one of the many algorithms like nearest neighbour , k-means clustering etc. SURF finds usage not only in object detection. It is used in many other applications like 3-D reconstruction.

Here are a few more screenshots of object recognition using surf:

21 thoughts on “SURF in OpenCV

  1. Hi I am currently writing a surf program to detect multiple objects, however when i tried to detect more than 4 objects or above, my cpu is almost hit 100% and hang there, i not sure which part is wrong or any other ways to solve this. do you mind to help me? my email is jinhonglu89@gmail.com

  2. I am usig SURF to extract features.I need to print the descriptors..can somebody help me how to convert the descriptor vector to matrix.I hav used reshape command..but its not working..this is wat i hav used..
    cvExtractSURF( image, 0, &imageKeypoints, &imageDescriptors, storage, params );
    and i hav used
    CvMat desc = CvMat(imageDescriptors).reshape(0, keypoints.size());
    But its not working…

  3. Hello! I am trying to implement object detection as a hobby project. I came across SURF features and wrote some code based on OpenCV's tutorials that tries to find my object in the video coming from my laptop webcam.

    Works, but the false-positive rate is too high. I mean even if my object is not present in front of the camera, the program manages to find scores of matches.

    Do you know of a remedy?

    Thanks!

  4. I am doing a project on object recognition.ie to recognize object from video samples.I tried to use SURF algorithm to recognize object.But as the object is not scale invariant i am not able to extract the features accurately..is there any good algorithm to extract features even if the objects are not scale invariant..

  5. i have been using SURF implementation to recognize object from images..Can somebody tell me how to get the descriptors rather than getting the number of descriptors..

  6. Hey can you guide me on how to use these features in classification of objects. I am presently working on a project where i need to classify car in the image according to their make(eg. Ford, Fiat, Tata etc.) How can i proceed after getting the features from the car image.

  7. Hello there. I'm from Brazil too and started using SURF just a few weeks ago. Anyway, i got some faster results using OpenMP with it (like 28% faster extraction). Congratulation for your tutorial, i'm really looking foward the second part.

  8. Hi, I’m brazilian, I’m based in your work to develop an program for my monography. I would like an opinion. I need to make an program to recognize banknotes. Do you recommend that I do this with HaarTraining? Do you think I will have success? Otherwise, I would like an opinion how could I have a good result! Thank you.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s