Create Your Own Haar Classifier for Detecting objects in OpenCV


The steps for training a haar classifier and detecting an object can be divided into :
  • Creating the description file of positive samples
  • Creating the description file of negative samples
  • Packing the positive samples into a vec file
  • Training the classifier
  • Converting the trained cascade into a xml file
  • Using the xml file to detect the object

Let us see all these steps in detail.
First of all,we need a large number of images of our object. One would end crazy if they resort to shooting   all the positive and negative images, which can run into  thousands. Here ffmpeg comes to rescue. We can shoot a small video (if possible 360 degree) of our object and then use ffmpeg to extract all the frames.  It is very helpful because a 25fps video of 1 minute will yield 1500 pictures….!!!  Thats cool…. isn’t it???.. To do that using ffmpeg, use the following synopsis of ffmpeg:

ffmpeg -i Video.mpg Pictures%d.bmp

For details about that visit this. It is better to use all the images in Bitmap format for improved performance, even though it takes a little more space since it is uncompressed image format.

Once the positive and negative images are prepared,it is better to put them in two different folders named something like positive and negative. The next step is the creation of description files for both positive and negative images.The description file is just a text file, with each line corresponding to each image.The fields in a line of the positive description file are: the image name, followed by the number of objects to be detected in the image, which is followed by the x,y coordinates of the location of the object in the image.Some images may contain more than one objects.

The description file of positive images can be created using the object marker program

The code of objectmarker is given below:

/***************objectmarker.cpp******************

Objectmarker for marking the objects to be detected  from positive samples and then creating the
description file for positive images.

compile this code and run with two arguments, first one the name of the descriptor file and the second one
the address of the directory in which the positive images are located

while running this code, each image in the given directory will open up. Now mark the edges of the object using the mouse buttons
  then press then press "SPACE" to save the selected region, or any other key to discard it. Then use "B" to move to next image. the program automatically
  quits at the end. press ESC at anytime to quit.

  *the key B was chosen  to move to the next image because it is closer to SPACE key and nothing else.....

author: achu_wilson@rediffmail.com
*/

#include
#include
#include

// for filelisting
#include
#include
// for fileoutput
#include
#include
#include
#include
#include

using namespace std;

IplImage* image=0;
IplImage* image2=0;
//int start_roi=0;
int roi_x0=0;
int roi_y0=0;
int roi_x1=0;
int roi_y1=0;
int numOfRec=0;
int startDraw = 0;
char* window_name="add save and load next exit";

string IntToString(int num)
{
    ostringstream myStream; //creates an ostringstream object
    myStream << num << flush;
    /*
    * outputs the number into the string stream and then flushes
    * the buffer (makes sure the output is put into the stream)
    */
    return(myStream.str()); //returns the string form of the stringstream object
};

void on_mouse(int event,int x,int y,int flag, void *param)
{
    if(event==CV_EVENT_LBUTTONDOWN)
    {
        if(!startDraw)
        {
            roi_x0=x;
            roi_y0=y;
            startDraw = 1;
        } else {
            roi_x1=x;
            roi_y1=y;
            startDraw = 0;
        }
    }
    if(event==CV_EVENT_MOUSEMOVE && startDraw)
    {

        //redraw ROI selection
        image2=cvCloneImage(image);
        cvRectangle(image2,cvPoint(roi_x0,roi_y0),cvPoint(x,y),CV_RGB(255,0,255),1);
        cvShowImage(window_name,image2);
        cvReleaseImage(&image2);
    }

}

int main(int argc, char** argv)
{
    char iKey=0;
    string strPrefix;
    string strPostfix;
    string input_directory;
    string output_file;

    if(argc != 3) {
        fprintf(stderr, "%s output_info.txt raw/data/directory/\n", argv[0]);
        return -1;
    }

    input_directory = argv[2];
    output_file = argv[1];

    /* Get a file listing of all files with in the input directory */
    DIR    *dir_p = opendir (input_directory.c_str());
    struct dirent *dir_entry_p;

    if(dir_p == NULL) {
        fprintf(stderr, "Failed to open directory %s\n", input_directory.c_str());
        return -1;
    }

    fprintf(stderr, "Object Marker: Input Directory: %s  Output File: %s\n", input_directory.c_str(), output_file.c_str());

    //    init highgui
    cvAddSearchPath(input_directory);
    cvNamedWindow(window_name,1);
    cvSetMouseCallback(window_name,on_mouse, NULL);

    fprintf(stderr, "Opening directory...");
    //    init output of rectangles to the info file
    ofstream output(output_file.c_str());
    fprintf(stderr, "done.\n");

    while((dir_entry_p = readdir(dir_p)) != NULL)
    {
        numOfRec=0;

        if(strcmp(dir_entry_p->d_name, ""))
        fprintf(stderr, "Examining file %s\n", dir_entry_p->d_name);

        /* TODO: Assign postfix/prefix info */
        strPostfix="";
        //strPrefix=input_directory;
        strPrefix=dir_entry_p->d_name;
        //strPrefix+=bmp_file.name;
        fprintf(stderr, "Loading image %s\n", strPrefix.c_str());

        if((image=cvLoadImage(strPrefix.c_str(),1)) != 0)
        {

            //    work on current image
            do

    {
                cvShowImage(window_name,image);

                // used cvWaitKey returns:
                //    =66        save added rectangles and show next image
                //    =27        exit program
                //    =32        add rectangle to current image
                //  any other key clears rectangle drawing only
                iKey=cvWaitKey(0);
                switch(iKey)
                {

                case 27:

                        cvReleaseImage(&image);
                        cvDestroyWindow(window_name);
                        return 0;
                case 32:

                        numOfRec++;
                printf("   %d. rect x=%d\ty=%d\tx2h=%d\ty2=%d\n",numOfRec,roi_x0,roi_y0,roi_x1,roi_y1);
                //printf("   %d. rect x=%d\ty=%d\twidth=%d\theight=%d\n",numOfRec,roi_x1,roi_y1,roi_x0-roi_x1,roi_y0-roi_y1);
                        // currently two draw directions possible:
                        //        from top left to bottom right or vice versa
                        if(roi_x0<roi_x1 && roi_y0<roi_y1)
                        {

                            printf("   %d. rect x=%d\ty=%d\twidth=%d\theight=%d\n",numOfRec,roi_x0,roi_y0,roi_x1-roi_x0,roi_y1-roi_y0);
                            // append rectangle coord to previous line content
                            strPostfix+=" "+IntToString(roi_x0)+" "+IntToString(roi_y0)+" "+IntToString(roi_x1-roi_x0)+" "+IntToString(roi_y1-roi_y0);

                        }
                        else
                                                    //(roi_x0>roi_x1 && roi_y0>roi_y1)
                        {
                            printf(" hello line no 154\n");
                            printf("   %d. rect x=%d\ty=%d\twidth=%d\theight=%d\n",numOfRec,roi_x1,roi_y1,roi_x0-roi_x1,roi_y0-roi_y1);
                            // append rectangle coord to previous line content
                            strPostfix+=" "+IntToString(roi_x1)+" "+IntToString(roi_y1)+" "+IntToString(roi_x0-roi_x1)+" "+IntToString      (roi_y0-roi_y1);
        }

                        break;
                }
            }
            while(iKey!=66);

            {
            // save to info file as later used for HaarTraining:
            //    \bmp_file.name numOfRec x0 y0 width0 height0 x1 y1 width1 height1...
            if(numOfRec>0 && iKey==66)
            {
                //append line
                /* TODO: Store output information. */
                output << strPrefix << " "<< numOfRec << strPostfix <<"\n";

            cvReleaseImage(&image);
            }

         else
        {
            fprintf(stderr, "Failed to load image, %s\n", strPrefix.c_str());
        }
    }

    }}

    output.close();
    cvDestroyWindow(window_name);
    closedir(dir_p);

    return 0;
}

Now its time to create the description file of negative samples.The description file of negative samples contain only the filenames of the negative images. It can be easily created by listing the contents of the negative samples folder and redirecting the output to a text file, ie, using the command:

ls > negative.txt

Now we can move on to creating the samples for training. All the positive images in the description file are packed into a .vec file. It is created using the createsamples utility provided with opencv  package. Its synopsis is:

opencv-createsamples -info positive.txt -vec vecfile.vec -w 30 -h 32 

my positive image descriptor file was named positive.txt and the name chosen for the vec file was vecfile.vec. Since a bottle was taken as a sample, minimum width of the object was selected as 30 and height as 32. the above command yielded a vec file. The contents of a vec file can be seen using   the following command:

  opencv-createsamples -vec vecfile.vec -show 

it opens the images in the vec file and use SPACEBAR to see the next image

Now everything is ready to start the training of the classifier. For that, we can use the opencv-haartraining utility. Its synopsis is:

opencv-haartraining -data haar -vec vecfile.vec  -bg negative.txt -nstages 30 -mem 2000 -mode all -w 30 -h 32

It creates a directory named haar and puts the training data into it. The arguments given defines the name of vecfile, background descriptor file,number of stages which is given here as 30, memory allocated which is 2 Gb, mode, width, height etc. There are many more options for the haartraining.This step is the most time consuming one. It took me days to get a usable classifier. Actually, I aborted training at 25 th stage because the classifier was found satisfactory at that stage.

Once the training is over, we are left with a folder full of training data (named haar in my case). The next step is to convert the data in that directory to an xml file. it is done using the convert_cascade program given in the opencv samples directory.

convert_cascade --size="30x32"   data  bottle.xml

Here data is the directory containing the trained data and bottle.xml is the xml file created. Now its time to use our xml file. The following code grabs frames from webcam and uses the classifer to detect the object.

/******************detect.c*************************/
/*
opencv implementation of object detection using haar classifier.

author: achu_wilson@rediffmail.com
*/


#include
#include "cv.h"
#include "highgui.h"

CvHaarClassifierCascade *cascade;
CvMemStorage            *storage;

void detect( IplImage *img );

int main( int argc, char** argv )
{
    CvCapture *capture;
    IplImage  *frame;
    int       key;
    char      *filename = "bottle.xml"; //put the name of your classifier here

    cascade = ( CvHaarClassifierCascade* )cvLoad( filename, 0, 0, 0 );
    storage = cvCreateMemStorage(0);
    capture = cvCaptureFromCAM(0);

    assert( cascade && storage && capture );

    cvNamedWindow("video", 1);

    while(1) {
        frame = cvQueryFrame( capture );

        detect(frame);

        key = cvWaitKey(50);
        }

    cvReleaseImage(&frame);
    cvReleaseCapture(&capture);
    cvDestroyWindow("video");
    cvReleaseHaarClassifierCascade(&cascade);
    cvReleaseMemStorage(&storage);

    return 0;
}

void detect(IplImage *img)
{
    int i;

    CvSeq *object = cvHaarDetectObjects(
            img,
            cascade,
            storage,
            1.5, //-------------------SCALE FACTOR
            2,//------------------MIN NEIGHBOURS
            1,//----------------------
                      // CV_HAAR_DO_CANNY_PRUNING,
            cvSize( 30,30), // ------MINSIZE
            cvSize(640,480) );//---------MAXSIZE

    for( i = 0 ; i total : 0 ) ; i++ )
        {
            CvRect *r = ( CvRect* )cvGetSeqElem( object, i );
            cvRectangle( img,
                     cvPoint( r->x, r->y ),
                     cvPoint( r->x + r->width, r->y + r->height ),
                     CV_RGB( 255, 0, 0 ), 2, 8, 0 );
                    
            //printf("%d,%d\nnumber =%d\n",r->x,r->y,object->total);


        }

    cvShowImage( "video", img );
}



36 thoughts on “Create Your Own Haar Classifier for Detecting objects in OpenCV

  1. How to use cascade.xml generated by opencv_traincascade. If we use directly give following error..
    OpenCV Error: Unspecified error (The node does not represent a user object (unknown type?)) in cvRead, file /build/buildd/opencv-2.1.0/src/cxcore/cxpersistence.cpp, line 4720
    terminate called after throwing an instance of 'cv::Exception'
    what(): /build/buildd/opencv-2.1.0/src/cxcore/cxpersistence.cpp:4720: error: (-2) The node does not represent a user object (unknown type?) in function cvRead

    Aborted

  2. use ./convert_cascade –size=”30×32″ data bottle.xml and get the convert_cascade code from opencv/samples/c folder….

  3. Mr. Wilson i have a seminar on this topic can you please give me the point to point explanation of the above program..please its a request..

  4. I thank you for the wonderful post. For my project i have to create xml file for hand detection.

    opencv-createsamples -info positive.txt -vec vecfile.vec -w 64-h 150

    after this step i'm getting the followong error. what is minimum width and height that has to be specified.

    Assertion failed (rect.width >= 0 && rect.height >= 0 && rect.x width && rect.y height && rect.x + rect.width >= (int)(rect.width > 0) && rect.y + rect.height >= (int)(rect.height > 0)) in cvSetImageROI, file /home/divyajs/Downloads/OpenCV-2.3.1/modules/core/src/array.cpp, line 3006
    terminate called after throwing an instance of 'cv::Exception'
    what(): /home/divyajs/Downloads/OpenCV-2.3.1/modules/core/src/array.cpp:3006: error: (-215) rect.width >= 0 && rect.height >= 0 && rect.x width && rect.y height && rect.x + rect.width >= (int)(rect.width > 0) && rect.y + rect.height >= (int)(rect.height > 0) in function cvSetImageROI

  5. Hi achu,
    when i am trying to execute final command – convert_cascade –size=”30×32″ data bottle.xml
    it gives following error in linux..

    convert_cascade: command not found
    Please help me….

  6. If i put the name of my classifier,this code (detect.c) can detect the object(in my case it's a pepsi can)?
    thanks in advance

  7. After this, I m unable to load the Image. It shows…

    Opening directory…done.
    Examining file .
    Loading image /home/shree/positive/.
    Examining file ..
    Loading image /home/shree/positive/..
    Examining file image9.jpeg
    Loading image /home/shree/positive/image9.jpeg
    Failed to load image, /home/shree/positive/image9.jpeg
    Examining file image11.jpeg
    Loading image /home/shree/positive/image11.jpeg
    Failed to load image, /home/shree/positive/image11.jpeg
    Examining file image8.jpeg
    Loading image /home/shree/positive/image8.jpeg
    Failed to load image, /home/shree/positive/image8.jpeg
    Examining file image7.jpeg
    Loading image /home/shree/positive/image7.jpeg

  8. Amazing work Achu!

    Just wondering, I'm using a c# wrapper for opencv (opencvsharp) – so I'm a little unfamiliar with the commands you are using. If I would want to follow the above post, what complier/software would I execute the commands in?

    Thanks in advance!

    SK

  9. i'm getting the following error . can u please help

    Data dir name: haar
    Vec file name: vecfile.vec
    BG file name: negative.txt, is a vecfile: no
    Num pos: 2000
    Num neg: 2000
    Num stages: 30
    Num splits: 1 (stump as weak classifier)
    Mem: 2000 MB
    Symmetric: TRUE
    Min hit rate: 0.995000
    Max false alarm rate: 0.500000
    Weight trimming: 0.950000
    Equal weights: FALSE
    Mode: BASIC
    Width: 30
    Height: 32
    Applied boosting algorithm: GAB
    Error (valid only for Discrete and Real AdaBoost): misclass
    Max number of splits in tree cascade: 0
    Min number of positive samples per cluster: 500
    Required leaf false alarm rate: 9.31323e-10

    Tree Classifier
    Stage
    +—+
    | 0|
    +—+

    Number of features used : 234720
    OpenCV Error: Unspecified error (Unable to read negative images) in cvCreateTreeCascadeClassifier, file /home/divyajs/Downloads/OpenCV-2.3.1/modules/haartraining/cvhaartraining.cpp, line 2425
    terminate called after throwing an instance of 'cv::Exception'
    what(): /home/divyajs/Downloads/OpenCV-2.3.1/modules/haartraining/cvhaartraining.cpp:2425: error: (-2) Unable to read negative images in function cvCreateTreeCascadeClassifier

  10. When i try to run this command its getting killed???!!!

    opencv_haartraining -data haar -vec vecfile.vec -bg /home/manoj/Videos/Negative/negative.txt -nstages 30 -mem 2000 -mode all -w 150 -h 80

    Data dir name: haar
    Vec file name: vecfile.vec
    BG file name: /home/manoj/Videos/Negative/negative.txt, is a vecfile: no
    Num pos: 2000
    Num neg: 2000
    Num stages: 30
    Num splits: 1 (stump as weak classifier)
    Mem: 2000 MB
    Symmetric: TRUE
    Min hit rate: 0.995000
    Max false alarm rate: 0.500000
    Weight trimming: 0.950000
    Equal weights: FALSE
    Mode: BASIC
    Width: 150
    Height: 80
    Applied boosting algorithm: GAB
    Error (valid only for Discrete and Real AdaBoost): misclass
    Max number of splits in tree cascade: 0
    Min number of positive samples per cluster: 500
    Required leaf false alarm rate: 9.31323e-10

    Tree Classifier
    Stage
    +—+
    | 0|
    +—+

    Killed

  11. You have to change:

    strPrefix=dir_entry_p->d_name;

    to:
    strPrefix=input_directory+dir_entry_p->d_name;

    For me it works. Good luck

  12. I am also having the same problem as Sadik above when I compile the code through terminal in Linux, this is what i get:

    Loading image simon9.jpg
    Examining file mmanson9.jpg
    Loading image mmanson9.jpg
    Examining file mmanson4.jpg
    Loading image mmanson4.jpg
    Examining file lisa10.jpg
    Loading image lisa10.jpg
    Examining file marie6.jpg
    Loading image marie6.jpg
    Examining file chris1.jpg
    Loading image chris1.jpg
    Examining file mmanson2.jpg
    Loading image mmanson2.jpg
    Examining file iroy10.jpg

    I then made the change that was mentioned above by replacing a line of code with strPrefix=input_directory.append(dir_entry_p->d_name);

    When I done this, i got:

    peter9.jpgamellanby3.jpgamellanby13.jpgolive2.jpgmartin13.jpgian1.jpgdhawley1.jpglisa4.jpgamellanby17.jpgpeter8.jpgandrew!22.jpggfindley2.jpgdavid6.jpggraeme17a.jpgmichael18.jpghin1.jpgmartin5.jpgpeter6.jpgirene2.jpgmartin8.jpgdavid2.jpgdhands11.jpggraeme4.jpgdpearson1.jpgiroy14.jpgpeter1.jpgdave_faquhar1.jpgjohn_thom1.jpglisa9.jpgpaul11.jpggordon4.jpgkirsty11.jpgcatherine18.jpghack.jpgpat18.jpgtock12.jpgian10.jpgtock11.jpgmartin1.jpgmarie12.jpgdlow17.jpgdlow18.jpgian9.jpgdhands8.jpgtock7.jpgpat10.jpgjim14.jpgneilg.jpgkim18a.jpgdpearson2.jpgmmanson6.jpgamellanby8.jpgpeter10.jpgdpearson14.jpgandrew5.jpgpaul4.jpgandrew13.jpgdavid13.jpgcatherine6.jpgandrew7.jpgtracy14.jpggeorge1.jpgkieran2.jpgstephen5.jpgkirsty3.jpgmichael1.jpgkim3.jpgamellanby12.jpgmnicholson2.jpgalister1.jpgcatherine5.jpgsimon11.jpgkieran4.jpgpat6.jpgdavid_imray1.jpgmarie13.jpgkay8.jpgkay1.jpgstephen7.jpgiroy11.jpgmilly4.jpglisa5.jpgbfegan16a.jpgstephen13.jpgkirsty9.jpgkay15.jpgbarry8.jpgjenni12.jpgchris3.jpgdlow7.jpgsimon5.jpg

    How can I get this to work?
    Any help would be greatly appreciated

  13. Object Marker: Input Directory: raw/data/directory Output File: devin1.txt
    Opening directory…done.
    Examining file .
    Loading image raw/data/directory.
    Failed to load image, raw/data/directory.
    Examining file ..
    Loading image raw/data/directory…
    Failed to load image, raw/data/directory…
    Examining file image0.bmp
    Loading image raw/data/directory…image0.bmp
    Failed to load image, raw/data/directory…image0.bmp
    Examining file output_info.txt
    Loading image raw/data/directory…image0.bmpoutput_info.txt
    Failed to load image, raw/data/directory…image0.bmpoutput_info.txt
    Examining file Thumbs.db
    Loading image raw/data/directory…image0.bmpoutput_info.txtThumbs.db
    Failed to load image, raw/data/directory…image0.bmpoutput_info.txtThumbs.db

    the object marker fails and cannot load the image
    how to fix it?
    thanks for your help

  14. This is really help, just what i was looking for. A question though, do i put any information in the description file when i run objectmarker or something? i ran everything but when i open the output file it is empty.

  15. It is using the key capital 'B' not lowercase 'b'. If you change all references to the key number '66' to another number (e.g. 98 for lowercase 'b') then it will change the key used for that function.

  16. Nice tutorial …. but what about “B” key it is not working in the objectmarker ,,and so the data is not saved in the output file !!!!

  17. It is called just in an OpenCV way – see the initial lines:

    cvNamedWindow(window_name,1);
    cvSetMouseCallback(window_name,on_mouse, NULL);

    This creates the window, and sets the mouse callback function to on_mouse

  18. You should take a look at opencv_traincascade.exe which has better support for multi-core processing, and runs much much faster than opencv-haartraining.exe

  19. The ObjectMarker is trying to load those images from the relative directory of the project. You can try modifying the following to load the absolute path of the images:

    Change:

    strPrefix=dir_entry_p->d_name;

    To:

    strPrefix=input_directory.append(dir_entry_p->d_name);

    Assuming you have set the input_directory to be the correct location of your images, it will load the images from the absolute path instead of the relative path to the project.

  20. It looks like you are not loading your cascade XML file:

    char *filename = “bottle.xml”; //put the name of your classifier here

    cascade = ( CvHaarClassifierCascade* )cvLoad( filename, 0, 0, 0 );

    Have you loaded up your xml file instead of the default 'bottle.xml' which will not exist with the project?

  21. Hello

    When i run the above code of detect.c, then debug was successful but then error occur in command prompt “Assertion failed: cascade && storage && capture, file: , line 22
    The Application has requested the run time to terminate it in an unusual way. Please contact the applications support team for more information”

    Please help me to resolve this error.

    Thanks in advance

  22. Thanks for shairing this document.

    How ever I can't use “object marker”. I compile it and try to use it from cmd on win7. Just I see such a things and finishes.

    Examining file img (6).jpeg
    Loading image img (6).jpeg
    Examining file img (6).JPG
    Loading image img (6).JPG
    Examining file img (7).jpeg
    Loading image img (7).jpeg
    Examining file img (7).jpg
    Loading image img (7).jpg
    Examining file img (8).jpeg
    Loading image img (8).jpeg
    Examining file img (8).jpg
    Loading image img (8).jpg
    Examining file img (9).jpeg
    Loading image img (9).jpeg
    Examining file img (9).jpg
    Loading image img (9).jpg
    Examining file img 1.jpeg
    Loading image img 1.jpeg

    How can I use ObjectMarker effectively?
    Thanks for your help.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s