OBJECT DETECTION IN OPENCV USING HAARTRAINING


OPENCV is an open source image processing library developed by Intel and currently maintained by WillowGarage. It is very powerful and can be coded in C/C++ or Python. I chose C, as it is more comfortable to me.
Bottle Detection

Object detection can be done using SURF or by using  Haartraining using Viola Jones algorithm to form a cascade file. Even though detection using Haartraining is more difficult to train  compared to SURF, it is less error prone and of course faster to run in real time.I used Haartraining to detect  a bottle. I just experimented. The first step was to  take some positive and negative images, I took about 180 photos containing the bottle to be detected (positive images) and about 260 background images which did not have the bottle (negative images). It was really a small  number of images compared to about 5000 images used to train the classifier for face detection.STEP

The next step was to mark the object in the positive set of images using Objectmarker tool to yield a description file, a description file for negative samples was also created. Then using createsamples utility provided with opencv, a training sample , ie, a vec file was created. Now I started Haartraining using the haartraining tool. 25 stages was defined. It was going really fast upto the 15th stage, then gradually it slowed down, in fact training the 25th stage took me about 2 days. The whole process  took about 4 days, after which the xml file was created. the xml file can also be created in between using the convert_cascade program provided with opencv. The screenshots at various stages are given below.
[

Output of the createsamples tool[/

An intermediate stage  during Haartraining

Intermediate output showing the haartraining progress

Here comes the outputs, the bottle in my hand is identified and encircled

some more............

The final outputs………

I am now working on an image processing robot that will track down objects, go near to them and pick them  and  handle them to the operator, upon his command to take something. It is for our college technical fest. Now I have to train some more cascades to detect various household objects

43 thoughts on “OBJECT DETECTION IN OPENCV USING HAARTRAINING

  1. hi
    nice tutorial i just detected the bottle and some other objects now if i want to know which object is detects how can i do this in haar cascading … how to recognize which image is this ? i will be very great full to if you will help me

  2. hey,
    we have a b-i-g problem in our project. we try to run the haartraining on 3 pictures(3 negitive and 3 positive) just to try if it’s working’ and it’s stuck in cluster 11, we read that it took to u 2 days too, but we used only 3 pictueres, so we think maybe it’s stuck for other reason.. do u have an idea?

    thanks,
    sagit.

  3. Hey man. My project is quite similar to yours. So I would be really really glad if you can send me your bottle.xml file. I know I am asking a lot, but my laptop was working for 3 days now and still the classifier is not accurate enough. Also I have to submit the project in a few days. So thanks in advance mate 🙂

  4. hi dude..
    can u help me out in “HOW TO MAKE THE ROBOT MOVE TOWARD THE DETECTED OBJECT AND HOLD IT AND BRING IT BACK.”
    i have no idea how to do it.i have detected the object using your above suggested method.
    i dont know how to proceed.
    it would be helpful if u can give a detailed specification for my problem.
    thank you

  5. hello
    i am akshat and i am doing a project on computer interaction [ emulate mouse etc] using hand gesture recognition in openCV.
    do you have any projects on such topic
    if you have plz share
    since me and my partner are not very familiar with opencv and linux environment
    and our project mentor is useless.
    it will be really helpful ..

  6. hi i am new to opencv. can u plz elaborate the process. my project is to move robot based on gesture. i am getting how to detect different gesture. It is similar like ur project. THANKS

  7. hi. I'm new to opencv and i am doing a similar project for my undergrad thesis. What do you mean when you're talking about “stages for classifying”?

    Your help will be very much appreciated. Thank you.

  8. Hi, I would be happy to help you with the coding, but now I am a little busy with my exams. Detecting tennis balls wont be a big challenge. You dont need to do Haartraining and such stuff for this. They can be detected by simple color and shape based detection. You have to
    1)grab a frame from the camera
    2)Convert the RGB image to HSV so that intensity variations does not affect much
    3)Threshold the HSV image for the color ranges of the tennis ball. Now you will get those areas of the frame having the color of the tennis ball
    4) Smooth the image
    5)Apply Hough transform to get areas of circular shape. you can specify the maximum and minimum size limitations too.
    6) Get the pixel coordinates of the circle
    Better try it in python if you are a beginner.

  9. Hey,

    When using haartraining how do you find the correct value to put into the -npos parameter? I have a list of images in a text file with the additional crop info, eg;

    ./Images/Postives/frontal_sitting_0006.jpg 2 112 60 36 28 142 76 25 20
    ./Images/Postives/frontal_sitting_0007.jpg 2 107 66 32 25 134 82 26 20
    ./Images/Postives/frontal_sitting_0008.jpg 2 95 74 31 24 123 89 27 21
    ./Images/Postives/frontal_sitting_0009.jpg 2 86 78 34 27 113 92 28 22
    ./Images/Postives/frontal_sitting_0010.jpg 2 107 97 23 18 77 83 32 25
    ./Images/Postives/frontal_sitting_0011.jpg 1 247 2 53 42
    ./Images/Postives/frontal_sitting_0012.jpg 3 75 109 26 20 48 96 31 24 202 8 61 48
    ./Images/Postives/frontal_sitting_0013.jpg 3 38 97 32 25 66 108 23 18 187 6 54 43
    ./Images/Postives/frontal_sitting_0014.jpg 3 172 11 56 44 33 100 29 23 58 108 25 20
    ./Images/Postives/frontal_sitting_0015.jpg 3 24 101 29 23 47 111 26 20 152 17 56 44

    I assumes the vec file contains the number of positive samples? Do I just leave the npos parameter out?

    Thanks

  10. Hi,

    I am trying to use haartraining to detect my hand. I have used 500 positive samples and 1000 negative samples. Haarperformance report was not good. None of the testing images detected. All were missed. Error was high. It created some mark in testing photo. But each photo had different place of marking and it was multiple detection in wrong place. I have also tried by taking very few images as both positive and negative. But all are same problem. haarperformance result is poor. Can you give some tips for getting better result by using haartraining?

  11. Hi There;

    Have you experimented with Positive Builder. If so can you guide me with the following question. I am using Positive Builder and instead of using an avi video file I want to use a folder that contains many images and I want it to load my images one by one just like taking frames from video. I know its not a big deal but if it is already available in Positive Builder this work would be redundant.

    Cheers.
    Touqeer

  12. I think it would be enough. You can use the convert cascade tool to create the xml file and use it to detect your object. You need not stop the training for using the convert cascade tool. If found successful you can stop the training…

  13. Hi, I did haar training on my laptop (ACER ASPIRE 5738) powered by intel core2duobprocessor 2.2Ghz, 4GB DDR2 RAM, Ubuntu 10.04 Lucid Lynx LTS OS.. and I allocated 2 GB of RAM for haartraining

    I also faced a situation similar to that described by you. The training of initial stages went on smoothly and fastly, after about 12 th stage, it became slower. It took me about 2/3 days to complete the 20th stage. I opted for 30 stages but I found that the classifier was satisfactory, so the training was aborted. The whole training took about 1 week, and killed my battery since the lap was continuously plugged in for 1 week.

  14. HI could you tell me what are the features of the pc that you used to do the haartraning? right now i`m using a pentium core i7 processor 2.8 GHZ, 12 gigabytes RAM, OS windows 7 and my sample consist of 1492 positive and 5000 negative images(aprox).

    curiosly as you, it`s gettin slower once i get through the 15th stage, right know i`m on 17(it took 6 hours aprox to calculate this levels) so i wonder if it will take the same time as you.

    thanks.

  15. Hi, for hand gesture recognition, all you have to do is to detect and get the x,y coordinates of the hand, and then map the changes in those x,y values when a gesture is performed. You would better use haar classifier to detect the hand. You may first try the classifier given in this link with the code given in my blog. Then you try to create your own classifier. Its all simple and best wishes.

  16. hi, i have just started using opencv2.1 on vs2010 for my college hand gesture recognition project search a lot online but the picture is not clear how to and what method to follow plz help

  17. Hi, I feel that using HaarTraining would be an overkill for recognizing banknotes. The training of the classifier is little difficult. It would be better if you try out template matching.

  18. Hi, I'm brazilian, I'm based in your work to develop an program for my monography. I would like an opinion. I need to make an program to recognize banknotes. Do you recommend that I do this with HaarTraining? Do you think I will have success? Otherwise, I would like an opinion how could I have a good result! Thank you.

  19. Hi, I haven't encountered an error like this, but I think that I can help.. Can you please describe something about your background description file?. My background description file constituted of just the negative sample image names, one filename per line. And that background descriptor file was placed in the same folder of the negative images. You can easily create the negative descriptor using the following command:
    ls > negative.txt
    in the linux terminal. It lists all the files and writes their names into a text file named negative. Then while running haartraining, the path of the negative file descriptor is given. Thats all, and it worked for me.

  20. Hi,

    I successfully created the vec file but when i try haartraining, I get the below error
    Invalid background description file.
    Wonder if you encounter this before?
    I would appreciate some pointers, thank you.

  21. Hii,I did haartraining on my laptop powered by Intel Core 2 Duo (2.2 GHz) and with 4 Gb DDR2 RAM and running ubuntu 10.04 64 bit. The vec file and a text file containing the name of negative images (background images) was created. The number of stages was set to 30, although I stopped haartraining at 26th stage because it was taking too much time and the results at 26th stage was satisfactory . 2000 Mb of memory was allocated to haartraining.

    In short the arguments given to haartraining are “opencv-haartraining -data h -vec positive/bmp/h.vec -bg negative/bmp/negative.txt -nstages 30 -mem 2000 -mode all -w 30 -h 32”

  22. Hi I'm Marco.

    I see that you used an small amount of images, I thought that haartrining would require more, but it looks pretty well (enough for me).

    I'm thinking of doing something similar, with a similar amount of images also, so I would like to ask you something: How do you recomend to set the parameters in haartrining.exe?

    Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s