This week, I finally managed to get some OpenCV working in C++.  Borrowing someone else’s laptop, I managed to write a program that captured skin-colored blobs and also sent data to Scratch in one go.  I’ll take a few screenshots when I get a chance!

Skin Segmentation

Initially, I attempted to use OpenCV’s dynamic skin detection example to filter skin color out of a video stream.  I experimented a bit with some of the detector’s parameters, but found that I was getting worse results that the results I got filtering in an HSV color space.  So, I went back to my previous method of sking detection.

Just as I did in my previous Python program from a few weeks ago, I was able to use OpenCV to transform an image into HSV color space and perform a basic skin filtering operation.  Instead of using static images like in the Python program I wrote, I was able to use images from a webcam.  I’m planning on doing a little tweaking of my filter values, but these seemed to work fairly well for a first pass.

Mat hueMask;

Mat satMaskLow;

Mat satMaskHigh;
threshold(frameChannels[0], hueMask, 50, 1, THRESH_BINARY_INV);
//mask sat 0.23-0.68
threshold(frameChannels[1], satMaskLow, 59, 1, THRESH_BINARY);
threshold(frameChannels[1], satMaskHigh, 173, 1, THRESH_BINARY_INV);
filterFrame = hueMask & (satMaskLow & satMaskHigh);

I did notice that I seemed to have a bit of noise in my filtered image, but I think that may have been bad lighting conditions.  In any case, my HSV color segmenter seemed to perform better than the dynamic one from OpenCV…although it also detected a skin-colorish pillow.  Perhaps I’ll refine my filtering to only include *moving* skin-colored objects.

Blob Detection

The next step was to use OpenCV’s blob detector to grab the largest areas of skin in the webcam stream.   I was able to get this up and running and could adjust the parameters to include only fairly-large blobs.  I attempted to create KeyPoints of all blobs larger than a certain size and get the blobs’ radius.  The radius values didn’t seem to be correct, however (they all seemed small!).   Going to investigate this further.

Sending Data to Panther/Scratch

As I did previously, I used a nice socket library to send data from my C++ program to Scratch.  This seemed to be working, but I noticed a bug!  It seems like I am only able to send one command from C++ to Scratch – all subsequent commands are ignored.  I think this may be an issue with how I format/terminate my commands in C++.  I did not have this problem when I sent commands from a Python program to Scratch – maybe I am overlooking some subtlety of string formatting in C++?  I plan to investigate this further this week.

Next week is spring break, so I’ll have lots of time to work!

To Do

-Fix Scratch socket issue (must be able to send more than one message)

-Experiment with blob detection

-Track only *moving* blobs (background subtraction from previous frame

-Get this running on *my* computer (have an error, but getting that sorted out tonight with some help)