This week, I finally managed to get some OpenCV working in C++. Borrowing someone else’s laptop, I managed to write a program that captured skin-colored blobs and also sent data to Scratch in one go. I’ll take a few screenshots when I get a chance!
Skin Segmentation
Initially, I attempted to use OpenCV’s dynamic skin detection example to filter skin color out of a video stream. I experimented a bit with some of the detector’s parameters, but found that I was getting worse results that the results I got filtering in an HSV color space. So, I went back to my previous method of sking detection.
Just as I did in my previous Python program from a few weeks ago, I was able to use OpenCV to transform an image into HSV color space and perform a basic skin filtering operation. Instead of using static images like in the Python program I wrote, I was able to use images from a webcam. I’m planning on doing a little tweaking of my filter values, but these seemed to work fairly well for a first pass.
Mat hueMask;
Mat satMaskLow;
Mat satMaskHigh;
threshold(frameChannels[0], hueMask, 50, 1, THRESH_BINARY_INV);
//mask sat 0.23-0.68
threshold(frameChannels[1], satMaskLow, 59, 1, THRESH_BINARY);
threshold(frameChannels[1], satMaskHigh, 173, 1, THRESH_BINARY_INV);
//59-173
filterFrame = hueMask & (satMaskLow & satMaskHigh);
I did notice that I seemed to have a bit of noise in my filtered image, but I think that may have been bad lighting conditions. In any case, my HSV color segmenter seemed to perform better than the dynamic one from OpenCV…although it also detected a skin-colorish pillow. Perhaps I’ll refine my filtering to only include *moving* skin-colored objects.
Blob Detection
The next step was to use OpenCV’s blob detector to grab the largest areas of skin in the webcam stream. I was able to get this up and running and could adjust the parameters to include only fairly-large blobs. I attempted to create KeyPoints of all blobs larger than a certain size and get the blobs’ radius. The radius values didn’t seem to be correct, however (they all seemed small!). Going to investigate this further.
Sending Data to Panther/Scratch
As I did previously, I used a nice socket library to send data from my C++ program to Scratch. This seemed to be working, but I noticed a bug! It seems like I am only able to send one command from C++ to Scratch – all subsequent commands are ignored. I think this may be an issue with how I format/terminate my commands in C++. I did not have this problem when I sent commands from a Python program to Scratch – maybe I am overlooking some subtlety of string formatting in C++? I plan to investigate this further this week.
Next week is spring break, so I’ll have lots of time to work!
To Do
-Fix Scratch socket issue (must be able to send more than one message)
-Experiment with blob detection
-Track only *moving* blobs (background subtraction from previous frame
-Get this running on *my* computer (have an error, but getting that sorted out tonight with some help)
