OpenCV and Webcam

The first time I installed OpenCV, I intended to use it with Python and thus put the files in my Python27 directory.  At that point, the C++ sample files were able to use my on-board webcam to capture and display video.

After uninstalling/rebuilding OpenCV in a different directory, I was no longer able to get my webcam working.  I tried switching version of OpenCV and rebuilding, installing OpenCV patches, and reinstalling my webcam drivers – nothing seemed to work.  Ultimately, I ended up using a different webcam (borrowed from the ER) and was able to show video.

C++

This week, I also started learning the basics of C++ and generally just refreshing my programming knowledge.  I went through a tutorial and did a couple of small sample programs.  I also started experimenting a tiny bit with C++ and OpenCV (once I finally got my webcam working).

Sending Data to Panther/Scratch

In order to use OpenCV with Scratch (or any modified versions of Scratch, like Panther), I needed to find a way to transmit data from a C++ program to a Scratch program.  Handily, Scratch has an easy way to do this.  If you right click on one of Scratch’s sensor blocks, you have the option of enabling remote sensing.  This allows you to send data to Scratch from another program.  The Scratch Wiki provides a few code examples for sending data from programs written in Python, Java/Processing, and Flash.  They also provide a remote sensing protocol so that you can figure out how to properly format messages to send to Scratch in any language of your choosing.  With the protocol, you can create messages that will change variables in Scratch, broadcast a message, or get access to a Scratch program’s global variables.

I used the remote sensing protocol and a handy C++ socket class to successfully send commands to a Scratch program.   Here’s  a function I wrote to format a message to Scratch:

string sendScratchCommand(string cmd){

int n = cmd.length();
string header = “”;

//calculate message length data

char b1 = (char) ((n >> 24) & 0xFF);
char b2 = (char) ((n >> 16) & 0xFF);
char b3 = (char) ((n >> 8) & 0xFF);
char b4 = (char) (n & 0xFF);

header+=b1;
header+=b2;
header+=b3;
header+=b4;
header.append(cmd);

return header;
}

And here’s where I connect to Scratch via a socket connection:

int main() {

//enclose in a try/catch later

SocketClient s(“127.0.0.1″, 42001);

for (int i = 0; i<10; i++){

string to_send = sendScratchCommand(“broadcast \”beat\””);
s.SendLine(to_send);

}

return 0;

}

Finally, here’s a simply program I wrote to respond to my C++ program:

The Scratch cat changes color when it receives my message!

 

 

 

 

Splitting Video Stream

Because OpenCV AND Scratch will most likely both need webcam data, I needed to find a way to “split” my webcam stream.  I got some recommendations online and installed a free program, SplitCam.  SplitCam is supposed to allow you to use webcam data in many, many applications at once.  So far, I haven’t been able to route webcam data through SplitCam but I will keep exploring.  Ideally, I’d like to be able to split webcam data through my C++ programs, but I have no idea if that’s possible.

Accomplished:

-Use sockets in C++ to send broadcasts/data to Scratch/Panther

-Complete an introduction to C++

-Successfully use a webcam with OpenCV (had to use an external webcam)

-Explore options for splitting webcam stream (so that both Panther and OpenCV can access video data)

-Begin playing with OpenCV and webcam

To Do:

-Split webcam data!

-Get back on track with doing gesture recognition with OpenCV

-Talk to Nancy Hechinger about constructivism/borrow books