This week was pretty frustrating on the whole, but I made some last-minute advances!  Installing OpenCV for C++ was a pain and now it seems that OpenCV is having problems accessing my webcam (it was working with a previous compilation).  Here’s a summary of what went on:

Installing OpenCV for C++

Because OpenCV with Python was 1) potentially slow and 2) starting to become more cumbersome by the minute, I decided to switch to using OpenCV with C++.  One caveat is that I don’t actually know C++ (not to mention I haven’t used OpenCV), but I have access to people who OpenCV professionally (yay!).

With some help, I got OpenCV installed (which included installing MingGW, helping me use CMake, and building and compiling).  After all of that I tried to run a sample face tracking program.  Unfortunately, my webcam is showing up as a black box!  The webcam was working previously, so I need to do some more investigation…

WebCam with Scratch-like languages:

I spent a lot of time looking at Squeak, being confused, and looking at other blocks-language alternative.  Integrating video into a Scratch-like language seemed really intimidating, and I was worried that it would be impossible for me to complete by myself.  However, after some searching, I found a solution (Panther, a Scratch extension).  Here are all of the options I considered:

OpenBlocks -A Java-based blocks language generator released by MIT.  OpenBlocks seemed very versatile and (I’m assuming) you could use it to create a visual representation of any other language.  I thought I might be able to use a more familiar language, handle video input/machine learning, and use OpenBlocks to create a nice blocks version.  HOWEVER, after downloading OpenBlocks (and emacs for editing purposes), I did not have much luck.  I also found OpenBlocks confusing – they had some Java Docs, but there was so much to go through.  Ick.

Squeak – Ahh Squeak.  Squeak is such a strange language (at least to me), so I was at a total loss when thinking of how to integrate video input.  The most recent version of Squeak is version 4.3, but Scratch runs on a modified version of Squeak 2.8.  There are several pre-written Squeak libraries, some even for video, but they were spottily documented and none worked with Squeak 2.8!

Scratch – I spent some time looking at the latest version of Scratch (and its Squeak code).  Scratch offers some webcam support.  Users can take a picture with a webcam and use the image as a sprite’s costume.  The only downside is that you cannot access the webcam in code or do any kind of streaming.  Still, I figured if you could take a picture with a webcam, there must be SOME support for streaming video.  Now I just need to have some convenient blocks for USING this functionality…which brings me to Panther.

(As a side note, I also made a simple block in Scratch)

Panther – Panther is an extension of Scratch that integrates more advanced features – including manipulation of the webcam!  Panther also lets you do file I/O, which may be useful for getting data to/from OpenCV.  I still have to explore Panther a lot more, but I think it’s a great platform for this project!

Here’s an example of me using webcam data in a Scratch-like program:

The blocks I used:

To summarize…

Things accomplished:

  • Install OpenCV for C++
  • Learn to make a block in Scratch
  • Find a way to integrate webcams and a Scratch-like language (WHEW)
  • Get a decent text editor for C++ programming (Notepad++ with added modules)

Things to do:

  • Get OpenCV to work with my webcam
  • Re-implement skin detection in C++
  • Use blob tracking in C++
  • Start working on gesture recognition
  • Send data from C++ program to Panther
  • Create more custom blocks in Panther