This week was pretty frustrating on the whole, but I made some last-minute advances! Installing OpenCV for C++ was a pain and now it seems that OpenCV is having problems accessing my webcam (it was working with a previous compilation). Here’s a summary of what went on:
Installing OpenCV for C++
Because OpenCV with Python was 1) potentially slow and 2) starting to become more cumbersome by the minute, I decided to switch to using OpenCV with C++. One caveat is that I don’t actually know C++ (not to mention I haven’t used OpenCV), but I have access to people who OpenCV professionally (yay!).
With some help, I got OpenCV installed (which included installing MingGW, helping me use CMake, and building and compiling). After all of that I tried to run a sample face tracking program. Unfortunately, my webcam is showing up as a black box! The webcam was working previously, so I need to do some more investigation…
WebCam with Scratch-like languages:
I spent a lot of time looking at Squeak, being confused, and looking at other blocks-language alternative. Integrating video into a Scratch-like language seemed really intimidating, and I was worried that it would be impossible for me to complete by myself. However, after some searching, I found a solution (Panther, a Scratch extension). Here are all of the options I considered:
OpenBlocks -A Java-based blocks language generator released by MIT. OpenBlocks seemed very versatile and (I’m assuming) you could use it to create a visual representation of any other language. I thought I might be able to use a more familiar language, handle video input/machine learning, and use OpenBlocks to create a nice blocks version. HOWEVER, after downloading OpenBlocks (and emacs for editing purposes), I did not have much luck. I also found OpenBlocks confusing – they had some Java Docs, but there was so much to go through. Ick.
Squeak – Ahh Squeak. Squeak is such a strange language (at least to me), so I was at a total loss when thinking of how to integrate video input. The most recent version of Squeak is version 4.3, but Scratch runs on a modified version of Squeak 2.8. There are several pre-written Squeak libraries, some even for video, but they were spottily documented and none worked with Squeak 2.8!
Scratch – I spent some time looking at the latest version of Scratch (and its Squeak code). Scratch offers some webcam support. Users can take a picture with a webcam and use the image as a sprite’s costume. The only downside is that you cannot access the webcam in code or do any kind of streaming. Still, I figured if you could take a picture with a webcam, there must be SOME support for streaming video. Now I just need to have some convenient blocks for USING this functionality…which brings me to Panther.
(As a side note, I also made a simple block in Scratch)
Panther – Panther is an extension of Scratch that integrates more advanced features – including manipulation of the webcam! Panther also lets you do file I/O, which may be useful for getting data to/from OpenCV. I still have to explore Panther a lot more, but I think it’s a great platform for this project!
Here’s an example of me using webcam data in a Scratch-like program:
The blocks I used:
To summarize…
Things accomplished:
- Install OpenCV for C++
- Learn to make a block in Scratch
- Find a way to integrate webcams and a Scratch-like language (WHEW)
- Get a decent text editor for C++ programming (Notepad++ with added modules)
Things to do:
- Get OpenCV to work with my webcam
- Re-implement skin detection in C++
- Use blob tracking in C++
- Start working on gesture recognition
- Send data from C++ program to Panther
- Create more custom blocks in Panther