For my Research Algorithms course, I’m investigating the possibility of making AI more accessible to children, especially in a constructionist-oriented learning environment.  More specifically, I want to allow children to train and use machine learning techniques in the games and programs that they can write with languages like Scratch.

I have a few main goals:

  • Successfully simplify 1-2 machine learning techniques and make them easy to train and use.
  • Integrate these techniques into a Scratch-like programming language.
  • Allow novices to create programs using the simplified machine learning techniques.

Tasks I need to complete include:

  • Implement chosen machine learning techniques (gesture recognition and potentially facial feature tracking).  I’ll most likely be using OpenCV with Python for this task.
  • Learn how to create new blocks for a Scratch-like language, either by using Squeak or a Javascript-based block generator.  I will explore both options and ultimately choose the option that allows for the best integration with my python script while still letting kids make fun programs.
  • Integrate webcam data, code for training/running machine learning algorithms, and Scratch-like language.

Initial Plan for the Semester:

Feb 9 – 16 – Work with OpenCV to implement basic gesture recogntion with HMMs (requires skin filtering, and implementing forward-backward algorithm) – may be offline.
Feb 16 – 23 – Continue working on gesture recognition.  Try to get an online method working with webcam.  Explore training and identifying different gestures – how can users be guided to train gestures that are not overly complex?
Feb 23 – 29 – Investigate both Squeak and Javascript-based blocks.  Examine how to send data from a Python script to desired platform.  For Squeak, gauge how feasible embedding video in a Scratch-like program would be (email around as well).
March 1-8 – Begin creating my own blocks in either Squeak or JavaScript.

March 8-15 – Attempt to use either Squeak or JS blocks to create a simple program using gesture recognition (two programs need not be well-integrated).

March 15-22 – Work on more seamlessly integrating blocks language, webcam stream, and Python script.

March 22-29 -If feasible, work on implementing a second machine learning algorithm (most likely facial feature tracking).

March 22 – April 5 – Make adjustments to programs and write a couple of sample programs.’