For my Research Algorithms course, I’m investigating the possibility of making AI more accessible to children, especially in a constructionist-oriented learning environment. More specifically, I want to allow children to train and use machine learning techniques in the games and programs that they can write with languages like Scratch.
I have a few main goals:
- Successfully simplify 1-2 machine learning techniques and make them easy to train and use.
- Integrate these techniques into a Scratch-like programming language.
- Allow novices to create programs using the simplified machine learning techniques.
Tasks I need to complete include:
- Implement chosen machine learning techniques (gesture recognition and potentially facial feature tracking). I’ll most likely be using OpenCV with Python for this task.
- Learn how to create new blocks for a Scratch-like language, either by using Squeak or a Javascript-based block generator. I will explore both options and ultimately choose the option that allows for the best integration with my python script while still letting kids make fun programs.
- Integrate webcam data, code for training/running machine learning algorithms, and Scratch-like language.
Initial Plan for the Semester:
Feb 9 – 16 – Work with OpenCV to implement basic gesture recogntion with HMMs (requires skin filtering, and implementing forward-backward algorithm) – may be offline.
Feb 16 – 23 – Continue working on gesture recognition. Try to get an online method working with webcam. Explore training and identifying different gestures – how can users be guided to train gestures that are not overly complex?
Feb 23 – 29 – Investigate both Squeak and Javascript-based blocks. Examine how to send data from a Python script to desired platform. For Squeak, gauge how feasible embedding video in a Scratch-like program would be (email around as well).
March 1-8 – Begin creating my own blocks in either Squeak or JavaScript.
March 8-15 - Attempt to use either Squeak or JS blocks to create a simple program using gesture recognition (two programs need not be well-integrated).
March 15-22 - Work on more seamlessly integrating blocks language, webcam stream, and Python script.
March 22-29 -If feasible, work on implementing a second machine learning algorithm (most likely facial feature tracking).
March 22 – April 5 – Make adjustments to programs and write a couple of sample programs.’
