Waldo Success                                                                                     

The green LED  is viewed as white!

It turns out that the green LED was not the best way to detect Waldo’s hand position. The bright LED caused the camera to go to total light saturation at that spot, so the camera registered pixel RGB levels of 255, 255, 255, which is white, not green. The background is already white. I could blink the LED and look for the spot that changed brightness but I would need to capture and process two images per move and would at least double the detection time.

Black ink has some blue pixels!

What I need to use for location detection, is a unique color. I have covered Waldo’s hand with a Child’s red glove that has green lettering on the back, so those colors are no good. I tried seeking out the color blue, but unhappily, the black pen ink, when looked at by the camera, is digitalized into a whole host of pixel colors, including blue. 

Yellow target between thumb & finger,.

There are very few yellow pixels in the images, so I glued a bright yellow 3mm plastic disk at he head of the pen and used an algorithm looking for small blocks of pixels (3x3) that had values of blue < 20, Green > 120 and Red > 120. Since most pixels in the images have lots of blue, those blocks are quickly eliminated and the hand position detection takes only about a half second per captured image.

Yellow target pixels.

I had to make a crazy math function that uses the current stepper position of Waldo’s hand, the current position of the yellow dot and the Sudoku image cell’s target position to calculate an estimate of where to next move Waldo’s hand. The XY steppers moves are not exact but they get the hand to within 90% of where it should be. I capture another image and move the hand again to within 99% of the target position. I do three consecutive moves to each target position.

Waldo filled-in puzzle.

 So the steppers move the pen to the center of an empty Sudoku square and the hand servos fill in the number. The process is very similar to how humans write and the hand has an uncanny human look about it. It takes about three seconds to move to each cell location and write a digit. The writing process takes about 3.5 minutes for a Sudoku puzzle. The whole process, from capturing the first image of the Sudoku to completely filled in puzzle, takes 4 minutes and 40 seconds.

Check out the amateur video.