This robot uses AI to find Waldo, thereby ruining Where’s Waldo

If you’re totally stumped on a page of Where’s Waldo and ready to file a missing persons report, you’re in luck. Now there’s a robot called There’s Waldo that’ll find him for you, complete with a silicone hand that points him out.

Built by creative agency redpepper, There’s Waldo zeroes in and finds Waldo with a sniper-like accuracy. The metal robotic arm is a Raspberry Pi-controlled uArm Swift Pro which is equipped with a Vision Camera Kit that allows for facial recognition. The camera takes a photo of the page, which then uses OpenCV to find the possible Waldo faces in the photo. The faces are then sent to be analyzed by Google’s AutoML Vision service, which has been trained on photos of Waldo. If the robot determines a match with 95 percent confidence or higher, it’ll point to all the Waldos it can find on the page.

Google’s Cloud AutoML has been available since January to let users train their own AI tools without any previous coding knowledge. The drag-and-drop tool lets anyone create an image recognition tool, which has a variety of use cases such as categorizing photos of ramen by the shops they came from. You can catch a glimpse of this process in the video above, in which different photos of Waldos are fed into the software.

Matt Reed, the Creative Technologist at redpepper who shepherded the project, explained via email: “I got all of the Waldo training images from Google Image Search; 62 distinct Waldo heads and 45 Waldo heads plus body. I thought that wouldn’t be enough data to build a strong model but it gives surprisingly good predictions on Waldos that weren’t in the original training set.” Reed was inspired by Amazon Rekognition’s ability to recognize celebrities, and wanted to experiment on a similar system which supported cartoons. He had no prior experience with AutoML, and it took him about a week to code the robot in Python.


To me, this is like the equivalent of cheating on your math homework by looking for the answers at the back of your textbook. Or worse, like getting a hand-me-down copy of Where’s Waldo and when you open the book, you find that your older cousin has already circled the Waldos in red marker. It’s about the journey, not the destination — the process of methodically scanning pages with your eyes is entirely lost! But of course, no one is actually going to use this robot to take the fun out of Where’s Waldo, it’s just a demonstration of what AutoML can do. Reed listed a few more possible applications: “Maybe a fun use would be seeing what cartoon character the AI thinks you look closest to? Maybe could detect comic book forgeries?”

Redpepper’s video description boasts: “While only a prototype, the fastest There’s Waldo has pointed out a match has been 4.45 seconds which is better than most 5 year olds.” If this is a competition, we really can’t win against the machines.


The hand slamming the page is awesome lol

Where’s Wally sounds so much better than Where’s Waldo

fight me Americans

I came to say a similar thing. I will fight with you!

At first I always thought Where’s Waldo was a cheap knock off…

I also concur.

Attaching a hand that looked more like a pointing finger than something pulled of a 6 month old dead body would’ve been nicer.

But not as funny

I use to sell books and had parents who hated the Waldo books because supposed "hours of fun" were over in a few minutes because their five year old would open the books and go "there he is" turn page "there he is" turn page "there he is" and so on.
So "yay" a computer can now do what any preschooler could do years ago.

if they didn’t explain to the kids the other stuff they had to find then it’s their loss.

besides I remember as a kid being in awe of the art style. loved it then, love it now.

I was waiting for it to identify the huge Waldo on the top left of the page near the postcard.

It also apparently uses a waldo to find a Waldo.

(Those robot arms used to be called ‘waldos’ when they were hand powered. )

View All Comments
Back to top ↑