Known for their proficiency at making breakfast foods, Technical University of Munich robots James and Rosie have expanded their repertoire to include sandwiches and popcorn. The robots are given the steps required to perform simple cooking tasks, then make decisions based on inferential reasoning and external inputs rather than relying on preprogrammed commands. To prepare the bread for the sandwich, for example, Rosie is programmed to know that it must be toasted, and that it is located on the cutting board. It then finds the bread with a Kinect sensor and slots it into the toaster. To find tools and ingredients, Rosie matches the video input to shapes or colors associated with things like forks or salami.

The system is far from perfect — Rosie, for example, wasn't able to find a piece of bread after fumbling it, and the process is a little slow. Nonetheless, the robots represent a step towards machines that can deduce how to complete tasks rather than being given specific, step-by-step instructions. See the full cooking process in the first video, or the sped-up version below that.