Earlier this year it was rumored that Apple had bought Faceshift, a Swiss startup which specializes in facial animation using real-time motion capture. Now, TechCrunch reports that Apple has confirmed the acquisition, handing out its customary statement: "Apple buys smaller technology companies from time to time, and we generally do not discuss our purpose or plans." Apple later confirmed the same to The Verge.
Faceshift uses hardware like Kinect and Intel RealSense
However, while Apple isn't saying anything about its intentions for Faceshift, the smaller firm's technology speaks for itself. Faceshift's software analyzes human facial expressions in real-time and transposes them onto CGI avatars — anything from robots to orcs to dogs. It does so without the need for facial reference markers and using only commercially available hardware. The company first showed off its methods using Microsoft's Kinect camera in 2012, and earlier this year demoed its tech with Intel's RealSense 3D cameras:
"Our background is in animation for content production, so we have Hollywood studios that use the technology to make animated characters come alive," says Brian Amberg, the company's CFO, in the promotional video above. Faceshift's technology was even reportedly used in the making of Star Wars: The Force Awakens (the company's software can be seen onscreen briefly in this video at 40 seconds, although it's not clear how extensively the tech was used).
Although Faceshift never released any consumer products using software, the most obvious application is creating expressive avatars for video calls and video games. "About two-thirds of our communication is non-verbal," says Faceshift's creative director Nico Scapel, "and digital characters can express these range of expressions." Apple already has considerable expertise in this area, with its purchase of Swedish facial recognition startup Polar Rose in 2010, and Israeli 3D sensor company PrimeSense in 2013. Combined, these technologies could be used for anything from facial authentication (like Windows Hello) to animated avatars for chatting in FaceTime.
Update, November 25th, 11.30AM ET: Updated to include Apple's verification of the purchase to The Verge.