Skip to main content

    MIT team developing gesture recognition for unmanned aircraft

    MIT team developing gesture recognition for unmanned aircraft


    MIT researchers have developed a gesture recognition system for use on Aircraft Carriers.

    Share this story

    MIT aircraft gesture recognition
    MIT aircraft gesture recognition

    MIT scientists are developing a camera system to enable an unmanned aircraft to respond to gestures from the crew on the deck of an aircraft carrier. The team uses a combination of proprietary software and a stereo camera to detect posture, arm, and hand movements, and compares the data against standard Air Force gestures. The system is very similar to Microsoft's Kinect, which unfortunately wasn't available when the project began. Although the technology certainly belongs in the same family tree, the researchers face some unique challenges in accurately tracking the crew members: an aircraft carrier's deck is not a static environment, and the crew is always on the move. Because of this, it's not possible for the system to be constantly tracking movement, so instead it focuses on key body-pose sequences.

    An algorithm has been developed that focuses on a finding one of 24 gestures in sequences around 60 long. It analyses video in small chunks, looking at overall body shape and arm position to weigh up the probability that one of the frames contains the start of a gesture. Once it thinks it has found a key frame, it then analyses those immediately after it to weigh up the probability of a gesture taking place. After identifying the movement, it then looks at hand shape to determine the exact gesture taking place. Vetting the data in this way helps the system to improve its accuracy, but it still isn't quite up to scratch. The system is currently only 76 percent accurate — a long way off the level needed for military applications. Moving forward, the researchers are going to retool the software to analyze arm and hand position simultaneously, which they believe will fix the current issues.

    Today’s Storystream

    Feed refreshed 23 minutes ago Midjourneys

    Richard Lawler23 minutes ago
    Punt, Pass and Kick.

    Ken Dorsey’s blow up on Sunday was just one incident in a string of violent outbursts between NFL coaches, players, and those conveniently tossable Microsoft Surface tablets.

    Alex Cranz6:50 PM UTC
    After DART smashed into Dimorphos, I can’t stop thinking about the best “blow up an asteroid” story.

    LucasArts and Steven Spielberg came up with The Dig, a game about an astronaut, scientist, and journalist blowing up an asteroid and finding a spaceship inside, and they did it years before Bruce Willis, or NASA. You can still buy and play it on Steam!

    Asian America learns how to hit back

    The desperate, confused, righteous campaign to stop Asian hate

    Esther WangSep 26
    Richard Lawler6:38 PM UTC
    Everything looks better in slow motion.

    Apple’s Dynamic Island alert system isn’t sitting still around your iPhone 14’s front-facing camera array. We’ve been enjoying its contextual animations — and even an Android copycat — since it was unveiled, but take a look at it here, captured at 240fps, to see exactly how iOS applies animations that make it feel a bit more lively.

    External Link
    Russell Brandom5:47 PM UTC
    Oracle will pay $23 million to settle foreign bribery charges.

    The SEC alleges that Oracle used a slush fund to bribe officials in India, Turkey and the United Arab Emirates.

    This behavior is sadly common among software companies doing business overseas, and it’s not unique to Oracle. In March, a former Microsoft executive claimed the company spent as much as $200 million a year in bribes for foreign officials.

    External Link
    Emma Roth3:16 PM UTC
    Celsius’ CEO is out.

    Alex Mashinsky, the head of the bankrupt crypto lending firm Celsius, announced his resignation today, but not after patting himself on the back for working “tirelessly to help the company.”

    In Mashinsky’s eyes, I guess that means designing “Unbankrupt yourself” t-shirts on Cafepress and then selling them to a user base that just had their funds vaporized.

    At least customers of the embattled Voyager Digital crypto firm are in slightly better shape, as the Sam Bankman-Fried-owned FTX just bought out the company’s assets.

    Mary Beth Griggs2:46 PM UTC
    NASA’s SLS rocket is secure as Hurricane Ian barrels towards Florida.

    The rocket — and the Orion spacecraft on top — are now back inside the massive Vehicle Assembly Building. Facing menacing forecasts, NASA decided to roll it away from the launchpad yesterday.

    External Link
    Andrew J. Hawkins1:30 PM UTC
    Harley-Davidson’s electric motorcycle brand is about to go public via SPAC

    LiveWire has completed its merger with a blank-check company and will make its debut on the New York Stock Exchange today. Harley-Davison CEO Jochen Zeitz called it “a proud and exciting milestone for LiveWire towards its ambition to become the most desirable electric motorcycle brand in the world.” Hopefully it also manages to avoid the cash crunch of other EV SPACs, like Canoo, Arrival, Faraday Future, and Lordstown.

    The Verge
    Andrew Webster1:06 PM UTC
    “There’s an endless array of drama going on surrounding Twitch right now.”

    That’s Ryan Morrison, CEO of Evolved Talent Agency, which represents some of the biggest streamers around. And he’s right — as you can read in this investigation from my colleague Ash Parrish, who looked into just what’s going on with Amazon’s livestreaming service.