The Brooklyn Museum's app acts like AI, but there are humans behind it

If this visitor had previously wanted an answer to his question, his best bet would have been emailing the museum or hoping Google had some ideas. Ask fills that void, Shelley Bernstein, the museum’s vice director of digital engagement and technology, told The Verge. Hawkes and his five coworkers spend their days messaging visitors, answering their questions, and virtually riding along on visits. The app, developed in-house, has been out on iOS for nearly a year now in beta, and the museum launched its Android version this past week, along with the debut of its redesigned permanent collection.

There's an immense amount of brain power in the specialists’ back room, where they’ve worked for the past year. It's the ultimate communal work space. Books on Egyptian art are scattered around their shared table. A library cart sits in the corner, filled with even more books. Their massive Mac screens obscure their faces, unless they pop out from behind them to ask for help on a visitor's question. It’s like a 911 call center, but for burning art questions.

For all their effort, the team still often gets mistaken for robots. "Sometimes people think we’re machines," Hawkes said. "People ask if I’m real and to prove it." It’s harder than it sounds. He’s offered to tell jokes, but Siri can tell jokes, too. Maybe the only way to prove his humanity would be showing off his tax return, he said.

The team is constantly focused on learning more about the museum and how to assist visitors more efficiently. They go on walk-throughs with curators and one another when the museum's closed on Tuesdays to learn about the works and history behind them. Other lulls are spent putting together a Wiki page and compiling "snippets," which are essentially snapshots of the most interesting questions. These help them scale the whole process in the event that hundreds of visitors decide they want to use the app — many questions come up again and again. For the painting of a woman named Deborah Hall, for instance, visitors most often simply want to know who Deborah Hall is. The moderators put this question’s snippet and others on the museum’s website. (Hall was the daughter of a printer who also was briefly Benjamin Franklin’s business partner.) Each person can comfortably handle between six and seven conversations, and they typically talk with around 20 people on weekdays and 30 on weekends. A conversation can consist of a one-off question or a complete dissection of the art.

While there are a lot of recurring questions, there’s still plenty of room for surprises, and the moderators don’t always know what they’re getting into. Like once when a visitor kept frantically sending pictures of the art back to Megan Mastrobattista, a decorative arts specialist on the team. She wasn’t sure what the visitor wanted, so she kept responding to the photos with go-to blurbs about the art. The conversation continued like that for the entirety of their visit. It wasn’t until the end that Mastrobattista got a clue as to who was on the other side of the screen. The visitor, a young boy, sent her a selfie, along with a thanks for tagging along on his trip. He was, apparently, just excited to use the app, which doesn’t really come with a prescriptive plan for use.

The questions visitors ask the app also help the museum plan its future. The data ultimately influenced the redesign of the museum’s permanent collection. The Egyptian exhibit, for instance, often prompted questions about the room’s ceiling, which displayed the country’s zodiac. The museum never intended for the ceiling to be a focal point, so when the time came to rethink how the Egyptian art would be displayed, the zodiac was nixxed. Yes, the museum cut out visitors’ favorite part of the room based off data, but the curator wanted to "refocus the experience," Bernstein said.

collected Data influences the museum's exhibits

As the line slowly blurs between bot interactions and human ones, menial tasks and questions will consistently be relegated to bots. There will be a bot for everything, with virtual assistants Siri and Cortana now being joined by Facebook’s chat bot initiative. Finding an answer to a question has never been easier. But sometimes those answers aren’t readily available, and humans have to help. Facebook’s M, which seemed impressively smart when it debuted, has its replies reviewed by humans. As the Brooklyn Museum’s specialists know, if a chat program seems extremely intelligent, it might be because there are humans in a back room somewhere finding you the answer.

Photography: Amelia Holowaty Krales / The Verge