The Verge - Features

Google Glass apps: everything you can do right now

2013-05-20T16:00:06Z

Google Glass isn’t ready for prime time. Even Google knows this, which is why it hasn't shipped to the masses yet. Instead, Google floated a few units to “Explorers,” glorified guinea pigs who can enjoy the joys and trials of this cuttingest edge of cutting edge technologies. But nascent or not, Glass exists, and it works. Or at least it "works." Developers are still getting their feet wet, high-profile apps like Twitter and Facebook feel more like experiments than finished products, and bugs aren’t the exception, they’re the rule. But, you know, the thing turns on, and hears you say "Okay, Glass," and eagerly awaits your next command. Beyond the home screen, it’s up to Glass Explorers to wade through the good apps, the bad apps, and the broken apps, and we're right there with them. We went exploring, and this is what Glass can do; right here, right now.

And for those of you without Glass, hopefully we can help you live vicariously through our misadventures and humble GIF illustrations. Glass is a strange beast, and it can take a while to get used to and understand what it’s trying to do. Maybe we’ll make you jealous, or maybe you’ll decide Glass is a worthless piece of trash. Either way, we hope we can shed light on the mystery, the wonder, and the social awkwardness that is Glass.

We’ll be updating this list as new apps come out, and old apps are updated or made obsolete, so keep checking back. And if you see an app we haven’t covered yet, make sure to let us know!

Sticky TOC engaged! Do not remove this!

Glass basics

Glass’ basic paradigm is one of a "timeline" full of "cards." Each card, when tapped, can reveal more cards, or present actions like "reply" or "delete." Some cards can be "pinned," which places them to the left of the home screen. Otherwise, cards are sorted chronologically to the right of the home screen.

While Glass’ best known interaction method is the verbal "Okay, Glass" prompt, most of the UI can only be operated by swipes and taps. Outside of the homescreen, the only time you’ll be speaking is when you’re composing a text reply to a card in your timeline (like an email message, or a tweet). While voice might seem like a gimmick, it’s actually preferable to the hypersensitive touchpad at times — we’ve accidentally tapped to share a photo with the wrong Google+ contact a dozen times, simply because Glass registers a tap instead of a swipe. At least when you’re talking you get a chance to cancel the action if you’re misheard.

Glass’ very simplicity means there’s actually a steep learning curve: You have to use Glass how Google wants you to use Glass, or it just doesn’t work. Also, you have to be really good at swiping and tapping.

Setup

When you first boot up Glass, after a long stare at the Glass logo, you’re presented with a URL to visit on your phone or computer to set up the device: google.com/myglass. From there you log in to your Google account (@gmail.com addresses only, for now), and begin a breezy setup to pair your device to that account. The big moment is when you enter a Wi-Fi network, which generates a QR code. Glass’ camera will recognize the QR code automatically and then join that network. Congratulations, you’ve set up Glass!

From now on, the MyGlass site will be your home base for Glass configuration — the device itself offers little in the way of preferences, it’s more of a viewer for the content and interactions you enable from MyGlass. It's also easy to overload your Glass with contacts and apps, and since deleting cards from your timeline is a pain, you want to make sure all your incoming content is quality.

We recommend you do the setup from a computer, because the next step is a pain: adding contacts. You have to enter a name manually, and then Glass pulls a contact card from your Gmail account. If your Gmail contacts are a mess (ours sure are), you’ll probably have to tweak each Gmail contact by hand to make sure the correct email address is at the top. And that it’s the right version of that contact — it’s all a little mysterious, but you’ll know you’ve got the right one if a profile pic shows up in MyGlass. Even then, Glass won’t play nice with everybody: for instance, the voice recognition won’t register "Jordan Oplinger" no matter how hard we try.

Confusingly, there is a second set of contacts, called Sharing Contacts, that’s simply comprised of your Google+ friends and Circles. Those require less management, but some Glass apps work only through adding themselves to your Sharing Contacts, so you’ll have to watch out for that and be sure to switch off any apps or friends you don’t want to accidentally share to.

Next you’ll want to install some apps. MyGlass offers a few apps by default, like Twitter, Facebook, New York Times, CNN, Google Now, and Path. You simply click the "on" button and the app will come alive. If you want other apps, there’s no app store to go to — you’ll have to hunt them down and install them yourself. Or you could refer to this handy little list.

We recommend you do the setup from a computer

MyGlass Android app

Google’s Android app for Glass doesn’t offer anything beyond the web app in terms of device management, but it does include two very important features: tethering and mirroring. Because Glass doesn’t have a cell radio, tethering to your phone is necessary for internet service on the go, and the MyGlass app makes it very simple. MyGlass finds nearby Glass devices and attempts to pair with them. A tap on your Glass to accept finishes the job. Unfortunately, while the setup is easy, keeping a connection isn’t, and we hope the Bluetooth reliability is a software problem that future updates can solve. Until then, a Wi-Fi hotspot might be your preferred solution, if your phone can handle the battery drain.

Mirroring is just a way to show what’s happening on your Glass screen on your Android screen — a "wow your friends feature" that’s much appreciated.

While the setup is easy, keeping a connection isn’t

Built-in apps

Glass is not an all-purpose computer, it's a Google computer. All of its core experiences are based on Google+, Google Search, and Gmail, and there's no way to change that. The more of a Google ecosystem adherent you are, the more utility you'll get out of Glass. If you're okay with that, Glass can be a really slick, natural experience. If you're not okay with that, you should probably be in the market for a different wearable computer.

"ok glass" launcher

The main screen of Google Glass shows a simple clock, and below it the words "ok glass." If you say "Okay, Glass," you’re sent into a launcher that lists Glass’ primary functions. There’s no need to pause after saying "Okay, Glass" before the next command. For instance, you could say "Okay, Glass, send a message to Joshua Topolsky" in one breath and Glass will keep up with you (if Joshua Topolsky is one of your contacts). Unfortunately, it’s easy to misspeak and watch Glass eagerly perform the wrong command — like sending a message to the wrong Joshua. Worse, Glass can hear other people talking in the vicinity, and is likely to pepper some of their words in as well. Depending on your friend set, tapping through the menus might be safer. At least Glass filters out expletives and "unsafe" searches, but a clever troll can still find ways to prompt a disgusting Google Image search if you're not careful — we only barely dodged a "horse diarrhea" query at the office the other day.

Glass can hear other people talking in the vicinity

Google Search

The first option in the launcher, Google Search, prompts you to "ask a question," but Glass doesn’t flinch if you just speak a regular search term. The results are wild cards, because for certain queries Glass delivers six beautiful, Glass-appropriate results, and others are simply a list of six unclickable links with short, usually useless descriptions. Glass Search works best when you do actually ask a question, like "How tall is Ryan Gosling?" instead of just "Ryan Gosling." When Glass knows the actual answer, it speaks the result out loud. (Ryan Gosling is 6’1", by the way). If you search for "images of Ryan Gosling" you get six photos of the man, and if you ask when the Knicks game is on Glass will tell you, but it doesn’t know how fast a cheetah or a giraffe runs.

Glass Search works best when you ask questions

"take a picture"

If you say "Take a picture," Glass will take a picture. It happens instantly, without any preview of what you’re about to get — the idea is to frame with your eyeballs, so hopefully you have Glass on straight. The photo is then flashed up in front of you, stashed in a new card, and synced privately to Google+. When you swipe over to the card and tap it, you get the option to share the photo with your Sharing Contacts (comprised of Google+ friends, Google+ Circles, and apps), or to delete it. Additionally, the photo is stashed in Glass’ physical storage, and can be manually transferred to your computer over USB.

The idea is to frame with your eyeballs

"record a video"

When you start recording, Glass will automatically start recording for 10 seconds, and with a tap you can cut the video short or extend recording indefinitely. After it’s captured, the video is stored on Google+ and shareable in the same way photos are — though not every service will accept shared video, and it's not always clear which ones will.

With a tap you can extend the recording indefinitely

"get directions to…"

If your search results include an item with an address, you can tap on it to get directions. Alternatively, you can request directions to an address or point of interest by speaking, "Get directions to" and your destination. The first time you do this, Glass will sternly warn you, "Do not manipulate this application while in motion," which you will tap to ignore and then be taken through turn-by-turn directions. The compass responds to your head motions, so you know exactly if you’re pointed the right way. You can tap again to see the entire route, or swipe over to choose between walking, driving, and biking directions — there are no public transportation directions available as of yet. If you exit out of the map, the destination will be pinned to the left of the "ok glass" screen so you can resume your trip at any time.

"Do not manipulate this application while in motion."

"send a message to…"

Saying "Send a message to" presents you with your list of hand-picked contacts. Then you simply speak your message and Google will transcribe it. When you stop talking you have a brief moment to swipe and cancel, and then the message is sent automatically — typos and all. It’s sent as a regular email from your Gmail account, with a "Sent through Glass" signature, and a card will show up in your timeline. Replies show up inside that card, just like with all Gmail conversations on Glass, and you can continue the conversation at any point by tapping "reply."

Speaking of Gmail, Glass shows some messages from your Gmail inbox, but not all of them. It seems queued off of emails that Gmail flags for your priority inbox, but that can be a little mysterious — like with all things, Glass won't become your primary Gmail interface, only a small supplement. The account we used Glass with wasn’t a high-volume account, but we’re curious how others will handle the influx of emails into their timelines.

The message is sent automatically — typos and all

"make a call to…"

If you’ve paired your phone with Glass, you can make a phone call to one of your contacts. It works fine, but you’ll have to decide for yourself if Glass’ bone conduction speaker and microphone are sufficiently loud for your conversation — much more than a quiet conversation nearby can easily drown you out.

Much more than a quiet conversation can easily drown you out

"hang out with…"

Hangouts couldn’t be easier on Glass. From the home screen you speak "Hang out with…" followed by the name of your chosen friend or circle, and they’ll be invited automatically. Conveniently, Google just unified Hangouts across its services, so your friend is more likely to notice your call — in our experience, an unsolicited hangout is typically an unfulfilled hangout. Also in our experience: Hangouts are horribly buggy and unreliable, so be sure to pick a very patient friend that doesn't mind attempting a dozen calls before one connects.

Once connected, you’re treated to a view of one of your Hangout participants at a time, based on whoever is speaking, while they see what your camera sees. With a tap you can enable the names of the participants, in case you’re chatting with strangers or your friends are wearing masks. Like with phone calls, it can be a little hard to hear Hangouts participants, especially when you’re talking to multiple people at once, but the novelty factor is high enough to tolerate the impracticality of it all.

The novelty factor is high enough to tolerate the impracticality

Google Now

Our Google Now card seems mostly concerned with the local restaurants available at any moment, but every Now experience is different, based on your location, activities, and whatever web activity of yours Google is tracking and Now-ifying these days — there are at least 25 "cards" that have been developed so far. Glass also has a very beautiful, simple weather widget that’s always available to the left of the home screen. It's Glass' reliance on serendipity that's both its greatest strength and greatest weakness, and Now is the case in point.

Every Google Now experience is different

Third-party apps

All third-party Google Glass apps ("Glassware," as Google calls them) are developed with what’s called the "Mirror API," which basically means all apps are presented in the form of cards in your timeline, not as core functionalities available from the main "ok glass" menu. No developer would be able to build an equivalent of Glass’ Map app, but they will be able to build a lot of New York Times app clones. Google wants developers to push HTML info to you, receive photos, videos, and text from you, and be happy about it. If history is any indication, developers won't be happy about it — just look at how long the iPhone lasted with web apps before Apple broke down and built a real SDK. Still, for now Glass is a content consumption and content sharing device, and it might stay that way for a while.

Of course, Google Glass can also be hacked to sideload Android apps (APKs), which can circumvent these limitations. We’ll mostly focus on legit apps, but some hacks are too good to pass up.

New York Times

Before Twitter and Facebook showed up, New York Times was the flagship third-party launch app for Glass — which was a little sad. Not that there’s anything wrong with the app: once activated from your MyGlass page, the Times will give you a new card once an hour with a roundup of new stories (there aren’t any preferences, so you can’t lower that frequency), along with an occasional lone breaking story. Inside the card you can scroll through the stories, and tap to have Glass read you a summary out loud, but that’s all there is. A share button wouldn’t go amiss, nor would an option to filter out subjects you just don't care about when you're wearing Glass — like politics, for instance.

A share button wouldn't go amiss

Twitter

Twitter’s app adds a Sharing Contact, which lets you share photos from Glass to your adoring public. Unfortunately, the app doesn’t let you do standalone text tweets, and you can’t add a description before you upload. You can, however, compose text tweets when you engage with @ replies from people who you follow. But you can’t @ reply yourself, so nothing you say will be visible to all your followers — just your photos. Any tweet can be favorited or retweeted as well.

It doesn’t let you do standalone text tweets

Facebook

Facebook’s Glass implementation is even more minimal than Twitter’s. Once activated, the app simply installs three new Sharing Contacts for distributing photo to the appropriate audience: Public, Friends, and Only Me — good luck never tapping the wrong one. Video uploads aren’t available, and while you can add a description to a photo after it’s uploaded (the posted photo boomerangs back to your timeline), you can’t see or respond to comments or likes. Also, Steven's name is Stephen, and we're not sure how to convince Glass of that fact.

Even more minimal than Twitter

Evernote

Evernote lets you share photos with your Evernote account, which show up in your inbox as a "Note from Glass." While that’s the extent of the sharing functionality — you can’t tag or comment on photos after they’re uploaded, though Evernote can run text recognition to transcribe your handwriting — it’s kind of a relief to have a service on Glass that’s not dedicated to publicizing your life. Perhaps more interestingly, from Evernote you can push a note to your Glass timeline — like a shopping list, for instance — which could help sync up your Glass life with your real life.

A relief to have a service not dedicated to publicizing your life

Tumblr

Like everyone else, Tumblr adds a Sharing Contact to beam photos to. Unlike everyone else, Tumblr actually offers a preferences pane on its site to customize your experience: you can pick which of your Tumblrs you’d like to share with (the name of that Tumblr is what the contact will be), and set the frequency of Dashboard updates. Unfortunately, if you follow a lot of Tumblrs, your timeline will be inundated with posts, even if you select the "some" option instead of the "all" option. At least GIFs work, which beats Google's own apps — even stills don't show up in Gmail.

Tumblr actually offers a preferences pane

Elle

When the Elle app installs you get a preference pane to pick what news you want to receive from the fashion magazine. You can also input your sign to receive your daily horoscope. Elle’s updates come in various forms, from the basic single news item (which can be read aloud), to a gallery of photos (you can add clothing items to your "wish list"), to a fancy horoscope card. Coincidentally, our first horoscope? "Geek is chic today…" Boy, we sure hope so.

Our first horoscope? "Geek is chic today…"

CNN

CNN offers news alerts on Glass, much like New York Times, but with more customizability. The CNN preference pane lets you pick which topics you’re interested in, set a range of times to receive alerts, and even shows you how many alerts to expect per topic. What sets CNN apart is the video it delivers alongside the report, which loads when you scroll over to it. As bizarre as many of Glass’ functions are, CNN video news pushed to your eyeball really feels like you’re living in the future.

CNN video news pushed to your eyeball

Glassagram

Despite the name, Glassagram isn’t an Instagram-connected service, it’s simply a method to apply Instagram-like filters to your photos before you share them with others. After you install Glassagram, it shows up as a Sharing Contact. Once enabled, it allows you to share a photo with Glassagram, which will then reply with a batch of cropped and filtered photos. You select from one of those photos and share it in turn with another Sharing Contact, like Twitter for instance — it's not elegant, but it works. Too bad Instagram will never see the results.

Apply Instagram-like filters to your photos

GlassTweet

While you can’t tweet text, GlassTweet makes it dead simple to tweet photos from Glass’ so-so camera. Once you’ve authorized the app and enabled it as a Sharing Contact, you simply tap to share a photo with it and it’s tweeted with the traditional #throughglass hashtag. The app feels pointless now that there’s an official Twitter app, but as we’ve seen on other platforms, people always like options in their Twitter clients. Perhaps GlassTweet can differentiate itself with the true Twitter killer app we're all waiting for: the one that makes text tweets.

Tweet photos from Glass' so-so camera

Fullscreen BEAM

This is a YouTube upload app, which adds a Sharing Contact to upload your videos to YouTube through the typical tap-to-share interface. Fullscreen BEAM is one of the few sharing apps with "features," which is to say it has a preferences screen on the website that allows you to choose your timezone, default your videos to public or private, and tweet a link to the video once it’s uploaded.

Upload your videos to YouTube

Glass To Facebook

Like GlassTweet, Glass To Facebook couldn’t be simpler… or more limited. Once enabled, you simply share a photo with Glass To Facebook and it will post the photo to your timeline with a "Posted through Glass" description. Another app made irrelevant by the introduction of official apps.

Couldn't be simpler… or more limited

ThroughGlass

While unofficial, the Facebook updater app ThroughGlass easily outstrips the functionality of Glass’ official Twitter and Facebook apps. The features sound simple and logical enough, but they’re a breath of fresh air on Glass: You can send photos and text status updates to Facebook, and you can also see and reply to comments on the items you post. All you need to do differently is pin the ThroughGlass card so it stays next to your homescreen, ready for your submissions. Hopefully the big boys are taking notes, because so far none of the official sharing apps let you pin them as a card, and we're getting tired of scrolling through our Sharing Contacts.

Reply to Facebook comments on the items you post

Glassnost

Glassnost works exactly like every other single-minded photo upload app, except instead of uploading to Twitter or Facebook, it shares photos to its own nascent Glass-only social network. Perhaps a fully fledged app experience is forthcoming, but for now if you’d like to +1 or comment on a photo, you have to visit Glassnost in a web browser, which sort of defeats the purpose.

Its own nascent Glass-only social network

Glass To Do

A simple to-do list app which installs two cards in your timeline. Glass To Do’s "Add a New To Do Item" card lets you "reply" to it with a spoken message, which is then added as a to-do item in the "Your To Dos" card — the to-dos are stored in the cloud somewhere, but only accessible from Glass. To complete a task, you simply delete it. Unfortunately, while the "Add" card can be pinned, the list of to-dos doesn’t pop to the front of your timeline when updated, so it’s quickly lost in the hubbub of Glass life. Hopefully that’s a bug, and we’ll keep an eye out for an update.

It's quickly lost in the hubbub of Glass life

Thirst

Thirst, a Flipboard-like virtual newspaper already available in web and app form, delivers news that’s trending on social networks, based on which topics you want to track. We set it up to track Google Glass news, naturally, and have received a handful of stories in our timeline over the past two days. Despite the "trending" status, the selections and represented publications feel a little arbitrary, but Thirst makes up for it in quality presentation. With Thirst you can have Glass read an entire article aloud to you, like or dislike the item, and email it to a friend. Only one word of warning: Glass can fall asleep while reading to you, so you’ll need to tap the touchstrip on occasion to keep it awake.

You can have Glass read an entire article aloud to you

Hacks

Like all Android devices, Google Glass has a "debug" mode, and once that’s enabled it’s relatively easy to run unapproved apps on your device. Obviously, there’s always the chance you could mess up your unit, but that’s what "Factory Reset" is for, right? At this early stage, you might have to compile these apps yourself, or run them on Glass from the Android development kit, but it’s nothing you can't handle with a couple hours of how-tos and tinkering. There's also a Glass root available, if you're feeling really brave.

Winky

The most controversial and interesting app to come to Glass so far, Winky allows you to take a photo at any time with a simple wink of your right eye. It relies on a little-known "gaze detector" sensor on the inside of Glass’ frame, pointed right at your eye. For a "hack," the app is surprisingly slick and reliable. When it’s first run, Winky offers a configuration screen where you wink a couple times to calibrate Winky’s wink tolerances. From then on, whether Glass is awake or not, a similar exaggerated wink will set it off and a picture will be snapped immediately.

It relies on a little known "gaze detector" sensor

Final thoughts

Final thoughts on first impressions

We know Glass is still in its early stages. We know "the apps aren’t there yet," so we shouldn’t be so quick to judge. We know Google is just getting started with this experiment. Still, as of May 20th, 2013, we can’t help but be judgmental. It’s sort of our job, and so we’ll let you know what we think anyway.

Glass is a slick implementation of the wrong idea. Google has worked so hard at making the device simple that it has succeeded too well — Glass simply can’t do anything Google hasn’t allotted for. Right now there’s little room for innovation by developers, because all third-party apps are trapped in a simple API that’s only capable of pushing and pulling data to and from the device. The API could grow, and Glass doesn't have to be this limited forever, but without the ability to process information on the device, most of the true power of apps on smartphones is lost here. Yes, most of the popular apps on mobile devices are games and social services, but the whole point of a wearable computer is that it should open up new opportunities: augmented reality, serendipity, ways to comprehend your surroundings, way to improve interactions with the people in front of you, and at least some sort of notetaking capability beyond sending emails to yourself. A game like Ingress should be Glass’ killer app, not Google+ sharing. Services like Google Search and Google Now will have to be smarter, not just simpler, on a device like this.

A game like 'Ingress' should be Glass’ killer app, not Google+ sharing

For a device that’s so likely to induce ridicule for the mere fact that you’re wearing a computer on your face, Google seems unconcerned with disabusing people of the notion that you’re a self-absorbed nerd who's fading from reality and becoming part of the Borg. There's not much of a "Hey, look what I can do!" feature, it's more like: "Hang on, let me wait for a push notification."

We expect many of Glass’ practical issues and bugs to fade over time: the lack of an app store, the unreliable connectivity, the constant computer usage required to manage your device, and the confusion between contacts, Sharing Contacts, and apps masquerading as Sharing Contacts. And we have to compliment Google on building a wearable computer that’s truly usable. But when will someone build a wearable computer that’s truly useful?

For now, Glass is the only viable consumer wearable on the horizon, our greatest hope. We’ll be watching carefully, updating often, and installing apps indiscriminately. We want to be proven wrong about Glass, but it just hasn’t happened yet.

Photography by Michael Shane and James Chae. Art Direction by James Chae.

Using Google Glass: stuck in traffic
Using Google Glass: at a Justin Timberlake concert
I used Google Glass: the future, but with monthly updates

Exclusive: Inside Hangouts, Google's big fix for its messaging mess

2013-05-15T17:58:08Z

By Ellis Hamburger and Dieter Bohn

Skydivers equipped with futuristic glasses live-broadcasted their descent into the Moscone Center in downtown San Francisco during last year’s Google I/O. Developers lined up to receive not one, but four free devices costing $300 or more. Google announced the Nexus Q, an exercise in over-produced gadgetry meant to stream music to your home theatre system.

Away from the spectacle, during a quiet “fireside chat” for a product that was not receiving any major updates, Google admitted it had a serious messaging problem — or rather, a messaging app problem. When faced with a question about Google’s fragmented communication tools, director of real-time communications Nikhyl Singhal responded quickly and honestly, as if he’d prepared for the question.

“I think we've done an incredibly poor job of servicing our users here.”

What he didn’t say was that Google had already been working on a solution for a year, and that the results of those efforts were still a year away. Since then, Google’s fragmented messaging story has become Google’s behind-the-times messaging story with apps as diverse as iMessage, WhatsApp, and Facebook Messenger grabbing both the spotlight and the market share.

But today, the wait is over as Google introduces a new messaging platform it’s calling Hangouts. It spans Android, iOS, Chrome, and Gmail. It’s a fusion of Google’s strengths in cloud computing, search, and mobile.

It’s also late to the game.

The story of Google Hangouts’ fractured development and eventual launch reveals quite a bit about how the company is changing from a place where tiny projects are initiated at random into one that’s being forced to organize and coordinate across products. At the new “One Google,” major projects require deep collaboration across multiple teams. Hangouts is more than just a way for Google to take on SMS. It could be a core product that stands next to Search, Gmail, and Docs, acting as a key part of Google’s suite of services. Here’s how it came to be... and why it took so long.

A history of hangouts

A history of Hangouts

Once upon a time, Google was at the forefront of the messaging game. In 2005, the company launched Talk, essentially the best experience in what was then the hottest category: instant messaging. Since then, a new category of mobile messaging has emerged: always-on chat with rich media attachments like voice memos, read receipts, and group chats. BlackBerry Messenger drove millions to buy a BlackBerry, and in recent years tiny startups like WhatsApp and established players like Facebook have grown to process billions of messages a day. In April of this year, people sent more messages using chat apps than SMS while Google sat on the sidelines with its little-used Google+ Messenger app.

Google instead spent its time building out mobile apps like Gmail, Google Now, Google Drive, and Google+, allowing its early lead to evaporate. As Singhal puts it, "When we started looking at the project we realized, and this sounds obvious in retrospect, that we built a lot of the communication products at Google without smartphones, without social networks in people’s lives."

Google’s lack of a unified messaging platform became something of a mystery, since it seemed like Google had all the necessary pieces

Talk, for example, was built to help enterprise users communicate better, Singhal says. "The notion of creating something that’s social and that’s always available wasn’t the same charter as we set out with when we created Talk." With Hangouts, Singhal says Google had to make the difficult decision to drop the very "open" XMPP standard that it helped pioneer.

Google’s lack of a unified messaging platform became something of a mystery, since it seemed like Google had all the necessary pieces. "There is a lot of code that existed that from a high-level point of view, if we just crunched it together, [it theoretically] should’ve turned into one product," says Singhal. The messaging experience was neither consistent or unified, and it wasn’t even easy to use. Google+ Messenger, a real-time communication platform that launched last year, was the first piece, but it only existed inside of Google+. Talk was another, but it was based on an old standard that predated the advent of cloud computing. Hangouts was third, a real-time video chat product embedded in Google+. Gmail had its own real-time communication team. Google Voice was the final piece, an infrequently updated relic that seemed destined for Google’s infamous spring cleaning chopping block.

Ben Eidelson, Nikhyl Singhal, Kate Cushing, and Randall Sarafa

Although Google Talk seemed like a good foundation for a messaging app, Google decided it needed to start from scratch. "We had to essentially rebuild everything," says Singhal. To create Hangouts from the guts of a decade’s worth of insular communication products, Google had to pull in engineers from across the company — many of whom were working separately on this very problem. From the outside, the company looked like a crystal palace of servers and engineers, but on the inside those same engineers weren’t working in concert, especially on messaging. "What’s funny is that most of them actually were trying to build this unified product independently," says Singhal, but separately, no one could get it done.

The watershed moment came with Google+, which proved that Google could — and should — organize a company-wide effort to implement a single idea. Years of work had shown that this core product couldn’t be created by just one team, or a handful of engineers working on their 20-percent time. "We started pulling the team together about two years ago," says Singhal. "It’s like The Avengers — we’ve had to pull together different styles and different cultures [from across Google]." The Hangouts team as it exists today is a combination of thiese pieces, a new backbone to hold together the real-time infrastructure that will power the next age of Google. Like with Google+, Singhal explains, the difficulty in messaging was "to integrate, aggregate, and crunch these different systems together."

Cloud messaging

On the surface, Hangouts is essentially a messaging app in the same vein as WhatsApp and Facebook Messenger. Hangouts replaces Google Talk, Google+ Messenger, and the classic Google+ Hangouts video chat.

The app — available on Android, iOS, and Chrome (but not Windows Phone or BlackBerry) — starts with text conversations. You're presented with a list of your recent conversations instead of a contact list. That's the first sign that this is more of a mobile messenger than a traditional instant messaging client, a distinction that becomes even clearer once you dive into a group chat or one-on-one conversation. Conversations get names, like chat rooms, and it's simple to add an image or one of Google’s 850 new hand-drawn emoji.

Major Google messaging moments

8/24/2005 Google Talk launches
2/7/2006 Google Talk integrated into Gmail, adds XMPP support
10/22/2008 Android launches with Google Talk app included
11/11/2008 Google Chat launches in Gmail , includes voice and video chat
3/11/2009 Google Voice launches
5/28/2009 Google Wave announced
4/15/2010 Google adds chat to Google Docs
4/28/2011 Google adds live video chat to Android
8/25/2010 Google Voice integrated into Gmail
6/28/2011 Google+ launches with video Hangouts and Huddle, an Android messenger
9/20/2011 Google launches mobile Hangouts for Android
7/30/2012 Google launches Hangouts in Gmail

Google says it's put a lot of thought into reconsidering presence, and it actually works better in Hangouts than on other apps. Instead of using "read receipts" (first popularized on BBM and coming this summer to iOS and Android) or a blanket online / offline indicator, Hangouts inserts tiny little square avatars into the chat history, called "watermarks." These watermarks show when somebody else is typing, but they also indicate how far others have read in the conversation.

The watermarks create a new sense of immediacy missing in ordinary text conversations

The watermarks create a new sense of immediacy missing from ordinary text conversations — you can see who is around and who isn't at a glance. Product manager Ben Eidelson says watermarks — like video Hangouts — reproduce the signals and cues we receive in face-to-face conversation. "It’s kind of like making eye contact," says Randall Sarafa, another product manager on Hangouts. The flip side of this new system is that you lose the more traditional "Active / Away" presence indicators that Google Talk users have grown accustomed to. It’s a hybrid of instant messaging and mobile messaging, though Hangouts will on some levels remain interoperable from Google Talk.

Hangouts keep your messages in the cloud — which isn’t exactly revolutionary, but since it's Google's cloud, there are some unique benefits. Every Hangouts conversation is stored online (and is accessible from any Hangouts app), but there is an option to toggle off history if you'd like to go off the record. The service’s Google+ integration is one of the best features in the entire product: every photo that you or a friend posts is automatically saved in a private, shared album on Google+. For example, after a year of using Hangouts, it will be easy not just to trace the text conversations your budding relationship has produced, but to track the photos you’ve shared over time. The feature is so obvious and so compelling that it could theoretically do what few Google initiatives have managed thus far — give a huge number of users a real sense of affinity with Google+.

Hangouts also benefits from Google+’s video conference tool — the "old" Hangouts. You can immediately start a video chat with up to 10 people with one tap, but voice-only conversation is a hassle since you’ll have to "mute" your video feed. Aside from audio calls, Hangouts has some other glaringly obvious missing features, like SMS. Google hasn't integrated SMS with Hangouts, either because Google doesn't want to annoy carriers or simply because it hasn't gotten around to it yet. Yet, both Apple and Facebook have proven with iOS’ Messages and Android’s Facebook Messenger that it can be tastefully done. Also, most of Google’s competitors let you broadcast messages, or embed video, voice messages, location, or any of the "stickers" that have become strangely popular. Finally, Google Voice, a product that could have been revolutionary but has instead languished, has been pulled into the Hangouts team with the promise of future integration. Singhal says that "this is the future for Google Voice," but offers no timeline for the integration of the the two products.

Aside from the missing features we’ve come to expect from new messaging platforms, Hangouts is also lacking your friends, who are using a great variety of messaging apps from WhatsApp to Kik. Hangouts hasn’t had years to cultivate hundreds of million of users worldwide like industry leader WhatsApp, which makes it a tough sell considering it’s short on features. However, most of the top apps on the scene don’t have desktop companions — a space ripe for the taking. If the masses decide that they do value a desktop app to complement their mobile one, Hangouts will be among the first to provide one, alongside Facebook Messenger and Viber. Plus, Google's size and ubiquity has proven to be an advantage when it wants to move into new areas.

Ultimately, Google has built a solid messaging app with a strong foundation that’s sure to win many converts, but it’s hard to ignore how late the company is to the game. With Facebook, Apple, BlackBerry, WhatsApp, and now many others well established in the messaging game, it’s a very crowded space. If Hangouts wants to make a stand, it will do so on the backs of Google’s 425 million Gmail users.

One Google

Google in 2013 is a very different company than the one most of us still envision. Larry Page’s ascension to CEO was the first step towards crafting a company more focused on big, integrated products. Gmail, Docs, and Maps now have a consistent look on both web and mobile, and although Google still has its share of experimental products like Glass and driverless cars, the days of the world-changing 20-percent projects are waning. The next Gmail just isn’t coming from two or three engineers on lunch break.

Hangouts takes massively powerful cloud services and marries it to a consumer product

Hangouts, like Google+, grew out of this internal shift, a revelation that was cultural as much as it was technical. Touchstone examples of the "One Google" product philosophy, they both serve as a "spine" for the company’s various products. "Google+ really was the template and the critical lesson that helped us believe that this type of project was possible," Singhal says. Hangouts may have taken years to build, but it’s devoid of the siloed approach Google has taken thus far with new products like Wave and Buzz. Hangouts takes what's best about Google — massively powerful cloud services — and marries it to a consumer product.

With Hangouts, Google has created a unifying backend for the future of its real-time products; Singhal describes it as "the single communication app that we want our users to rely on." It's a platform in an almost literal sense — something to build on that's not yet finished. In a way, Hangouts is an extension of Gmail’s product philosophy: to create a conversation between people instead of a chaotic flurry of messages. Google radically changed email in 2004 with its inclusion of "threaded conversations" in Gmail, but it never translated its success when the world went mobile. Google may have missed the boat on the mobile messaging revolution, but with its new real-time infrastructure in tow, it should be ready for the next revolution, whether it be holographic Hangouts or virtual email. "We don’t see Hangouts as a messaging product, we see it as a communication product," says product manager Kate Cushing. The only question is, when the next communication app trend hits, will Google be first or last?

Video by Jordan Oplinger & Stephen Greenwood
Edited by: Jordan Oplinger
Additional Editing: Billy Disney
Audio Mixing: John Lagomarsino

Body Message: GI Joe and the invention of the viral video

2013-05-09T15:45:04Z

2003 was an important year for American culture: Baghdad fell, the Human Genome Project was completed, Britney and Madonna French-kissed at the MTV Music Video Awards. And no less significantly, at least as far as internet culture is concerned, it was also the year of the “GI Joe PSAs”: 25 weird, short videos made from re-edited versions of ‘80s GI Joe cartoons. Before YouTube, Facebook, and Twitter were alive to launch a meme in a minute, the GI Joe PSAs went “viral” in a time when that idea didn’t even exist.

Created by a young, Chicago-based aspiring filmmaker named Eric Fensler, the series was constructed out of the public service announcements that were originally tacked on to the end of each show (which ran from 1985-1986), instructing viewers not to play around construction sites or leave stuff on the stove unattended. But in Fensler’s tweaked versions, the GI Joe characters drop bizarre non sequiturs in voices and accents not their own. (For instance: two kids approach a downed power line and are stopped by Roadblock, who pulls up in a Jeep and purrs, “Who wants a body massage?”)

Although there were earlier breakout online videos, “Baby Cha-Cha” for instance (1996), The GI Joe PSAs stand out because they don’t seem dated. They’re still just as funny and weird now as they were then. In fact, their impact can be felt in TV shows, commercials, and music videos being made today. And looking back at them a decade later reveals clues about the mystery of the viral video.

A meme is born

Eric Fensler didn’t set out to make a meme. Twenty-six years old in 2003 and working at a Chicago commercial post-production house, he made little movies on the side. Cartoons from his youth were a particular obsession: he loved The Ren & Stimpy Show so much that he’d record the show’s audio and listen to it in his car. After rewatching GI Joe: The Movie on DVD, he got the idea to do something with the public service announcements, which were extra features at the end of the movie. A friend helped him crack the DVD’s encryption so they could edit the PSAs.

“I would just dump all the footage in there, watch it with the sound off, then kinda go from there,” says Fensler.

His friends were enlisted to do the voiceovers, sometimes receiving specific instructions, other times just riffing. In one segment, a boy slides across a frozen lake and hurts himself, while his two friends watch in horror. Snow Job arrives, as if to help, and says, “’Give him the stick!” then, inexplicably, “DON’T give him the stick!” in a Cockney accent. The segment quickly ends, with Snow Job singing a sustained: “Aaaaahhhhh!”

“Give him the Stick!”“DON’T give him the stick!”

“All Eric told me to say was ‘Give him the stick / don’t give him the stick’,” remembers Fensler’s then-roommate P.K. Hooker, an LA-based sound effects editor, who played the part of Snow Job. “To this day, anytime anybody realizes I was one of the people who did this ancient internet meme, they ask me to do the voice.”

Fensler’s friend, Doug Lussenhop, now an editor for the show Portlandia, helped with the first GI Joe PSAs, then got them screened at an art gallery in Chicago called Heaven Gallery. As Fensler and his friends made more of the videos, there were more screenings, which became increasingly packed with fans. Fensler handed out VHS copies of the shows to people who asked, then the gallery uploaded them to its webpage. The traffic they generated crashed its server. After that, the PSAs were picked up by two entertainment sites that were popular at the time, eBaum’s World and Heavy.com. Articles were written.

Q: If you could design a cool toy, what would it be?
Fensler: A special pill you take that expands in your mouth as flavored foam… so it looks like you’re barfing.

In September of 2004, Hasbro sent Fensler a cease-and-desist order.

It was official: the GI Joe PSAs had gone viral.

Certain phrases became famous. Fensler later sold T-shirts printed with the words “Pork Chop Sandwiches!”, a line from one of the segments.

“We were just friends who knew each other on this dorky level, and half of those jokes are from us playing video games and joking around,” says Fensler’s friend Mat Biscan, who helped with a few episodes. “It’s really weird to hear people quoting these inside jokes.”

Hitting the big time

Riding on the success of the videos, Fensler, Hooker, and Lussenhop decided to seek their fortunes in LA. Fensler found himself pigeonholed.

“People saw me as this person who remixed cartoons, and I’d get a lot of requests to do that,” says Fensler. “But that’s not the only thing I wanted to do.”

In that vein, he made some animated promo spots for Sealab 2021 on Cartoon Network’s Adult Swim and a full episode that never aired. He tried and failed to make a live-action soap opera with mannequins called Sunset Feelings.

But their luck turned around in 2004, when Lussenhop answered a Craigslist ad for an internship on the Adult Swim show Tom Goes to the Mayor, and struck up a friendship with its creators, the comedians Tim Heidecker and Eric Wareheim. Lussenhop became their writing / editing collaborator, and hooked Fensler up with a writing gig on their next venture, The Tim and Eric Awesome Show, Great Job! Fensler stayed with the show for five seasons, and collaborated with Tim and Eric on subsequent projects, such as the risqué Flying Lotus music video, “Dance Floor Dale,” which also went viral.

For the last two and a half years, Fensler has been working in the creative department of the Portland, OR advertising agency Wieden + Kennedy, best known for the Nike “Just Do It” tagline and the Old Spice campaign.

“He keeps a very low profile,” says Robin Rosenberg, Wieden + Kennedy’s creative resource manager. “It’s not like I’ve taken a poll, but I would guess that most people here, even though they’re familiar with the GI Joe PSAs, don’t know Eric made them.”

Feelin’ viral

The GI Joe PSAs occupy a uniquely liminal space between traditional word-of-mouth sharing and the dawn of mass online sharing, as they were initially screened and distributed at the art gallery, but then also were shared online. “The screenings were really the origin of how it happened,” says Hooker. “Human beings showing up to an actual place in an actual room.”

But it’s a pretty good bet they would have gone viral without the person-to-person jumpstart.

“People respond to seeing familiar images with a weird voice or something non sequitur thrown in, and GI Joe had that formula,” says Lussenhop. Recently, Lussenhop and Eric Wareheim released a video with an original rap about hippie pubic hair called “Backpacker Bush,” that has yet to crack 100,000 views – not exactly “Gangnam Style.”

“We came up with the idea for this on a trip to New Zealand, and of all our videos, we would have put all our chips on this one going viral,” says Lussenhop, who also writes and stars in his own video series, Pound House. “I think if we’d had 50 Cent rapping, maybe it would have, but we didn’t have that familiarity quotient.”

Fensler insists that he “didn’t really crack the code” on what makes a video viral, but his protestations provide a decent directive. “I didn’t really set out to do all those things. I make stuff I enjoy, and I have a sense of what surprises me,” he says. “Why not just make something that you like?”

Design by Scott Kellum

Who am I? Data and DNA answer one of life’s big questions

2013-05-07T15:02:41Z

In March nearly 7,000 people traveled to the Salt Palace Convention Center in Salt Lake City, Utah, to spend the weekend at RootsTech, a yearly technology-focused genealogy conference sponsored by FamilySearch and a few other big names in the family history industry. Genealogy — the search for and documentation of one’s ancestors — and “technology” haven’t always been kissing cousins, but this conference speaks to and encourages a growing relationship between the two. The hobby, traditionally picked up near retirement age and most often by women, is now a billion-dollar industry with a growing younger demographic.

In the past few years, finding and charting one’s family history has become trendy because it’s also become a lot easier to get started. Companies like Ancestry.com and FamilySearch have spent the last decade or so making all of their tools, records, and data available on the internet, revolutionizing genealogical research — and significantly lowering the barrier to entry in the process. What was once a pastime for older people or professionals with disposable income is quickly becoming a more mainstream pursuit. Taking a peek into the past now requires nothing more than a decent internet connection and a laptop. DNA testing, which just a few years ago cost thousands of dollars and offered little information for genealogists, is now a growing consumer option, reaching back hundreds of years to provide undreamed of amounts of information about our ancestors.

Sticky TOC engaged! Do not remove this!

Genealogy’s next phase, which is quickly approaching, is actually its end game. The massive accumulation, digitization, and accessibility of data combined with recent advances in DNA testing mean the questions we have about our families — who they were, how they got here, and how they’re related to us — will soon be instantly solvable. Realistically, the pursuit of family history as it exists now probably won’t be around in 20 years: most of the mysteries are disappearing, and fast.

So, who are we? How did we get here? Where did we come from? And where are we going?

A Tree Grows in Provo

A tree grows in Provo

Ancestry.com’s headquarters are nestled at the foot of a mountain in Provo, Utah, just 45 miles from downtown Salt Lake City. Founded in 1990 by Paul B. Allen (not the Paul Allen of Microsoft) and Dan Taggart (both Mormon graduates of Brigham Young University), the company was initially known as Infobases, and distributed Latter-day Saints (LDS) publications on floppy disk. From its earliest days, Ancestry.com was a software company, selling disks of LDS archives for around $300 out of Allen’s car. By 1995, the two were putting their wares online. Ancestry.com quickly became a leading destination for online genealogical research. Though the tools to create online family trees, indexes, and records were free, actual scanned images of historic documents were behind a paywall, and the company made much of its money through subscriptions. Today, an all-access membership to the service costs $298 a year — around $35 a month — for its over 2 million paying subscribers.

Years ago, the information would have filled a shelf of handwritten binders

The company’s extensive records mean anyone can construct a family tree. Once you find a record — say a 1940 census image — that you believe has your grandfather's name on it, you can link that record to his name in the tree. Your tree can be private, or you can share it and link your tree to others’. It's a powerful, centralized place where almost all of your research can be consolidated. Years ago, the same amount of information would have filled a shelf of handwritten and photocopied paper binders and wouldn’t have been easily shared.

Ancestry.com
is the most recognizable brand in genealogy today

Tim Sullivan, Ancestry.com’s CEO, gave a keynote address at RootsTech. He has been with the company for 10 years. Before that he had been first the COO, then the president of Match.com. He’s also worked for TicketMaster and Disney. Under Sullivan's stewardship, Ancestry.com has become the most recognizable — and probably also the most successful — brand in genealogy today. Still, he is approachable in a way that most CEOs are not and, as we walk through the halls of the convention center looking for a quiet place to talk, people wave, smile, and occasionally approach him. "Search, that’s what the past five years have been about," he says. But now, he says, "Family history is truly social." People work together — whether they know it or not — to improve their own personal trees, and to improve Ancestry.com’s data, by "stitching together" their information.

Sullivan is right that the company’s early success hinged almost entirely upon search. Ancestry.com provides an unmatched and ever-improving search algorithm. A generic search engine such as Google can’t distinguish between, say, a first and last name, which can mean all the difference in this kind of work, especially if your ancestor’s first name was something common like "Smith" or "Taylor." But Ancestry.com (and other companies like it) has built a search engine with a specific, single-minded purpose. It can handle, in one request: a first name associated with a last name (including a vast array of alternate spellings); a range of dates; a specific or broad range of documents to search; a geographic location as broad as a country or as specific as a town; a number of birth dates; a birth location; and additional names such as those of a relative’s children. The engine — which processes around 45 million searches a day (Google sees around three billion) — isn't perfect, but it is very powerful, and it’s constantly being tweaked and upgraded. The search can return hundreds of results within these specifications, ranked according to how well they match. By mining its massive database of archives and connections, Ancestry.com also automatically delivers "hints" — denoted by a shaking, illustrated leaf — based on your family tree that point to potential relatives and primary sources. Recently, it also debuted a Facebook sharing feature, where you can link yourself and your family’s Facebook accounts to profiles in your tree, too. This has also increased the power of Ancestry.com’s search.

All God’s Children

All God’s children

Interest in one’s ancestors dates back as far as history itself, but for most of mankind’s time on earth, the study of kinship and descent was reserved for royalty and the super-rich, usually with the aim of consolidating power and wealth. Modern hobbyist genealogy as it is practiced today, however, has its roots in the founding of the New England Historic Genealogical Society in Boston, Massachusetts in 1845. The Society popularized the system of charting relationships on "trees" as originally developed by John Farmer in the 1820s and still practiced today. In the next decade, a similar society was founded in New York, and it became common to seek ties to the Founding Fathers and other Revolutionary War figures. "When the Daughters of the American Revolution started up, and The Mayflower Society," Thomas MacEntee, a genealogist well known throughout the hobby’s large and growing online community, says, "that's really what I call the first phase of genealogy." As the American Republic was born, so was genealogy in the US.

Less than 50 years after the founding of the New England Historic Genealogical Society, what would arguably become the most important player in the genealogy game came into existence, way out west in Salt Lake City, Utah.

Thomas MacEntee says that Salt Lake City is the "Mecca" of family history research. This is because it is also the home of The Church of Jesus Christ of Latter-day Saints and its Family History Library, founded in 1894 as the Genealogical Society of Utah. It is the largest library in the world dedicated to genealogy. Its online portal, FamilySearch, sees about 10 million visits a day.

One of the Mormons’ fundamental tenets is doing genealogy

Salt Lake City was founded by Brigham Young and several other Latter-day Saints in 1847, and now has a population of just over one million, about half of whom are members of the LDS Church. Donald Anderson, the senior vice president of patron and partner services at the Family History Library, says that the Mormon church believes in "eternal families," and in the ability of those families to "continue beyond this life." So identifying ancestors, is, he says, a "significant part of the doctrine of the church." Standing in-between giant banks of filed microfilms, he says, "We’re all God’s children."

One of the Church's fundamental tenets is doing genealogical research because its members believe that Mormons can baptize ancestors in their absence. The act of baptizing family by proxy — i.e., without the knowledge or permission of the ancestor, usually because they're deceased — has been fairly controversial, but it’s not a focus for most genealogists. FamilySearch and The Family History Library’s staff welcome Mormons and non-Mormons alike. That’s because the library’s usefulness reaches far beyond its own religious goals, and the Latter-day Saints believe in spreading their information far and wide, all free of charge.

The Family History Library is an angular, jarringly modern building, open to the public six days a week. All of its services are free. Many — if not most — of its half-million yearly visitors are hobbyist or professional genealogists with no Mormon affiliation. They come because the Family History Library has amassed the largest collection of documents, books, and microfilms related to genealogical research in the world.

Beginning in the late 1930s, the Latter-day Saints undertook a massive project, finding and microfilming genealogical records on a global scale. Using an army of volunteers and missionaries, the LDS visited governments and churches (where most vital records were kept until the turn of the 20th century) all over the world, amassing 2.4 million rolls of microfilm. The Family History Library also operates 4,600 volunteer-staffed Family History Centers worldwide; these are smaller research facilities where patrons can order microfilms and books from the main library and have them delivered for off-site work close to home. But the days of digging in dusty libraries (the Family History Library is state-of-the-art, and not, in fact, dusty at all) for long-forgotten and yellowed documents are quickly coming to an end, thanks to the internet.

So You Want to Make a Family Tree

So you want to make a family tree... now what?

"It seems like the internet was invented for genealogy."

Until the internet came along, researching your family was a grueling and often unrewarding process. If, like many people, you were starting from scratch — maybe you know your four grandparents’ names and not much else — beginning could be nearly impossible. Thomas MacEntee got his start in the ‘70s and says, "You had to go down to an archive or repository" straight away. Luckily for him, he went to college in Washington, DC, home of The National Archives, which house an immense collection of United States census and military records. "It was all very paper-based," he says. The records were either paper or microfilm, and access required travel or, failing that, a mail order. The records typically weren’t indexed, either, so you had to know exactly what you wanted: a tall order if you were looking for a great-grandparent’s death certificate without knowing what day they died. All the charting of the family tree, of course, was paper-based too, so it was also often hard to figure out the relationship between, say, one cousin and another. Until the mid-2000s, almost none of this information was readily available online. Today almost all of it is, with the exception of some vital records (state laws determine their availability), and many military service files.

Katie’s Family Tree

Katie Notopoulos is just the kind of genealogist that these advances in technology have made possible. A self-described "hobbyist genealogist" and editor at BuzzFeed, Katie says she got started about five years ago when a friend told her how interesting and fun it was. She doesn’t go to genealogy conventions or scour cemeteries for missing dates on tombstones. "I do all of my research online," she says. This simply wasn’t possible until a few years ago.

To construct a family tree, one had to be a historian, a detective, and a linguist

"Genealogy is an industry that I think has lagged behind technology," MacEntee says, likely because the record holders — small churches and local governments — didn’t have the funds to microfilm and exhaustively catalog (and later, digitize) their records. Often small county courthouses have just one person dedicated to processing requests for family history records. Progress in digital photography, scanning software, and OCR (optical character recognition) technologies have only recently brought down costs.

In the early days, constructing a tree could be a lonely road, often with just an overworked librarian or archivist here or there to help you make sense of your findings. In order to be successful, you've also historically had to possess a broad, working knowledge of geography, history, world events, and immigration patterns over the last 200 years. A lot of that has changed.

Katie has constructed, for the most part, a tree reaching back about five generations, which includes British, Greek, and German ancestry. She has mostly accomplished this using Ancestry.com. "Very early on," she says, "I had a breakthrough where I found someone else who had been doing research for years and years on a branch of my family." Finding another person — however distantly related to her — to work on the same project with, she says, "Was immensely helpful and made it a lot easier to quickly trace far back, which all seemed very romantic and exciting." She likes to do her research in tiny chunks of time; sitting on her couch, watching TV. "What makes it a good hobby," she says, is that she can solve "mini-mysteries." It's an activity that takes her away from the "real world, temporarily." She even found a third cousin who was researching one of her family lines on Ancestry.com, and met them for dinner when they visited New York. This is a regular phenomenon in the online world of ancestral research.

"We’re all related" is a sentiment you’ll often hear

"We're all related" is a sentiment you hear often in the genealogy community, and it's not completely untrue: go back just 10 to 15 generations and many of us will find common links. But not until the internet was that widely held wisdom actionable in any useful and organized way. "It seems like the internet was invented for genealogists," Thomas MacEntee says. He's at his birthday party in a ballroom at the Radisson in Salt Lake City, and it's the third night of RootsTech. It's a large, friendly gathering, and there are even some celebrities of the genealogy world in attendance: Cyndi Howells, who invented Cyndi's List, is there, along with people who work for FamilySearch and the Israeli company MyHeritage. Thomas knows everyone, and the community is tight-knit, meeting up a few times a year at conferences.

The rest of the time, though, they're online, networking and helping one another dig. At its core, genealogy's draw is in the hunt, in searching — sometimes for years — for a clue that holds the key to another ancestor. The search is, of course, essentially infinite: most people are lucky to get back five or six generations, at which time their tree will contain upward of 5,000 people.

Katie’s DNA

Graphic from 23andMe

In the earliest days of the internet, the best place for genealogists to meet up was on email lists and message boards, where they pooled resources and helped each other look up newspaper clippings or birth records. In 1984 the Latter-day Saints published the open standard format for a genealogical file, known as the GEDCOM. A simple plain text file with metadata linking records to one another, GEDCOMs can be read by different types of proprietary software and remain the standard file format even today. This meant that people could begin to share large amounts of information — their findings, their families — with one another in an easy and portable way online. Small websites focusing on single families or on compiling lists of obituaries in one small town popped up all over the internet. Some people walked entire cemeteries, documenting every headstone, painstakingly transcribing them and putting them online. What was an essentially data-driven hobby wasn’t going to lag behind forever. "We knew it would catch up, eventually," Thomas says. And it did. By the mid-90s, small startups began to see that the internet could mean big business for genealogy, and the Latter-day Saints were taking notice, too.

The Search Continues

The search continues

Search is the key. Scott Sorensen, the CTO of Ancestry.com, explains that each search result is tied to a series of metadata — an index, or basic information such as the person's name. It’s also usually tied to a high-quality scan of a document which may be hundreds of years old. "We've got 10 billion records, four petabytes of data," he says, tied to the results returned from a search. Any search might dig up between 10 and several hundred results, weighted according to how well they match your terms. And with each search, the engine improves: "All of the interactions that our customers have with our site, we're able to learn from those." Using "machine learning technology, we can observe customer behaviors in the aggregate and over time learn and improve our algorithms, because they continue to add structure to our data" he says. The Ancestry.com users, he says, "Continue to make judgements about the records, which we're able to learn from." Finally, the indexes and records are tied, through customer interaction, to 38 million separate user-generated trees, which can further be linked to one another in an ever-expanding giant matrix of data representing peoples’ families.

Genealogy Toolbox

There are several basic types of records essential to any American starting a family tree.

Birth, marriage, and death records: Since the turn of the 20th century they’ve been kept by municipal, county, and state courthouses; before that, they were usually found in churches.
United States census records: Dating back to the 1840s, the census records enumerate each person living, and are conducted every ten years.
Immigration records: Usually passenger manifests, lists of people on ships arriving in major US ports such as Ellis Island, which saw 12 million immigrate to the United States from 1892 to 1954.
Naturalization records: Located at the National Archives in Washington DC, and in city courts. Documents filed over a series of months or years in which non-US immigrants were granted citizenship.
Military service records: At the National Archives, documenting military personnel from the Revolutionary War forward, pension files and other supplementary documents.
The Social Security database: A unique identification number, with date and state of death, for each person in the United States. Started in 1936, but with most numbers being assigned from 1965 onward.

So where are the records coming from? Unsurprisingly, many of them come from the Latter-day Saints' Family History Library. Ancestry.com forges agreements with companies large and small, granting access to the valuable records behind its paywall. Their data is particularly useful since the LDS "got there first," in many cases — for instance, by microfilming census data. FamilySearch CEO Dennis Brimhall says that because the organization is a nonprofit (as an arm of the church), it’s easy for it to share records. "We’re just interested in people finding records," he says. "We hope that works with their financial model. It probably works with ours because we don’t really have a financial model, but what we really want to do is make more records available to more people throughout the world." This thinking drives most of the companies in the genealogy business: access is key, regardless of who owns what, so companies share their data rather than force each other to "duplicate efforts" by digitizing redundant copies. Ancestry.com also has relationships with the non-profit JewishGen, the largest destination for Jewish genealogy, and Find A Grave, the most comprehensive user-generated database of cemetery transcriptions in the world. Some of these relationships give users direct access to the records and data without ever leaving Ancestry.com’s portal; some, such as the indexes of vital records and censuses from the United Kingdom, allow users to see names and other basic information. If access to the actual images is required, however, users have to go to the site, which controls them directly.

But Ancestry.com is also actively buying. "We spend over $20 million a year acquiring new content," Scott Sorensen says. On the day of our visit to a clean laboratory, employees are using digital cameras and proprietary software to create high-resolution scans of high school yearbooks. "Yearbooks are incredibly important for genealogy," Thomas MacEntee says, "because they are a great source of finding maiden names for women," a tricky problem when married women often take their husband’s surname. Ancestry.com has acquired a huge number of similar "secondary" sources such as city directories, phone books, and church directories. Once the images are scanned, names are transcribed, metadata is embedded, and the images are uploaded with indexes to Ancestry.com’s website. The company has also bought several other genealogy and archive businesses — smaller competitors — in order to bolster its resources. In April of 2012 it paid $100 million for Archives.com, and that October acquired the photo digitization service 1000memories.

FamilySearch has a website with very similar capabilities, where everything is free. The search isn’t quite as powerful as Ancestry.com’s, and their family-tree making software is not as robust, but the massive collection is growing literally by the day. This growth is fueled by over 150,000 volunteer transcriptionists using a proprietary Java application the company developed itself. Anyone at home can download the app, and in a few minutes transcribe a series of birth, death, or marriage records. This process, called "Indexing," is one of FamilySearch’s most prized and valuable tools. Each year, using its sophisticated transcribing and indexing system, FamilySearch adds 400 million indexed, organized images to its website. The company — which used to distribute its records via microfilm and CD-ROMs — can now move incredibly fast to make its data available to genealogists. The process, from capturing images in the field to making the records available to customers, used to take about 18 months. Now it’s usually less than two, and of course — it’s online, not on rolls of microfilm.

Family history is big business

Ancestry.com and FamilySearch may be the biggest names in online genealogy, but they’re not the only ones by far, and the newer players are moving fast to try to eat up a piece of the growing market. MyHeritage is an Israeli company founded in 2003 whose service operates more like a social network for family members — both the living and the dead — than traditional family tree approaches. The site recently raised $25 million in funding, and is available in 38 languages. Because its early focus was on places such as eastern Europe — where Ancestry.com’s holdings are somewhat weaker — MyHeritage arguably offers something quite unique to American audiences, where it is now making an aggressive push into the market. UK-based FindMyPast.com is also making headway in the American market because exclusive relationships with the governments of England, Scotland, and Wales have essentially given it a monopoly on vital records in those countries. FindMyPast.com’s CEO Chris van der Kuyl is also the president of 4J Studios, the company responsible for bringing Oblivion to the PlayStation 3 and Minecraft to the Xbox. He describes himself as a "technology geek" and thinks about genealogy from that perspective. He got into the family history business by accident when a friend asked him to apply some of his user-experience building skills to a genealogy company’s software. Five years later, he’s still here, working at the helm of the UK’s most powerful family history source. "Technology is empowering," he says, "and the more people [who] have access to the right technology and bring their own data and their own experiences, the more exponentially things get better for everyone. Our mission is to create the most amazing family history experience, to give as many people access to their own story as possible."

By 2010, Ancestry.com had forged a relationship with NBC to bring the UK television series Who Do You Think You Are? to mainstream US audiences. The show featured professional genealogists paired with celebrities like Sarah Jessica Parker, Steve Buscemi, and Spike Lee, sending them on journeys to find the stories of their ancestors. In October of 2012, Ancestry.com — then a publicly traded company — was acquired by a group of investors including CEO Tim Sullivan and European private equity firm Permira Advisers LLP for $1.6 billion dollars. Family history is big business, sure, but searching documents online isn’t the only way to figure out who you are. If you want to get serious and look to the future, well then you’re going to have to spit.

Spitting Image

Spitting image: DNA can solve this for you

In Mountain View, California, just around the corner from Google, rest the unassuming headquarters of 23andMe. The company was founded in April of 2006 by a small group including biologist Anne Wojcicki, who is now married to Google cofounder Sergey Brin. 23andMe was created as a personal genome company, its main goal being to put the "power of a person’s health into their own hands," says Catherine Afarian, the company’s public relations manager. While that sounds like a simple mission, it was unheard of just a few years earlier.

Katie recently took the 23andMe DNA test, as well an AncestryDNA test. She did so, she says, because she was curious to see how well science matched up with what she had found in her research. She simply signed up for an account on the website, ordered the DNA test, spit into a tube once the test had been shipped to her, and then registered its barcode number on the website. About eight weeks later, personalized ancestry and health results showed up in her inbox.

At launch, the test cost $999. It was fairly cheap, all things considered, but not something that everyone could afford. This past December, after announcing it had amassed a database of 180,000 DNA tests, 23andMe permanently lowered the cost of the test to $99 in the wake of a large round of financing, and set its sights on getting 1 million tests in its database this year. Though just two percent of Americans have taken such a test, a study conducted by 23andMe indicates that nearly 71 percent of those who haven’t taken one are interested in doing so.

How DNA is inherited

Both Ancestry.com and 23andMe’s genealogical DNA results have similar features. Once your results have been processed — both companies send their tests out to a lab for extraction, then do in-house analysis — you can log into your account and see an approximate composition of your ancestral DNA, which dates back around 500 years. For example, if your grandparents were half Polish and half Irish, your DNA results wouldn’t necessarily reflect that closely, but they would show you roughly where your family came from 10 generations ago. The results for both tests are displayed in a map format (as seen in the diagrams above). "It’s a little bit confusing," Katie says, "because the Ancestry.com test shows that I have about 17 percent Scandinavian DNA, and I haven’t found any Scandinavians in my own research." This opens new, often previously unconsidered, territory for a genealogist to pursue.

Though 23andMe delivered some ancestry results at launch, its "Ancestry Composition" feature — which delivers fairly specific, sophisticated information based on 22 worldwide populations — was launched in August of 2012, just three months after Ancestry.com launched its new DNA testing feature. Both 23andMe and Ancestry.com now offer the same type of test: an autosomal DNA test which delivers specific ancestry information for anyone. Earlier tests for females tested only MtDNA, and delivered only ancestry results from their mothers: a much less specific and useful type of test. The release of a more powerful test by both companies, and the subsequent decrease in cost, means that many people are now signing up. Ancestry.com announced in March that its database comprises of more than 120,000 DNA tests.

But how is the analysis done? Unsurprisingly, it’s complicated and, according to 23andMe’s senior director of research, "not very interesting." Basically, your DNA is tested using several hundred "markers," and compared using the "signal" those markers share strongly in common with geographic populations worldwide. Some markers have a very strong association with a specific location, making the results much more reliable, while others — such as those associated within central Europe, France, and Germany — are much less so, making that fine of a distinction often difficult to assume with a high level of accuracy. The process is further complicated by the fact that most people living today have multiple ancestries, as populations have inevitably migrated and mingled over the course of centuries.

Brother and sister: Katie Notopoulos' great great aunt and great great uncle

So, if you take the spit test, your DNA is then compared to a set of "reference" tests — the DNA of thousands of people whose origins are well-documented and mapped to geographic locations. In the simplest of terms, where your DNA matches with those reference sets of data, a percentage of your ancestry can be extrapolated to be from that region, too.

A secondary, and possibly more powerful feature of both 23andMe and Ancestry.com’s DNA sites provide something else entirely through similar methods of comparison: they show you other members who have taken the same test who are likely related to you. Both sites give a percentage of confidence in the match, so an example match might say, "We are 95 percent confident that member X is between a fourth to sixth cousin." Now, a sixth cousin is pretty far back in your family tree, but a second or third cousin (and many people who take the test, Ancestry.com tells me, have one or two matches at that level of closeness), is not. A second cousin is the child of your first cousin (your parent’s sibling’s child); a third cousin would mean that you and the other person share great-grandparents. On average, 23andMe says that each person who takes their test has more than 1,000 genetic matches found in the database. You have the option of contacting them — first anonymously — to compare information. Obviously, the more people who take the test, the more matches will be found, and the accuracy of those matches will increase too; hence the big push from both Ancestry.com and 23andMe to get many, many people to take their tests. It also helps explain the recent deep discount — in both cases permanent — to $99.

We’re approaching a future where the mysteries of our ancestral past will simply no longer exist

Katie Notopoulos and her parents

"We can create a whole new market of people who can make family history discoveries without having to do original research in old historical documents," Tim Sullivan says, calling the recent developments a "revolution in personal genomics." Ancestry.com links your DNA test directly into your tree, and 23andMe offers a less robust feature where you can upload a GEDCOM file to the site, also linking your data to a tree. Personal DNA tests for genealogy aren’t yet widespread, and they haven’t yet realized their full potential. But that’s not very far in the future.

So what does that mean for people who just love digging through documents, whether online or off, searching for tiny clues to link them to the past? In the shortest of short terms, the search continues. But realistically, in the next five to 10 years, it will become increasingly simple to find out who your ancestors were even several generations back, with relatively little effort: genealogy questions are a problem that technology is going to solve, and the foundation has already been laid. And a bit further in the future, it’s entirely realistic to believe that those questions of bloodline, like "Who was my great-grandmother?" simply won’t exist. The role that social networks like Facebook play in laying the groundwork for the future documentation of relationships is an important one — we’re all making more data than ever before. It’s not hard to imagine a future where the mysteries most of us have in our ancestral past will simply no longer exist.

The "stories," they say: that’s what all this data leads us to. A link to our past, not just in birth certificates, dates, and names on a chart, but through stories about who we are via who came before us. In the past decade, genealogy as a hobby has grown exponentially because of the vast amount of searchable data accumulated online: by companies like Ancestry.com, by the government, and by individuals. That trend will only accelerate in the coming years, making the research far more accessible for people with limited time or resources. "For me," Katie says, "I quickly moved past any sense of ‘these are my relatives’ and just fell in love with discovering these completely regular lives from the past, and learning history. I can't ever imaging thinking, "Well, I've found out everything I wanted to know; that's a wrap." Unfortunately for those who love the hunt, the future is about to get way less mysterious.

Video by Billy Disney, Stephen Greenwood, Ryan Manning, and Sam Thonis
Design by Scott Kellum
Infographics by James Chae and Scott Kellum
Photo credits: National Archives and Records Administration, and Katie Notopoulos

Offline: how to use the internet

2013-05-03T21:25:01Z

Whew! What a week.

First came Monday, and then Tuesday, and then there was the internet. You know how in Star Trek when they engage the warp engines and the Enterprise kind of stalls for a moment while its projection blurs toward the future, toward the stars, and then it’s gone? I’m in the blur phase.

I feel severely disoriented, totally overwhelmed, and kind of… happy about it?

At 12:00AM on Wednesday, May 1st, I rejoined the internet. I guess I thought I’d just start using the internet again, see some funny cat videos, and that would be that. Instead, I almost had a panic attack as I attempted to pull off basic 21st-century maneuvers like managing multiple tabs in a single browser window. Of course my inability to cope, which involved me shouting at my coworkers and incoherently stumbling through a discussion of pornography, was livestreamed to the internet.

I couldn’t be more proud. Because the internet was scary, and the livestream told that story.

I looked like I needed a hug, because I did

After the livestream ended, I went outside for a cigarette to settle down — because I’m apparently a huge smoker now, which wasn’t a big deal when it was just a thing I did instead of a thing open for discussion by all ‘net users. Coworkers exited the office a couple at a time, and gave me handshakes that turned into hugs, or just straight-up hugs. I looked like I needed a hug, because I did. I felt traumatized by the internet.

And yet, somehow enticed. I stayed at the office until 3AM, clicking, scrolling, and tabbing. When I got home at 3:30AM I wanted to get back online, but mercifully I didn’t know the Wi-Fi password yet, and my roommate was asleep, and so I went to bed. My mind churned for half an hour before I slept.

Maybe there’s something about information that makes you want more of it. Something about the fear I have of the internet that produces adrenaline. Something that makes me love the internet as much as I try to hate it.

The next morning, armed with the Wi-Fi password, I attempted to “work” from home. I’m sure it was cute for my coworkers to observe, like an old-time newspaper man getting his first WordPress blog. When I arrived at the physical office, I felt a little more useful. I know I want to “be with people” on the internet, but I have to relearn that skill. For now, I need to be face-to-face with someone to feel like an interaction is actually working.

Unfortunately, face-to-face was rare, because it seemed almost impossible to step away from the internet. In a meeting I found myself poking around on an iPad, catching myself, setting it in my lap and trying to listen really hard, and then diving back into the iPad a minute later. I had a recent Vine that was really "blowing up," so I was tracking the likes, then I’d swipe over to my email and become utterly bewildered, then I’d see if I had any new snapchats. My distraction was obvious to everyone in the room, despite the fact they all had laptops open in front of them. I just can’t split my focus like them yet. It hurts to even consider.

In fact, I started to feel alone at the office. Because I couldn’t be in two places at once, I chose to be inside the computer. I went full Tron; I jacked into the Matrix. Snapchats became reminders of moments I wasn’t present for, work emails felt like a foreign language I couldn’t converse in, and on Twitter I was just a madman shouting on a street corner.

That evening I watched funny YouTube videos with my sister and my roommate, but I had trouble setting the internet down when they turned on an actual movie. I sunk into my iPad and lost track of the conversation, and humanity.

On Thursday I was worse. I started to talk loud and fast, and felt panicked constantly. I sent emails to the wrong places, put calendar entries on the wrong days, and jogged around the office like a lost kid in a supermarket, clutching my laptop to my chest. Also, I felt like I might have lasers inside of my eyes (I forgot my glasses at home), and so it was weird to look at people. I got really hungry, and a coworker brought me food, but then I didn’t eat much of it.

I want to be a part of it all, but the trouble is I still want to be a little apart from it, too

But at least I was working. At 8PM there were four of us left in the office and I proudly looked up from the computer: I’d managed to process my work email all the way back to April 30th. Only 21,076 unread messages left to go.

At home at last, I Skyped with my parents after 15 minutes troubleshooting my Wi-Fi router. They told me I looked tired, which is something parents are very good at. And it was so good to see them, and it was so good to be on the internet. I knew it then. In two mere days I’d made a 20-year technological leap. The future is amazing. We have so much cool stuff. I want to be a part of it all, but the trouble is I still want to be a little apart from it, too.

Afterwards I meant to chill and listen to internet-streamed music, but instead I watched StarCraft videos and ate cereal. And then I finally closed the YouTube tabs, put the music on, pulled up the text editor on my Mac, and began to write.

Everything is moving so fast, so I’m not sure how to explain what’s happening. I recently half-seriously told someone I want to be an "internet caterpillar" when I get back, not an internet butterfly. But I feel like the metamorphosis is happening without my consent. I guess I should get some sleep.

Still, it feels good to write. I know how to do this.

I’m still here: back online after a year without the internet

2013-05-01T14:40:45Z

I was wrong.

One year ago I left the internet. I thought it was making me unproductive. I thought it lacked meaning. I thought it was "corrupting my soul."

It's a been a year now since I "surfed the web" or "checked my email" or "liked" anything with a figurative rather than literal thumbs up. I've managed to stay disconnected, just like I planned. I'm internet free.

And now I'm supposed to tell you how it solved all my problems. I'm supposed to be enlightened. I'm supposed to be more "real," now. More perfect.

But instead it's 8PM and I just woke up. I slept all day, woke with eight voicemails on my phone from friends and coworkers. I went to my coffee shop to consume dinner, the Knicks game, my two newspapers, and a copy of The New Yorker. And now I'm watching Toy Story while I glance occasionally at the blinking cursor in this text document, willing it to write itself, willing it to generate the epiphanies my life has failed to produce.

I didn't want to meet this Paul at the tail end of my yearlong journey.

In early 2012 I was 26 years old and burnt out. I wanted a break from modern life — the hamster wheel of an email inbox, the constant flood of WWW information which drowned out my sanity. I wanted to escape.

I thought the internet might be an unnatural state for us humans, or at least for me. Maybe I was too ADD to handle it, or too impulsive to restrain my usage. I'd used the internet constantly since I was twelve, and as my livelihood since I was fourteen. I'd gone from paperboy, to web designer, to technology writer in under a decade. I didn't know myself apart from a sense of ubiquitous connection and endless information. I wondered what else there was to life. "Real life," perhaps, was waiting for me on the other side of the web browser.

My plan was to quit my job, move home with my parents, read books, write books, and wallow in my spare time. In one glorious gesture I'd outdo all quarter-life crises to come before me. I'd find the real Paul, far away from all the noise, and become a better me.

My goal would be to discover what the internet had done to me over the years

But for some reason, The Verge wanted to pay me to leave the internet. I could stay in New York and share my findings with the world, beam missives about my internet-free life to the citizens of the internet I'd left behind, sprinkle wisdom on them from my high tower.

My goal, as a technology writer, would be to discover what the internet had done to me over the years. To understand the internet by studying it "at a distance." I wouldn't just become a better human, I would help us all to become better humans. Once we understood the ways in which the internet was corrupting us, we could finally fight back.

At 11:59PM on April 30th, 2012, I unplugged my Ethernet cable, shut off my Wi-Fi, and swapped my smartphone for a dumb one. It felt really good. I felt free.

A couple weeks later, I found myself among 60,000 ultra-Orthodox Jews, pouring into New York's Citi Field to learn from the world's most respected rabbis about the dangers of the internet. Naturally. Outside the stadium, I was spotted by a man brandishing one of my own articles about leaving the internet. He was ecstatic to meet me. I had chosen to avoid the internet for many of the same reasons his religion expressed caution about the modern world.

"It's reprogramming our relationships, our emotions, and our sensitivity," said one of the rabbis at the rally. It destroys our patience. It turns kids into "click vegetables."

My new friend outside the stadium encouraged me to make the most of my year, to "stop and smell the flowers."

This was going to be amazing.

I dreamed a dream

And everything started out great, let me tell you. I did stop and smell the flowers. My life was full of serendipitous events: real life meetings, frisbee, bike rides, and Greek literature. With no clear idea how I did it, I wrote half my novel, and turned in an essay nearly every week to The Verge. In one of the early months my boss expressed slight frustration at how much I was writing, which has never happened before and never happened since.

I lost 15 pounds without really trying. I bought some new clothes. People kept telling me how good I looked, how happy I seemed. In one session, my therapist literally patted himself on the back.

I was a little bored, a little lonely, but I found it a wonderful change of pace. I wrote in August, "It's the boredom and lack of stimulation that drives me to do things I really care about, like writing and spending time with others." I was pretty sure I had it all figured out, and told everyone as much.

As my head uncluttered, my attention span expanded. In my first month or two, 10 pages of The Odyssey was a slog. Now I can read 100 pages in a sitting, or, if the prose is easy and I'm really enthralled, a few hundred.

I learned to appreciate an idea that can't be summed up in a blog post, but instead needs a novel-length exposition. By pulling away from the echo chamber of internet culture, I found my ideas branching out in new directions. I felt different, and a little eccentric, and I liked it.

Without the retreat of a smartphone, I was forced to come out of my shell in difficult social situations. Without constant distraction, I found I was more aware of others in the moment. I couldn't have all my interactions on Twitter anymore; I had to find them in real life. My sister, who has dealt with the frustration of trying to talk to me while I'm half listening, half computing for her entire life, loves the way I talk to her now. She says I'm less detached emotionally, more concerned with her well-being — less of a jerk, basically.

Additionally, and I don't know what this has to do with anything, but I cried during Les Miserables.

It seemed then, in those first few months, that my hypothesis was right. The internet had held me back from my true self, the better Paul. I had pulled the plug and found the light.

Back to reality

When I left the internet I expected my journal entries to be something like, "I used a paper map today and it was hilarious!" or "Paper books? What are these!?" or "Does anyone have an offline copy of Wikipedia I can borrow?" That didn't happen.

For the most part, the practical aspects of this year passed by with little notice. I have no trouble navigating New York by feel, and I buy paper maps to get around other places. It turns out paper books are really great. I don't comparison shop to buy plane tickets, I just call Delta and take what they offer.

In fact, most things I was learning could be realized with or without an internet connection — you don't need to go on a yearlong internet fast to realize your sister has feelings.

But one big change was snail mail. I got a PO Box this year, and I can't tell you how much of a joy it was to see the box stuffed with letters from readers. It's something tangible, and something hard to simulate with an e-card.

In neatly spaced, precisely adorable lettering, one girl wrote on a physical piece of paper: "Thank you for leaving the internet." Not as an insult, but as a compliment. That letter meant the world to me.

But then I felt bad, because I never wrote back.

And then, for some reason, even going to the post office sounded like work. I began to dread the letters and almost resent them.

As it turned out, a dozen letters a week could prove to be as overwhelming as a hundred emails a day. And that was the way it went in most aspects of my life. A good book took motivation to read, whether I had the internet as an alternative or not. Leaving the house to hang out with people took just as much courage as it ever did.

By late 2012, I'd learned how to make a new style of wrong choices off the internet. I abandoned my positive offline habits, and discovered new offline vices. Instead of taking boredom and lack of stimulation and turning them into learning and creativity, I turned toward passive consumption and social retreat.

A year in, I don't ride my bike so much. My frisbee gathers dust. Most weeks I don't go out with people even once. My favorite place is the couch. I prop my feet up on the coffee table, play a video game, and listen to an audiobook. I pick a mindless game, like Borderlands 2 or Skate 3, and absently thumb the sticks through the game-world while my mind rests on the audiobook, or maybe just on nothing.

People who need people

So the moral choices aren't very different without the internet. The practical things like maps and offline shopping aren't hard to get used to. People are still glad to point you in the right direction. But without the internet, it's certainly harder to find people. It's harder to make a phone call than to send an email. It's easier to text, or SnapChat, or FaceTime, than drop by someone's house. Not that these obstacles can't be overcome. I did overcome them at first, but it didn't last.

It's hard to say exactly what changed. I guess those first months felt so good because I felt the absence of the pressures of the internet. My freedom felt tangible. But when I stopped seeing my life in the context of "I don't use the internet," the offline existence became mundane, and the worst sides of myself began to emerge.

I would stay at home for days at a time. My phone would die, and nobody could get ahold of me. At some point my parents would get fed up with wondering if I was alive, and send my sister over to my apartment to check on me. On the internet it was easy to assure people I was alive and sane, easy to collaborate with my coworkers, easy to be a relevant part of society.

So much ink has been spilled deriding the false concept of a "Facebook friend," but I can tell you that a "Facebook friend" is better than nothing.

My best long-distance friend, one I'd talked to weekly on the phone for years, moved to China this year and I haven't spoken to him since. My best New York friend simply faded into his work, as I failed to keep up my end of our social plans.

I fell out of sync with the flow of life.

there's a lot of "reality" in the virtual, and a lot of "virtual" in our reality

This March I went to, ironically, a conference in New York called "Theorizing the Web." It was full of post-grad types presenting complicated papers about the definition of reality and what feminism looks like in a post-digital age, and things like that. At first I was a little smug, because I felt like they were dealing with mere theories, theories that assumed the internet was in everything, while I myself was experiencing a life apart.

But then I spoke with Nathan Jurgenson, a ‘net theorist who helped organize the conference. He pointed out that there's a lot of "reality" in the virtual, and a lot of "virtual" in our reality. When we use a phone or a computer we're still flesh-and-blood humans, occupying time and space. When we're frolicking through a field somewhere, our gadgets stowed far away, the internet still impacts our thinking: "Will I tweet about this when I get back?"

My plan was to leave the internet and therefore find the "real" Paul and get in touch with the "real" world, but the real Paul and the real world are already inextricably linked to the internet. Not to say that my life wasn't different without the internet, just that it wasn't real life.

Family time

A couple weeks ago I was in Colorado to see my brother before he deployed to Qatar with the Air Force. He has a new baby, a five-month-old chubster named Kacia, who I'd only seen in photos mercifully snail mailed by my sister-in-law.

I got to spend one day with my brother, and the next morning I went with him to the airport. I watched dumbfounded as he kissed his wife and kids goodbye. It didn't seem fair that he should have to go. He's a hero to these kids, and I hated for them to lose him for six months.

My coworkers Jordan and Stephen met me in Colorado to embark on a road trip back to New York. The idea was to wrap up my year with a little documentary, and spend the hours in the car coming to terms with what had just happened and what might come next.

I thought hard about whether I could succeed online where I'd failed offline

Before we left, I spent a little more time with the kids, doing my best to be a help to my sister-in-law, doing my best to be a super uncle. And then we had to go.

On the road, Jordan and Stephen asked me questions about myself. "Do you think you're too hard on yourself?" Yes. "Was this year successful?" No. "What do you want to do when you get back on the internet?" I want to do things for other people.

We stopped in Huntington, West Virginia to meet a hero of mine, Polygon's Justin McElroy. I met with Nathan Jurgenson in Washington DC. I thought hard about whether I could succeed online where I'd failed offline. I asked for tips.

What I do know is that I can't blame the internet, or any circumstance, for my problems. I have many of the same priorities I had before I left the internet: family, friends, work, learning. And I have no guarantee I'll stick with them when I get back on the internet — I probably won't, to be honest. But at least I'll know that it's not the internet's fault. I'll know who's responsible, and who can fix it.

Late Tuesday night, the last night of the trip, we stopped across the river from NY to get "the shot" from New Jersey of the Manhattan skyline. It was a cold, clear night, and I leaned against the rickety riverside railing and tried to strike a casual pose for the camera. I was so close to New York, so close to being done. I longed for the comfortable solitude of my apartment, and yet dreaded the return to isolation.

In two weeks I'd be back on the internet. I felt like a failure. I felt like I was giving up once again. But I knew the internet was where I belonged.

12:00AM, May 1st, 2013

I'd read enough blog posts and magazine articles and books about how the internet makes us lonely, or stupid, or lonely and stupid, that I'd begun to believe them. I wanted to figure out what the internet was "doing to me," so I could fight back. But the internet isn't an individual pursuit, it's something we do with each other. The internet is where people are.

the internet isn't an individual pursuit, it's something we do with each other

My last afternoon in Colorado I sat down with my 5-year-old niece, Keziah, and tried to explain to her what the internet is. She'd never heard of "the internet," but she's huge on Skype with the grandparent set. I asked her if she'd wondered why I never Skyped with her this year. She had.

"I thought it was because you didn't want to," she said.

With tears in my eyes, I drew her a picture of what the internet is. It was computers and phones and televisions, with little lines connecting them. Those lines are the internet. I showed her my computer, drew a line to it, and erased that line.

"I spent a year without using any internet," I told her. "But now I'm coming back and I can Skype with you again."

When I return to the internet, I might not use it well. I might waste time, or get distracted, or click on all the wrong links. I won't have as much time to read or introspect or write the great American sci-fi novel.

But at least I'll be connected.

Video by Jordan Oplinger & Stephen Greenwood
Editing by Jordan Oplinger
Audio mixing by Brendan Murphy
Special thanks to Billy Disney, John Lagomarsino, Regina Dellea, Ross Miller, Ryan Manning, Sam Thonis, and Thomas Houston

Photography by Michael B. Shane
Art Direction by James Chae

First Person Shooter: Taser's new cop-cam takes aim at perps and privacy

Matt Stroud

Who am I? Data and DNA answer one of life’s big questions

Laura June

How emoji conquered the world

Jeff Blagdon

Seduced by ‘perfect’ pitch: how Auto-Tune conquered pop music

Lessley Anderson

I used Google Glass: the future, but with monthly updates

Joshua Topolsky

Drones over US soil: the calm before the swarm

Ben Popper

iTunes Store at 10: how Apple built a digital media juggernaut

2013-04-26T15:56:54Z

Additional reporting by Greg Sandoval

Ten years ago this month, a music sector ravaged by Napster and largely ignorant of digital distribution found a savior of sorts in what was then called the iTunes Music Store. With its 99-cent unbundled songs, the service quickly became the only significant source for acquiring music legally online.

With iTunes, Apple had drawn the blueprint for distributing music, movies, books, and apps over the web. By supplying and tying together a music player, online store, and song-mangement software, Apple drastically simplified the entire music experience, defying the odds to build a music-retailing dynasty even as file sharing skyrocketed. A decade ago, Apple started to answer what would become an all-important question: how do you get consumers to pay for content again?

"They invented the digital music business," said Michael Nash, the former digital chief at Warner Music Group. "Apple really created the convergence of music and technology and showed everyone what the connected economy around content looks like."

Now known simply as the iTunes Store, the music, movie, TV, book, and app marketplace celebrates its 10-year anniversary on April 28th. Few should be singing Happy Birthday with more zeal than those at the major entertainment companies. And now, as the iTunes Store enters its second decade, there’s a growing sentiment that iTunes has become bloated and stagnant, that Apple is resting on its laurels and failing to innovate while a new generation of music services begin to find an audience. Over the next ten years, will the company be able to evolve its longstanding business model and keep dominating in the face of upstart competitors?

A new model

In the early 2000s, web piracy began to mushroom and digital music services launched by the large record labels — MusicNet and Pressplay — were busts. There was a "scramble in the music industry to create a service to answer the marketplace," according to Paul Vidich, the former executive vice president of Warner Music Group and the first label exec to cut a licensing deal with iTunes.

In April of 2003, the iPod was already drawing intense consumer interest — Apple claimed the device was the number one MP3 player in the world with over 700,000 sold. (For context, Apple sold 5.6 million iPods in the last quarter, despite the continued downturn in the MP3 player market.) Getting content for that iPod, however, was a bit of a mess — it seemed nobody knew how to build an easy-to-use web music service. Napster may have shut down in late 2001, but P2P music sharing was already out of the bag — and the options for legally acquiring music online were poor.

However, Apple aimed to change all that with the iTunes Music Store, the first real a la carte download music service, built directly into the software. At launch, it was Mac-only and offered a relatively tiny catalog: 200,000 songs (it currently has 26 million). But it did have the support of the major record labels of the day: Universal, EMI, Warner, Sony, and BMG. The partnerships were key to helping Apple take control of music distribution — without the songs, the iPod was a nicely designed but empty box.

Steve Jobs' salesmanship helped close those deals. Vidich remembers the Apple CEO flying to New York to demonstrate iTunes for Warner execs sometime around September 2002. CEOs didn't often handle product demos but Jobs "was so exuberant about iTunes and its simplicity," Vidich said. "We were too. The other products out there just weren't simple to use."

Jobs certainly had his challenges. Vidich said he's the one who suggested that iTunes charge 99 cents per track and he remembers Jobs nearly hugged him. At the time, Sony Music execs wanted to charge more than $3 a track, according to Vidich. No doubt a $3 song price would have tied an anchor around iTunes' neck, stifling growth. 99 cents, on the other hand, was below the sub-$1 psychological barrier — and has continued to be an important price point for not only music but the wide swath of 99-cent iOS apps in the store.

Eventually all the major record companies signed one-year deals, and at the conclusion of the store’s first year, the labels found themselves captives. iTunes sales had grown so fast and the buzz was so electric that Jobs held all the leverage in subsequent negotiations. Apple sold one million songs in the first week and 10 million by September of 2003. In its first year, the company sold 50 million. "If you were in that space and you weren't supplying iTunes," Vidich said, "you weren't cool."

Apple’s most important move to make itself cool was to create of one of the most memorable ad campaigns of the last decade: the famous dancing silhouette ads.

In contrast to Apple’s typical spots focusing on simplicity and design elements, its iPod / iTunes ads featured loud music, quick movements — and no clear shot of the product itself. The ads sold you music and white earbuds, with the rest left to your imagination. When the iPod was first released, those white headphones were an exotic rarity that pointed to someone carrying a piece of bleeding edge technology — but after Apple’s silhouette commercials, they soon became a constant reminder of Apple’s dominance in the music world.

All the major record companies signed one-year deals, and found themselves captive to iTunes explosive growth

"They were the undisputed leader with no one touching them," said Jason Hirschhorn, a former exec at MTV, MySpace, and Sling Media. "They were the hottest thing in pop culture. They were the devices that all the media execs carried." Media execs were hardly the only ones. The iPod was already the top-selling MP3 player on the market, but the iTunes Store launch helped increase the size of that market exponentially — and Apple never came close to losing its dominant position.

At the core of Apple's success was the airtight combination of iTunes software, the iTunes Store, and the true ace up its sleeve: the iPod. The power of this combo can’t be overstated — everyone wanted an iPod, and they already used the iTunes software to manage the iPod’s content. The DRM included in all iTunes files (until May of 2007, when Apple began the process of going DRM-free) meant that its files couldn’t be used on other MP3 players, and most other stores’ music wasn’t compatible with the iPod. Competitors were locked out.

Apple bet that the majority of consumers wouldn’t have an issue with its lock-in tactics, and it bet correctly. They got to use the hardware they wanted and had a best-in-class online music store to fill it up. Of course, the iPod continued to support standard, DRM-free MP3 files, so critics couldn’t say that Apple was forcing users to buy music from the iTunes Store. Still, the airtight hardware and software model made it easy for consumers to increasingly spend more of their music budget with Apple.

One million songs were downloaded in the store’s first week, 25 million by the end of 2003, and one billion by February of 2006. iPod sales responded in kind, jumping from under one million in 2003 to over four million in 2004 to a staggering 22.5 million in 2005. By the time iPod sales reached their peak at nearly 55 million in 2008, the iTunes Store had supplanted Best Buy as the number one music retailer in the US. Less than two years later, in February of 2010, iTunes became the number one music retailer on the planet.

The legacy

As the iTunes Store became the model for selling digital music, Apple’s competition scrambled to find a way to keep up with the company’s runaway success. Since most competitors didn’t have the same three-pronged store / software / hardware system, most took a different tactic — they focused on Apple’s lock-in.

Microsoft spearheaded the movement with “PlaysForSure,” a partnership between compatible hardware manufacturers and music stores (most of which focused on subscription services). It launched in October 2004 and was an abject failure almost from the get-go. Most PlaysForSure-compatible music stores closed after a few years, and hardware partners like Archos, SanDisk, iRiver, and Creative eventually found themselves niche players.

"To say that Microsoft can just decide to copy (iTunes) and copy it in six months — that's a big statement. It may not be so easy."

As for most of the storefronts of the day, such as Aol MusicNow, MSN Music, Yahoo Music Unlimited, Sony Connect, MTV Urge, SpiralFrog, or Ruckus, few survived longer than a year or two. Microsoft itself abandoned PlaysForSure after less than three years and changed strategies with the launch of the Zune — an MP3 player that came with an iTunes-esque store-and-software combination. It never quite found a market, either, though a lot of what Microsoft learned from the experienced has gone into Windows Phone and Xbox Music.

That some of his competitors would stumble trying to duplicate iTunes success didn't surprise Steve Jobs. He told Rolling Stone not long after iTunes launched: "To say that Microsoft can just decide to copy (iTunes) and copy it in six months — that's a big statement. It may not be so easy."

As iTunes’ success grew, so did its content offerings — one of the biggest facets of Apple’s dominance in the digital landscape was how it took its findings from selling music and applied them to TV shows, movies, and eventually apps. The company built both a distribution model and a usage model (the now-loathed iTunes syncing process) and used it to keep adding more and more media to the three prongs of its ecosystem.

The ‘iTunes killers’

Aol MusicNow: Purchased from Circuit City in 2005, shuttered in 2007. Offered a la carte downloads as well as subscriptions.
Napster: Originally known for being the first P2P music sharing service to gain a widespread following, relaunched numerous times by Roxio and Best Buy before eventually being purchased by Rhapsody.
Yahoo Music Unlimited: Opened in May 2005, shuttered in September 2008. Users could transfer their subscriptions to Rhapsody. Subscription based; users could pay extra to download individual songs to burn to CD or put on a portable player.
SpiralFrog: Opened in September 2007, shuttered in March of 2009. Ad-supported and allowed free downloading of music. Only offered songs from two of the top labels. Tracks couldn’t be burned to CD or played on an iPod or on Macs.
MTV Urge: Opened in May 2006, merged with Rhapsody in August of 2007. Was built into Windows Media Player 11 and offered subscriptions as well as single-track purchasing.
MSN Music: Launched in 2004, closed in November 2006 as Microsoft began focusing efforts on the Zune hardware and Zune Marketplace. Started with 1.5 million songs, but trimmed to 1.1 million songs due to licensing problems and lack of support.

Imeem: Founded in 2003, the service evolved into a social network that let users share tracks with each other. Like SpiralFrog, it offered free listening to users and then tried to support itself through ad sales. MySpace acquired the company's assets and shut down the service in 2009.
Project Playlist: Opened in 2006, it let users locate freely available songs online (typically pirated) and create playlists that could be posted to MySpace and Facebook. The labels sued and then prompted MySpace to block the playlists. The company went bankrupt in 2010.
Sony Connect: This was Sony's failed attempt to duplicate iTunes. While Sony could never get Sony's music label, software, and hardware divisions to cooperate, its efforts were plagued most by poor software. The store closed in March 2008.
MOG: Founded in 2005, the paid subscription service competitive with Rdio and Spotify sold out cheap to Beats Electronics, the company behind the ubiquitous headphones.
Zune Marketplace: Microsoft’s attempt to recreate Apple’s integrated strategy launched in 2006 alongside the Zune hardware, offered a la carte downloads as well as the “Zune Pass,” a music subscription that also included free song downloads each month. Transitioned to Xbox Music in October 2012, though desktop, mobile, and Xbox 360 apps still work.

The App Store

While the iTunes Store was a near-instant hit in the music space, it didn’t spark the same revolution in video. Originally, use cases for TV shows and movies from iTunes were rather limited — you could watch on your iPod’s tiny screen or on your computer, but Apple didn’t have a straightforward way into taking over the living room. The launch of Apple TV in 2007 helped, but it was famously a “hobby” product for the company for years. Recent revisions and improvements to the store and Apple TV have made it a strong contender for the living room, but it was a while before video was considered one of Apple’s strengths.

If music was a hit and video was more of a slow burn, the launch of the App Store in 2008 is probably best described as a rocket igniting. The June 2007 launch of the iPhone was even more disruptive than the iPod, and consumers quickly clamored for the ability to extend the expensive device’s features beyond the 16 apps Apple included when it launched. It didn’t take long for Apple to get the message. Despite Steve Jobs’ insistence that web apps in Safari would be the ideal mobile solution for developers and consumers alike, Apple announced the forthcoming availability of a full iPhone OS SDK in October of 2007. Over the App Store’s first weekend in July 2008, consumers downloaded a staggering 10 million apps — the familiarity with Apple’s digital marketplace, the abundance of high-quality free selections, and the pent-up demand amongst iPhone owners meant the App Store was an immediate success.

Having a vibrant, third-party app ecosystem became the defining feature for a smartphone

“The App Store changed everything,” said Jeremy Olson of Tappity. “It made selling software so easy that anyone could do it, and it made buying software so simple and affordable that everyone does it.” Part of the App Store’s power was that it leveled the playing field between giant companies and independent developers. “My dinky three-person team has built apps that at certain points in time were some of the highest selling apps on the whole store, dominating all the huge competing brands,” said Olson.

For some developers, the App Store provided a significant financial windfall. “Without much to lose, I founded App Cubby on a $20k loan from family members,” says David Barnard, who was “completely broke” when the App Store launched. “Over the next five years App Cubby grossed well over $1 million.”

Almost overnight, having a vibrant, third-party app ecosystem became perhaps the defining feature for a smartphone — and the lack of one would quickly lead to hard times, as seen in the struggles endured by Palm’s webOS and eventually even the powerful BlackBerry brand. Google, Apple’s main competitor in the mobile device space, certainly took the lesson to heart. Its Google Play market for the Android OS covers nearly all the same bases as iTunes, with vast selections of music, movies, TV shows, books, magazines, and apps.

Side effects

Despite its success, Apple's iTunes has received its share of criticism. Those signature white earbuds delivered poor-quality sound (though they were better than many pack-in headsets of the time). More concerning to artists was the concept of selling compressed files — the quality they had painstakingly crafted was lost in Apple’s 128kbps AAC compression. As for the overall health of the music sector, two years before iTunes launched the labels generated $14 billion in revenue. Sales last year were half of that. Some critics feel that Apple helped strip the value out of music.

Indeed, iTunes hacked away at the dominance of the album as a sales unit and simultaneously tapped into consumer desire to be more selective about the music they owned. Apple’s business model brought back the single, which up until the early 1990s was one of the primary formats for the recording industry. The single all but vanished with the rise of the CD, and music fans were forced to pay for entire albums to get the songs they wanted. Apple unbundled songs, sold them for less than a buck — and paved the way for the CD’s eventual extinction.

The ability to buy nearly any single song without needing to buy the whole album really pushed the concept of the “digital mixtape” into high gear. Music listeners could now easily experience what Apple promised back in 2001 with its “Rip, Mix, Burn” commercial, and playlist curation and sharing has only grown in popularity since then. If you’ve ever spent any time using Spotify, you’ve likely come across all types of user-created playlists in a wide variety of themes — the iTunes Store helped popularize that concept. Plenty of artists didn't like it — AC/DC, Jon Bon Jovi, and Kid Rock were among those that criticized Apple's practices or withheld their songs from the service.

Of course, iTunes wouldn’t still be here if the Store and the iPod hadn’t been easy and fun to use, but Apple nailed both the hardware and the user experience out of the gate. It’s easy to forget a decade later as both the iTunes software and Store have become bloated, but once upon a time iTunes was a far superior option to most other music-management players. There’s no doubt that the iPod’s excellent UI, small size, and solid battery life were a major step forward from the clunky “Jukebox” players of the day that offered high storage capacity but little else.

As someone who grew up dealing with a number of pre-iPod MP3 players, I found the simple experience of buying new music on iTunes and plugging in the iPod to automatically sync new content to be a vastly improved user experience. Even now, just holding the iPod hardware, with its signature shiny, scratch-prone back and then-ubiquitous click wheel, brings back memories of a time when digital music made the huge leap beyond laptop speakers and burnt CDs. And being able to wake up, download a brand-new album you’ve been waiting for, and immediately take it out the door with you made iTunes’ tradeoffs well worth it.

The next decade

Looking forward, Apple remains in a healthy position with content creators and publishers, despite the occasional tensions. While the top film studios and major labels have sought to nurture Apple's competitors to help loosen Apple's control of retail sales, the links between the company and its suppliers are strong, says former Warner Music Group exec Michael Nash. He cites Apple's decision to wait to launch iTunes Match, its $25-per-year cloud music locker, until it had obtained all the necessary licenses — even while Amazon and Google launched unlicensed cloud services free of charge.

"Amazon and Google didn't believe in the value proposition around paid lockers," Nash says. "Apple was willing to wait to get the deal done and go into the market with a paid locker service. That represented their commitment to the content owners. They were more interested in getting it right and believed that consumers wanted the features of a licensed service. Apple is a good partner and has shown it values content."

"We think subscriptions are the wrong path. We think people want to own their music."

While Apple may still be the dominant player in the digital music arena, cracks are starting to show in the company’s strategy. When introducing the iTunes Store 10 years, ago Steve Jobs said that ''these [subscription] services treat you like a criminal. We think subscriptions are the wrong path. We think people want to own their music.” While download sales are growing modestly, subscription music is finally making a dent and seeing white-hot growth — companies like Spotify and Rdio are doing subscriptions right after the mistakes of their predecessors. These services, which the labels call "access models," have grown in a very short time to make up 15 percent of the industry's total revenue (pdf).

Apple may now let users load songs from the cloud and access them via the internet, but the iTunes business model has remained largely unchanged. It may be a proven strategy, but Apple appears to have no interest in finding a way to communicate with those who don’t care about "owning" music. While many consumers are still happy to buy music from iTunes, there’s a growing population moving to more innovative services that are finally doing streaming subscriptions without the many bad decisions made by the many "iTunes killers" that failed to make a dent. Of the top subscription and streaming services, not one has reported profits, so it isn't certain whether these services will pan out. But there's no doubt that they're starting to find an audience at the same time as iTunes — when it comes to music — appears to be standing still.

That said, Apple is still keeping pace with its main competition, and neither Amazon nor Google have launched music subscription services yet. Evidence is mounting that the company is ready to take its first stab at streaming music with a Pandora-like “iRadio” service this year, which would mark the most notable new feature for iTunes in years. An iRadio launch would certainly break iTunes’ long period of stagnation and cause the entire music industry to take note, but the question is whether or not it’ll be enough for the iTunes Store to continue a second decade of dominance.

Lead photo courtesy of Matt Olson

They're watching: why city-wide surveillance failed to stop the Boston bombing

2013-04-24T17:25:02Z

All day Sunday, police directed traffic around a blocked-off section of Boylston Street in downtown Boston where bombs had gone off nearly a week earlier, killing three and wounding hundreds. A makeshift memorial had been set up to honor the dead with personal messages and flowers, and old running shoes hung from metal barricades. Similar makeshift memorials were set up in suburbs outside the city proper, at least one designed in the dried blood of a dead suspect.

About a mile northwest from Boston’s ground zero, across the Charles River in Cambridge, a small but notable memorial had been set up. This one lay on the Massachusetts Institute of Technology’s campus, on a concrete walkway beside MIT’s Ray and Maria Stata Center. Famous for its Frank Gehry-designed architecture, Stata is all kitty-cornered frames and weirdly metallic boxes, like something out of a futuristic claymation video or Pee-wee’s Playhouse. This memorial seemed out of place. It was set up to honor an MIT police officer, Sean Collier, who had been shot and killed here while on duty, allegedly by the brothers suspected of carrying out the bombings downtown days earlier.

At MIT, as the sun shined brightly in a cloudless sky in the mid-afternoon on Sunday, a man named Robert Burke, who runs a small business video production company out of a Boston suburb, sat with a camera pointed at a pile of T-shirts, flowers, teddy bears, and candles. His complaint, like the complaints of many others this past week, was with the media establishment that had descended on his hometown first for the Boston Marathon, then for the aftermath of a tragedy, without much context.

“All these media people in town this week, and none of them have the real story,” he told me in a thick Boston accent. “Well, I’ll tell you the real story here: two terrorists assassinated a good cop.”

If the Boston bombings told us anything, it’s first that everyone has his own story. And second, it’s that cameras such as Burke’s — both those lodged in people’s hands and those installed by government agencies on buildings and roads — can allow everyone’s own story to do as much harm as it can good.

Too many theories

What we know so far — what we can really verify — is that the FBI charged 19-year-old Dzhokhar Tsarnaev, the lone surviving suspect, for detonating two improvised bombs with his 26-year-old brother, Tamerlan. We know five people died, including Tamerlan, in related events. Nearly 300 were injured. That’s all verifiable.

But there are many unanswered questions.

Just one example: when Collier, the MIT police officer, was shot on Thursday evening a little after 10:30PM, the circumstances of his murder were odd. This was during the post-bombing manhunt, 80 hours or so after the marathon ended prematurely. The officer was killed in his car, which meant he hadn’t attempted to chase anyone on foot. If we accept FBI allegations that the Tsarnaevs were involved in that shooting, we know they didn’t take the officer’s police car anywhere. And they didn’t rob him. So was it simply anger? Was he shot because he was a cop in the wrong place at the wrong time? Or was something else afoot?

A conspiracy theory — and again, it’s just one associated with this case — is that the Tsarnaevs had sought to toss one of their improvised explosives into MIT’s “little nuclear reactor” near the corner of Massachusetts Ave. and Vassar Ave. Officer Collier had stumbled upon their plot, the theory goes, and so he had to die.

AMERICA'S FIRST HIGH PROFILE TERRORIST ATTACK IN THE AGE OF OMNIPRESENT CONNECTIVITY AND NEAR-UBIQUITOUS SURVEILLANCE

Like many of the theories floating around Boston right now, chances are that’s not what happened. What the MIT shooting theory represents is just one example of a grassy knoll, second shooter-type guess offered mere days after the event took place. And again, that’s just one theory, from one event. Alex Jones offered another. He said the Tsarnaevs were patsies, set up by the FBI, the event a “false flag.” The Tsarnaevs' father is making similar claims. There are many more.

Everyone’s got their story, after all.

Especially now. The Boston Marathon bombing was the first time America has experienced a high profile terrorist attack in the age of omnipresent internet connectivity, in an environment of near-ubiquitous surveillance.

Reddit users responded immediately. The first post was a photo from Twitter that captured white smoke billowing skyward. That was followed by a live video feed of the finish line, and gruesome photographs of blood spilled on the street, people scrambling, police officers and EMS workers racing toward petrified people scattering for their lives. In turn, more videos, more news reports, and more tweets appeared by the second. The National Guard cleared Copley Square and established a 15-block crime scene. The events continued to unfold, live, across the internet, thanks to America’s conditioned response to anything that happens, good or bad: plastering it all over social media.

Not even law enforcement could resist the lure of the internet. Realizing they needed as much information as they could gather, and that the CCTV cameras near the finish line perhaps didn’t provide all the evidence they needed to identify a suspect, Boston police put out a call at around 4:30PM for video and images of the finish line. Shortly thereafter, when the FBI announced that it had no suspects, Reddit users had plenty of “leads”: suggestions about who the bombers may have been, as well as photos and videos that might lead police to the culprits. It wasn’t long before the Reddit approach to the investigation — guesses based on images and video floating around the web — would spill into the mainstream. On Friday, Boston police said they kept citizens abreast of updates via Twitter and governmental websites because the public speculation had reached dangerous levels.

On Wednesday, CNN’s John King erroneously reported that an arrest had been made, prompting both Fox News and the Associated Press to follow suit. On Thursday morning, the now infamous New York Post front cover was released. “BAG MEN: Feds seek these two pictured at Boston Marathon,” it screamed in bold, capital letters. A photo of two local high school students sat beneath the headline. The students had no connection to the plot whatsoever.

“[E]veryone thinks they’re an investigator,” said Howard Levinson, president of Expert Security Consulting in Norton, Massachusetts. “I would suggest that people should just move on with their lives and be more careful. But I think we know that’s not going to happen.”

The tech used to capture the Boston suspects

by Adrianne Jeffries

FINGERPRINT SCANNER (used by the FBI)

Early Friday morning, police engaged in an intense shootout with suspected bombers Dzhokhar Tsarnaev, 19, and Tamerlan Tsarnaev, 26. The chase ended when the older brother jumped out of the car and ran toward police wearing a bomb on his chest; he was gunned to the ground. Using a portable scanner, an FBI agent took his fingerprints and ran them through the agency’s database, which quickly gave them the suspect’s name.

The mobile fingerprint scanner came out in August 2011, according to the federal tech trade publication FCW, as part of its Next Generation Identification (NGI) system. The database, called the Repository for Individuals of Special Concern (RISC), contains the prints of more than 2.5 million people.

NGI is replacing the FBI’s old fingerprints database, adding the capability for “additional biometric data, such as facial and iris data,” GCN reported.

This is also the system that failed to make a match based on facial recognition, even though both suspects were in the database: Tamerlan because of his previous brush with the FBI, and Dzhokhar because he had a Massachusetts drivers license.

Cost: The NGI costs about $1 billion, according to the FBI.
ROBOTS (FBI, Mass and Boston police, Navy)

Robots were used throughout the investigation. A robotic arm attached to a vehicle reportedly pulled the tarp off the boat to reveal the second suspect. The navy’s bomb squad also brought in at least one robot to defuse bombs and inspect suspicious items that could have been bombs, although the number and model have not been revealed.

PackBot, a defense and security robot with multiple purposes include bomb disposal, was used to search the suspect’s car Friday morning.

PackBots were also used for rescue after 9/11.

Cost: While the price of a PackBot can vary depending on its configuration, the cost averages about $100,000, a representative from maker iRobot told The Verge.

THERMAL IMAGING (MA police)

An infrared camera attached to a helicopter was used to confirm that a human form — Dzhokhar Tsarnaev — was hiding inside a boat on Friday night.

The infrared camera was made by Oregon-based FLIR (forward looking infrared) which makes government and commercial surveillance and security systems. The Massachusetts State Police department bought multiple cameras from the company’s Star Safire line, first in 2004 and most recently in 2010, according to FLIR. This camera is specifically designed for airborne surveillance and search-and-rescue, and includes a thermal camera, TV camera, laser range finder (which determines how far away target is), and navigation computer.

This sensor can’t “see” through walls, roofs, or glass, but it was able to detect heat through the very thin tarp stretched over the top of the boat the suspect was hiding in.

Cost: Around $500,000 each, and the Massachusetts State Police own “a few” of them, a FLIR representative told The Verge.
CCTVs

Boston’s extensive CCTV network assisted in identifying the alleged bombers. In 2007, the ACLU found that 147 total cameras had been installed in Boston. A year later, that number increased to 183, and that number has grown as the city received additional federal grants for more cameras. There are now more than 500 cameras in the MBTA system.

Cost: The total cost is difficult to say, as the city won’t release data on how many cameras are in place. However, the first round of installations, which took place before the 2004 Democratic Convention, cost upwards of $6 million dollars.

We got the bad guys

This is not how it was supposed to play out.

In the wake of September 11th, 2001, we have been constantly promised safety and security in exchange for submitting to increased surveillance. The grant application for the Federal Emergency Management Agency’s Homeland Security Grant Program — the principle program that helped bring all those surveillance cameras to Boston — is very clear on this. Millions of dollars were ultimately spent to prevent any “acts of terrorism and other catastrophic events” that might occur.

Boston’s CCTV infrastructure was installed in 2004, ostensibly to deter terrorists from attacking that year’s Democratic National Convention. It was paid for, according to a December 2008 Boston Globe report, as part of a $4.6 million federal grant from the US Department of Homeland Security. By the end of 2004, there were 59 cameras installed in Boston metro area. By 2007, according to the American Civil Liberties Union of Massachusetts, there were 147.

Few in the Boston metro area questioned this program. The Globe report quoted a police official: “There was no debate in Boston.”

But debate did, in fact, occur in a suburb west of Boston called Brookline. With a population of about 60,000 people, Brookline is not a huge or particularly unusual place. But its local legislative arm threatened to reject federally provided CCTV cameras within its borders.

The ACLU of Massachusetts spoke out in the town’s favor.

“The overarching concern is what kind of society are we creating, where general police surveillance cameras are in operation,” an ACLU attorney told the Globe. “You cannot assume that we will always be a free society, and we are putting the structures in place that would allow a very different United States of America from the one we have lived in.”

But Brookline — much like the rest of Boston, much like the rest of the US, and much like cities worldwide — acquiesced. In 2009, they agreed to take the cameras. Brookline’s concerns — unique as they were — ended in compromise. It would take the CCTV cameras. But it would only keep them running from 10PM to 6AM.

Which is consistent. Concerns about privacy have a tendency to diminish in the face of fear.

Despite those concerns — and studies, and anecdotes, and journalism showing that CCTV cameras don’t deter crime — the ACLU estimates that more than 500 of the cameras have been installed in the greater Boston area.

But what have they done for Boston?

Most news outlets have focused on the bombings and the investigation. The results: first, we got the bad guys. And second, we got the bad guys because CCTV cameras allowed law enforcement agencies to find and hunt down two terror suspects.

But that was never the point of FEMA’s Homeland Security Grant Program. The main goal was to “protect against” and to prevent terrorist attacks in the first place.

The marathon bombings represented the first real test of that program.

And it failed.

IT WAS THE FIRST REAL TEST OF FEMA'S SURVEILLANCE GRANT PROGRAM — AND IT FAILED

View Shootings 4/19 MIT && Carjacking related to boston bombers in a larger map. Created by Sam Twining.

The trove

The capture of two suspected terrorists for killing four people with “weapons of mass destruction” was perhaps made easier with the use of cameras. But it was not worth what Boston gave up in privacy and funding over the past few years. And if you look at it in a certain light, it may have actually made Boston a much more dangerous place.

At the very least, all that surveillance will severely complicate a court battle that’s only just beginning.

Kelly Currie is a former assistant United States attorney in the Eastern District of New York where he was chief of the eastern district’s Violent Crimes and Terrorism Section. He tells me last week’s bombings, and the unbelievable amount of video and photo evidence collected, will set new standards for how criminal information is processed.

“The sheer scale of the data available is monumental,” he said. “And we don’t even know the extent of it yet” — the extent of the video and photos from nearly unlimited sources, the extent of the tips being filtered in more forms than ever before, and the extent of mistakes, misinformation, and conspiracy theories being broadcast in all directions.

The sheer scale of the data available is monumental, and we don't even know the extent of it yet

This is going to be a problem for the courts because there are limited precedents for how this scale of information can be processed. A 2011 Vancouver investigation offers a model, but last week’s bombings and investigation were different. First off, the events that lead to an arrest in this case are not discrete, Currie said. There are hundreds, if not thousands, of witnesses who may come forward about the bombings as well as the carjacking, the shooting on the MIT campus, and the events that lead to Dzhokhar Tsarnaev finally being captured. All of these people could potentially provide video and photo evidence. Not to mention footage from 500-plus CCTV cameras. And all that evidence needs to be assessed by prosecutors as well as the federal public defender's office, which will handle Dzhokhar’s defense.

That might be impossible.

Commentators such as Farhad Manjoo have argued that the Boston bombings confirm the need for expanded federal funding for CCTV coverage in places such as Boston. Of “all the measures we might consider to improve security in an age of terrorism,” he wrote in Slate on Thursday, “installing surveillance cameras everywhere may be the best choice.” But that’s not a universal opinion. And some experts are saying the opposite.

The executive director of the ACLU in Boston, Carol Rose, is one of them.

“These terrible attacks confirm that surveillance doesn’t deter people from these kinds of serious crimes,” she said. “The Boston Marathon is about the most heavily surveilled event anyone could’ve targeted in Boston. And it didn’t deter anyone from setting off a bomb there.”

And while she admits that video evidence was involved in locating potential suspects in this case, it seems the best images provided to police were not provided by government-installed cameras but by cell phone photos taken by individuals.

She also suggested something darker: perhaps the network of cameras turned everyone in the city into a suspect, greatly contributing to the success of the terror attack.

Inaccurate reporting

by Adrianne Jeffries
April 15

AP: cell networks shut down
NBC: police guarding possible suspect at hospital
CBS: police have surveillance video of suspects with backpacks
WSJ: five undetonated bombs
AP: two undetonated bombs were found and defused
NBC: third unexploded device found

April 16

NY Post: “10 to 12 fatalities”
NY Post: police have identified a Saudi Arabian man as a suspect
NBC, CBS, and LA Times repeat the claim about the Saudi being questioned
April 17

CNN, then Fox News and AP, report an arrest has been made
Boston Globe tweets that a suspect is en route to the courthouse

April 18

New York Post cover pictures two innocent high school students under the headline, “BAG MEN: Feds seek these two pictured at Boston Marathon”

April 19

Reporters including BuzzFeed’s Andrew Kaczynski, Politico’s Dylan Byers, and Newsweek’s Brian Ries tweeted that missing Brown student Sunil Tripathi was named as a suspect on the police scanner; claim was repeated by local television news
NBC: says brothers are ages 19 and 20
NBC: brothers robbed a 7-Eleven

Section TOC Title

THE NETWORK TURNED EVERYONE INTO A SUSPECT, CONTRIBUTING TO THE ATTACK'S SUCCESS

She points out that the deceased suspect, Dzhokhar’s older brother, was on the FBI’s radar. He was even interviewed. But because of the overwhelming amounts of information at their disposal, FBI officials failed to follow up. “Part of the problem,” Rose said, “is that when everyone is surveilled, then authorities lose their focus” and civil liberties can be violated as a matter of procedure. Political speech can be targeted. People can be profiled by their race. People can be wrongly accused of perpetrating a horrific crime. Or investigated dubiously in place of a legitimate threat.

“We need to think about the purpose of these cameras,” she said. “If the purpose is to have a camera at an ATM to find people who are stealing, that’s one thing. But putting everyone into a database or recording every moment of every day from more than 500 cameras, the systems become overwhelmed.”

Rose is “not suggesting there needs to be a ban on surveillance cameras,” she said. “But if, in fact, the reason for using them is public safety, then there’s no reason all the innocent people who have been victimized by these awful attacks at the Boston Marathon need to end up in a criminal database or need to be used as a justification for expanded surveillance.”

“We’re part of a surveillance state that has made everyone into a suspect,” she said. “And if everyone becomes a suspect, then nobody is a suspect.”

Robert Burke, who sat for hours on a blue Coleman cooler filming Officer Collier’s memorial at MIT, wasn’t so sure about that. He, like many others in Boston, was just glad the bombing suspects — “the terrorists,” he was sure to clarify — are in custody. And as for adding his story to the media dump, he said he’s not interested. I prodded him repeatedly about who he was doing his video for, where he was planning to post it online. He told me nowhere.

“This video is for one person,” he said. “The officer’s father. He wanted to know why no one was covering his son’s death. So I said I’d do that for him.”

Burke had only met Collier’s dad once, randomly, at another memorial last week.

“Sometimes a story just isn’t for everyone.”

Adrianne Jeffries, Adi Robertson and Nathan Ingraham contributed to this report.

Photo Credits: Getty Images, C. Holmes, Shawna England, and Vjeran Pavic.

Sick idea: how rabies spawned vampires and zombies

2013-04-18T17:45:01Z

“What disturbs me is I smashed his mouth off, I smashed his teeth in, but he still wanted to continue in the attack mode. I was terrified at [its] resilience.”

This gory description could have been one of many zombie survivor stories from the novel World War Z, but it’s actually a man’s factual description the tenacity of a rabid raccoon he beat to death with a hammer in the non-fiction book Rabid. Many people are familiar with Max Brooks’ 2006 best-seller World War Z: An Oral History of the Zombie War (soon to be a film starring Brad Pitt), a novel about a global pandemic. People are probably less familiar with the 2012 non-fiction book by Wired Senior Editor Bill Wasik and veterinarian Monica Murphy, Rabid: A Cultural History of the World’s Most Diabolical Virus 9 (a 5,000 word excerpt can be found here). If Rabid walks us through humanity’s two thousand year history of trying (and failing) to cure this horrifying disease, World War Z shows us what the consequences of failing could be.

Connecting zombies to rabies isn’t a stretch, it turns out. Rabies is a real and terrifying neurological disease that, when contracted in humans, can make the victim appear zombie-like (fatigue, vision disturbances, slurred speech, loss of coordination) and, if the infection reaches the brain, it is one of the few diseases that is 100 percent fatal and for which there is still no known cure. Once bitten by an infected animal, the human victim begins showing symptoms much like the flu, becoming weak, feverish, and plagued by headaches for several days. If caught in the early acute stages, the virus can be treated with a vaccine and human immune globulin injections, but left untreated, the victim will begin exhibiting stronger feelings of anxiety and confusion, alternating between mania and sluggishness. As the virus spreads and infects the central nervous system, delirium and hallucinations set in, and the brain and spinal cord ultimately become fatally inflamed. The most recent World Health Organization numbers, updated in 2013, show that there are still over 55,000 deaths due to rabies every year, primarily centred in Asia and Africa. Fewer than five cases are reported per year in the US.

If World War Z weren’t so liberal with the use of the word zombie, one could be lulled into thinking it’s a non-fiction book. Told through a series of interviews with survivors of a global pandemic ten years after the initial outbreak, the novel’s clinical approach was inspired by Studs Terkel’s oral history of World War II, The Good War. While Max Brooks’ previous book Zombie Survival Guide was packed with practical knowledge (“The trench spike is the best compact anti-zombie weapon on earth”), World War Z opts for a more analytical perspective and a series of unreliable narrators. It begs the reader to think harder about the subject than they might otherwise. Where World War Z entertains, Rabid educates.

Rabies is a real and terrifying neurological disease that can make the victim appear zombie-like

Viral histories of the undead

In 1953, Richard Matheson, inspired in part by the movie Werewolf of London, wrote the iconic novel I Am Legend (yes, the one that became the Will Smith movie). It was a novel nominally about vampires, but not the rural cape-wearing paramours popular up until then. Matheson pitted a lone protagonist researching and testing a cure for a global pandemic that created a horde of base, animalistic vampires. Sound familiar? Matheson’s vampires are the embryonic version of the modern day zombie. Zombie master George Romero even admits he “basically ripped off” Matheson in writing one his first short stories about a dystopian world of the undead.

In Rabid, Bill Wasik and Monica Murphy actually name check World War Z in a chapter-length overview of the history of zombies and how they connect to rabies through their sister-monster: the vampire. Yes, zombies, part of our culture for less than a century, find their roots in the thousands of years old myths of vampires. There’s not just a casual relationship between the virus and the monster: zombies and their much older cousins, vampires, share a causal connection.

The myth of vampires, imbued with traits of rabies victims, was popularized long before we fully understood rabies

Spanish physician Juan Gomez-Alonso explains four connections between rabies and vampire myths in a 1998 Neurology journal article, the most obvious being infection through the blood via bites. Rabies victims also often suffer from facial spasms, lending them an animalistic appearance. The third connection is the time frame: vampire lore had them living for forty days before being turned, the same amount of time it usually takes for the victim of a rabies attack to die after their initial bite. The final connection is probably the most surprising: sex drive. The insatiable sexual desire that’s a trademark of both traditionally gothic and sparkly modern vampires can also be traced to rabies. Male rabies victims often get involuntary erections and have spontaneous orgasms. Unsurprisingly, this was not often spoken of outright, but was alluded to in much of the early medical literature, with one eighteenth century Austrian physician noting “his seed and his life were lost at the same time.”

The myth of vampires, imbued with traits of rabies victims, was popularized long before we fully understood rabies. Even further back than the vampire is a tale that is part myth, part metaphor: in Homer’s epics, the Trojan hero and Achilles’ foe Hector has a strong fury akin to primal possession that made him terrifying and superhuman on the battlefield. The word used to describe his state is lyssa. As Wasik and Murphy describe it in Rabid: “Lyssa was rare, terrifying, violent, and animalistically destructive of self. It made creatures maim and kill those closest to them. It hollowed out reason and left nothing but frenzy.” In Attic Greek (the prestige dialect of Ancient Greek that was spoken in Attica) lyssa literally translates to rabies. World War Z has its own version of the lyssa crazed warriors: quislings. Quislings were once humans, reduced below a feral state into thinking and acting as zombies. It’s an involuntary coping mechanism on the part of some weak-willed percentage of the population. They were so ingrained with the idea they were zombies, they’d lay motionless while they were eaten alive.

We personify and mythologize what we fear to try to understand it in whatever way we can.

Placebos, comforts, and cures

After a week-long coma and several weeks of rehabilitation, Jeanna was able to leave the hospital

Considering the high (100 percent) rate of fatality for rabies in the advanced stage, it’s seen its own share of placebo ‘cures’ throughout the ages, most of which have amounted to a mixture of palliative treatment and wishful thinking. Ancient recommended treatments outlined in Rabid include bleeding and then cauterizing the wound with clarified butter which the patient would then drink, applying a sesame paste to the wound, and then being fed a cake made of rice, roots, and leaves. The cauterization and bleeding make as much sense as anything, the rest is a comfort.

Even today the best treatments are still unproven. In 2004, 15 year-old Jeanna Giese was transferred to the care of Milwaukee physician Rodney Willoughby who confirmed with the Center for Disease Control (CDC) that she had rabies. Knowing there was no proven effective treatment, he proposed something unorthodox: since rabies seems to kill the brain faster than it kills the body, put the patient into a medically induced coma to give the body a fighting chance to kill the disease on its own. To everyone’s surprise it worked. No one knows whether it was the coma, the drug used, or Jeanna’s innate predisposition to survive. But after a week-long coma and several weeks of rehabilitation, Jeanna was able to leave the hospital. This treatment is now called the Milwaukee protocol.

World War Z never specifies exactly what disease caused the zombie plague or whether a cure was developed. One chapter, however, features a pharmaceutical bigshot who had developed a vaccine that turns out to be a glorified placebo.

“We knew it would work against rabies, and that’s what they said it was, right, just some weird strain of jungle rabies. […] We never lied, you understand? They told us it was rabies, so we made a vaccine for rabies. We said it had been tested in Europe, and the drugs it was based on had been tested in Europe. It protected them from their fears. That’s all I was selling.”

Just because the “cure” in World War Z may have been a placebo doesn’t mean it wasn’t a panacea. For people who took the placebo and who were then bitten by quislings, the zombie play-acting humans, the fact that they didn’t become infected was enough to stave off a national panic. Which goes to show, as anyone who’s seen the destruction caused by panic will tell you, a placebo can be almost as good as a cure.

World War Z doesn’t attempt to cure the zombies of their rabies-like disease, with author Max Brooks preferring to find hope in the silver lining of a society ably adapting to a world with zombies.

Which is why the scariest part of either book is this section towards the end of Rabid about where rabies research stands today, despite some early success with the Milwaukee protocol:

“Nevertheless, the medical establishment remains largely skeptical. At the time of his original publication, Willoughby declared his intention to set up animal studies to test some of his claims. […] Six and a half year later, these studies have yet to materialize. The basic reason is financial: Willoughby has not received enough funding to undertake the research himself, and meanwhile no other rabies researcher has made such efforts a priority.”

A deadly but overlooked disease. No money for scientific research. A lone doctor throwing up his hands. The medical establishment turning away from him.

… Isn’t this how zombie movies start?

Zombie photos courtesy Jere Keys, Tobias M. Eckrich and David Fulmer.

Talking heads: how a late-night hack turned into Facebook's next big thing

2013-04-16T16:17:43Z

On a Wednesday almost one year ago, Facebook product designers Joey Flynn and Brandon Walkin decided to work from home. They discussed how frustrating it is that modern smartphones aren't designed with texting in mind, since that’s what we’re doing most of the time. It's impossible to multitask while texting with friends — who are, more often than not, faceless entities organized by row inside a texting app.

"We had always talked about how apps with messaging components inside them are always the best," Flynn says. "We thought that it would be awesome if every app could have a messaging component." Or, what if your friends could somehow be one tap away as you found directions, looked up a restaurant, or responded to an email? Nearly two days later, "Chat Heads" started coming into focus — icons of your friends that stay with you no matter what app you're using. Tapping a friend pops your conversation with them into the foreground. With a flick, the conversation zips into the background.

They spent hours mocking up the idea, putting faces inside circles, and then squares, and then rounded rectangles. They decided on circles, a recent trend for avatar pictures, then added white borders, and then scrapped the borders. They stacked faces vertically, and then horizontally, and tooled with rubber-banding animations inside Apple’s Quartz Composer software. "It was one long night, one crazy idea, and that's how it started," says Flynn.

The duo threw together a pitch video and received a green light to begin production on Chat Heads: a new form of interactive and omnipresent notifications for your phone. CEO Mark Zuckerberg was impressed. "Everyone liked the idea," Flynn says. Chat Heads were to be the very literal faces of Facebook's new messaging platform on Android and iOS. These faces were impossible to miss, representative of a new vision of Facebook where your friends are their own floating heads.

Chat Heads started as an "experiment," like most new Facebook products, but was quickly turning into something much bigger. "Status updates and photo posts get more visibility, so people think that we only think about News Feed," Flynn says, "but private sharing is a really, really, important thing for Facebook." That message came through most clearly at the company's Home launch event, where Chat Heads were shown to sit (and bounce) on top of Home, a piece of Android software CEO Mark Zuckerberg called "the best version of Facebook ever." The News Feed may get stale, but Facebook's betting that private messaging never will.

Hacked together

In early 2012, Mark Zuckerberg initiated a company-wide shift towards putting "mobile first." From Messenger to News Feed, each team was responsible for its own mobile experiences across all platforms. The Photos team had recently launched Facebook Camera (in late-May 2012), and Facebook wanted to create more experiences that felt as natively mobile. But among all of Facebook’s mobile products, one was growing fastest: Messenger, which had launched in August 2011. Today, more than ten billion messages are sent each day using Facebook — up from one billion back in 2009. Facebook will not disclose what portion was sent using Messenger, but says that mobile messages have quadrupled since last year. Also worth noting is the number of private messages sent using Facebook for Every Phone, a pared down version of the social network for feature phones, which has has increased by 10x in the last year, Facebook says.

Deng saw the writing on the wall: the next age of Facebook might not be about the public web space, but about the more private mobile space. While Mark Zuckerberg's mission has always been "to make the world more open and connected," users were flocking to private mobile experiences like Messenger and apps like WhatsApp, Kik, Viber, Skype, KakaoTalk, and others. At the time, the mobile version of Facebook was essentially a mobile web News Feed filled with photos that took forever to load. Aside from a late-summer update that improved Facebook’s notoriously slow mobile app, the company seemed to be stuck in a rut, turning out new mobile features a few weeks after competitors proved them viable. The company’s first mobile phone collaborations with HTC were essentially feature phones with blue buttons to post status updates.

"If you think about the problem we're trying to solve, we can solve it on Android 100 percent of the time." Henry Bridge "Private sharing is a really, really, important thing for Facebook." Joey Flynn "We thought that it would be awesome if every app could have a messaging component." Joey Flynn "These devices are always logged on as you. They're always on and always with you." Peter Deng "What Poke showed us was that there’s a desire for both intimacy and reach." Aaron Goldsmid

"We realized there's a huge opportunity here," Deng says, "because these devices are always logged on as you. They're always on and always with you." New private messaging services continued to pop up all over the board, each with its own special feature. Voxer popularized sharing voice clips, Snapchat introduced self-destructing "ephemeral" pictures, and LINE introduced stickers, enormous emojis you can buy by the pack. One by one Facebook assimilated these concepts into Messenger.

The next age of Facebook might not be about the public web, but about the more private mobile space

"Move fast and break things," read countless signs on the walls of Facebook headquarters. A long string of experiments led Facebook to imitate others, but also to build a holistic vision for how messaging should work across platforms. Zuckerberg described the "Hacker Way" in a letter to investors in February 2012: "Hackers try to build the best services over the long term by quickly releasing and learning from smaller iterations rather than trying to get everything right all at once."

Poke, a shameless Snapchat clone, notoriously took just ten days to produce — suggesting that Facebook had plenty of great engineers, but lacked inspiration. "At the end of the day, we look at what users are demanding," Deng says. "Every week, I get an email from the user operations team saying 'here are the top requested features and here are the top bugs.'" Yet when asked about what make "ephemeral" sharing apps like Snapchat appealing to Facebook, Deng responded: "I don't really think much about, to be honest." He's more interested in finding out what makes a Snapchat conversation richer than a text conversation.

Poke may be more polished than Snapchat, but it didn't go viral and received mostly negative publicity. Yet, what matters most to the ex-Googler Deng is gathering data and making the most of his experiments, whether they succeed or fail. "What Poke showed us was that there’s a desire for both intimacy and reach," says Aaron Goldsmid, Product Manager on iOS. Facebook discovered that Poke users often sent the same message to multiple friends, and subsequently built an as-.yet-unnamed feature that lets you contact multiple friends individually within Messenger. The new feature may be useful when it launches in the coming weeks, but it sounds a lot like "broadcast," a feature WhatsApp and BlackBerry Messenger have offered for years.

Stickers, also launching today on iOS and Android, help paint yet another picture of Facebook miles behind mobile rivals. The company partnered with the Berkeley Social Interaction Laboratory, as well as with some top-tier digital illustrators like David Lanham, but as with Poke, well-constructed features have never been the issue. Snapchat hit the streets first, as did LINE, which reportedly earns more than $3.75 million per month selling stickers.

The goal is to encompass "the full spectrum of communication," as Goldsmid says. "When you look at what people are gravitating towards, the things we should focus on are very obvious," Deng says, but what about trying something new, or something innovative, or something bold? Along with turning out features that ape other messenger clients, Facebook was also working on something unique and innovative. Chat Heads were ready for prime time. It was a fresh UI idea combining SMS and Messenger to form something much more important: an always-on connection with friends.

Facebook finds a new home

Two weeks ago, Facebook made clear that the next big step for Messenger, but also for Facebook, lies on Android. The company partnered with HTC to produce a smartphone — a phone that elevates messaging above sharing statuses and taking photos. It spent months rewiring Android to serve one end goal: keeping you better connected with Facebook friends. "If you think about the problem we're trying to solve, we can solve it on Android 100 percent of the time, because we can have Chat Heads available inside other apps," says Henry Bridge, Product Manager on Facebook Messenger for Android and Chat Heads. Since the summer of 2012, what’s changed is the problem Facebook is addressing. During the presentation, Zuckerberg repeated over and over that 23 percent of the time spent on smartphones is spent using Facebook.

The first step was designing a new user interface from scratch. "I'd always heard these horror stories, where everybody [says] ‘are you gonna learn all the Android paradigms?’ and ‘Is it gonna take a long time to get up to speed on designing for Android?’" Flynn says, "and really, none of it makes sense. You're designing a product for people, and it doesn't matter if it's on Android or iPhone or Windows Phone or whatever it is." Facebook approached building Home and Chat Heads not as if they were apps, but as if they were something new. "We built an entire physics engine that everything runs on," he says. "It's more like your app is built like a game, as opposed to built with app components."

"Nobody was shouting at us asking ‘Where's the blue bar? This is a Facebook product!'"

"There's a difference between the aesthetic we've chosen for Facebook Home and for Chat Heads from other traditional Android apps," Bridge said, "but really we just wanted to create something that felt great and looked beautiful for users." And what they came up with is barely recognizable as Facebook. "Nobody was shouting at us asking ‘Where's the blue bar! This is a Facebook product!’" Flynn says.

The second step was actually building Home and Chat Heads using the Android SDK. Bridge and the Android team spent months rewriting key parts of Android like "scroll view," "list view," and "pagers" so popping in and out of chat conversations felt silky smooth. They also had to optimize system resources so an always-running version of Messenger wouldn’t zap battery life. Flynn insists that they’ve only scratched the surface of what Facebook could do with Android. Messenger for Android already integrates your onboard SMS texts, a smart way to consolidate your conversations while keeping you inside Facebook. Google Maps, Flynn says, could let you send a location by picking up a map and dropping it on a friend’s Chat Head.

The next all-nighter

In Facebook’s latest commercial for Home, a nameless employee idly flips through his Cover Feed and texts friends as Mark Zuckerberg drones on, unable to tear his eyes away from his phone’s screen. The nameless employee is none other than Joey Flynn. Chat Heads wasn’t the first time Flynn had his hands (and his mouse) on something big. He previously led design work on Timeline for mobile, a key part of Facebook’s mobile strategy. "We pulled so many all nighters," Flynn once said. "Everyone's there at 4AM on a Saturday night, but there's no second guessing your life choices. A real sense of camaraderie got us through it."

The story doesn’t seem all so different with Chat Heads, which in hindsight feel like much more than handy icons for access to texting. They feel like part of the "next Facebook," a social network — but also a utility, like Gmail, for communicating with friends. With Home and Chat Heads, Facebook is preying off the most addictive experiences it can currently provide: News Feed and private messaging.

Back when Flynn designed Timeline for mobile, he remarked that Android presented a great challenge. Today, Android is a big opportunity for Facebook. Flynn and co's invention works best on Android, and it’s likely to stay that way for the foreseeable future. Facebook, like Google, will continue dedicating resources to its iOS apps, but Android offers a more potent strategic opportunity. It also offers a chance for Facebook to be creative in ways it hadn’t before, to build a mobile communications experience from the ground up. A year and a half later, Facebook is finally mobile-first, but may have just found its first killer app.

Additional reporting by Dieter Bohn
Video shot by Jordan Oplinger and Billy Disney
Edited by Billy Disney
Sound mix by John Lagomarsino
Design by Scott Kellum

The Verge - Features

Google Glass apps: everything you can do right now

Glass basics

Setup

MyGlass Android app

Built-in apps

"ok glass" launcher

Google Search

"take a picture"

"record a video"

"get directions to…"

"send a message to…"

"make a call to…"

"hang out with…"

Google Now

Third-party apps

New York Times

Twitter

Facebook

Evernote

Tumblr

Elle

CNN

Glassagram

GlassTweet

Fullscreen BEAM

Glass To Facebook

ThroughGlass

Glassnost

Glass To Do

Thirst

Hacks

Winky

Final thoughts on first impressions

Read More:

Using Google Glass: stuck in trafficUsing Google Glass: at a Justin Timberlake concertI used Google Glass: the future, but with monthly updates

Exclusive: Inside Hangouts, Google's big fix for its messaging mess

A history of Hangouts

Cloud messaging

Major Google messaging moments

One Google

Read next First look: Google+ shifts the focus to better photos and big data

Body Message: GI Joe and the invention of the viral video

A meme is born

Hitting the big time

Feelin’ viral

Who am I? Data and DNA answer one of life’s big questions

A tree grows in Provo

All God’s children

So you want to make a family tree... now what?

Katie’s Family Tree

Katie’s DNA

The search continues

Genealogy Toolbox

Spitting image: DNA can solve this for you

How DNA is inherited

Offline: how to use the internet

I’m still here: back online after a year without the internet

I dreamed a dream

Back to reality

People who need people

Family time

12:00AM, May 1st, 2013

Read next

First Person Shooter: Taser's new cop-cam takes aim at perps and privacy

Matt Stroud

Who am I? Data and DNA answer one of life’s big questions

Laura June

How emoji conquered the world

Jeff Blagdon

Seduced by ‘perfect’ pitch: how Auto-Tune conquered pop music

Lessley Anderson

I used Google Glass: the future, but with monthly updates

Joshua Topolsky

Drones over US soil: the calm before the swarm

Ben Popper

iTunes Store at 10: how Apple built a digital media juggernaut

A new model

The legacy

The ‘iTunes killers’

Using Google Glass: stuck in traffic
Using Google Glass: at a Justin Timberlake concert
I used Google Glass: the future, but with monthly updates