Inside the labs where Netflix is trying to make televisions suck less

The designation program was unveiled back in January, but it’s only now beginning to make its way into this year’s crop of TV sets, something the company showed off to a small group of reporters during a day-long tour around its headquarters in Los Gatos, California, yesterday. Right now, only about 20 sets have been blessed with the designation, a figure Netflix hopes will grow throughout the rest of this year. Looking ahead, Netflix also plans to make its requirements a moving target, not only to keep up with new TV technologies, but to help push the ones it thinks should become new standards.

Getting Netflix’s seal of approval is not a simple affair. The sets need to pass certain benchmarks, many of which are based on speed. For instance, Netflix’s app needs to start up within a certain amount of time, as do its videos once you hit the play button on your remote. Netflix is also counting how long it takes when you come back to its app after doing something else on your TV. Does it take too long to get back to your show after jumping out to watch sports or catch the local news? Better try harder next year.

While those are examples of software, Netflix is also judging TV makers on their hardware. Its program requires TVs to turn on instantly, and it rewards sets that come with remotes with a dedicated Netflix button. Netflix considers these things necessary additions to help bring TV watching in line with what modern viewers have become accustomed to on mobile devices like smartphones and tablets.

"A lot of this was inspired by innovation that was naturally happening in the phone and tablet space," says Brady Gunderson, who is Netflix’s director of product development. "When I turn my phone on, I never really turned it on because it was never off — it just comes right back where I left off, in the app I’m on. If I’m reading a news article, I’ll be right back to that article. TVs, meanwhile, when you turn them off and turn them on again, there’s some time to reboot, they lose all context, and the network has to come up."

"They weren’t purpose built for internet TV," he adds, saying that all that’s changing this year. "There’s some quantum leaps in terms of improving that experience. We think of this as the year the smart TV got smart."

To illustrate how quickly things have progressed, Gunderson — who is joined by his colleague David Holland, a director of business development at Netflix — stage a competition between last year’s crop of smart TVs to this year’s, some of which have been designated with the new standard. In each case, the newer set handily beats the old one on things like turning on, locking onto a Wi-Fi signal, keeping users from digging to find the Netflix app, and eventually starting up a show. In one particular matchup between Sony’s old Bravia TV set and its newer one running Google’s Android, we were already watching the opening credits of a film while the older TV had only just put some text on the screen to let us know it was warming up.

The exercise is painful to watch: the older sets lag and struggle by comparison. Gunderson is careful to say that the program isn’t meant as a threat that people should just give up watching Netflix on older TVs, or set their busted old 4K TV from 2014 on fire just to put it out of its obsolete misery. "The message is not that this is bad," he says, pointing to the older set. "It’s just about how much better these newer TVs are."

Much of the improvement has centered around a real sea change in the software TV makers are using. New in this year’s TV sets are things like Mozilla with its Firefox OS, and LG’s WebOS. There’s also Samsung with Tizen (which really only this year became ready for the world), and Sony embracing Google’s Android TV platform.

Netflix’s recommendation could carry a lot of weight with hardware and software makers. It commands a paying army of ravenous viewers 62 million strong worldwide, and which is set to grow as Netflix expands into new territories like Japan. Not only is the company bringing its customers movies and TV shows from studios, but it’s also making its own programming that can’t be had elsewhere (unless you’re willing to wait for the DVD version, that is). And some of those shows are being shot in 4K and with high-dynamic-range, things that TV makers like because it means that people who weren’t even thinking about buying a new TV set a year or two ago now might suddenly want one to experience these things.

While the side-by-side tests seem like a remarkably clear-cut way to test things, it’s not very scientific. That brings us back to the Shu. New TVs are bought from stores once they’ve been released, a process that like Consumer Reports’, involves sending out employees to make the purchases unbeknownst to the TV maker or any particular store that might try to weed out dud units. It’s then brought back to the cage of horrors, and run through a simulated home Wi-Fi network (they don’t do wired tests since most people with these TVs simply hook them up over Wi-Fi instead) that’s been throttled to match up with both good and bad speeds that can be expected in homes in America and elsewhere.

The entire recommendation program was born out of what Gunderson says became a common, and often bothersome question from people at barbecues and other social gatherings. "Because you work in TVs, they say ‘what TV should I buy?’ My answer was always ‘don’t buy one this year, wait until next year ‘ and it was never quite there. [Now] I do feel like it’s there — buy a TV this year."

That may be a fair assessment, but even some new model sets from major TV makers have not managed to make Netflix’s cut for one reason or another. Of the aforementioned requirements (things like the fast start and fast resume), Netflix is looking for at least five of seven criteria. That gives manufacturers some wiggle room to be approved, which Netflix points out does not cost set makers any money or require a fee for them to advertise their designation. But there have been cases where even high end, flagship sets from some TV makers are not approved. Right now that notably includes Samsung, which is the top TV maker in the entire world, along with budget set maker Vizio, neither of which have had any sets approved. Gunderson mentions both of these examples, then points to a small, inexpensive TV from HiSense that’s sitting in the Shu. It’s just been added to Netflix’s recommended list, despite its low cost and lesser picture quality compared to the sweeping Sony and LG sets mounted to the walls in the other room.

That brings up one shortcoming of Netflix’s big thumbs up for TVs: it’s not looking at things like image quality, sound, ease of use, ecosystem, and myriad other factors that can be really important when you’re buying a new TV set. Instead, Netflix’s program has been designed only to address what the company believes are things that make using its software on those sets the best possible experience. If you don’t even plan to use the TV’s built-in Netflix app, and believe you’ll use it through a set top box, or HDMI stick like Roku, then this will have little impact.

But it might not stay that way for long. Netflix has already begun pushing the technologies it’s using to present its programming. First it was 4K video, and now its high-dynamic range (HDR), technology designed to make the image on screen look more realistic, with brighter brights and darker darks, all intermingling in a way that is more pleasing to the eye. Holland says the company is incredibly wary about requiring these features in its evaluations given how new they are.

"We have a principle that in order for something to be included in the Recommended TV program, it needs to impact most of our members most of our time," he says. "For the same reason that 4K is not part of the spec today, [HDR is] being limited to very high-end TVs and there’s very little content available."

Two things that are likely to be part of next year’s requirements? More stringent requirements for speed, and making sure that users can easily return to Netflix if there’s an interruption. "We think the TV instant on feature could be faster. They take several seconds now," Gunderson says. "Next year, we think they could be that much faster." It’s a similar story for when you have to take a sudden break from watching something. "Someone’s calling you downstairs and you turn the TV off, ‘did someone call me?’ [you ask] No. You go to turn the TV on, and the app comes back to snow, or live TV, or your row of apps."

The unspoken genius about all this is that Netflix is very carefully pushing TV makers to design what it believes to be the perfect Netflix experience while avoiding the need to become its own hardware maker. That’s something the company briefly contemplated in 2006 in the early days of its streaming service, before realizing it was better to focus on making software for computers and mobile devices made by others. While no money is being exchanged in this new program, the benefits are being pitched as a two-way street: TV makers who play ball might snap up sales from consumers who trust Netflix and believe they are getting what Netflix deems an optimal experience. Netflix, in turn, has made sure that the buyers won’t get so fed up with the TV software that they give up trying to use Netflix. In fact, if Gunderson and Holland are doing their jobs right, Netflix will be the best part of using your TV.