Skip to main content

Frog's Mark Rolston: the 'Minority Report' interface is a 'terrible idea'

Frog's Mark Rolston: the 'Minority Report' interface is a 'terrible idea'


The Chief Creative Officer sounds off on a future dominated by voice, not indistinct black slabs

Share this story

If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.

Mark Rolston Frog
Mark Rolston Frog

Mark Rolston holds the title of chief creative officer for Frog, the venerable design company responsible for the look of Sony Trinitron televisions in the seventies and Apple's minimalistic "Snow White" design language of the eighties. In 1999, it was Mark who personally drove the creation of to become the top-grossing e-commerce site in the world. Now, as a C-level executive with 18-years at Frog, Mark's responsible for driving the global creative vision for the 43-year old organization. More specifically, Mark's helping companies remove the "computer" from computing, making it a diffused and transparent part of our daily lives.

Design is something we see every day thanks to the democratization of the art form. How can a consumer differentiate between design and decoration?

Good design is defined by a product's ability to address people's wants and needs. As long as we're focused on that as designers, we're good. I don't believe people consciously discern between whether a product's quality comes from "true" design or mere decoration. How we label things doesn’t really matter. It's academic. What matters is "did we solve the problem?"

Will the recent Samsung v. Apple decision help force a resurgence in creative innovation or contribute to stifling it?

"In terms of hardware design, that [Android handset] industry is stuck in a rut."

I expect that it will help, at least in the world of Android handsets. In terms of hardware design, that industry is stuck in a rut. Check out a retail store display of those things. It's a wall of indistinct black slabs with rounded edges. Consumer patience with this would likely have run out eventually, even if the courts had forced the issue later than they did. People want variety in life. As far as the software side of things, even without the court decision, each OEM is finding itself needing more competitive leverage in the Android market. They can't compete on price and technical innovations alone. As the Android OS matures, OEMs are learning to create their own variations without breaking the core compatibility. This is something the press rarely gets right: Customizing a device's OS is not a monolithic notion anymore. The platforms are getting sophisticated enough that we can customize key aspects such as the core apps, the desktop, and the unlock screen without breaking the underlying OS or app compatibility. It's a layered story.

With regard to mobile computing, what will replace the dominant "black slab" touchscreen uniformity of smartphones and tablets?

Certainly if you look back at the history of feature phones, you can trace a similar path from simple, black/grey hardware to more colorful, decorative, jewel-like designs. That's likely to happen again. However, this time around, there's something very different going on. The hardware is no longer really the story. The hardware is really just a minimalist frame for that touchscreen. It's where the action is and we're finding what really works is to get the hardware out of the way. That tends to drive a lot of similar designs, in terms of very simple and very similar devices. The focus is simply shifting away from the physical object. But I still believe we're going to see a lot of variation.

The bigger story is that we're going to eventually start to see devices, essentially computers, in new shapes. Some designed for our bodies, and others designed to be part of the rooms we live in. I'm looking forward to this future. I think we all love what computing does for us, but we don't like computers. We're babysitting them way too much.

When will we see the first Mark Rolston Kickstarter project, unencumbered by the "let’s play it safe" compromise that clients often seek?

Actually a good number of our clients are doing anything but "playing it safe." Many of them are asking for really amazing things. So I'm not exactly longing for a bigger challenge. We're in a very privileged position to be able to share in such a variety of challenges. So any outside project would have to be one hell of a moonshot. But don't count me out.

It's hard to imagine using a smartphone or tablet that lacks gesture support. However, the arm-extending gestures used to control home theater apps on TVs are a novelty at best, while many people cringe at the idea of reaching out to a touchscreen monitor to control Windows 8. Is it too soon to draw a conclusion on what that means for the computer interaction model famously depicted in the film Minority Report?

"Sigh... I hate that model."

(sigh...) I hate that model. On one hand, it's been great for a user interface to have become so recognizable by the general public. But really, it's a terrible idea. Few people are going to stand up and wave their arms around like that to operate a computer. In the movie, Tom Cruise is doing a lot of object selection, sorting, and editing. Those things work best with small hand movements. We require more motor control for that kind of work. But there is an emerging field of computing that does take advantage of more 'phatic' interactions. Doing simple things like turning lights on, opening doors, and signaling yes/no can be done very effective at the kind of full body level that we saw in Minority Report. Actually, we've been experimenting with this ourselves. We created a prototype called RoomE, where we can control basic events with our bodies and our voice. It's really cool. Check it out here.

The touchscreen existed for 20 years before it went mainstream. What technologies do you see around us now that could be mainstream if only a company had the courage to embrace it?

Voice control. I believe it's going to be big. So much of our interaction with computers has been with boxes. At first they were big, and now they are small, but we still have to to tappity-tap in one way or another to get these boxes to do things for us. But we're getting to a point where those boxes are smart enough that we can try putting them away and just using voice to control them for many situations. If you look at the full range of experiences you have with computers throughout your day, you'll find that many of those interactions are very small discreet tasks that might be replaced by voice, gesture, or other new models. Of course, our workstation activities, email, spreadsheets, web browsing, aren't going away. We just need a wider range of ways to interact with machines.

We've been discussing a so-called "internet of things" for decades, where everything is connected and we're inside of the machine. What's most important in driving that forward, and what are we still lacking to get there?

Well, as William Gibson said, "the future is already here, it's just not very evenly distributed." In that sense, a lot of what the internet of things promises is happening around us right now. Our phones make us persistent nodes on the network. So many of our homes now have tens of computers, but most of them are are small, single purpose devices. Wearable computing was once a fanciful concept. But today people wear wristbands and watches that track their body metrics and movement. All of this has becoming ordinary.

What's really exciting is that the data-side to the equation is really starting to step up. For example, the infrastructures of cities are becoming a source for massive amounts of useful information — social, commercial, traffic, security...There's enough information about people, places, and things now available on the internet that we can start to do really interesting things with all of our computing horsepower.

"We have more information than we have skills to turn it into useful knowledge."

What's still lacking is the interface. We have more information than we have skills to turn it into useful knowledge. It's a human problem, not for lack of the technology. We are still using computers that require a ton of babysitting and human guidance to get much done with them. We need more background, policy-driven computing. The real goal of the vision is a deep extension of our senses — more knowledge and more control of our world. We want to know more about people, more about the places we're in and where we are going, and more about the things we have and might acquire.

What will Frog be focusing on in ten years? In 2050?

Ten years is nothing. We've been around more than 40 years. If I look back ten years, we were pursuing primitive forms of some of the same problems. What then was merely a concept, today is a real development project. Ten years ago we were working with Motorola, Nextel, and others to create cellphones that had value beyond making phone calls. It was a struggle to do much of anything. Today a project like our Feel UX for Sharp is easily possible. So I can imagine that in ten more years some of our latest concepts will become real. Concepts such as RoomE might actually be in the market. I can hope so.

As for 2050, if you buy into the vision that computing is destined to disappear into the woodwork, then we'll likely be adopting new skills such as body choreography, voice dialog design, and more artificial intelligence-oriented design. We're already doing these things. The skills just aren't formalized yet. We're making it up as we go. But we've been here before. WiMP apps, websites, touchscreen software — we were there in the beginning for each of these.

"Industrial design is hot again."

Another trend that will play out in the next 40 years is in hardware design. We've seen the evolution of many forms of hardware, from direct expressions of function to just being window-frames for software. It might seem as if the importance of hardware is fading, but ironically, hardware design is becoming more valuable to us. Industrial design is hot again. It turns out that as things become more software-driven, more ephemeral, our need to connect with the visceral, touchable stuff increases.

If you think about it, 2050 is closer to today than when we started Frog in 1969. So we’ve already seen what this kind of change looks like. We thrive on it.

Read more 5 Minutes on The Verge here.