Skip to main content

Frog's Mark Rolston: the 'Minority Report' interface is a 'terrible idea'

Frog's Mark Rolston: the 'Minority Report' interface is a 'terrible idea'


The Chief Creative Officer sounds off on a future dominated by voice, not indistinct black slabs

Share this story

Mark Rolston Frog
Mark Rolston Frog

Mark Rolston holds the title of chief creative officer for Frog, the venerable design company responsible for the look of Sony Trinitron televisions in the seventies and Apple's minimalistic "Snow White" design language of the eighties. In 1999, it was Mark who personally drove the creation of to become the top-grossing e-commerce site in the world. Now, as a C-level executive with 18-years at Frog, Mark's responsible for driving the global creative vision for the 43-year old organization. More specifically, Mark's helping companies remove the "computer" from computing, making it a diffused and transparent part of our daily lives.

Design is something we see every day thanks to the democratization of the art form. How can a consumer differentiate between design and decoration?

Good design is defined by a product's ability to address people's wants and needs. As long as we're focused on that as designers, we're good. I don't believe people consciously discern between whether a product's quality comes from "true" design or mere decoration. How we label things doesn’t really matter. It's academic. What matters is "did we solve the problem?"

Will the recent Samsung v. Apple decision help force a resurgence in creative innovation or contribute to stifling it?

"In terms of hardware design, that [Android handset] industry is stuck in a rut."

I expect that it will help, at least in the world of Android handsets. In terms of hardware design, that industry is stuck in a rut. Check out a retail store display of those things. It's a wall of indistinct black slabs with rounded edges. Consumer patience with this would likely have run out eventually, even if the courts had forced the issue later than they did. People want variety in life. As far as the software side of things, even without the court decision, each OEM is finding itself needing more competitive leverage in the Android market. They can't compete on price and technical innovations alone. As the Android OS matures, OEMs are learning to create their own variations without breaking the core compatibility. This is something the press rarely gets right: Customizing a device's OS is not a monolithic notion anymore. The platforms are getting sophisticated enough that we can customize key aspects such as the core apps, the desktop, and the unlock screen without breaking the underlying OS or app compatibility. It's a layered story.

With regard to mobile computing, what will replace the dominant "black slab" touchscreen uniformity of smartphones and tablets?

Certainly if you look back at the history of feature phones, you can trace a similar path from simple, black/grey hardware to more colorful, decorative, jewel-like designs. That's likely to happen again. However, this time around, there's something very different going on. The hardware is no longer really the story. The hardware is really just a minimalist frame for that touchscreen. It's where the action is and we're finding what really works is to get the hardware out of the way. That tends to drive a lot of similar designs, in terms of very simple and very similar devices. The focus is simply shifting away from the physical object. But I still believe we're going to see a lot of variation.

The bigger story is that we're going to eventually start to see devices, essentially computers, in new shapes. Some designed for our bodies, and others designed to be part of the rooms we live in. I'm looking forward to this future. I think we all love what computing does for us, but we don't like computers. We're babysitting them way too much.

When will we see the first Mark Rolston Kickstarter project, unencumbered by the "let’s play it safe" compromise that clients often seek?

Actually a good number of our clients are doing anything but "playing it safe." Many of them are asking for really amazing things. So I'm not exactly longing for a bigger challenge. We're in a very privileged position to be able to share in such a variety of challenges. So any outside project would have to be one hell of a moonshot. But don't count me out.

It's hard to imagine using a smartphone or tablet that lacks gesture support. However, the arm-extending gestures used to control home theater apps on TVs are a novelty at best, while many people cringe at the idea of reaching out to a touchscreen monitor to control Windows 8. Is it too soon to draw a conclusion on what that means for the computer interaction model famously depicted in the film Minority Report?

"Sigh... I hate that model."

(sigh...) I hate that model. On one hand, it's been great for a user interface to have become so recognizable by the general public. But really, it's a terrible idea. Few people are going to stand up and wave their arms around like that to operate a computer. In the movie, Tom Cruise is doing a lot of object selection, sorting, and editing. Those things work best with small hand movements. We require more motor control for that kind of work. But there is an emerging field of computing that does take advantage of more 'phatic' interactions. Doing simple things like turning lights on, opening doors, and signaling yes/no can be done very effective at the kind of full body level that we saw in Minority Report. Actually, we've been experimenting with this ourselves. We created a prototype called RoomE, where we can control basic events with our bodies and our voice. It's really cool. Check it out here.

The touchscreen existed for 20 years before it went mainstream. What technologies do you see around us now that could be mainstream if only a company had the courage to embrace it?

Voice control. I believe it's going to be big. So much of our interaction with computers has been with boxes. At first they were big, and now they are small, but we still have to to tappity-tap in one way or another to get these boxes to do things for us. But we're getting to a point where those boxes are smart enough that we can try putting them away and just using voice to control them for many situations. If you look at the full range of experiences you have with computers throughout your day, you'll find that many of those interactions are very small discreet tasks that might be replaced by voice, gesture, or other new models. Of course, our workstation activities, email, spreadsheets, web browsing, aren't going away. We just need a wider range of ways to interact with machines.

We've been discussing a so-called "internet of things" for decades, where everything is connected and we're inside of the machine. What's most important in driving that forward, and what are we still lacking to get there?

Well, as William Gibson said, "the future is already here, it's just not very evenly distributed." In that sense, a lot of what the internet of things promises is happening around us right now. Our phones make us persistent nodes on the network. So many of our homes now have tens of computers, but most of them are are small, single purpose devices. Wearable computing was once a fanciful concept. But today people wear wristbands and watches that track their body metrics and movement. All of this has becoming ordinary.

What's really exciting is that the data-side to the equation is really starting to step up. For example, the infrastructures of cities are becoming a source for massive amounts of useful information — social, commercial, traffic, security...There's enough information about people, places, and things now available on the internet that we can start to do really interesting things with all of our computing horsepower.

"We have more information than we have skills to turn it into useful knowledge."

What's still lacking is the interface. We have more information than we have skills to turn it into useful knowledge. It's a human problem, not for lack of the technology. We are still using computers that require a ton of babysitting and human guidance to get much done with them. We need more background, policy-driven computing. The real goal of the vision is a deep extension of our senses — more knowledge and more control of our world. We want to know more about people, more about the places we're in and where we are going, and more about the things we have and might acquire.

What will Frog be focusing on in ten years? In 2050?

Ten years is nothing. We've been around more than 40 years. If I look back ten years, we were pursuing primitive forms of some of the same problems. What then was merely a concept, today is a real development project. Ten years ago we were working with Motorola, Nextel, and others to create cellphones that had value beyond making phone calls. It was a struggle to do much of anything. Today a project like our Feel UX for Sharp is easily possible. So I can imagine that in ten more years some of our latest concepts will become real. Concepts such as RoomE might actually be in the market. I can hope so.

As for 2050, if you buy into the vision that computing is destined to disappear into the woodwork, then we'll likely be adopting new skills such as body choreography, voice dialog design, and more artificial intelligence-oriented design. We're already doing these things. The skills just aren't formalized yet. We're making it up as we go. But we've been here before. WiMP apps, websites, touchscreen software — we were there in the beginning for each of these.

"Industrial design is hot again."

Another trend that will play out in the next 40 years is in hardware design. We've seen the evolution of many forms of hardware, from direct expressions of function to just being window-frames for software. It might seem as if the importance of hardware is fading, but ironically, hardware design is becoming more valuable to us. Industrial design is hot again. It turns out that as things become more software-driven, more ephemeral, our need to connect with the visceral, touchable stuff increases.

If you think about it, 2050 is closer to today than when we started Frog in 1969. So we’ve already seen what this kind of change looks like. We thrive on it.

Read more 5 Minutes on The Verge here.

Today’s Storystream

Feed refreshed Sep 24 Striking out

External Link
Emma RothSep 24
California Governor Gavin Newsom vetoes the state’s “BitLicense” law.

The bill, called the Digital Financial Assets Law, would establish a regulatory framework for companies that transact with cryptocurrency in the state, similar to New York’s BitLicense system. In a statement, Newsom says it’s “premature to lock a licensing structure” and that implementing such a program is a “costly undertaking:”

A more flexible approach is needed to ensure regulatory oversight can keep up with rapidly evolving technology and use cases, and is tailored with the proper tools to address trends and mitigate consumer harm.

Andrew WebsterSep 24
Look at this Thing.

At its Tudum event today, Netflix showed off a new clip from the Tim Burton series Wednesday, which focused on a very important character: the sentient hand known as Thing. The full series starts streaming on November 23rd.

The Verge
Andrew WebsterSep 24
Get ready for some Netflix news.

At 1PM ET today Netflix is streaming its second annual Tudum event, where you can expect to hear news about and see trailers from its biggest franchises, including The Witcher and Bridgerton. I’ll be covering the event live alongside my colleague Charles Pulliam-Moore, and you can also watch along at the link below. There will be lots of expected names during the stream, but I have my fingers crossed for a new season of Hemlock Grove.

Andrew WebsterSep 24
Looking for something to do this weekend?

Why not hang out on the couch playing video games and watching TV. It’s a good time for it, with intriguing recent releases like Return to Monkey Island, Session: Skate Sim, and the Star Wars spinoff Andor. Or you could check out some of the new anime on Netflix, including Thermae Romae Novae (pictured below), which is my personal favorite time-traveling story about bathing.

A screenshot from the Netflix anime Thermae Romae Novae.
Thermae Romae Novae.
Image: Netflix
Jay PetersSep 23
Twitch’s creators SVP is leaving the company.

Constance Knight, Twitch’s senior vice president of global creators, is leaving for a new opportunity, according to Bloomberg’s Cecilia D’Anastasio. Knight shared her departure with staff on the same day Twitch announced impending cuts to how much its biggest streamers will earn from subscriptions.

Tom WarrenSep 23
Has the Windows 11 2022 Update made your gaming PC stutter?

Nvidia GPU owners have been complaining of stuttering and poor frame rates with the latest Windows 11 update, but thankfully there’s a fix. Nvidia has identified an issue with its GeForce Experience overlay and the Windows 11 2022 Update (22H2). A fix is available in beta from Nvidia’s website.

External Link
If you’re using crash detection on the iPhone 14, invest in a really good phone mount.

Motorcycle owner Douglas Sonders has a cautionary tale in Jalopnik today about the iPhone 14’s new crash detection feature. He was riding his LiveWire One motorcycle down the West Side Highway at about 60 mph when he hit a bump, causing his iPhone 14 Pro Max to fly off its handlebar mount. Soon after, his girlfriend and parents received text messages that he had been in a horrible accident, causing several hours of panic. The phone even called the police, all because it fell off the handlebars. All thanks to crash detection.

Riding a motorcycle is very dangerous, and the last thing anyone needs is to think their loved one was in a horrible crash when they weren’t. This is obviously an edge case, but it makes me wonder what other sort of false positives we see as more phones adopt this technology.

External Link
Ford is running out of its own Blue Oval badges.

Running out of semiconductors is one thing, but running out of your own iconic nameplates is just downright brutal. The Wall Street Journal reports badge and nameplate shortages are impacting the automaker's popular F-series pickup lineup, delaying deliveries and causing general chaos.

Some executives are even proposing a 3D printing workaround, but they didn’t feel like the substitutes would clear the bar. All in all, it's been a dreadful summer of supply chain setbacks for Ford, leading the company to reorganize its org chart to bring some sort of relief.