Tim Cook says Apple wants to use AR to make conversations better

Photo by Dieter Bohn / The Verge

Apple CEO Tim Cook rarely provides details on unannounced products, but he offered some hints about Apple’s thinking on augmented reality and cars in an interview with Kara Swisher for The New York Times this morning.

When it comes to augmented reality, he agreed with Swisher’s framing that the tech is “critically important” to Apple’s future and said it could be used to enhance conversations.

“You and I are having a great conversation right now. Arguably, it could even be better if we were able to augment our discussion with charts or other things to appear,” Cook said. He imagines AR being used in health, education, retail, and gaming. “I’m already seeing AR take off in some of these areas with use of the phone. And I think the promise is even greater in the future.”

Apple has been rumored for years to be working on an augmented reality headset, and the latest leaks suggested a mixed reality device could launch next year. Augmented reality features are already available on the iPhone and iPad, but outside of some fun Snapchat filters, augmented reality hasn’t become all that widely used yet.

Cook also talked broadly about Apple’s approach to products during a question about cars. Leaks from Apple have made it unclear if the company is developing self-driving tech that it could license to other companies or if Apple plans to develop an entire car by itself. Cook’s latest comments suggest the latter, assuming the project comes to fruition.

“We love to integrate hardware, software, and services, and find the intersection points of those because we think that’s where the magic occurs,” Cook said. “And so that’s what we love to do. And we love to own the primary technology that’s around that.”

Cook referred to “autonomy” as a “core technology” and said there are “lots of things you can do” with it in connection with robots. But he warned that not every Apple project eventually ships. “We investigate so many things internally. Many of them never see the light of day,” Cook said. “I’m not saying that one will not.”

Swisher also asked Cook about Elon Musk’s comments about a failed attempt to discuss selling Tesla to Apple around 2017. “You know, I’ve never spoken to Elon,” Cook said, “although I have great admiration and respect for the company he’s built.”


If Apple’s vision for AR is looking at charts during a conversation I don’t think they’re going to be much more transformative than Hololense was.

I highly doubt Tim Cook summed up their vision for AR in one off-hand comment.

I was kind of joking but I do wonder if they have a good comprehensive vision for AR or if it will turn out like the Apple Watch where they release a few versions before they really figure out what it’s most useful for.

Apple Watch is useful for a lot of things, and this is a trend with most Apple products.

Telling people and being useful are two different things. The watch is still mostly fashion. Heard bout the health thing, but mostly seen people justify it to me. Without me even asking….
OK so you didn’t pull out your phone for a text… You saw me saw you then you explained….

Alternative view: if you’re going to start talking up your product, you put your best foot forward. I highly doubt that it was an "off-hand" comment in an interview with the press, and if there was something more interesting, then he would have mentioned it other than "charts".

Your alternate view betrays a lack of understanding of the way Apple deals with products prior to announcing them.

That argument doesn’t really address my point.

Yes it does.

No more tech that reduces eye contact please

Isn’t the point that this would be the only interface that would allow you to maintain eye contact instead of looking down at your watch?

I mean you’d be looking into a corner somewhere probably, as it wouldn’t be pasting graphs right over someone’s face

You say that, but I watched enough Dennou Coil to know that I wont be making eye contact in AR because there’s a giant demon jellyfish that’s trying to eat up the AR landscape and I need to collect AR pebbles and draw out summoning circles to stop the virtual jellyfish.

That sounds fun tbh

I’m curious to see if Apple will include a nice 6 dof controller (ideally even two) with their VR/AR device or whether they will make it all hand gestures in air or on hand/arm only (which would be a bummer for many game types, especially for a VR device)

Apple can’t compete in VR games, unless they reinstate support for OpenGL and/or Vulkan. No one is making serious games for devices that need to programmed in Metal (which still has very poor documentation btw, unlike DirectX).

Windows has great VR games already (like HL: Alyx) because of GPUs, excellent steam headset and controller (Index), and very accessible APIs. I can see them competing with Oculus for mobile quality VR games. though.

I can see them competing in AR space though, as there is no major player in this market yet.

I think that Apple competing with oculus in the VR space would be a real success for them. Oculus is the #1 VR headset and is streets ahead of the competition.

I agree partially with you.
First, yeah, i can see them competing in the AR space, too, there is basically no western company which has a strong AR glasses for consumers story/product right now, so that whole market is completely up for grabs still, and it is the biggest market.
There Apple generally has high chances i’d say, for multiple reasons.
Let’s say Apple and Facebook both make exactly the same AR glasses with exactly the same cameras etc etc, whom would most consumers trust more on privacy etc aspects?
Obvious answer.
I feel Facebook shot a major own goal there by insisting on abandonding the way more trusted Oculus brand and forcing everyone to use an FB account on Quest now, which will fire back bigtime wen they want to introduce AR glasses as a lot of people don’t trust FB on privacy aspects anymore, which is an even more important factor when one runs around with glasses with cameras on them in the outside world constantly instead of headset with cameras in the own four walls only, because in the outside world one also has to have all bystandars etc around one accept that one has such glasses on, which will be a tough sell with FB’s privacy reputation).
If i was at Oculus/FB, i would instantly reverse that decision to force using FB accounts, sorry FB, that ship has pretty much swailed regarding privacy acceptance, as bigger wise strategy move best thing to do is to not force FB accounts anymore.
Maybe the closest thing in appeal to Apple consumers AR glasses could be the snap glasses.
Of course all theoretical as we haven’t seen either of them in full feature set yet, could all still turn out to be disappointing, just talking about up front chances based on current market reputation and stance of company.
Also on tech side Snap is in good shape with their AR filters etc at least and Apple in good shape with their AR api and AR/ tracking/mapping tech in their iOS devices so far.
FB in great/best shape regarding tracking tech and some other key tech aspects, but in worst shape by far in privacy trust.

MS has an ok device with the hololens 2 for enterprise but it basically gave up on consumer market, atleast temporarily, and while that turned out as shortterm wise goal for them as by selling it in high price to enterprise they can have a profit margin while stepwise advancing the tech there, it also means they will likely loose out in consumer market and maybe then even in enterprise once there is a mass market ready compelling device by the competition which really captures a large portion of the consumer market.
See basically same story as with windows phone etc where by the time they made the switch to a device/OS which could be appealing to end consumers, it was already too late as iPhone and Android devices had alreadsy grabbed up the consumer markets largely and were too dominant together and so windows phone lost out bigtime in consumer market which then also with employes bringing their own devices to work cobbled up the enterprise market, too, away from more enterprise focus things like older windows phone and blackberry etc.
Ok, so that’s my short thoughts on AR, as i said, i think short term best chances for Snap and Apple, even if too early to say something definitive until we see their offerings in full.

Now with VR it is more complicated to predict anything.
Why? VR needs ideally highes end graphics hardware for the best fidelity in content and that is currently only available on PC.
At the same time what is the best selling VR device? The oculus Quest 2.
On PC i’d still say we have the best VR games and apps, but one can play them on Quest, too with link cable or even wireless with virtual desktop etc.
Then Quest also has the biggest and best standalone device software library and standalone app/game eco system.
So it wins massively in amount of content avaailable while also having the best inside out standalone tracking tech and also among the top 2 best controllers.
And for those who care also the best hand tracking for controllerless play.

Now personally, me, i don’t expect Apple to release a VR headset with great controllers on Oculus Quest/Rift or Valve Index controllers level.
Honestly, i’d be suprised if Apple even ships it with a controller which is way more than a remote shaped thing with very few buttons and no proper analogue stick, comparable to maybe let’s say a 3-6 dof oculus go kinda controller.

And that to me would automatically make it still ok for AR glasses usage but automatically quite bad to use for a lot of game types we already have for VR right now.
The 6 dof controllers with the analogue sticks and all the buttons, triggers etc are a big part of what makes controlling all those games so immersive and fun (and even feasible).
Hand tracking only using stuff can be fun, too, but is not suited for nearly as many games and also less immersive and more finnicky and less reliable and slower to use.
(When one holds some virtual object in the hand in VR it feels much better to hold an actual physical object like the controller in the hand, much more immersive, and the analogue sticks allow one to move around bigger distances better and the multitude of buttons are needed in many games, too and are the fastest most precise inout method, way more so than doing hand gestures in air).

So yeah, i most give Apple low chances in VR for in depth VR games due to the controller aspect basically, i would change my mind there if they surprise me with bundling it with two awesome 6 dof controllers.
Seeing how their only "controller" for games shipped in the last few years was the apple tv remote, i rate the chances as very low, because that is already as tv remote the worst remote ever invented by anyone in the world and as a game controller a laughing stock and overall in sum one of the main reasons the Apple tv largely went nowhere as a gaming and apps platform (next to other badly implemented aspects like extremely bad management of app eco system)
So yeah, i hope Apple surprises me with bundling good controllers, but upfront: chances: very low.

Where i don’t agre fully with you is that a major fail reason for them regarding VR would be that they don’t have OpenGL/Vulkan support, mainly because i think that aspect would not matter as much for this anymore.
Let me be clear, i have several times in the past moaned about/blamed Apple deprecating OpenGL while at the same time not jumping on board with Vulkan and instead doing their own proprietary (and way more closed black box) thing with Metal.
It has had many bad effects, like for example Mac loosing support for many opengl games of their back then already small game library and many other things.
So i get where you’re coming from.
But that all said, Apple has done some very clever moves in the past few years regarding bringing iOS apps and games to mac and streamlining their OS and and hardware pipeline and has delivered in great way on the hardware side at least with their first M1 macbook.
Now that is basically their entry level first laptop with own chip, and they still have a lot to prove regarding higher end models also/especially on graphics capabilities sides, but at least they already managed to make an extremely compelling first low end entry device case there.
On the software side it is completely nonsense right now because iOS apps are largely trash on mac currently because no touchscreen on mac and most iOS apps being trash to control without touch, but let’s see/hope, maybe they still address that sometime soon finally.

So anyway, back on the topic, that strategy overall is very powerful for Apple, for many reasons, for example the major game engines like Unity and Unreal already have full metal deploy support for a good while and with macs and then likely VR and AR devices using the same low end api, there will be engine support on day 1 at least by the major biggest engines. I would be surprised if in their unveiling they don’t right away have support for Unreal and Unity either on that day or within a month.

Of course it would still be better if they ran on Vulkan, because well, others generally push more on most cutting edge graphics things faster than Apple, but well, it would still be usable.

Where i see their biggest potential downfall for the VR headset (if they ship one before AR glasses and actually have any longterm support intent for the VR one, too) is next to the controllers if they screw up the app eco system.
Because some uninformed ones may still think the mobile app stores are a fine eco system because they are bringing in huge amounts of money overall and a small amount of apps is making huge amounts of money, too, but the reality is the mobile app stores are a a completly shit market place for the large majority of most app developers and both Apple and Google have not just not done anything to improve the situation, they actively made it worse and worse for most devs over the years.
Hard to get exact numbers but i’m pretty sure over 99% of apps there which are not free to play milking mechanics based don’t even make their dev investments back.

So yeah, there, as dev, there new Apple devices have A LOT to prove to me as a dev before i would invest anything into developing for them.
they would basically have to forbid free to play apps there or completely hide them on the store for me to even just consider ever developing for any of their new platforms ever again with own investment.

people moan about facebook this, facebook that, but besides many such concerns valid (see my above points for some) FB/Oculus has done a better job at creating and keeping up a sustainable quality markletplace for content for both devs and consumers over the past few years tfor the Quest han basically all others.
That could still go to shits, like, again, the push to enforce FB accounts usage was a very bad move for example (and there are a bunch of other stories i could tell where things go less than ideal) , but yeah, still, at least not (already) free to play or subs implicitly enforced like on mobile app stores due to completely f*ed up market place on the mobile app stores.

So yeah, long story short, Apple has strong chances on some ends, but also many points which aregue for how they could still massively screw this up, so it is not a sure hit homerun thing at all, so i will be watching this one closely.

A way to disengage ourselves from actual, face-to-face interaction? Apple, would you like that next trillion in small or large bills?

I’m curious to see if Apple will wait for Facebook to launch their AR Glasses first, see the adoption and release a killer upgrade later

View All Comments
Back to top ↑