It seems almost certain Apple will be betting heavy on AR and maybe VR in the near future. Is the M series that we have now up for task GPU-wise? More AR than VR? Does it remain to be seen once Apple comes out with their software and frameworks and vision and whatnot? I assume eGPUs are off the table, right?
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Silicon and VR/AR
- Thread starter subjonas
- Start date
- Sort by reaction score
Apple likely wants to do fully portable untethered AR/VR. That is probably a big part of the reason they have chosen to go at it alone with their own GPUs, since neither AMD nor Nvidia are especially concerned with power consumption at the high end.
Apple has all the cards for a successful VR push. Their GPUs are very power- and bandwidth-efficient, which makes them uniquely suited for the task. Everyone else is years behind in this regard. They also have very energy efficient ML accelerator tech (needed for eye tracking and AR), advanced spatial audio, low-power CPU technology etc. So yes, their VR product could be something else entirely.
Fun fact: Apple has been adding some Metal features that seem to be designed specifically for VR.
Fun fact: Apple has been adding some Metal features that seem to be designed specifically for VR.
Yeah that’s probably true the headset will be untethered. I forgot that the rumor is it’s getting some variation of an M1 (article link below). Surely with its performance to power draw ratio, it will leave current competitor standalone headsets far behind.
But as powerful as it may be, all standalone headsets will always be less powerful than headsets tethered to powerful desktops, and therefore standalone applications will always be more limited in potential. I believe that’s why some standalone headsets have a PC tethered option, to give them access to the more powerful experience. So I wonder if Apple would offer the same both options. Wouldn’t developers need a tethered headset anyway? If that is the case, then I wonder 1) how do current M series (I’m particularly curious about the Max) GPUs stack up against current PC GPUs for VR? And 2) even if they aren’t as powerful, might Apple have other advantages in hardware or software to offer an equivalent or better VR graphical experience compared to current tethered VR headsets? Or will Apple just not bother with anything that can’t be done with a standalone headset?
Either way, excited to see what Apple will bring to the AR/VR landscape.
Apple Planning to Replace the iPhone With AR Headset in 10 Years
Apple is planning to replace the iPhone with an augmented reality (AR) headset in 10 years, a process that is apparently due to start as soon as next...
www.macrumors.com
My personal speculation that they will prioritize smarts and everyday usability before raw power. They already have technology in place that allows them to focus GPU processing only on the areas where the user attention is, massively reducing the memory bandwidth and amount of computation needed. But I won’t be too surprised if ultra-high-end graphics won’t be the focus
I agree there’s a need for a tethered VR setup but I’m pretty sure that it’s not something Apple is going to do. I myself have a Valve Index connected to a Windows PC with a 3080 Ti and it is in fact superior to the untethered Oculus system but you’re already seeing in the VR space that Oculus is significantly outselling PC VR with their easier to use all-in-one systems. If Apple gets into this, it’s going to be to take it much more mainstream than Oculus which means something that doesn’t require an expensive desktop.
I think Oculus Quest 2 (rift isn't available anymore) is selling well due to it's price (relative to Vive and Index). Maybe Apple will surprise us with closer to Quest 2 pricing and be able to take over.
Apple is pretty likely going to take a "work smarter , not harder" approach. Their VR/AR isn't going to be about building the biggest, hot, fire-breathing dragon GPU. With foveated rendering you don't rendered the whole frame at equal fidelity. There is only a narrow range where the eye can see the most detail. Out of that cone of gaze rending precise details that no one will ever see is a waste of time and effort.
VR would be much, much better (practical) with foveated direction of the work has a researched and known issue for over a decade (or two). Somewhere along the way there has been a trend of "gaming == VR and VR == gaming engine" that has been somewhat of a legacy boat anchor distraction.
The iPhones/iPads have been doing augmented reality (AR) for a couple of generations now. There is little need for a tethered via a wire to a personal computer to do that part of the functionality.
There are tetherless VR units by competitors already. Reportedly part of Apple's delay is that they (or at least some influential few inside the company . In part , the "we hate wires" faction. ) didn't want to ride the early evolutionary curve until could do it wirelessly.
What likely will see is tweaked "GPU" that is more power efficient in this niche than in driving a generic LCD panel with 2-3 people looking at the screen results.
Again the AR frameworks have been out for years. FaceID has been looking at people's faces and eyeballs for years ( not in exactly the same level of detail, but looking. )
Differentiated area rendering got released 2-3 years ago ( can't recall at the moment which WWDC but it wasn't last year. )
There are missing pieces , but Apple has been incrementally building a foundation for several years.
I just did a quick Wikipedia read on foveated rendering (eye-tracking-dependent rendering). That does indeed sound very efficient and the way Apple will go, especially because I vaguely recall rumors of the headset having eye cameras. I don’t know what percentage of the industry uses it now but I can only assume it will be ubiquitous soon if not already because of the efficiency.
I wonder if they’ve gotten it to a point where it’s completely unnoticeable, or if one can still tell the rendering is a little delayed. I imagine any lag would be a frustrating UX.
Edit-
Here is that article about eye-tracking cameras. It explicitly mentions it will be used for focused rendering, but also for some sort of user input control.
Kuo: Apple's Mixed Reality Headset to Feature Eye Tracking System, Iris Recognition a Possibility
Apple's upcoming mixed reality headset will be equipped with an advanced eye tracking system, Apple analyst Ming-Chi Kuo said this evening in a...
www.macrumors.com
Last edited:
the screens being relatively hyper close to the eyeballs is a problem for "retina display". Rumors are that Apple is going to throw 4K microLED screens with some specialized lens design at each eye. ( and throw "affordable price" out the window. At least short to intermediate term. ) You don't absolutely "have to" render a 4K scene to present on a 4k monitor.
But yes with AI driven upscaling ( AMD , Intel , and Nvidia all have variants) can upscale to point folks won't notice 'touched up' peripheral areas much.
IF have a subset of cores and/or fixed function logic for eye data processing. Another set of cores for other camera data processing of outside views . and another set for running virtual object generation and a common scratchpad RAM area where they can all communicate quickly... there shouldn't be much lag. If spend some money on specialized hardware there should be much lag. eye's move with muscles ... they aren't making ns changes.
Which would be very Apple. You can’t always see where Apple is going to land based on the incremental steps they are taking today. You can make pretty good guesses though. But you can always see the trail of incremental steps that got them to where they are now.
This is part of the reason why we're very unlikely to see third party GPUs again. Beating the 3090 right now is a speeds-and-feeds battle that Apple really only cares about as a rhetorical flourish in a keynote, but having a superior low-power GPU architecture and an extensive ecosystem of software developers lined up to develop for it's Apple-specific APIs is where they are able to enter a market with massive leverage. That leverage quickly dissipates if they offer choice around GPUs, especially if those GPUs drive API fragmentation.
I think the consensus is Apple’s headset will be untethered and it will have a huge graphical advantage over other untethered headsets. That’s a pretty obvious safe bet.
And there are other advantages around Apple’s ecosystem and other technologies, which a little less obvious, or I need to look more into them anyway.
I suppose my current remaining questions is, is there any reason Apple might offer a tethered option (to Mac)? What would that look like as far as the software ecosystem? I believe Oculus has separate VR apps for tethered and untethered, but that doesn't seem like something Apple would do, but who knows.
And there are other advantages around Apple’s ecosystem and other technologies, which a little less obvious, or I need to look more into them anyway.
I suppose my current remaining questions is, is there any reason Apple might offer a tethered option (to Mac)? What would that look like as far as the software ecosystem? I believe Oculus has separate VR apps for tethered and untethered, but that doesn't seem like something Apple would do, but who knows.
TBH, I’m just giving you my perspective on what I think Apple is going to do. That said, the tradeoff is that you’re going to have a completely separate set of developers building for PC VR and Apple VR. There are a lot of interesting VR titles in the PC VR world that may never go to Apple VR, just the same way that most of the game studios don’t port Windows games to Mac. And, the VR titles for Apple VR are going to assume whatever number of GPU cores Apple puts in lower priced consumer offering whereas a lot of PC VR titles are aimed at people with pretty high end setups. That said, Oculus is already much more about entry level users with inexpensive equipment.
Hm yes, with Apple’s headset being untethered, it would probably not be able to run the most demanding PC VR apps currently out there (if they were ported). But yes of course Apple doesn’t care about that. They always do their own thing and act like other companies don’t exist haha.
Of course if Apple’s headset gets popular enough, PC VR developers will want to get in on it and bring their apps over, and if need be, pare them down to Apple’s gpu limits. And if the headset really shakes up the market, maybe the whole industry might move toward untethered headsets and less demanding apps that are cross platform. Though there will probably always be some tethered headsets and apps, but more niche.