What features differentiate a real-time processor such as Apple's R1 from those used by IPhone and Mac?
A real-time processor is one without an MMU (virtual memory). There's now a hybrid core called the Cortex R82 that has an MMU, so it can run a normal OS like Linux or in Apple's case xnu. xnu can be running and when a real-time event occurs, the processor can switch to handle the real-time event, then switch back to xnu. The Apple R1 is probably something like 8x Cortex R82 cores, DSP cores and a 16 core Apple Neural Engine and a large system level cache. Deterministic performance.
It would do the eye tracking and gesture recognition before the M2 even receives the image. LiDAR Mapping, oncoming person detection and fusion of the two camera images together would also be performed by it. These can't occur on the M2 as they need to occur constantly and in real-time. Better to use highly optimised code that can continuously had over a continuous stream of image data to the M2, with eye tracking and gesture recognition already done.
This means visionOS can work more like the windowing compositor on macOS, and inputs appear to visionOS like touch events on iOS.
Last edited: