Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Gnattu

macrumors 65816
Original poster
Sep 18, 2020
1,107
1,670
9E1A8E66-7BEB-4C3B-BC54-273286832A67.png


The link to Apple’s official doc: https://developer.apple.com/documen...ning_intel_binaries_in_linux_vms_with_rosetta

Great news for Linux VM users.

Please note this is a user space emulation and will not enable you to install x86 OS directly, plus this will not work for Windows.
 

altaic

Suspended
Jan 26, 2004
712
484
Woah, that's unexpected and really interesting!

Edit: I wonder what the target market is, or if this is a case study for Windows Rosetta support. When I think "Linux VM," I don't imagine people clamoring to run some proprietary/abandonware x68 binaries on aarch64. It's odd. OTOH, I wonder how x86 Steam runs in an aarch64 Linux VM with Rosetta... I suspect the Asahi Linux folks will be publishing some interesting tidbits soon.
 
Last edited:

Gnattu

macrumors 65816
Original poster
Sep 18, 2020
1,107
1,670
I wonder how x86 Steam runs in an aarch64 Linux VM with Rosetta...
In fact running games is indeed a very huge use case for the x86 userspace emulators, especially the FEX emulator.
 
  • Like
Reactions: kc9hzn

altaic

Suspended
Jan 26, 2004
712
484
In fact running games is indeed a very huge use case for the x86 userspace emulators, especially the FEX emulator.
Huh, I hadn't heard of FEX. Looks interesting, thanks.

I am still however a bit confounded by Apple going out of its way to support the interoperability of Linux x68 binaries on Linux aarch64 VMs. The Asahi Linux folks had already discovered some friendly design decisions by Apple that made their work easier, but I feel like Rosetta support is way more purposeful. Is this Apple signaling some kind of a warm embrace for Linux?
 

senttoschool

macrumors 68030
Nov 2, 2017
2,626
5,482
Huh, I hadn't heard of FEX. Looks interesting, thanks.

I am still however a bit confounded by Apple going out of its way to support the interoperability of Linux x68 binaries on Linux aarch64 VMs. The Asahi Linux folks had already discovered some friendly design decisions by Apple that made their work easier, but I feel like Rosetta support is way more purposeful. Is this Apple signaling some kind of a warm embrace for Linux?

Apple seems to be actively supporting Linux on AS.

It does seem to make sense to me. There's really no downside to supporting Linux.
 

Malus120

macrumors 6502a
Jun 28, 2002
696
1,456
Great job Apple. Now just provide official drivers for Linux and GPU passthrough for Linux guests and we'll be set. Seriously though really looking forward to seeing what else has changed in the virtualization framework! :)
(hopefully a lot)
 

Gnattu

macrumors 65816
Original poster
Sep 18, 2020
1,107
1,670
Great job Apple. Now just provide official drivers for Linux and GPU passthrough for Linux guests and we'll be set. Seriously though really looking forward to seeing what else has changed in the virtualization framework! :)
(hopefully a lot)
In fact, macOS 13 does add a virtio GPU to its Virtualization framework, but we do not know if it is hardware accelerated, yet. If it is then we probably don’t need extra drivers because virtio-gpu is in the kernel and virgl is in mesa.
 

altaic

Suspended
Jan 26, 2004
712
484
Great job Apple. Now just provide official drivers for Linux and GPU passthrough for Linux guests and we'll be set. Seriously though really looking forward to seeing what else has changed in the virtualization framework! :)
(hopefully a lot)
Looks like they made quite a few additions and changes. They have an API diff view that makes it really easy to see:
 

Malus120

macrumors 6502a
Jun 28, 2002
696
1,456
In fact, macOS 13 does add a virtio GPU to its Virtualization framework, but we do not know if it is hardware accelerated, yet. If it is then we probably don’t need extra drivers because virtio-gpu is in the kernel and virgl is in mesa.
Nice. Although Looking at the documentation posted by @altaic below it looks like it may be for macOS guests only...
Looks like they made quite a few additions and changes. They have an API diff view that makes it really easy to see:
Thanks for posting this!

Unfortunately (?)

API Collection

Graphics

Configure a device for a macOS guest to use hardware-accelerated graphics.

Hopefully I'm missing something or its still a work in progress and support will be added for Linux (and other OS) before release
 

altaic

Suspended
Jan 26, 2004
712
484
The more I think about it, I think Rosetta 2 (and beyond) is going to be around for a long time. Clearly x86_64 support is a priority at Apple, on all platforms. They’re positioning themselves to be very flexible.

The Virtualization API looks really easy to use and looks to be feature-full; the new virtio graphics, console, trackpad, shared clipboard, and Rosetta support go a long way to make for a potentially turnkey experience. I wouldn’t be surprised if they release an example project that does it all for Linux VMs.
 

Gnattu

macrumors 65816
Original poster
Sep 18, 2020
1,107
1,670
Nice. Although Looking at the documentation posted by @altaic below it looks like it may be for macOS guests only...

Thanks for posting this!

Unfortunately (?)

API Collection

Graphics

Configure a device for a macOS guest to use hardware-accelerated graphics.

Hopefully I'm missing something or its still a work in progress and support will be added for Linux (and other OS) before release
The virtio gpu is going be used with Linux guests because it is stated by this Apple document:

 
  • Like
Reactions: altaic

altaic

Suspended
Jan 26, 2004
712
484

Graphics

Configure a device for a macOS guest to use hardware-accelerated graphics.

Hopefully I'm missing something or its still a work in progress and support will be added for Linux (and other OS) before release
Hmm, I missed that. Is virtio useful for macOS VMs, though? I’ve always used it for Linux VMs.

Edit: Looks like @Gnattu figured it out. Tomorrow’s presentation should be interesting.
 
Last edited:

Malus120

macrumors 6502a
Jun 28, 2002
696
1,456
The virtio gpu is going be used with Linux guests because it is stated by this Apple document:

Thanks for posting this. Could you point our roughly where in the document you see that. I scanned it and didn't see any mention, but I'm not an expert so I may well have just missed it.

Hmm, I missed that. Is virtio useful for macOS VMs, though? I’ve always used it for Linux VMs.

Edit: Looks like @Gnattu figured it out. Tomorrow’s presentation should be interesting.
Indeed, I'm looking forward to more details.
 

Gnattu

macrumors 65816
Original poster
Sep 18, 2020
1,107
1,670
Thanks for posting this. Could you point our roughly where in the document you see that. I scanned it and didn't see any mention, but I'm not an expert so I may well have just missed it.


Indeed, I'm looking forward to more details.
It calls createGraphicsDeviceConfiguration() to initialize a graphics device config for the VM, this was only used for macOS guests before macOS 13, and the only configurations classes inherits VZGraphicsDeviceConfiguration was VZMacGraphicsDeviceConfiguration and VZMacGraphicsDisplayConfiguration on macOS 12. macOS 13 added VZVirtioGraphicsDeviceConfiguration, and it is the only option if we want to use a graphic device for a Linux guest, because the other 2 has "Mac" inside their names.
 
  • Love
  • Like
Reactions: altaic and Malus120

Malus120

macrumors 6502a
Jun 28, 2002
696
1,456
It calls createGraphicsDeviceConfiguration() to initialize a graphics device config for the VM, this was only used for macOS guests before macOS 13, and the only configurations classes inherits VZGraphicsDeviceConfiguration was VZMacGraphicsDeviceConfiguration and VZMacGraphicsDisplayConfiguration on macOS 12. macOS 13 added VZVirtioGraphicsDeviceConfiguration, and it is the only option if we want to use a graphic device for a Linux guest, this would be the only choice as the other 2 has "Mac" inside their names.
Thanks for the detailed reply. If we actually get hardware accelerated graphics AND Rosetta 2 support in Linux VMs... 🤤
 

altaic

Suspended
Jan 26, 2004
712
484

Apple seems to be actively supporting Linux on AS.

It does seem to make sense to me. There's really no downside to supporting Linux.
Hector Martin is part of the Asahi Linux crew that I wrote about in my post you quoted. And, “supporting Linux” is a very deep rabbit hole, especially considering adding transpilation capabilities (with a different binary format, mind you, plus dynamic linker quirks and god knows what else). Deep rabbit holes by definition have downsides.
 

senttoschool

macrumors 68030
Nov 2, 2017
2,626
5,482
Hector Martin is part of the Asahi Linux crew that I wrote about in my post you quoted. And, “supporting Linux” is a very deep rabbit hole, especially considering adding transpilation capabilities (with a different binary format, mind you, plus dynamic linker quirks and god knows what else). Deep rabbit holes by definition have downsides.
I'm well aware that Hector Martin is the lead developer on Asahi.

When I say "support", I don't mean official support. Just that Apple won't hinder AS on Linux development. They might sometimes work to make it easier to get Linux working.
 

deconstruct60

macrumors G5
Mar 10, 2009
12,493
4,053
Woah, that's unexpected and really interesting!

Edit: I wonder what the target market is, or if this is a case study for Windows Rosetta support.

Nothing really to do with Windows. If look at the overview of virtualization docs


And then look at the subsection entries with "Beta" next to them this is mainly about flushing out the robust virtualization support for Linux. About as close as get to Windows is some standard framework support for EFI for virtualized OS that require it.

That is about it. There are no VZWindowsBootloader class.

Rosetta requires the new host/virtual directory share features added.
In graphics , Some Virtio graphics features but that is mainly for minimal Linux GUI support.
In Keyboards and Input devices . Some trackpad support. (again macOS and Linux primary consumers)
In Storage : USBMassStorage support ( again macOS and Linux probably primary consumers )
In Consoles : more console support ( more akin to command line Linux as primary consumer .
In Clipboard Sharing .... macOS and Linux VM likley primary consumers .


Apple's virtualization frameworks is still relatively thin. Folks like Parallels and VMWare still have lots they can layer on top as a value add for more robust virtualization environments.



When I think "Linux VM," I don't imagine people clamoring to run some proprietary/abandonware x68 binaries on aarch64. It's odd. OTOH, I wonder how x86 Steam runs in an aarch64 Linux VM with Rosetta...

The target market is non full open source VM linux instances that someone want to migrate over from x86-64 Linux to Linux on M-series . [ Apple probably has more than several of these in house (**) . Other folks running cloud services operations would have them too. ] Most major software packages will have linux port but if there is some commercial vendor product that are using for some sub-service that hasn't ported to Arm yet . To da. Can use the binary on a VM machine instance that is mostly ported over. Very similar baseline reason as use it on macOS. Not all software vendors can port their software over at the same speed.

Apple barely has GUI linux running running well. This isn't primarily targeted for gaming.






I suspect the Asahi Linux folks will be publishing some interesting tidbits soon.

This has little bearing at all to Asahi Linux which nominally supersedes the macOS at the bare metal layer. This is a "call out" mechanism to Rosetta running on the host macOS instance from a virtualized Linus instance inside an Apple framework foundation based virtual machine. If there is no host macOS layer , it doesn't work.

Ahashi Linux probably does not have a large stack of commerical binaries that are only compiled for x86-64. So really has no use for this. If 99% of the apps are all ARM on that Linux distribution then there is no traction here.
Rosetta is owned by Apple so Asahi can't just take it and stuff it into their Linux distribution.

An noone has an intense need for the Asahi distribution to run on VM machine instance. There are several distributions that run fine on these virtual machines.



(**) Not trying to imply that Apple is going to wholesale move the bulk of their cloud services stack onto Mac running virtual machines. But like several niche areas where things are running on Intel Macs with Linux virtual machines on top ( e.g. engineering software that is Linux only. Or perhaps something in the XCode cloud stack that is mac based. ) . Suspect folks like MacStadium and Amazon AWS have customers with more "entangled" , complex software stack ecosystems that have been cobbled together over lots of years using several commercial vendor apps in the mix.
 
Last edited:

Gnattu

macrumors 65816
Original poster
Sep 18, 2020
1,107
1,670
It seems like there is a performance penalty for any VMs activated rosetta, even when NOT executing x86 code, because Apple is unable to toggle TSO mode at run time for a VM, so instead, they always running the VM in TSO mode.

* TSO: Total Store ordering, or the "x86-like memory ordering".

Always on TSO mode introduced huge performance hit and the single core GB5 score(I know it is a bad metric, but it is fast) is even about the same(actually lower) when executing arm64 native code and executing emulated x86-64 code.

arm64 native:
66255E3EEECBB3E7907DC9DCAD185F89.jpg


x86-64 emulated:
B32AF6C0499AB39A7DE1A72576E4785E.jpg



So don't activate Rosetta for your VM unless it is absolutely necessary to do so, otherwise you will get a slower VM.
 
  • Like
Reactions: jdb8167
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.