Ubuntu Linux virtualized on M1 (success!)

[MK500]

Using @mikeboss Ubuntu build, I ran a simple sysbench cpu comparison.
This is a single thread test against the fastest Intel processor I own.
Just thought you might find the results interesting.

Summary sysbench under ubuntu:
Events Per Second

10988.07 - Apple M1
1417.03 - Intel I7-8700B @ 4.6Ghz

Around 8 times faster? Am I doing something wrong?

Details:

Intel Mac Mini i7 spec:
I7-8700B
3.2Ghz Base Frequency
4.6Ghz Turbo Reached During Test

-------------
M1 MacBook Air Ubuntu VM
-------------

$ sysbench --test=cpu --num-threads=1 --cpu-max-prime=9999 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time


Prime numbers limit: 9999

Initializing worker threads...

Threads started!

CPU speed:
    events per second: 10988.07

General statistics:
    total time:                          10.0001s
    total number of events:              109889

Latency (ms):
         min:                                    0.09
         avg:                                    0.09
         max:                                    0.62
         95th percentile:                        0.09
         sum:                                 9986.17

Threads fairness:
    events (avg/stddev):           109889.0000/0.00
    execution time (avg/stddev):   9.9862/0.00
   
-------------
Intel Mac Mini i7 homebrew
-------------

$ sysbench --test=cpu --num-threads=1 --cpu-max-prime=9999 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time


Prime numbers limit: 9999

Initializing worker threads...

Threads started!

CPU speed:
    events per second:  1316.82

General statistics:
    total time:                          10.0002s
    total number of events:              13170

Latency (ms):
         min:                                    0.73
         avg:                                    0.76
         max:                                    2.76
         95th percentile:                        0.81
         sum:                                 9996.85

Threads fairness:
    events (avg/stddev):           13170.0000/0.00
    execution time (avg/stddev):   9.9968/0.00
   
-------------
Intel Mac Mini i7 Parallels ubuntu VM
-------------

$ sysbench --test=cpu --num-threads=1 --cpu-max-prime=9999 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time


Prime numbers limit: 9999

Initializing worker threads...

Threads started!

CPU speed:
    events per second:  1417.03

General statistics:
    total time:                          10.0007s
    total number of events:              14173

Latency (ms):
         min:                                    0.67
         avg:                                    0.71
         max:                                    3.10
         95th percentile:                        0.74
         sum:                                 9998.01

Threads fairness:
    events (avg/stddev):           14173.0000/0.00
    execution time (avg/stddev):   9.9980/0.00

 

[mikeboss]

@MK500

Macmini9,1 SimpleVM.app Ubuntu 20.10:

 

[MK500]

@MK500

Macmini9,1 SimpleVM.app Ubuntu 20.10:

View attachment 1681948

Puzzling. You ran same command I did? Your numbers seem extremely slow. Do you see a maxed out core in Activity Monitor when you run it? This is the aarch64 ubuntu image you uploaded to mega?

 

[wyrdness]

regarding performance: you might read this blog post -> http://hoult.org/arm64_mini.html
Linux on the M1 is plenty fast already.

32 cores ThreadRipper 2990wx is slower than M1 ? Unbelievable maybe fake .

There's definitely nothing fake about these. Let's take a closer look. He's posted 3 simple benchmarks. The first is one line of Perl code to sum numbers. This is a single threaded task, so will execute on only one core. So 31 of those 32 ThreadRipper cores will be unused. Geekbench says that the M1 is about 70% faster in single core than the 2990wx. His result is that M1 is about 64% faster in this test. So nothing odd about that at all.

His second test (Prime number sieve) is also single threaded. This time the M1 performs worse than expected, and is only about 25% faster than the 2990wx.

His third test is compilation, which is easily parallelizable, so should be using all (or most) available cores. In this case, the 2990wx is about 1.4 times faster than the M1. The Geekbench scores show the 2990wx being about 1.5 times faster than the M1 for multi-core. So these results are again close to what I would expect.

So perhaps the only "unbelievable" thing about these results is that the M1 didn't win by a larger margin in the Primes test

 

[Stephen.R]

this being the first time ever, me messing around with GitHub and Xcode etc, it didn't occur to me that I can do that ;-)
of course this is a way better solution to host the package, thank you for your suggestion! here you go:

Releases · mibosshard/SimpleVM

Sample code for Virtualization framework. Contribute to mibosshard/SimpleVM development by creating an account on GitHub.

github.com

No worries. Well done on your first foray into it.

 

[ADGrant]

Real 'men' use the CLI

Seriously though; I think you misunderstand the point of WSL2. I can happily run GUI applications that connect to WSL2 like Docker Desktop or Visual Studio Code. But also I can also run Linux side installed GUI apps in Windows like Intellij Idea.

It is not the same as a virtual machine or desktop replacement. That is not the point and is also not required. There is plenty of tools for that.

Actually WSL2 runs in a hypervisor and is therefore a virtual machine.

 

[ADGrant]

This is not for the general public. If you're already a seasoned Linux/Unix person, you should be able to make sense of most of this already, and you don't need hand-holding.

Hence why there are no steps. If you need something more user-friendly, you have to wait for Parallels or VMWare.

Personally I use Ubuntu Multipass to run Linux VMs on my Intel Mac. Fast, lightweight and free. Oracle has a product called VirtualBox which is also free.

 

[ADGrant]

My concern is that if this only runs CLI and not a GUI, then it's really not any different than Microsoft's inclusion of Ubuntu, SuSe, Mint, etc. in the Microsoft Store. Those can all be downloaded and run inside Windows, but they are all CLI only, which limits their usefulness for many people. I tried both Ubuntu and Mint on my MSI rig, and there were so many weird quirks with Microsoft's implementation of the Linux Subsystem that both were largely unusable.

What were you trying to use WSL for? Unlike Windows, MacOS is a Unix operating system with a lot of shared software with Linux, so the usefulness of Linux is mostly limited to building and testing software before deploying to a Linux production environment. A command line interface should be fine for that.

The bigger issue is that this is ARM Linux, currently most production Linux environments are build on x86 Linux though AWS does offer ARM VMs for Linux.

 

[bombardier10]

There's definitely nothing fake about these. Let's take a closer look. He's posted 3 simple benchmarks. The first is one line of Perl code to sum numbers. This is a single threaded task, so will execute on only one core. So 31 of those 32 ThreadRipper cores will be unused.

I expected that this benchmark is for ONE core of the CPU. But keep in mind that high end CPU (18 , 36 cores)
has lower clock for one core. This is due to the production technology. According to this measurement method we come to the absurd ; another CPU 4 or 8 cores has always better benchmark for one Core (usually higher clock for one core) !

Next we must remember that Rosetta, VM (no matter what you call it) these are emulators .
So emulator ALWAYS will be slower than native linux or windows on this same machine.
It can not be otherwise. Now we cant run native Linux or Windows on M1 machine so we
cannot compare. Maybe in future will be available or maybe not.

On PC computer I can run native Windows , Linux and even native MacOS . On separate drives or
on one drive divide into partitions. All works with 100% performance depending on power CPU/GPU.
No emulation but native.

Of course I can use VM too and emulate MacOS but I lose performance and probably will have less or more
errors , won't run some apps etc.

Conlusions. For max performance we must start Linux or Windows directly from computer start
without participation MacOS or any emulators.
(I dont known it will be possible on M1 computers)
But now we use first MacOS , second emulator apps and third Linux or Windows.
It is obvious that this cannot work perfectly.

 

[ADGrant]

I expected that this benchmark is for ONE core of the CPU. But keep in mind that high end CPU (18 , 36 cores)
has lower clock for one core. This is due to the production technology. According to this measurement method we come to the absurd ; another CPU 4 or 8 cores has always better benchmark for one Core (usually higher clock for one core) !

Next we must remember that Rosetta, VM (no matter what you call it) these are emulators .
So emulator ALWAYS will be slower than native linux or windows on this same machine.
It can not be otherwise. Now we cant run native Linux or Windows on M1 machine so we
cannot compare. Maybe in future will be available or maybe not.

On PC computer I can run native Windows , Linux and even native MacOS . On separate drives or
on one drive divide into partitions. All works with 100% performance depending on power CPU/GPU.
No emulation but native.

Of course I can use VM too and emulate MacOS but I lose performance and probably will have less or more
errors , won't run some apps etc.

Conlusions. For max performance we must start Linux or Windows directly from computer start
without participation MacOS or any emulators.
(I dont known it will be possible on M1 computers)
But now we use first MacOS , second emulator apps and third Linux or Windows.
It is obvious that this cannot work perfectly.

A VM is not an emulator and Rosetta is not a VM or an emulator. It pre-translates the x86 to arm64.

If you run MacOS or Windows in a VM you are not running it in an emulator. I/O devices are virtualized but the software is running directly on the CPU.

 

[MK500]

Now we cant run native Linux or Windows on M1 machine so we
cannot compare. Maybe in future will be available or maybe not.

The Linux VMs we are running through hypervisor are not emulated X86 code. These are Linux kernels and distros built fully in ARM binaries, which are native to the M1. It is “virtually” the same as booting up the M1 into Linux. Likely less than 5% performance degradation from the hypervisor. Most of this performance hit will come from virtualized device drivers like network and disk. Over time these will improve. Executable code should be running extremely close to bare metal speed, as it is just passed through to the CPU.

Hypervisor virtualization technology has gotten pretty advanced; since this is how most cloud computing functions. At this point the vast majority of computer processing worldwide is happening through some sort of virtualization. Amazon, Azure, Google, etc.

So while we may or may not ever boot up Linux instead of MacOS on this hardware, from a performance standpoint we are already at the equivalent place.

 

[mikeboss]

You ran same command I did? Your numbers seem extremely slow. Do you see a maxed out core in Activity Monitor when you run it? This is the aarch64 ubuntu image you uploaded to mega?

I copy-pasted your command. I saw very inconsistent results. at times as high as yours... this benchmark seems not to be very reliable...

 

[MK500]

I copy-pasted your command. I saw very inconsistent results. at times as high as yours... this benchmark seems not to be very reliable...

Hmm. Not my experience here. I’ve run it on multiple machines and multiple times in a row. It has been very consistent for me. Oh well; in the grand scheme of things benchmarking isn’t as important as actual performance anyway.

I’m just happy virtualization is going so well this early in the life of these new machines. I’m very optimistic about what we will be able to do in the future.

 

[mikeboss]

this is interesting: I re-ran the benchmark again, this time under vftool #66 (1 CPU / 1024 MB RAM) instead of SimpleVM.app. all with the latest images I uploaded (ubuntu 20.10). I was not able to get a score below 110'000 no matter how often I ran the benchmark (10 times at least):

 

[MK500]

I re-ran the benchmark again, this time under vftool #66 (1 CPU / 1024 MB RAM) instead of SimpleVM.app. all with the latest images I uploaded (ubuntu 20.10). I was not able to get a score below 110'000 no matter how often I ran the benchmark (10 times at least):

Cool! Just goes to show all of this software is pretty bleeding edge experimental

 

[Stephen.R]

Actually WSL2 runs in a hypervisor and is therefore a virtual machine.

To be pedantic, if you're using WSL2, both the Linux and Windows environments are running in Virtual Machines, under the Hyper-V hypervisor. This is why other virtualisation technologies such as VirtualBox, VMWare, etc generally can't be used if you're using WSL2.

 

[The Mercurian]

I have at least 8 Raspberry Pi's in my house, and 3 of them are model 4s. I love Linux. The reason I love running Linux on my M1 is as follows:

Raspberry Pi 4 GeekBench 5:
Single core: 229
Multi core: 661

My M1 GeekBench 5:
Single core: 1726
Multi core: 7522

An M1 Mac Mini 8GB costs $699
A Raspberry Pi 8GB + SD Card + Power Supply + HDMI adapter costs ~ $100

So 7x the cost for 7x the single core performance. Pretty easy math. Warning: Don't do the math on multicore, GPU accelerated video processing, neural engine work, disk IO, RAM IO, etc. or you might end up buying a Mini.

My life is too short to spend waiting for compilers.

So here is a question. Is it possible to make a compute cluster of Mac mini's running virtualised linux ?

 

[Argus500]

this being the first time ever, me messing around with GitHub and Xcode etc, it didn't occur to me that I can do that ;-)
of course this is a way better solution to host the package, thank you for your suggestion! here you go:

Releases · mibosshard/SimpleVM

Sample code for Virtualization framework. Contribute to mibosshard/SimpleVM development by creating an account on GitHub.

github.com

@mikeboss this has really unblocked me thank you so much (and the original author Khaos). Had been looking at Apples virtualisation API and so knew this was possible. But was stuck between getting my hands dirty in an unfamiliar dev environment or waiting for the Parallels beta. Now I have very interesting week ahead

 

[MK500]

So here is a question. Is it possible to make a compute cluster of Mac mini's running virtualised linux ?

Of course. If gigabit bandwidth is enough for you; it could be done today, as the network interface is already supported by the current revision of Apple's hypervisor. However I think a much more powerful/robust solution would utilize the thunderbolt port to allow some kind of 40Gbit backbone. You may have to wait a bit for the sofware/hardware/drivers needed to do that. It would be quite the amazing cluster, as it would also have significant neural processing capabilities.

As an aside, I believe the current fastest supercomputer on Earth is Japan's Fugaku. It runs on ARM processors.

 

[netkas]

It seems one can run ubuntu in qemu now, with full gui inside vm, not over Xquartz

 

[bill-p]

Yeah, follow this thread and replace the Windows 10 disk image with your Ubuntu one and profit:

[SUCCESS] Virtualize Windows 10 for ARM on M1 with Alexander Graf's qemu hypervisor patch

Ha, I missed the dependency. Fixed the pre-compiled binary package and Terminal command.

forums.macrumors.com

And now I have Ubuntu running both on M1 and my Raspberry Pi 4. Glorious!

 

[MK500]

It seems one can run ubuntu in qemu now, with full gui inside vm, not over Xquartz

Wow! This is all happening much faster than I expected!

[EDIT] Having watched the whole video, I now see they are using an interesting virtual graphics driver to make this work:

 

[bill-p]

Yeah, join us. It does work.

Big Buck Bunny 4K @ 60fps

Edit: OpenGL version info:

It's basically almost "perfect". I would advise against this if you're on the MacBook Air, though. CPU usage is pegged at 100% all the time for all 4 performance cores right now, so it may heat up your Mac significantly.

 

[netkas]

Wow! This is all happening much faster than I expected!

[EDIT] Having watched the whole video, I now see they are using an interesting virtual graphics driver to make this work:

View attachment 1682975

It's standard multithreaded software-only driver.

 

[jido]

Yeah, join us. It does work.

Big Buck Bunny 4K @ 60fps

Edit: OpenGL version info:

It's basically almost "perfect". I would advise against this if you're on the MacBook Air, though. CPU usage is pegged at 100% all the time for all 4 performance cores right now, so it may heat up your Mac significantly.

What do you not recommend on MacBook Air? Watching 4K movies or Qemu Linux in general?
What uses so much CPU?