Ubuntu Linux virtualized on M1 (success!)

[bill-p]

What do you not recommend on MacBook Air? Watching 4K movies or Qemu Linux in general?
What uses so much CPU?

So what happened was that an earlier build of the QEMU patch allowed us to create these VMs, but it did not include proper fix for CPU time, so all CPU cores are maxed out as if the VM is constantly using them. Even when there's nothing going on. Running all CPUs at max 100% all the time is not a good idea with the MacBook Air.

A new patch has been pushed to address CPU usage, and now these VMs should work just as well on a MacBook Air as they do on anything else.

More info here:
https://forums.macrumors.com/thread...s-qemu-hypervisor-patch.2272354/post-29327973

QEMU has now superseded SimpleVM for me.

 

[mehdiraza93]

i am getting this error state: Error and console is blank

 

[mikeboss]

i am getting this error state: Error and console is blank

booting and/or installing from ISOs does not work with SimpleVM.app. you should download the package I made:

https://github.com/mibosshard/SimpleVM/releases/download/20201128/SimpleVM_Ubuntu.zip

 

[The Mercurian]

Of course. If gigabit bandwidth is enough for you; it could be done today, as the network interface is already supported by the current revision of Apple's hypervisor. However I think a much more powerful/robust solution would utilize the thunderbolt port to allow some kind of 40Gbit backbone. You may have to wait a bit for the sofware/hardware/drivers needed to do that. It would be quite the amazing cluster, as it would also have significant neural processing capabilities.

As an aside, I believe the current fastest supercomputer on Earth is Japan's Fugaku. It runs on ARM processors.

So I'm thinking in terms of setting up RStudio Server to do distributed parallel computations for simulations studies, maybe building some MCMC (Stan) models and such - I wouldn't be pushing large amounts of data around by any means. I just want faster results. You are saying this could work even now with a couple of M1 Mac mini's running virtualised linux and a couple of ethernet cables? ?

 

[MK500]

So I'm thinking in terms of setting up RStudio Server to do distributed parallel computations for simulations studies, maybe building some MCMC (Stan) models and such - I wouldn't be pushing large amounts of data around by any means. I just want faster results. You are saying this could work even now with a couple of M1 Mac mini's running virtualised linux and a couple of ethernet cables? ?

I have no experience in this area; so my comment was hypothetical. However, I tested installing the following successfully:

apt-get install r-base

It seemed to install without errors.

What are your requirements in terms of RAM? Is everything you need to work functioning on ARM linux systems? I would maybe have someone with an M1 install everything you need and test before you buy your own boxes.

The more I think about this, the more I wonder if a bunch of people aren't considering this option. For similar per-core performance you would need something like the Ryzen 5600X. That's a $300 CPU if you can find one. Add case, power supply, motherboard, RAM, and SSD...seems like the Mini isn't a bad price point. And the Mini is likely going to be using less wattage and be far more reliable than a hodgepodge of parts.

 

[The Mercurian]

I have no experience in this area; so my comment was hypothetical. However, I tested installing the following successfully:

apt-get install r-base

It seemed to install without errors.

What are your requirements in terms of RAM? Is everything you need to work functioning on ARM linux systems? I would maybe have someone with an M1 install everything you need and test before you buy your own boxes.

The more I think about this, the more I wonder if a bunch of people aren't considering this option. For similar per-core performance you would need something like the Ryzen 5600X. That's a $300 CPU if you can find one. Add case, power supply, motherboard, RAM, and SSD...seems like the Mini isn't a bad price point. And the Mini is likely going to be using less wattage and be far more reliable than a hodgepodge of parts.

Honestly I don't know I've never done anything like this before. I was just wondering aloud so to speak after reading another thread: https://forums.macrumors.com/thread...devices-is-it-possible.2270916/?post=29308531 I've no experience with clusters myself!

 

[MK500]

I was just wondering aloud so to speak after reading another thread: https://forums.macrumors.com/thread...devices-is-it-possible.2270916/?post=29308531

Thanks for pointing out this thread! Thats a very good idea for recycling iOS devices. Now that these CPUs have gotten to this level, imagine a pile of iPhones and iPads with broken screens but that could be an incredibly powerful cluster. I don't have any actual cluster tasks; but if I was in research I would think this is pretty cool.

 

[MK500]

Just a quick repost of @bill-p instructions for Linux in case anyone missed them on the Windows thread:

So, here is how I got my Linux setup to work. A bit different from what you guys are doing, I'm guessing. I didn't have to pass in kernel/initrd:

#!/bin/bash
./qemu-system-aarch64 \
-M virt,highmem=off \
-accel hvf \
-m 4G \
-smp 2 \
-cpu max \
-drive file=storage.img,index=0,media=disk \
-serial stdio \
-netdev type=user,id=net0 \
-device virtio-gpu-pci -vga none \
-device nec-usb-xhci \
-device usb-kbd \
-device usb-tablet \
-device intel-hda -device hda-duplex \
-device virtio-net-pci,netdev=net0,romfile="" \
-bios QEMU_EFI.fd

Make sure there's a storage.img. I used qemu-img to create it.
If you need to install using the live CD, just add this (make sure to remove it after you're done with installation):

-device file=PATH_TO_LIVE_CD,index=2,media=cdrom

Boots every time. Though... you may lose the cursor if/when you get into the UI.

To fix that, either set "SWcursor" to "true" in xorg.conf (you may have to recreate this in Recovery if you're on a newer version of Ubuntu). Or... add this:

-display default,show-cursor=on

^ what that does is basically just "unhide" your mouse cursor when it enters the window. Would be useful in case it gets hard to see where you're pointing to. But I'd recommend doing "SWcursor" fix if you have the time to spare.

And yeah, V2 is far better. There are some other stuffs that can still be patched (like hidpi support, for instance), but it is... pretty much just what I'll want out of an eventual Parallels/VMWare release anyways. It has been useful and super stable!

 

[MK500]

Make sure there's a storage.img. I used qemu-img to create it.

I'm having a little trouble finding qemu-img. Anyone feel like zipping up a binary for me? Otherwise I will likely need to compile qemu; and the dependencies are a bit of a mess at the moment.

 

[jdb8167]

I'm having a little trouble finding qemu-img. Anyone feel like zipping up a binary for me? Otherwise I will likely need to compile qemu; and the dependencies are a bit of a mess at the moment.

It's part of the QEmu binaries that get built from GitHub. I have a version but I don't know how to use it to make sure it works. If you want to try it, it is attached.

 

[MK500]

It's part of the QEmu binaries that get built from GitHub. I have a version but I don't know how to use it to make sure it works. If you want to try it, it is attached.

Thanks for the file. No luck. I guess I'll have to compile everything myself. It's the right thing to do anyway. Too bad they didn't include qemu-img with the binaries that are going around in the other thread zip file.

% ./qemu-img
dyld: Library not loaded: /opt/local/lib/libglib-2.0.0.dylib
  Referenced from: /Users/MK500/Downloads/qemu-m1/./qemu-img
  Reason: image not found
zsh: abort      ./qemu-img

 

[tiiiecherle]

I am new to QEMU and I managed to convert my WIN10 virtualbox image to .qcow2 and start it up.

I use a 16 inch MacBookPro as host and installed qemu via homebrew.

I tried to find out my missing pieces but I hope someone can help me out with the following (general and off-topic) questions:

1 How can I get the clipboard to work for sharing between macOS host and win10 guest?
I installed the QEMU guest agent inside of the guest and confirmed its running with Get-Service QEMU-GA in Powershell but the clipboard is not working.

2 To use the right mouse click I need to use the command key instead of control and the windows menu pops up every time I press it. Can the windows menu be disabled or the right mouse click be remapped to control.

3 Is there a good GUI for qemu on macOS? I tried https://github.com/jeffreywildman/homebrew-virt-manager, but I only want to run two local VMs, win10 and archlinux, and no virtual viewer or vnc connections.

I start the VM with the following command:

qemu-system-x86_64 \
-m 8G \
-vga virtio \
-display default,show-cursor=on \
-usb \
-device usb-tablet \
-enable-kvm \
-drive file=/Users/"$USER"/qemu/win10_64.qcow2,if=virtio \
-device ich9-intel-hda -device hda-output \
-machine q35,accel=hvf \
-net user \
-net nic,model=virtio \
-cdrom /Users/"$USER"/Desktop/files/virtio-win-0.1.185.iso

Any help is appreciated. If this is too off-topic I will start another thread as perhaps a few more people would like to try QEMU and the documentation is not that easy for QEMU newbies.

Thanks

 

[Anonymous Freak]

My concern is that if this only runs CLI and not a GUI, then it's really not any different than Microsoft's inclusion of Ubuntu, SuSe, Mint, etc. in the Microsoft Store. Those can all be downloaded and run inside Windows, but they are all CLI only, which limits their usefulness for many people. I tried both Ubuntu and Mint on my MSI rig, and there were so many weird quirks with Microsoft's implementation of the Linux Subsystem that both were largely unusable.

It may limit usefulness for some people, but it covers 100% of what I use Linux for. I manage Linux servers, write code that is command-line-only Linux systems, compile, test, etc.

A command-line-only Linux does 100% of what I need. Right now, I use Linux in one of two ways:

1. Microsoft's "Windows Subsystem for Linux" - which does 99.9% of what I need.

2. SSH to a Linux server in our server farm - 100% command-line, does the remaining 0.1%, as well as just being faster for things like compiling, and saves on network bandwidth when doing code transfers.

A lightweight Linux VM on my new M1 Mac Mini would accomplish probably 95% of what I need. If it has USB passthrough for security tokens, it can do the 0.1% that my MS WSL can't do. If I were to take the effort to get an x86 emulator in there, it could do 100%.

 

[kjd2234]

I haven't reviewed the entire thread. Is a GUI available? Can this run the ARM version of Linux natively?

 

[MK500]

I haven't reviewed the entire thread. Is a GUI available?

Yes.

Can this run the ARM version of Linux natively?

It is a hypervisor inside a VM, so pretty close to bare metal speed. Linux ARM64 is running at mostly native speed.

 

[kjd2234]

Yes.

It is a hypervisor inside a VM, so pretty close to bare metal speed. Linux ARM64 is running at mostly native speed.

Thanks very much for the summary. Has anyone tried benchmarks? How does the performance compare to a VM?

 

[jdb8167]

Thanks for the file. No luck. I guess I'll have to compile everything myself. It's the right thing to do anyway. Too bad they didn't include qemu-img with the binaries that are going around in the other thread zip file.

% ./qemu-img
dyld: Library not loaded: /opt/local/lib/libglib-2.0.0.dylib
  Referenced from: /Users/MK500/Downloads/qemu-m1/./qemu-img
  Reason: image not found
zsh: abort      ./qemu-img

I was afraid of that. It came from the build directory and is probably not packaged as complete install. Also, I haven't applied the patches yet so it might not work anyway. Good luck. Getting the QEmu code to build wasn't that big a deal. I started with MacPorts instead of trying to deal with HomeBrew. Most of the dependencies are already working (I've been at this for days. I don't know which dependencies I had to hand build vs. letting MacPorts deal with it.) But everything works from ./configure to make. I just need to find the time to apply the patches and rebuild. Maybe later tonight.

 

[morelightning]

I'm not sure you understand what the hypervisor is.

This is not about Rosetta or "the real potential of the M1 gpu" in Big Sur.

This is about running different OSes and containers like Docker under macOS.

Supervisors is the same solution Windows and Linux use, it's the fastest solution, and there's no other as fast solution -- aside from not using macOS and real booting directly to Ubuntu on the Mac.

Well as it so happens, I was indeed referring to not using Mac OS at all and booting directly into another arm OS.

Because I write in C and Metal and squeeze the hardware to the limit, I am familiar with problems that exist in the core OS code, especially beginning in Mojave and that still exist to this day in Big Sur.

Virtualized solutions only would seem fast in context with the rest of the slow user space application strategies most everyone uses. - to me, those are inefficient by magnitudes than what I work with, as well as how I would implement linux without the standard user space - where the in house toolsets are themselves a custom OS.

Hypervisor is the carrot on the stick for the masses, for Apple to try to ignore and obfuscate the real complaints from the minority of developers. The more people who accept the results that hypervisor provides, the less likely the developers outside Apple will push for dual boot. This is what they are counting on, that you feel satisfied about hypervisor and the virtualized solutions, so that dual boot doesn't gain enough traction to ever happen.

 

[jido]

Well as it so happens, I was indeed referring to not using Mac OS at all and booting directly into another arm OS.

Because I write in C and Metal and squeeze the hardware to the limit, I am familiar with problems that exist in the core OS code, especially beginning in Mojave and that still exist to this day in Big Sur.

Virtualized solutions only would seem fast in context with the rest of the slow user space application strategies most everyone uses. - to me, those are inefficient by magnitudes than what I work with, as well as how I would implement linux without the standard user space - where the in house toolsets are themselves a custom OS.

Hypervisor is the carrot on the stick for the masses, for Apple to try to ignore and obfuscate the real complaints from the minority of developers. The more people who accept the results that hypervisor provides, the less likely the developers outside Apple will push for dual boot. This is what they are counting on, that you feel satisfied about hypervisor and the virtualized solutions, so that dual boot doesn't gain enough traction to ever happen.

Does Apple stop you from dual booting? There are secure boot settings you need to disable, but after that?

 

[Krevnik]

Does Apple stop you from dual booting? There are secure boot settings you need to disable, but after that?

The bigger issue is more that while switching away from EFI, Apple has moved to an undocumented, Apple-developed boot loader. It’s going to be a long slog to get stuff booting on M1 machines. And who knows how stable it will be in the long run.

Even those developing code for Raspberry Pi may appreciate being able to compile code many many times faster.

Yes, I’m actually one of those people. Raspbian’s recent AArch64 release in a VM would be great for building and deploying the Swift compiler, as well as my projects using Swift on the Pi.

 

[Rastafabi]

The bigger issue is more that while switching away from EFI, Apple has moved to an undocumented, Apple-developed boot loader. It’s going to be a long slog to get stuff booting on M1 machines. And who knows how stable it will be in the long run.

Actually the boot process is a known variable. It’s what jailbreakers used to tinker with on iOS for years, though actually far more open on these new Macs. As such a well established system I would consider it very stable. Furthermore Apple even provides tools to boot from non-Apple kernel images. You might also want to check out bputil.

 

[agrawalp86]

booting and/or installing from ISOs does not work with SimpleVM.app. you should download the package I made:

https://github.com/mibosshard/SimpleVM/releases/download/20201128/SimpleVM_Ubuntu.zip

Thank you, I really appreciate this! I was struggling with finding the right img files. I downloaded the arm64 img files from Ubuntu, but I was unable to boot (tried vftools and SimpleVM) because I did not know where to point root to (/dev/vda does not work on the ubuntu provided img files). How do you identify the right device for root in the img files?

I can run docker without any issues now and that will help get me over the bump while other developers catch up with M1. I am now using this as a development server since img file persists all the changes. I made a couple of changes, in case someone else is looking for the same -

1. I purged the openssh-server and reinstalled it. I was facing some issues when trying to ssh in to the linux vm.
2. I increased the size of the partition inside the img file to 100GB so I can install more tools and apps. I used this guide too resize - https://superuser.com/a/693186. Note - gunzip brings down the size the 100GB img file to 2GB, if like me, you are transferring files over ssh to another Linux machine so you can use parted.

Another thing I struggled with initially - vmlinuz kernel file. This is usually a compressed version but SimpleVM and vftools expect an uncompressed version. Steps to get that -
cp vmlinuz vmlinuz.gz
gunzip vmlinuz.gz vmlinuz

 

[VitoBotta]

Hi guys. I've read the thread and would love to try this to run Docker in a VM, but I am pretty confused. Which tools should I use between SimpleVM/vfTool/QEMU? I would like to run a VM with persistence. Thanks in advance for any clarification.

 

[mikeboss]

Hi guys. I've read the thread and would love to try this to run Docker in a VM, but I am pretty confused. Which tools should I use between SimpleVM/vfTool/QEMU? I would like to run a VM with persistence. Thanks in advance for any clarification.

I mostly use vftool. there's an interesting thread on GitHub containing helpful information:

Ubuntu Server fails · Issue #2 · evansm7/vftool

Hello, I am very interested in trying this. I am trying ubuntu-20.04.1-live-server-arm64.iso and would like to perform a install. so I also created a DMG. But it does not seems to launch the instal...

github.com

 

[VitoBotta]

I mostly use vftool. there's an interesting thread on GitHub containing helpful information:

Ubuntu Server fails · Issue #2 · evansm7/vftool

Hello, I am very interested in trying this. I am trying ubuntu-20.04.1-live-server-arm64.iso and would like to perform a install. so I also created a DMG. But it does not seems to launch the instal...

github.com

Thanks, will read it. The first time I tried vftool the VM crashed often. I followed https://iphonesdkdev.blogspot.com/2020/12/20201202.html. Are VMs started with vftool stable for you?