How well does Stable Diffusion perform on Macbook M4? How much RAM is recommended for UI/UX designers at minimum?

[Rumormac]

Can the base model M4 run Stable Diffusion? I tried using M1 (8GB/128GB) but it couldn't handle it at all. I'm wondering how the base M4 (16GB/512GB) would perform?
Or what are the recommended M4 specifications?
If anyone has experience running Stable Diffusion on other M4 configurations, please also share how the performance was.

I mainly use Midjourney for creating pictures at the moment and I'm wondering if Stable Diffusion could be a replacement. I'm not a heavy MJ user - I just occasionally use it when creating assets/materials.

Additionally, I'd like to ask designers who use Figma - when using a MacBook, what's the recommended minimum RAM? Please include your MacBook model. I currently use an M1(8G/128G) and work on two Figma projects simultaneously, sometimes with one to two windows open per project. I'm finding that the RAM is quite insufficient.

I also frequently use memory-intensive applications like CapCut, Photoshop, and note-taking software (not necessarily all at once, but I use them quite frequently).

I'm currently considering the M4 with 16GB RAM but I'm not sure if that's enough for my needs, or if I should upgrade to 24GB. I'd appreciate if anyone with relevant experience could share their insights. Thank you.

 

[artifex]

Good questions. I don't have answers, but I'm hoping my reply bumps the visibility

Was actually just considering putting off my m4 mini purchase another week so I could fit the 24/512 one in. More ram is better for models, that is always true. On the other hand, Apple rates the bandwidth at 120 GB/s, and the RTX 3060 card I have been using previously (on a pc) is supposedly 360 GB/s. So the actual iterations per second will likely be slower (except I won't have to move stuff back and forth between CPU and GPU memory since it's unified, so the whole job length might be similar) but I can do much bigger images. Definitely seems better for trying full Flux.dev and not just the cut down NF4 version I use now (3060 only has 12GB vram).

Looks like Mac people have been installing SD either using DiffusionBee or A1111 or ComfyUI+Homebrew (or maybe +Pinokio?), but I hope whoever pops in here can give us both guidance on that. Remember, though, you're probably going to have this computer for a few years, and so far you can't upgrade the RAM (expensive hacks on the Mini notwithstanding). So get as much as you think you might be needing then for all the other stuff you run, like Photoshop. (BTW, if you can afford an M4 Pro, the memory bandwidth on that is 273 GB/s.)

 

[AirThis]

I'm not an expert, but I'll try answering as I'm using Stable Diffusion and Fooocus on M series. Given the current lack of optimization for Apple silicon, SD is painfully slow to use. It also requires lots of memory. I have an M3Max 40C with 48GB memory.

- It takes me 1min 15 seconds to produce a 1024x1024 image at 30 steps when there is no swapping. Fortunately, I have 48GB memory so I avoid that most of the time. This is using regular models (Juggernaut XL, Stable diffusion base model, etc.).

- Adetailer takes a whopping 30 seconds to correct a face. And another whopping 30 seconds to correct each hand.

- For A1111 and Stable Diffusion, the Python Process alone takes 16GB to 24GB. Fooocus is even more memory hungry at times.

- According to its developpers, running Fooocus on an M1/M2 series Mac is 9 times slower than using an RTX 3XXX. Source: https://github.com/lllyasviel/Fooocus (check the minimal requirements table)

- As a test, I ran both Stable Diffusion and Fooocus to induce some swapping. It took me over 3 mins to generate a single 1024x1024 image with Stable Diffusion. 30 steps, SD base model, 1024x1024.

I tried a couple of the "Mac optimized" SD alternatives, but saw little benefits. I want to either run A1111 or ComfyUI for the features they offer. There is an existing plugin, solution, or workflow for practically anything you could think of.

A good source of info for running A1111 on M series when you run into trouble:

https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/5461

 

[picpicmac]

Given the current lack of optimization for Apple silicon

This is the operative issue.

There's plenty of software that runs better on a Mac than on a Windows machine, but this software that is designed for the platform.