Apple M1 CPU & GPU speed is very disappointing

[TiggrToo]

It's hilarious that so many people are expending so much time with all this metaphorical (ahem) measuring contest, they all seem to forget the fact that this is a v1 release.

This is Apple's first foray into this market and, by all accounts, they've done exceptionally well.

Most people with M1s will probably not give a flying fing over benchmarks: all they care about is real world usage. And if it succeeds there with all the other benefits the new M1 gives, then it's a win.

You can get into "my benchmark's better than your benchmark" all day long - it ultimatly means nothing of the overall result is success.

It's all rather funny really. Don't y'all have better things to do then see who's got the biggest whatever?

The overwhelming number of people buy devices without giving a tinkers pot over benchmark scores, and I've yet to see a benchmark score that has sold me on anything.

Sure the true single core (when one takes SMT & saturation into account) shows many other CPUs are faster than the M1.

Again, who cares? Really?

 

[dogslobber]

Benchmark: Stockfish (chess) speed

M1 CPU = 13000 kn/s

i7 3930k overclocked (from 2011 = 10 years old) = 13000 kn/s

Others = 40000 kn/s - 80000 kn/s

Desktop CPUs = 230000 kn/s and much stronger

Is this the standard app you run on your systems?

 

[dogslobber]

Well, I certainly hope that a desktop Rocket Lake chip would outperform Apple‘s M1. Otherwise it would be really embarrassing for Intel.

Lest we forget Performance Per Watt. M1 would win that I imagine.

 

[leman]

Again, who cares? Really?

Personally, I’m just having fun. It’s also a great opportunity to practice rhetorics with an uncooperative opponent. My students are too terrified of me to put up a good fight and my colleagues are always in boring agreement.

 

[TiggrToo]

Personally, I’m just having fun. It’s also a great opportunity to practice rhetorics with an uncooperative opponent. My students are too terrified of me to put up a good fight and my colleagues are always in boring agreement.

I'm sure you are - however there's one or two folks who really believe in this nonsense and think that their posts make a difference...

 

[cmaier]

I'm sure you are - however there's one or two folks who really believe in this nonsense and think that their posts make a difference...

I consider this progress. Years ago i setup a blog to post how apple was inevitably going to switch to arm. Nobody believed me. (http://www.applenews.zone/search/label/ARM)

Last year, once there was an announcement, it seems like it was me against almost everyone here, explaining over and over again that x86-64 had no inherent advantage over Arm, the Arm processors would be great, etc.

So if now it’s just a couple of people, then we’re getting somewhere.

 

[jeanlain]

Lest we forget Performance Per Watt. M1 would win that I imagine.

The rocket lake i7 consumes 43W for a single thread cinebench run. The M1 consumes 3.8W.

https://twitter.com/i/web/status/1328777333512278020

Yes, the M1 has fewer cores, which still consume power when idle, but the difference is massive.

 

[JMacHack]

Again, who cares? Really?

Sometimes you just feel like arguing.

I consider this progress. Years ago i setup a blog to post how apple was inevitably going to switch to arm. Nobody believed me. (http://www.applenews.zone/search/label/ARM)

Last year, once there was an announcement, it seems like it was me against almost everyone here, explaining over and over again that x86-64 had no inherent advantage over Arm, the Arm processors would be great, etc.

So if now it’s just a couple of people, then we’re getting somewhere.

I know I didn’t believe it until a year before they announced it. I didn’t believe Apple’s first Mac CPU would outpace Intel so significantly.

I was one of the people that thought there was something fishy with all the benchmarks. Yet here we are.

 

[Homy]

I've said this in similar discussions with the exact same people so it feels like Déjà vu and back to square one but I say it again (like TiggrToo). Whom are those Cinebench scores for? It's really pointless for most people and us. It's like "My dad is better than your dad". If you like/need Macs the latest M1 is your only option (soon). If you like/need PCs I guess AMD is your best option to buy (in this case). If you want/need both and can afford it then you buy both.

Mac lovers are not going to change their mind and buy PC because of some benchmarks. PC lovers won't buy Macs no matter how fast M1 is (as shown in this discussion). Mac lovers rather buy Parallels or Crossover than buy a PC. PC lovers build a Hackintosh? I guess the only ones who care about such numbers are those who don't know what they want or those who find satisfaction in trolling and claiming that "their dad is better than ours". Different hardware and different OSs for different people and different needs.

M1 is so fast that such comparisons don't matter in most cases. I don't "play" the game of Cinebench but if you want to go down that road and play games you should remember that the GPU performance is essential in gameplay and an APU is not only about its CPU but also its GPU (as I showed earlier with Metro Exodus as example).

 

[Joelist]

I agree that all the benchmarking gets silly. Also, I wonder how well these synthetic benches account for unusual architectures? For example Big-Little core arrangements. Also something like M1 which is a SOC not a CPU and has and uses blocks like ML and others in addition to CPU.

 

[dogslobber]

I agree that all the benchmarking gets silly. Also, I wonder how well these synthetic benches account for unusual architectures? For example Big-Little core arrangements. Also something like M1 which is a SOC not a CPU and has and uses blocks like ML and others in addition to CPU.

You know, the litmus test for the M1 Mini in my house was how well it played WoW for the wife. And it flies at max settings for everything. This is a true benchmark as it's a win-win. Happy wife makes an easy life.

 

[Joelist]

You know, the litmus test for the M1 Mini in my house was how well it played WoW for the wife. And it flies at max settings for everything. This is a true benchmark as it's a win-win. Happy wife makes an easy life.

Of course! I was just wondering how benches handle these unusual architectures.

 

[crazy dave]

Of course! I was just wondering how benches handle these unusual architectures.

For single core benchmarks, generally you bench the performance core. In macOS, you can sort of target the efficiency core by describing the benchmark as a background task to the OS which is how Anandtech gets its efficiency core performance estimates for Apple SOCs (and various Android SOCs using the Android equivalent scheduler - though I *think* that might allow for more explicit pinning of processes to a specific core).

For multicore benchmarks, you can often tell a benchmark how many cores you want utilize - the default being all of the ones available. Generally setting the benchmark for say 4 cores in the M1 will be enough to just benchmark on the P-cores as for big tasks the OS won’t spill over to the E-cores unless hinted to do so or if the P-cores are full ... sort of similar to hyperthreading in that way but obviously very different in most other ways.

 

[Appletoni]

Is this the standard app you run on your systems?

Stockfish is the standard app which I run on my systems.

 

[KingOfPain]

I consider this progress. Years ago i setup a blog to post how apple was inevitably going to switch to arm. Nobody believed me. (http://www.applenews.zone/search/label/ARM)

Last year, once there was an announcement, it seems like it was me against almost everyone here, explaining over and over again that x86-64 had no inherent advantage over Arm, the Arm processors would be great, etc.

I'm really sorry I missed this discussion (I hardly used my account here that I literally forgot that I had one).
I cannot exactly remember when I started thinking that Apple will at least migrate part of their computer hardware to ARM, but it was several years ago. I almost had a bet going with a colleague but he left the company before Apple actually switched.

I definitely remember that when Apple announced the passively cooled MacBook, two things kept me from buying it:
1. A single USB port that also was the power connector was a no-go for me and I hated the removal of MagSafe.
2. I really wanted it with an ARM processor not an Intel i3.

Every time I heard/read that ARM was only good for embedded use not for real computers, I wanted to tell people that I already had a desktop computer with an ARM at home (Acorn RiscPC) and that the company originally designed the CPU for their desktops. That the CPU was so low power that it became interesting for the embedded market was just an accident.

I certainly didn't expect Apple to migrate their whole lineup in two years, but it makes sense to only have one hardware architecture to support in the future.
I was absolutely sure that Apple Silicon would trounce Intel in an MacBook Air and that they wouldn't migrate if their CPUs wouldn't be at least as fast as the Intel CPUs they used before.
I'm still not totally sure how the Apple Silicon Mac Pro will look like, so that might be a surprise waiting to happen.

What is really laughable in this thread is comparisons of a 15 W 4+4 core with 65 W CPUs that often have twice as many cores...

 

[Appletoni]

After reading this, I now realize how poorly my Macbook M1 will perform with Chess. I am devastated and don't know what I will do to fix this situation. (What's Chess)?

It must be hard to live behind the moon. But that’s okay too.

From Google:

23 days
'The Queen's Gambit' Has To Be One Of Netflix's Most Popular Shows Ever. News and opinion about video games, television, movies and the internet. 23 days. That's how longit's been since The Queen's Gambit premiered on Netflix, and for almost all of those, it hasn't just been in the top 10, it's been at #1

Netflix boasted about the success of “The Queen's Gambit,” saying that the show made the Top 10 in 92 countries and ranked No. 1 in 63 countries, including the U.K., Argentina, Israel and South Africa. “The Queen's Gambit” premiered Oct. 23 on Netflix worldwide.

 

[Appletoni]

No, the issue is not downplaying unfavorable benchmarks, but the benchmarks must be fair and properly coded for the ARM/M1 platform. While I know nothing of Stockfish, it seems not to be coded well if the M1 is getting 1/10th the speed of comparable x86 processors, yet the M1 holds ground on virtually everything else known to be native ARM code.

Look here: https://github.com/official-stockfish/Stockfish/issues/3529 or better look at the Stockfish discord channel or the Talkchess discord channel or ask them. They will tell you Stockfish is coded perfectly for the ARM/M1 platform.

 

[Appletoni]

And that is what it's all about; real use performance.

Exactly and the real use performance using ARM/M1 and Stockfish is terrible.

 

[leman]

Look here: https://github.com/official-stockfish/Stockfish/issues/3529 or better look at the Stockfish discord channel or the Talkchess discord channel or ask them. They will tell you Stockfish is coded perfectly for the ARM/M1 platform.

I don’t see any developers in that issue. Just a bunch of really unprofessional folks bickering. If it was my repo I’d already delete that issue and ban most of the participants for bad conduct.

 

[Appletoni]

I have A10X chips that are just marginally slower than our 12X chips for 99% of workload. I’m now upgrading to the new M1, and really the only reason is because my youngest daughter; needs an new tablet and our old 10.5” pro is still better than the base line iPad(Promotion). The A10X was way overpowered, and still does everything; A12X was insanely overpowered, and 3 years later still pushes boundaries; M1 is game changing, and evades the boundaries of custom silicon.

I’m sorry a version of the M1 architecture: didn’t perform an older coding language, through two emulators, faster than a chip architecture it was directly designed for.

In real-world testing: the M1, allows almost perfect non-gaming 4K computing. Anything more, will not be needed for 99% of the customer base that falls into this category.

I know a lot of gamers which bought the M1 MBP to play many different games at 4K and also because it’s an all-in-one device. I hope that Apple will support better gaming experience at 4K or 4.5K and 120Hz.

 

[Appletoni]

Also, relatively few people actually play chess. I guess those that do should stick with Intel for now.

2016 = 605.000.000 „adults only“ play chess.
If you call this „relatively few people“ then you are right.

And don’t forget the kids.

Today it should be much higher.
Due to: much more kids are playing chess today, Queens Gambit, Corona, Home-Office, Twitch, Youtube…

 

[Appletoni]

The M1 does appear to perform as it should (= on par with the best cores) in "classic" stockfish. The underperformance appears specific to stockfish NNUE. I don't know how its coded, but is it more sequential that classic stockfish?
There's apparently some X86 specific instructions related to SIMD and bit manipulations. Maybe they've not been translated into ARM instructions yet.
I won't bother to look at the code as I don't know how to recognise those instructions anyway, but the Mac stockfish app has three different executables for X86-64, one whose name contains "sse41", another "bmi2" and a third one "vnni-256", which apparently corresponds to AVX 512. There is just one ARM executable: "stockfish-arm64".

As for SMT, it would be nice to have it on ARM Mac.

Feel free to download the code and take a look inside the makefile: https://github.com/official-stockfish/Stockfish

Create new page · official-stockfish/fishtest Wiki

The Stockfish testing framework. Contribute to official-stockfish/fishtest development by creating an account on GitHub.

github.com

Binary for macOS UCI engine · Issue #3251 · official-stockfish/Stockfish

Is it possible to download the binary for Stockfish 12 UCI engine for macOS (M1).

github.com

Compile StockFish optimized for arm64-v8a

I cloned StockFish from GitHub and did a make inside the src folder and found that supported archs are: x86-64-bmi2 > x86 64-bit with pext support (also enables SSE4) x86-64-modern ...

chess.stackexchange.com

https://www.reddit.com/r/macbook/comments/k3foze

 

[Appletoni]

I ran the benchmarks with the settings suggested by @mi7chy (stockfish bench 128 NUMCORES 24 default depth) with the following results (power usage checked using Apple's powermetrics):

M1 MBP 13":
~ 12000 knodes/s for 8 threads using 15W of power
~ 2500 knodes/s for 1 thead using 5W of power

Intel i9 16" MBP:
~ 13000 knodes/s for 16 threads using 65W of power
~ 1700 knodes/s for 1 thread using 35W of power

In a nutshell, M1 with it's 4 cores is 10% slower than top-shelf Intel mobile CPU with 8 cores while consuming 80% less power. Looking at the linked https://openbenchmarking.org/test/pts/stockfish, M1 is faster than Tiger Lake. 8-core desktop CPUs are considerably faster. Ah, and M1 is about 50% faster in the single-threaded variant than a 4.8ghz Intel Skylake refresh.

My conclusion: no, M1's performance is not disappointing at all. It's performance is comparable to much larger (and 4-5x hotter!) CPUs.

Few additional observations: AMD seems to do really well in stockfish (for unclear reasons) and this benchmark really loves SMT. Stockfish seems to support Neon (ARM SIMD) but it's not really clear how it is utilized. The fact that is scales so well with STM suggests to me that the SIMD code is suboptimal and could probably be improved to achieve better ILP.

Neon:

v8 phone:
vondele_v8 : 60 knps
sfndk.armv7 : 26.5 knps
sfndk.armv7-neon: 52 knps
sfndk.armv8-neon: 61 knps

error +/- 1 knps => 1) and 4) same speed

v7 phone:
vondele_v7 : 35 knps
sfndk.armv7 : 15.5 knps
sfndk.armv7-neon: 35 knps

error +/- 1 knps => 1) and 3) same speed

Startposition, 1 core, measured after several seconds, hash cleared before measurement, GUI=Droidfish
If something is not clear, let me know.

[NNUE] Android building issue · Issue #2860 · official-stockfish/Stockfish

Hello SF team! I have an issue with Android compilations. I'm using NDK's Clang 9.0.8 (last available), on Windows 64 bit. During the compilation i'm getting this error.. After I changed the line 3...

github.com

 

[Appletoni]

Just to make it clear, M1’s SIMD units are not weaker than those of x86 CPUs. Yes, M1 only supports 128-bit SIMD, but it has 4 of them and they operate with low latency. Most modern x86 CPUs can do two 256-bit SIMD AVX2 operations per clock, M1 can do 4 128-bit ones which has the same net result. And it you use 128-bit operations (e.g. SSE), M1 will be faster.

Now, newest Intel cores do have two 512-bit units, so they can achieve higher throughput under certain conditions and with software specifically written for them.

I dint know how exactly Stockfisch utilizes SIMD, but if M1 doesn’t perform well there my initial assumption would be that it’s simply not well optimized for ARM Neon. M1 usually does incredibly good on number-crunching benchmarks.

They said it’s well optimized for ARM Neon: https://groups.google.com/g/fishcooking

 

[Appletoni]

Just registered to voice my opinion. In this thread there are many people who are completely dismissing the OP, with sarcastic responses about divorce and offering a cookie. That's not productive.

It doesn't matter whether this "benchmark" is standardised or not, this is absolutely a real world usage. If you're a chess enthusiast, and you bought an M1 MacBook today, you will truly take a huge performance hit for now. This deserves to be publicly discoverable info as it can affect purchase decisions.

I have both in front of me and came to this thread because I was surprised to see that the new M1 is processing positions at 15-20% of the speed of my 2020 Intel MBP! A more productive discourse would be to find out what's the bottleneck here? I'd be interested to see if a rewrite of Stockfish to leverage the Neural Engine will help. I also saw some people saying only the NNUE mode of Stockfish is affected, sorry to say that's not the case. With both NNUE turned on or off, the Intel 2020 MBP gets around 7500 kN/s (8 cores) while the M1 only nets around 1200 kN/s. I checked various different hash sizes (RAM dependent) as well, no difference. I also tried both M1 and Rosetta, the result was about the same. My 8-core iMac actually nets 13000-14000 kN/sec even. So clearly something's very wrong.

Once again: do not dismiss this result just because it's not relevant to you. Chess is not so niche anymore. This needs more publicity and to be picked up by more people, so that the corrections/optimisations can be made to Stockfish and possible other engines. And it also goes to show that not all programs will automatically run faster, in fact some might end up running considerably slower. And that info is relevant to a lot of other developers. Don't just go by Geekbench score.

You should read about NNUE here: https://github.com/glinscott/nnue-pytorch/blob/f6a2e30d9393a7a8e62f0e3f8bfeecdf84b373c0/docs/nnue.md