Apple M1 CPU & GPU speed is very disappointing

[pshufd]

All positive benchmarks for M1 involve stuff for "creators". I find it funny that people consider chess a niche while "creators" are somehow common.

Also, most "creator" benchmarks are export only. How often do you export ffs. None compare file sizes btw.

Don't play to where the market is. Play to where the market is going.

I wouldn't call myself a creator but I've put out a few videos on optimizing Apple Silicon Macs for trading and garnered a number of followers along with somehow being someone to answer tech questions. We all have expertise in some areas and may want to communicate that expertise. Some have turned it into their main source of income and some have even done exceedingly well at it. There are even chess players, beginners and grandmasters alike, that are successful creators.

I did a Windows i7-10700 build in 2020 and the design goals were:

1. Efficiency
2. Thermals
3. CPU horsepower
4. 3x4k monitor support
5. Minimal SSD accesses

The approach was to use a lot of cores running at low frequencies for more efficiency so I got a CPU with 400% more CPU capacity than I would need. The goal was met though the fans can spin up and it can generate a lot of heat for video work. Clock speed really ramps up at about 40% CPU utilization.

This system has been replaced with an M1 mini and a M1 PRO MacBook Pro. The level of performance for these two systems is ridiculous for my purposes but Apple doesn't have a precise configuration for what I need and want. The M1 systems, though, have far more power than required and enough RAM though I have to reboot the MacBook Pro every other day because of the memory leak issue. The mini is running Big Sur which doesn't seem to have the memory leak issues.

The M1 systems never get even warm and I never hear the fans. I imagine that they use very little power as well.

We've been used to Intel and AMD laptops that are hot and noisy and the M1 MacBooks were a revelation that you could have high performance, great battery life and great thermals. Consumers have already spoken that they care more about those benchmarks over chess engine benchmarks.

 

[leman]

Apple's intel macs are known for overheating, and I wouldn't suspect anything different with heavy AVX2 workload, just as Stockfish has. There was A LOT of people that discovered their cooling solutions to be insufficient when trying to run Stockfish, not only on laptops.

Read the update, it turns out that binary distributed by homebrew is compiled without AVX2. If I build myself using most aggressive settings the i9 gets 16717512 nodes/sec.

And no, they are not "known for overheating". Where did you get that notion from? My i9 can run heavy throughput code at 2.6-3.0ghz or above it pretty much indefinitely, that's 10% better than the spec. Sustained CPU power dissipation in 2019 Mac chassis is 60W.

All positive benchmarks for M1 involve stuff for "creators". I find it funny that people consider chess a niche while "creators" are somehow common.

This is not true at all. M1 is currently the fastest laptop solution for developers and to a lesser degree people working with data analysis (unless you are heavy invested into ML where support for Apple Silicon is still lackluster). If you use R and/or STAN, M1 is crazy fast.

I don't care at all about "creator" workflows. But I care that an M1 laptop builds code, runs tests and does number crunching/data transforms as fast as a large desktop workstation. This is my daily bread and butter.

 

[Appletoni]

Ok, once my M1 16" has arrived I will be happy to contribute profiler output. Do you have instructions on how to collect it?

That’s great.
Thank you very much.

 

[Leifi]

It will be interesting to discuss when some of the excuse-ionists actually will be ale to get a device with Apple Silicon.. and actually run the bench and profiling. LOL

 

[cmaier]

It will be interesting to discuss when some of the excuse-ionists actually will be ale to get a device with Apple Silicon.. and actually run the bench and profiling. LOL

You mean like the person on this thread with an M1 who turned on avx in the compiler flags and now gets excellent results?

Like that person?

 

[leman]

You mean like the person on this thread with an M1 who turned on avx in the compiler flags and now gets excellent results?

It was an Intel Mac, not an M1 one. If you remember, I had a hiccup with my credit card that sent me to the end of the waiting list. Have been waiting since then ?

 

[Leifi]

You mean like the person on this thread with an M1 who turned on avx in the compiler flags and now gets excellent results?

Like that person?

Which person with M1 "turned on avx" ??? And did you bother to compare with results from 8-core AMD CPUs at all?

Seems to be no one here with an M1 that actually can do any real benching at all ... But a lot of fans here with strongly voiced "opinions" about what they "believe" the M1 is capable of.

It seems the discussion has more tilted from "the results are wrong", "the bench is not optimized enough", "bench is not relevant".. to... "I hate chess anyway so I don't care at all"

 

[pshufd]

Which person with M1 "turned on avx" ??? And did you bother to compare with results from 8-core AMD CPUs at all?

Seems to be no one here with an M1 that actually can do any real benching at all ... But a lot of fans here with strongly voiced "opinions" about what they "believe" the M1 is capable of.

It seems the discussion has more tilted from "the results are wrong", "the bench is not optimized enough", "bench is not relevant".. to... "I hate chess anyway so I don't care at all"

Why do you feel entitled to the free work of others?

It's your interest. Why don't YOU optimize stockfish for M1?

 

[Leifi]

Read the update, it turns out that binary distributed by homebrew is compiled without AVX2. If I build myself using most aggressive settings the i9 gets 16717512 nodes/sec.

So basically you even get a higher bench for your old MacBook, taking advantage of AVX2 instruction-set, than what people get from an M1 with NEON tweaks :-(

This should indicate to people here that there is ALOT of optimizations that could be done on most older Intel apps as well. Most of these are not remotely as efficiently coded as Stockfish.

The "benches" where M1 shines, mostly seem to be when running highly tweaked M1 specific code compared to generic, older un. optimized code for Intel CPUs (most commonly single-threaded type workloads). There is probably more room to tweak most old intel apps on Macs to better use the gen9 CPUs of Apple's intel models, than there is finding additional tweaks for optimized M1 code.

 

[bcortens]

Which person with M1 "turned on avx" ??? And did you bother to compare with results from 8-core AMD CPUs at all?

Seems to be no one here with an M1 that actually can do any real benching at all ... But a lot of fans here with strongly voiced "opinions" about what they "believe" the M1 is capable of.

It seems the discussion has more tilted from "the results are wrong", "the bench is not optimized enough", "bench is not relevant".. to... "I hate chess anyway so I don't care at all"

You ignored the follow up to this where clarification was issued.

Next: The bench is obviously doing something wrong, whether that is optimizations, assumptions, whatever... When the CPU resources of the M1 are able to be fully utilized it is able to easily keep up with Intel and AMD CPUs. At this point it is pretty well known that it has enough functional units and SIMD resources to do so, therefore we can conduced that something about the way this program is built is not utilizing those resources.
As I posted above, the SPEC chess benchmark shows the M1 easily keeping up with Intel and AMD.

Who is going to do the work to actually make sure stockfish runs well?

 

[pshufd]

So basically you even get a higher bench for your old MacBook, taking advantage of AVX2 instruction-set, than what people get from an M1 with NEON tweaks :-(

This should indicate to people here that there is ALOT of optimizations that could be done on most older Intel apps as well. Most of these are not remotely as efficiently coded as Stockfish.

The "benches" where M1 shines, mostly seem to be when running highly tweaked M1 specific code compared to generic, older un. optimized code for Intel CPUs (most commonly single-threaded type workloads). There is probably more room to tweak most old intel apps on Macs to better use the gen9 CPUs of Apple's intel models, than there is finding additional tweaks for optimized M1 code.

Just a load of supposition from someone with no experience in software optimization.

 

[pshufd]

You ignored the follow up to this where clarification was issued.

Next: The bench is obviously doing something wrong, whether that is optimizations, assumptions, whatever... When the CPU resources of the M1 are able to be fully utilized it is able to easily keep up with Intel and AMD CPUs. At this point it is pretty well known that it has enough functional units and SIMD resources to do so, therefore we can conduced that something about the way this program is built is not utilizing those resources.
As I posted above, the SPEC chess benchmark shows the M1 easily keeping up with Intel and AMD.

Who is going to do the work to actually make sure stockfish runs well?

The person complaining about it of course. But he's too lazy, uneducated or cheap to do it.

 

[bcortens]

So basically you even get a higher bench for your old MacBook, taking advantage of AVX2 instruction-set, than what people get from an M1 with NEON tweaks :-(

This should indicate to people here that there is ALOT of optimizations that could be done on most older Intel apps as well. Most of these are not remotely as efficiently coded as Stockfish.

The "benches" where M1 shines, mostly seem to be when running highly tweaked M1 specific code compared to generic, older un. optimized code for Intel CPUs (most commonly single-threaded type workloads). There is probably more room to tweak most old intel apps on Macs to better use the gen9 CPUs of Apple's intel models, than there is finding additional tweaks for optimized M1 code.

Stop ignoring SPEC, M1 shines in SPEC benchmarks additionally there are many examples of code that runs very well despite lack of optimization, such as Cinebench which is well known to run poorly on macOS and yet M1 is still faster than the Intel Macs they replace with similar core counts.

The opposite is true, M1 does poorly in only a few specific workloads but does as well as or better than Intel in most workloads.

 

[Leifi]

Why do you feel entitled to the free work of others?

It's your interest. Why don't YOU optimize stockfish for M1?

I don't. I just note that M1 is more expensive and a snail on demanding workloads compared to most cheap modern gaming laptops these days..

I have no desire to waste my time on the M1 if there is no tangible evidence that it can perform even closely on par with its asking pricepoint.

 

[JMacHack]

So lemme get this straight.

You’re telling me, that the damn near universal praise Apple Silicon has received, is wrong because of a single bench, which only you use or care about.

And this single chess benchmark, somehow proves the superiority of x86?

I read, really hope this is a damn good troll thread, and that we’ve all been had.

 

[JMacHack]

I don't. I just note that M1 is more expensive and a snail on demanding workloads compared to most cheap modern gaming laptops these days..

I have no desire to waste my time on the M1 if there is no tangible evidence that it can perform even closely on par with its asking pricepoint.

Lol.

“Wasting time”

Your time isn’t valuable if you’ve been spending it finding a single benchmark where AS performs poorly and arguing that’s the single most important metric.

 

[pshufd]

I don't. I just note that M1 is more expensive and a snail on demanding workloads compared to most cheap modern gaming laptops these days..

I have no desire to waste my time on the M1 if there is no tangible evidence that it can perform even closely on par with its asking pricepoint.

If you can't afford it, don't buy it. Apple can not keep up with demand for them so price isn't a problem.

 

[pshufd]

I have no desire to waste my time on the M1 if there is no tangible evidence that it can perform even closely on par with its asking pricepoint.

Yet you criticize it without understanding with a lack of understanding on the hardware capabilities and how to use them.

 

[pshufd]

Lol.

He doesn't want to waste his time.

Yet he wants us to spend our professional time.

And yet he wastes his time on this thread.

 

[JMacHack]

He doesn't want to waste his time.

Yet he wants us to spend our professional time.

And yet he wastes his time on this thread.

Consider me trolled, I was fooled into thinking this was another PC peen measuring contest. Now it’s just sad.

 

[Sopel]

Who is going to do the work to actually make sure stockfish runs well?

I will but I don't own an M1 mac so I'll need help. For what it's worth now there's nothing indicating that Stockfish is not optimized for M1.

It really feels like people just assume M1 is better, and based on that assumption dismiss all results to the contrary as bogus. That's weird.

 

[JMacHack]

**** it, I’m salty.

If I’m understanding the assertion correctly:

- The M1 is disappointing because this single benchmark has the M1 underperforming comparable x86 processors by a magnitude.
- Thus, the M1 is somehow “cheating” in other benchmarks where x86 is unnecessarily hobbled.
- Therefore x86 is superior.

At face value, the claim is stupid. But let’s examine the claims that the M1 only does well in optimized scenarios.

Let’s take for instance, any game available for MacOS, and compare it between an intel Mac and an Apple Silicon Mac of equivalent position. You’ll see that the games often run better through a translation layer on the Apple Silicon Macs. Furthermore, as has been repeatedly stated in Mac gaming threads, games are not well optimized for Mac at all, if a port is even released.

If Apple Silicon only “wins” in specific scenarios where it’s well optimized, then how does it beat x86 Macs in any game? Especially considering that ports often do not use any specific MacOS optimizations provided.

Now, you might say, that I should compare x86 PCs with Apple Silicon Macs, and you’d be right that they do “game” better. But even through a translation layer, M1 Macs are not an order of magnitude worse than x86 PCs, as shown by the stockfish chess benchmarks.

In summary:

- in other applications where there is no specific “optimizations” for Apple Silicon, it performs comparably and favorably to its equivalents.
- the only outlier in the performance metrics of Apple Silicon vs. x86, is this single bench, where it performs an order of magnitude worse.

Logically, this proves there’s some issue with that specific bench (as has been stated repeatedly ad nauseum in this thread), and that the praise given to this architecture is warranted.

 

[Taz Mangus]

It‘s always good to see good results.
But 16 vs 17 inch means 1 inch difference and 1 inch is 2.54 centimeter and that’s a lot for games, work, netflix and other stuff.

You know that he earns money by showing benchmarks of the new M1 Max to Apple fans?
Obviously no one will show bad results to (Apple) fans.
If I would sell VW cars, then I wouldn’t show bad results to the customers.

That video shows how crippled the AMD gets when it runs on battery only. While the 14" could achieve full performance on battery alone. That in itself is pretty amazing. That and the fact that the 14" won't sound like a jet engine.

 

[pshufd]

I will but I don't own an M1 mac so I'll need help. For what it's worth now there's nothing indicating that Stockfish is not optimized for M1.

It really feels like people just assume M1 is better, and based on that assumption dismiss all results to the contrary as bogus. That's weird.

Could you point out where I said I think that M1 is better?

1) I haven't seen the code
2) I don't know NEON (though I do know Intel SIMD fairly well - see my username)
3) I don't know the algorithms used
4) I haven't memorized the cyles and latencies of the ARM instruction set as implement on M1
5) It takes a while to learn an architecture
6) It take experience in optimizing for an architecture to get good at it
7) You have to figure out what tools are available for profiling and analysis
8) You have to learn the compiler and linker options to generate efficient code
9) You have to think about things like WPO, PGO, inlining
10) You may choose different algorithms based on things like the number and type of registers
11) Apple has a lot of specialized execution units and it would take a while to learn what they do and if and how you can use them.

Given a massive number of unknowns, I'd think that anyone with the skills and experience in software performance optimization wouldn't make an absolute statements. If you look at the architecture, though, you could make some estimations as to how it should perform based on what you've seen in optimizing for other architectures.

 

[Leifi]

Stop ignoring SPEC, M1 shines in SPEC benchmarks additionally there are many examples of code that runs very well despite lack of optimization, such as Cinebench which is well known to run poorly on macOS and yet M1 is still faster than the Intel Macs they replace with similar core counts.

It is Apple who is ignoring SPEC.. No offical tests published according to SPEC controlled tests.


About cinebench r23:

Mobile CPU	Score
Apple M1	9,569
Apple M1 Pro	12,390
Apple M1 Max	12,402
AMD Ryzen 5800H	12,788
AMD Ryzen 9 5900HS	12,794
Intel Core i9-11950H	12,836
AMD Ryzen 9 5900H	12,892
AMD Ryzen 9 5900HS Creator Edition	13,875
AMD Ryzen 9 5980HX	14,363