Apple M1 CPU & GPU speed is very disappointing

[Taz Mangus]

Written my own Apple supplied header includes??? LoL u are funny dude...

Now you are just being obtuse. What is your point, that you know where to find GCC header files?

 

[throAU]

Written my own Apple supplied header includes??? LoL u are funny dude...

No, written your own C/C++ software with your own libraries.

Or in any language...

Myself? Have done so in C, x86 assembler, Pascal, etc.

 

[bcortens]

Thats great.. Reality is always more interesting than wishful thinking from fanboys...

You can download the best chess engines on github... (can be compiled with homebrew)..

https://github.com/official-stockfish/Stockfish (stockfish)
https://github.com/LeelaChessZero/lc0 (Leela Zero, a spin of project with the origins from Googles Alpha Zero)

A very good page with lots of ready made compiles and stuff for Apple silicon is..

Apple | acepoint's home | Seite 3

acepoint.de

Preliminary results are quite dissapointing for the M1 Max as well.. But try for yourself... You can compare with other platforms... Especailly Leela should perform much better on M1 Max with OpenCL due to the beefier GPU parts.
View attachment 1920990

Given the known GPU compute performance of the M1 pro and Max I’m fairly certain that LC0 is not using the GPU on the M1 - if I have time tomorrow I’ll run it on my M1 Mac and check GPU usage.

 

[bcortens]

Thats great.. Reality is always more interesting than wishful thinking from fanboys...

You can download the best chess engines on github... (can be compiled with homebrew)..

https://github.com/official-stockfish/Stockfish (stockfish)
https://github.com/LeelaChessZero/lc0 (Leela Zero, a spin of project with the origins from Googles Alpha Zero)

A very good page with lots of ready made compiles and stuff for Apple silicon is..

Apple | acepoint's home | Seite 3

acepoint.de

Preliminary results are quite dissapointing for the M1 Max as well.. But try for yourself... You can compare with other platforms... Especailly Leela should perform much better on M1 Max with OpenCL due to the beefier GPU parts.
View attachment 1920990

Looking at the StockFish Makefile i see dedicated optimization paths for each version of the x86-64 SIMD instructions (SSE, SSE2 etc) and yet only a single path for NEON. How much time has really been devoted to one vs the other?

 

[Appletoni]

You would have a point if there was a small advantage between lets say a 5800 and a M1...

But it is like 200%-300% performance diff... The performace diff i HUGE.. Bigger than what you could "solve" by tweaking pipelining SIMD etc..

Please Keep in mind for reference, that the diff in these bencmarks is larger than ANY benchmarked diff between M1 and the latest M1 Max!! (only 40-70% jump) and those have even dubbled the high performance cores of the M1!!

Would you honestly argue that the M1 could be "optimized" to beat the M1 Max as well.. Of course not. this 4-copre (full speed) CPU cannot beat neither Apples 8-core versions, nor AMDs 8-core versions.. Its NOT a "coding" issue. lets get real.

It was Apples bad decision to build only 8 performance cores on the M1 Max chip.
A lot of people expected the same change like the gpu cores, which changed from 8 to 32 cores.
Looking at the heat, when running the MacBook Pro at the limit, I think it should be even possible to build 80 performance cores into the M1 Max. There will probably be no heat problem when you don‘t need the GPU for your work.
When the battery of the M1 Max reaches 0% after running Stockfish analysis for 1 hour, what will happen if you run LC0 (GPU) in the same time?
Will it reach 0% after 10 to 15 minutes?
It will be interesting to know how fast someone can get the M1 Max Battery from 100% to 0%.

 

[TSE]

UNIMPRESSED...? DESIGN YOUR OWN CPU & GPU, OP!!!​

 

[pshufd]

I would be more happy If Apple spent some time trying to improve their CPUs, GPUs, and stop being so d*mn overpriced, proprietary, and un-open .... More honesty and less dishonest sales-pitches, and less uninformative misleading graphs when presenting new stuff.. Thanks.... Please

Why? They are making money hand over fist. Their market cap is almost ten times that of Intel. If what you're doing works, do more of it.

 

[Leifi]

Broadly, I think we're in agreement that the M1 looks amazing and as soon as devs start taking advantage of it, performance will be great. I'm not sure I'd go quite as far as you in saying Intel and AMD have a great deal to worry about though. I think they'll be just fine.

I'd be grateful for any links to quality chess benchmarks though. I'm having a hard time finding any.

We are not in agreement on that , at least not from where I am sitting. I am quite disappointed in Apple Silicon's real-life performance. Despite all the hype, it gets hammered by cheaper older 7nm CPUs in benchmarks like these and many more. And of course, it is nowhere near a modern Nvidia+Amd or Nvidia+AMD laptop these days performance-wise :-(

It seems laptop beasts even with 3090 (and 4xxx GPUs) and Zen4s and AlderLakes with even more cores are inbound for early 2022 which will widen the gap down to Apple, even more, I am afraid.

 

[mr_roboto]

asmFish Benchmark - OpenBenchmarking.org

openbenchmarking.org

The newest binaries for macOS were posted August 2019, a bit over 2 years ago, and are x86 only. Apple had not yet announced Apple Silicon Macs.

asmFish/MacOS_binaries at master · lantonov/asmFish

A continuation of the nice project asmFish by Mohammed Li. Latest version: 07.08.2019 - lantonov/asmFish

github.com

The arm64 source code is all 4 years old, which is so long ago that it's plausible Apple's chip architects might not have even begun work on the A14/M1 generation of chips when these commits were pushed to github.

asmFish/arm at master · lantonov/asmFish

A continuation of the nice project asmFish by Mohammed Li. Latest version: 07.08.2019 - lantonov/asmFish

github.com

The arm64 port only supports Linux (unsurprising given those dates):

asmFish/arm/fish.arm at master · lantonov/asmFish

A continuation of the nice project asmFish by Mohammed Li. Latest version: 07.08.2019 - lantonov/asmFish

github.com

(look at lines 5-7; if you look at the equivalent x86 file it expects constant 'L' for Linux, 'W' for Windows, 'X' for OSX, but the arm64 code only knows about 'L'inux)

So no, you haven't found a gotcha here. You've found someone mindlessly running PTS by punching a button and recording an emulated result.

This is one of the countless reasons why anyone who knows anything about benchmarking methodology distrusts Phoronix. The guy who runs that site operates purely on a quantity-over-quality basis, whether it's his news articles or how he designs his benchmark suite. It isn't useful data.

 

[throAU]

Why? They are making money hand over fist. Their market cap is almost ten times that of Intel. If what you're doing works, do more of it.

For what you get they're also not expensive.

When you take into account the enclosure, the screen, the speakers, the software, etc.

Now, they might be expensive if you don't want some of those things (you have no choice to delete things you don't want), but that's another issue.

 

[throAU]

The newest binaries for macOS were posted August 2019, a bit over 2 years ago, and are x86 only. Apple had not yet announced Apple Silicon Macs.

asmFish/MacOS_binaries at master · lantonov/asmFish

A continuation of the nice project asmFish by Mohammed Li. Latest version: 07.08.2019 - lantonov/asmFish

github.com

The arm64 source code is all 4 years old, which is so long ago that it's plausible Apple's chip architects might not have even begun work on the A14/M1 generation of chips when these commits were pushed to github.

asmFish/arm at master · lantonov/asmFish

A continuation of the nice project asmFish by Mohammed Li. Latest version: 07.08.2019 - lantonov/asmFish

github.com

The arm64 port only supports Linux (unsurprising given those dates):

asmFish/arm/fish.arm at master · lantonov/asmFish

A continuation of the nice project asmFish by Mohammed Li. Latest version: 07.08.2019 - lantonov/asmFish

github.com

(look at lines 5-7; if you look at the equivalent x86 file it expects constant 'L' for Linux, 'W' for Windows, 'X' for OSX, but the arm64 code only knows about 'L'inux)

So no, you haven't found a gotcha here. You've found someone mindlessly running PTS by punching a button and recording an emulated result.

This is one of the countless reasons why anyone who knows anything about benchmarking methodology distrusts Phoronix. The guy who runs that site operates purely on a quantity-over-quality basis, whether it's his news articles or how he designs his benchmark suite. It isn't useful data.

For my next trick, I'm going to go benchmark ARM64 Linux code inside a VM on x86 Windows via QEMU

 

[Taz Mangus]

We are not in agreement on that , at least not from where I am sitting. I am quite disappointed in Apple Silicon's real-life performance. Despite all the hype, it gets hammered by cheaper older 7nm CPUs in benchmarks like these and many more. And of course, it is nowhere near a modern Nvidia+Amd or Nvidia+AMD laptop these days performance-wise :-(

It seems laptop beasts even with 3090 (and 4xxx GPUs) and Zen4s and AlderLakes with even more cores are inbound for early 2022 which will widen the gap down to Apple, even more, I am afraid.

Based on a faulty benchmark test.

 

[bcortens]

We are not in agreement on that , at least not from where I am sitting. I am quite disappointed in Apple Silicon's real-life performance. Despite all the hype, it gets hammered by cheaper older 7nm CPUs in benchmarks like these and many more. And of course, it is nowhere near a modern Nvidia+Amd or Nvidia+AMD laptop these days performance-wise :-(

It seems laptop beasts even with 3090 (and 4xxx GPUs) and Zen4s and AlderLakes with even more cores are inbound for early 2022 which will widen the gap down to Apple, even more, I am afraid.

Performance isn’t disappointing for most people. My compile times are almost half what the were previously on a 2019 8-core MBP, all while silent. The intel machine gets very loud during heavy compilation and this is a tangible quality of life improvement.

 

[JimmyjamesEU]

We are not in agreement on that , at least not from where I am sitting. I am happy with Apple Silicon's real-life performance. Even with all the hype, it dominates the opposition.

I’m a little confused then. We’re both saying it’s an excellent chip that beats traditional systems. I don’t see why the need for aggression. I just want to see some benchmarks for chess before I declare the death of x86 as you have.

 

[Appletoni]

Hi. Really interesting thread. I've been glued to it from the first page. Thanks to all those involved.

I've been wondering if it might be worthwhile performing some chess benchmarks on the M1 Pro/Max? It's obvious that these new chips are great performers on a wide variety of tasks. It'd be interesting to see some numbers for chess though. Once they get a chance to optimise these benchmarks for Apple Silicon, I think they'll do great.

Also, can anyone point me to some numbers for hashcat runs on the the M1 Max?

That’s a very good idea

 

[Leifi]

Why? They are making money hand over fist. Their market cap is almost ten times that of Intel. If what you're doing works, do more of it.

The good ol' - "10 million flies can't be wrong ... **** taste good" argument does not really fly with me.

I talked about what would make me happy, not what makes Apple happy... I fully realize no one really cares what I think and If Apple gets a better revenue by selling mediocre hw at exorbitant prices to many people who may or may not know better, then It is fully understandable that they will not make an effort to deliver value with lower margins... It's Not just for me, I like to get what I pay for... I also like openness, open hw-access, and open-source and little DRM and restrictions, code-signing, etc. Apple is sort of an anti-christ in this regard ;-)

 

[throAU]

Performance isn’t disappointing for most people. My compile times are almost half what the were previously on a 2019 8-core MBP, all while silent. The intel machine gets very loud during heavy compilation and this is a tangible quality of life improvement.

In actual real life use this is the first machine I genuinely think could replace my desktops. Because it can actually run fairly high end workloads without the noise.

VAST contrast to my previous MacBook Pros, or any other 14-16" laptop I've used since the late 90s.

 

[Appletoni]

Thats great.. Reality is always more interesting than wishful thinking from fanboys...

You can download the best chess engines on github... (can be compiled with homebrew)..

https://github.com/official-stockfish/Stockfish (stockfish)
https://github.com/LeelaChessZero/lc0 (Leela Zero, a spin of project with the origins from Googles Alpha Zero)

A very good page with lots of ready made compiles and stuff for Apple silicon is..

Apple | acepoint's home | Seite 3

acepoint.de

Preliminary results are quite dissapointing for the M1 Max as well.. But try for yourself... You can compare with other platforms... Especailly Leela should perform much better on M1 Max with OpenCL due to the beefier GPU parts.
View attachment 1920990

Don’t forget Ceres. This chess engine uses cpu + gpu in the same time for analysis

GitHub - dje-dev/Ceres: Ceres - an MCTS chess engine for research and recreation

Ceres - an MCTS chess engine for research and recreation - dje-dev/Ceres

github.com

 

[Leifi]

I’m a little confused then. We’re both saying it’s an excellent chip that beats traditional systems. I don’t see why the need for aggression. I just want to see some benchmarks for chess before I declare the death of x86 as you have.

It's an excellent chip..ARM is great and big little type architectures is great for battery-life. But it doesn't really beat the competition currently.. That just reality... If you like things like, chess, Go, Shogi, graphics, AI, pytorch, gaming etc. etc. it does not offer a good price/performance proposition compared to x86 alternatives.

 

[Appletoni]

Given the known GPU compute performance of the M1 pro and Max I’m fairly certain that LC0 is not using the GPU on the M1 - if I have time tomorrow I’ll run it on my M1 Mac and check GPU usage.

That’s also a very good idea.

 

[Taz Mangus]

The good ol' - "10 million flies can't be wrong ... **** taste good" argument does not really fly with me.

I talked about what would make me happy, not what makes Apple happy... I fully realize no one really cares what I think and If Apple gets a better revenue by selling mediocre hw at exorbitant prices to many people who may or may not know better, then It is fully understandable that they will not make an effort to deliver value with lower margins... It's Not just for me, I like to get what I pay for... I also like openness, open hw-access, and open-source and little DRM and restrictions, code-signing, etc. Apple is sort of an anti-christ in this regard ;-)

So you are complaining because you can't afford the price Apple is charging. As you have stated previously, there are other options for you that are far cheaper for you. Take your own advice.

 

[mr_roboto]

It was Apples bad decision to build only 8 performance cores on the M1 Max chip.
A lot of people expected the same change like the gpu cores, which changed from 8 to 32 cores.
Looking at the heat, when running the MacBook Pro at the limit, I think it should be even possible to build 80 performance cores into the M1 Max. There will probably be no heat problem when you don‘t need the GPU for your work.
When the battery of the M1 Max reaches 0% after running Stockfish analysis for 1 hour, what will happen if you run LC0 (GPU) in the same time?
Will it reach 0% after 10 to 15 minutes?
It will be interesting to know how fast someone can get the M1 Max Battery from 100% to 0%.

Why was it a bad decision? Eight P cores is good enough for Apple to have taken the crown for laptop multithreaded performance by a very comfortable margin.

You seem a bit confused about heat. Each M1 P core can use about 5W, so someone looking to design a system with eighty of them on a single chip would be looking at trying to supply and cool 400W. Absolutely insane in a desktop, just forget about it in a laptop, it's a terrible idea. And there wouldn't be any die area left over for GPU cores.

 

[JimmyjamesEU]

It's an excellent chip..ARM is great and big little type architectures is great for battery-life. But it doesn't really beat the competition currently.. That just reality... If you like things like, chess, Go, Shogi, graphics, AI, pytorch, gaming etc. etc. it does not offer a good price/performance proposition compared to x86 alternatives.

I’m not sure why you keep pushing this “M1 as Saviour” narrative. I don’t appreciate your tone or aggression. If you can’t show any chess benchmarks to show the putative superiority of the M1 over the competition, I’d ask you politely to calm down. Seriously, there is no need for this. We can have a mature discussion surely?

 

[Taz Mangus]

But it doesn't really beat the competition currently.

A meaningless statement.

 

[Leifi]

No, written your own C/C++ software with your own libraries.

Or in any language...

Myself? Have done so in C, x86 assembler, Pascal, etc.

It's a moot OT discussion, and yes I do Rust, Go, C/C++, Ada, haskl, Java, x86, sparc, arm, 6502, 680xx, Z/OS HLASM, Ruby, Python, C# etc.. But for the non existant prospect of increasing Stockfish speed by 200-300% on a Apples ARMv8.4 CPU should be ovious for you if you even have a basic grasp of the coding you refer to.

Stockfish/simd.h at master · official-stockfish/Stockfish

UCI chess engine. Contribute to official-stockfish/Stockfish development by creating an account on GitHub.

github.com