Apple M1 CPU & GPU speed is very disappointing

[Appletoni]

Not really. Apple Silicon performs exceedingly well in most of real-world (or synthetic) tests. The problem of these chess engines tests is not that they "hammer the hw multithreaded big-time". The problem is that they either don't support Apple platform fully (e.g. no GPU acceleration, no native SIMD) or are not well optimized and or not tested. Same goes for benchmarks like Cinebench, of which we know that they rely on compatibility layers that generate suboptimal code.

When discussing performance, one must carefully analyze the reasons why something is slow/fast. Where is the bottleneck? Is it running the optimal path? You cannot assess hardware performance without talking about these things.

Interesting opinion.
Looking fast at the internet I found this:

Stockfish up to 8 times faster on Apple M1 - TalkChess.com

Use SpaCy 3.2 when running Stockfish on Apple devices · Issue #3788 · official-stockfish/Stockfish

https://spacy.io/usage/v3-2 It can make Stockfish up to 8 times faster on the M1, M1 Pro and M1 Max.

github.com

Apple MacBook Pro 16-inch with M1 Max chip - TalkChess.com

It looks more like it’s Apples fault.

 

[Appletoni]

What?

Hyperthreading is only necessary when you have so poorly designed your instruction dispatch that you can’t keep your ALUs busy. M1 has tremendous IPC. It’s execution units are always busy. Adding hyper threading would slow the cpu down because it would cause unnecessary context switching.

As per forum rules, please cite your source for the statement beginning “it is obvious…”

Adding hyper threading on intel cpus is increasing the performance of Stockfish by 33% more kn/s.
Some guys mentioned that on Apples M1 hyper threading will increase the performance at least a little, talking about worst case scenario.

 

[leman]

As per your question to me there are already references in this thread to multiple GitHub discussions etc. which you can look up discussion with the top-tier chess engine developers about the performance of Apple silicon.. for example:

M1 Apple Silicon for Chess? - Page 55 - TalkChess.com

Apple M1 chip is much better at everything but very slow in chess · Issue #3529 · official-stockfish/Stockfish

What are the reasons? How many things in detail cause this slowdown? -M1 chip -macOS Big Sur -Stockfish engine (source code) Any ideas how to fix the one or another problem? Would it be possible to...

github.com

etc.


Can you name one top programmer of chess engines that would support a claim that Apple silicon beats cheaper AMD CPUs currently? ?

I just had a quick look at this discussion - no idea who there is a top programmer of chess engines and who is not, I just see a lot of bickering. There is zero technical discussion. The thread basically boils down to the following: nobody there tests or develops their engines on Apple Silicon and if you want them to perform well you need to take matters into your own hand.

Not Only chess.... Just look at openbenchmarking... Chess is just one demanding use-case where the performanceof the Apple silicion is slower than much cheaper alternatives..

Average CPU perf on https://openbenchmarking.org/


View attachment 1919485

Apple M1 vs. AMD Ryzen 7 4800U vs. AMD Ryzen 7 4700U Benchmarks - OpenBenchmarking.org

openbenchmarking.org

I really hope you are not serious… this is based on six benchmarks, three of which are GPU benchmarks with poor macOS support, one is a test of an Intel raytracing library and two are browser tests.

Why don’t you look at SPEC2017 - an industry standard workstation CPU benchmark suite? Or standard browser tests? Or compiler performance tests (these are often good predictors for performance of complex software logic).

Interesting opinion.
Looking fast at the internet I found this:

Stockfish up to 8 times faster on Apple M1 - TalkChess.com

Use SpaCy 3.2 when running Stockfish on Apple devices · Issue #3788 · official-stockfish/Stockfish

https://spacy.io/usage/v3-2 It can make Stockfish up to 8 times faster on the M1, M1 Pro and M1 Max.

github.com

Apple MacBook Pro 16-inch with M1 Max chip - TalkChess.com

It looks more like it’s Apples fault.

And more threads of random bickering.

 

[Appletoni]

I just had a quick look at this discussion - no idea who there is a top programmer of chess engines and who is not, I just see a lot of bickering. There is zero technical discussion. The thread basically boils down to the following: nobody there tests or develops their engines on Apple Silicon and if you want them to perform well you need to take matters into your own hand.



I really hope you are not serious… this is based on six benchmarks, three of which are GPU benchmarks with poor macOS support, one is a test of an Intel raytracing library and two are browser tests.



And more threads of random bickering.

No one show results, because they are bad.

MacBook Pro 16-inch M1 Max Stockfish Benchmarks

Do we have some Stockfish Benchmarks to look at the results? Some people mentioned that the M1 Max chip will score a bad result.

forums.macrumors.com

 

[cmaier]

As per forum rules. Your claim a lot of things about SMT without a source.. let's start there. What your source claiming SMT-support hurts CPU performance (given the die size and same number of physical cores), and only help "poorly designed" code

As per your question to me there are already references in this thread to multiple GitHub discussions etc. which you can look up discussion with the top-tier chess engine developers about the performance of Apple silicon.. for example:

M1 Apple Silicon for Chess? - Page 55 - TalkChess.com

Apple M1 chip is much better at everything but very slow in chess · Issue #3529 · official-stockfish/Stockfish

What are the reasons? How many things in detail cause this slowdown? -M1 chip -macOS Big Sur -Stockfish engine (source code) Any ideas how to fix the one or another problem? Would it be possible to...

github.com

etc.


Can you name one top programmer of chess engines that would support a claim that Apple silicon beats cheaper AMD CPUs currently? ?

I never said anything about poorly designed “code.” I talked about “instruction dispatch.” That is not code.

As for my sources, here are some below. (As someone who actually designed real microprocessors, including for AMD and others, I won’t post to my own work, because that would be cheating):

Deep inside Intel's new ARM killer: Silvermont

Chipzilla's Atom grows up, speeds up, powers down

www.theregister.com

Hyper-Threading and Everything You Need to Know

Learn everything about Hyper-Threading and how it benefits your PC performance on HP® Tech Takes.

www.hp.com

How long does it take to make a context switch?

That's a interesting question I'm willing to spend some of my time on. Someone at StumbleUpon emitted the hypothesis that with all the impr...

blog.tsunanet.net

http://home.eng.iastate.edu/~zzhang/courses/cpre585/slides/Lecture25.pdf

Current wisdom on SQL Server and Hyperthreading?

Lots of articles out there (see Slava Oks's original SQL 2000 article and Kevin Kline's SQL 2005 update) recommend disabling hyperthreading on SQL servers, or at least testing your specific workload

serverfault.com

Hyper-Threading may be Killing your Parallel Performance

Hyper-Threading, hyperthreading, or just HT for short, has been around on Intel processors for over a decade and it still confuses people. I'm not going to do much to help with the confusion. I just want to point out an example from some testing I was doing recently with the ray-tracing...

www.pugetsystems.com

The Cost Performance of Hyper-Threading Technology in the Cloud Computing Systems

Hyper-Threading (or HT, for short) is a technology used in some Intel CPUs. Intel claims that it can use processor resources more efficiently. Many past studies have evaluated the performance of the technology in HPC clusters. In this paper, we discuss the advantages...

link.springer.com

Analysis of Hyper-threading and Its Implementation Technology--《Computer Engineering》2004年23期

Firstly, the basic concepts of process, thread and multithread are presented. Secondly, the work principle of hyper-threading and the condition of implementing hyper-threading are analyzed detailedly. Then, the advantage and disadvantage of hyper-threading technology are analyzed. Finally, an...

en.cnki.com.cn

Replay: Unknown Features of the NetBurst Core. Page 15 - X-bit labs

In the third part of our NetBurst Architecture investigation trilogy we are going to reveal the details of the Replay mechanism Implemented in Intel Pentium 4 processors, which Intel keeps quiet about. This particular mechanism and its working principles explain why Pentium 4 processors perform...

web.archive.org

Now that that’s out of the way, where are your sources?

 

[cmaier]

Adding hyper threading on intel cpus is increasing the performance of Stockfish by 33% more kn/s.
Some guys mentioned that on Apples M1 hyper threading will increase the performance at least a little, talking about worst case scenario.

Which is relevant how? It is well known that M1’s instruction dispatch is much much wider than Intel’s, and that M1’s IPC is much higher - the ALUs are already full.

Of course hyperthreading generally helps *Intel* - they have a terrible instruction fetch and dispatch design because of the problems decoding x86-64. And before you try and tell me that x86-64 is not terrible, I’m the guy who wrote the first draft of the AMD64 integer instruction set (which we now call x86-64).

 

[robco74]

No one show results, because they are bad.

MacBook Pro 16-inch M1 Max Stockfish Benchmarks

Do we have some Stockfish Benchmarks to look at the results? Some people mentioned that the M1 Max chip will score a bad result.

forums.macrumors.com

No one show result because the testing tool you linked to only supports Windows, it specifically states there is no macOS version. Why would you expect people who may not be interested in chess engines to track down a bunch of chess engines and then try to run benchmarks they don't understand and/or couldn't care less about?

If you earn your living by running chess programs, then you should, according to the sources given, purchase an AMD machine. But if you are one of the billions who doesn't, then perhaps there are other areas where Apple's chips can provide excellent performance.

In any case, I fail to see why you are here kvetching about M1, instead of continuing your circular self-gratification over at the chess forums.

 

[Juraj22]

I guess that this talk is about this project: https://github.com/LeelaChessZero/lc0

Project supports CUDA, Intel+AMD AVX libraries, OpenCL for AMD GPUs.

It is going to suck on Apple platform for those reasons:
1. No Metal support
2. Bad OpenCL on Apple platform. Apple deprecated that long time ago. Performance is suboptimal at best. Path forward is Metal.
3. project uses CBLAS-compatibile library, meaning that on Apple platforms they are not using Accelerate framework, resulting in bad performance.

If you need to use this project buy Intel + Nvidia. This is the best combination.

As a benchmark to quantify how fast M1 is, this is not the best or fair example.

I could make a model that needs to use 50GB GPU memory and than complain that it only runs on M1Max with 64GB of ram and that 3080 is bad card, because it is not even able to run the model. (Yeah, Apple did this in their marketing)
But that would be equally stupid.

 

[Appletoni]

Which is relevant how? It is well known that M1’s instruction dispatch is much much wider than Intel’s, and that M1’s IPC is much higher - the ALUs are already full.

Of course hyperthreading generally helps *Intel* - they have a terrible instruction fetch and dispatch design because of the problems decoding x86-64. And before you try and tell me that x86-64 is not terrible, I’m the guy who wrote the first draft of the AMD64 integer instruction set (which we now call x86-64).

But that means:
1. The bad designed Intel is with and without hyperthreading much faster than Apple M1 Max.
2. You are the guy who wrote the first terrible draft, that means it’s you fault xD.

 

[leman]

1. The bad designed Intel is with and without hyperthreading much faster than Apple M1 Max.

Much faster where? In selected chess benchmarks developed for Intel? Sure. In everything else? Hardly.

 

[cmaier]

But that means:
1. The bad designed Intel is with and without hyperthreading much faster than Apple M1 Max.
2. You are the guy who wrote the first terrible draft, that means it’s you fault xD.

Much faster than what? A random chess benchmark?

Ok, you got me.

But that’s a little bit like winning the “Best Dad in this house” award.

 

[throAU]

Yes, M1 CPU seems to be way below (its) average on this particular benchmark. If your main use for your M1 Mac is to run chess engines, you would be better off buying a different computer.

Yeah one would suspect the software is .... not optimised for this processor. Yet.

Likely if it was to use the AI/ML cores it would be faster.

 

[AdamNC]

Benchmark tests all have their own bias and weaknesses and strengths. Just like PC vrs Mac. To say ones a POS vrs another based on a imperfect test format is effing stupid. If I wanted to play Crysis all day long would I buy a Mac? Hell no. If I wanted to mix music and creat a awesome song would I buy a PC again no way. No one computer can do everything with perfection.

 

[uller6]

Adding hyper threading on intel cpus is increasing the performance of Stockfish by 33% more kn/s.
Some guys mentioned that on Apples M1 hyper threading will increase the performance at least a little, talking about worst case scenario.

M1 CPU’s do not use simultaneous multi threading (SMT), aka hyperthreading.

 

[George Knighton]

My first M1 is the MacBook Air that I'm using right now. Had it for a few months now.

Absolute base version, bought when I found it on sale at a good price...a rare thing at Apple.

I have no doubt at all that there will be things that it doesn't do well. Not all applications are going to be optimised for the M1, but there's every indication that is a temporary situation.

All I have to do is Mail, Safari and iMessages, pretty much, and it excels at that. It seems to render Web pages amazingly quickly. Occasional forays into Numbers and conversions to Excel.

Amazing battery life.

I'm so surprised at what it can do for so so little money that I went ahead and optioned out a two-fan 24" iMac to replace my 27" iMac that dates all the way back to 2015.

These are awesome little things.

 

[ddhhddhh2]

Usually, you can ask the software developer to optimize the M1 processor, and while there are many chess pieces, the one you love does not perform well on the M1.

I am sure that when you are willing to invest tens or millions of dollars, the developer will be willing to help you and we are waiting for your good news.

Checkmate!

 

[Leifi]

All I have to do is Mail, Safari and iMessages, pretty much, and it excels at that. It seems to render Web pages amazingly quickly. Occasional forays into Numbers and conversions to Excel.

Single thread netbook native-optimized apps and some occasional video editing is fine use-cases for Apples ARM versions.. But those are all use cases where even a $200 phone will get the job done perfectly well these days. Chess and other high-perf. tasks are a different matter all together.. You cannot hide poor overall pipelining and multithread performance (some lower clocked cores as well) by being fast on single-core benches with these kind of more higher-end workloads.

 

[Leifi]

Usually, you can ask the software developer to optimize the M1 processor, and while there are many chess pieces, the one you love does not perform well on the M1.

I am sure that when you are willing to invest tens or millions of dollars, the developer will be willing to help you and we are waiting for your good news.

Checkmate!

Seen on talkchess..

Frankly I find it a bit disgusting to claim that it is the delveopers fault that Apple Silicon performs poorly on cross compiled multiplatform open source code.

 

[Leifi]

Now that that’s out of the way, where are your sources?

Pretty lame to post a bunch of link that pretty much all of them blatantly contradicts your statements about SMT not benefitting performance, and no statements that this would be utterly useless on ARM .

Also remember there are ARM SMT architectures around like Neoverse-E1 etc.

 

[dugbug]

Seen on talkchess..

View attachment 1920102

Frankly I find it a bit disgusting to claim that it is the delveopers fault that Apple Silicon performs poorly on cross compiled multiplatform open source code.

I mean, you are armed with a MEME now. You win.

 

[Leifi]

I mean, you are armed with a MEME now. You win.

LOL

But the question remains. The pundits that claim chess engines can be improved vastly on Apple silicon should put up some proof. What exactly are the optimizations possible and how much performance will be estimated to gain. The chess engines are completely open-source so pointing out the details and estimated performance gain would be prudent. If all this "optimize for M1" talk isn't simply just a simple way to try to distract from the fact that M1 benches poorly on these types of workloads?

 

[jdb8167]

LOL

But the question remains. The pundits that claim chess engines can be improved vastly on Apple silicon should put up some proof. What exactly are the optimizations possible and how much performance will be estimated to gain. The chess engines are completely open-source so pointing out the details and estimated performance gain would be prudent. If all this "optimize for M1" talk isn't simply just a simple way to try to distract from the fact that M1 benches poorly on these types of workloads?

Apple silicon in a platform that would benefit from such optimization has existed now for almost exactly 12 months. How long has the x86 optimization been going on?

That’s not to say anyone cares enough to optimize the chess programs for M1 but even if someone wanted to put in the effort, how long would it take to catch up?

 

[Romain_H]

Single thread netbook native-optimized apps and some occasional video editing is fine use-cases for Apples ARM versions.. But those are all use cases where even a $200 phone will get the job done perfectly well these days. Chess and other high-perf. tasks are a different matter all together.. You cannot hide poor overall pipelining and multithread performance (some lower clocked cores as well) by being fast on single-core benches with these kind of more higher-end workloads.

You are aware that this is going to bite you? Because Apples multithreaded performance and pipelining are actually much much better than Intel‘s.
The whole argument revolves around that, because if the software does not take advantange of it - being unoptimized - the results will predictably be underwhelmig. What is excactly the case here.

Unoptimized software running poorly. What a surprise

 

[cmaier]

Pretty lame to post a bunch of link that pretty much all of them blatantly contradicts your statements about SMT not benefitting performance, and no statements that this would be utterly useless on ARM .

Also remember there are ARM SMT architectures around like Neoverse-E1 etc.

What do ARM SMT architectures have to do with anything? There‘s a difference between microarchitecture and architecture.

Now, once again, before you get to comment on my links to sources, where are your sources? The forum rules require that when challenged on a factual statement, you post proof. Where is your proof?

 

[Leifi]

"You are aware that this is going to bite you?"

So bite me.. prove me wrong..No one would be happier than me if Apple silicon performed well for chess. It seems there are some non-technical persons here claiming how great the performance the M1 (or Max) "could be" if only people "cared" about optimizing.. But its all talk no evidence,code samples or "proof".

Once again What exactly do you suggest could be done to improve performance, and how much do you estimate to gain for a program like LC0 or stockfish..

There is so much BS and claims and no one who can put up some real numbers or even code that would prove it "can" be optimized to get even close to a 4800 or 5800 7nm CPU..

"The whole argument revolves around that, because if the software does not take advantange of it - being unoptimized - the results will predictably be underwhelmig. What is excactly the case here."

You are aware that compilers etc. used to compile these chess engines on Apple silicon indeed optimizes the code for the CPU.. do you realize that? Or are you claiming that it have to be coded in native machine op-code hand-crafted and optimized much much more for an Apple CPU than for competing platforms to show how *good" Apple silicone is?? And what gain would you expect from this? 5-10%??