Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Status
Not open for further replies.

AppleB

macrumors 65816
Oct 18, 2011
1,156
1,380
Benchmarking is a waste of time. What matters, is real world day to day use.

True that.

I’m running 100 plus audio tracks in Pro Tools with plugins with a M1 Mini and it is smooth as silk zero issues.
I’m not racing my computer with another one with benchmarks. I’m using my computer. If work gets done that’s all that matters.
 
Last edited:

Apple_Robert

Contributor
Sep 21, 2012
35,666
52,486
In a van down by the river
True that.

I’m running 100 plus audio tracks in Pro Tools with plugins with a M1 Mini and it is smooth as silk zero issues.
I’m not racing my computer with another come with benchmarks. I’m using my computer. If work gets done that’s all that matters.
And that is what it's all about; real use performance.
 

jeanlain

macrumors 68020
Mar 14, 2009
2,462
956
Funny how people selectively preach dorkbench when M1 does well but downplay a more relevant real world workload when M1 doesn't. Chess has been a relevant workload and benchmark going back to IBM Deep Blue to current DeepMind AlphaZero that topple all the grandmaster human players.
The OP's result are only relevant it you intend to use some version of stockfish on the M1 (we don't even know which version as the OP has not bothered to give details).

Do you not know that Geekbench and other benchmark tools consists in "real world workloads", which include chess (in SPEC) ? There are just many of them, which makes these tools more relevant than a single algorithm like Stockfish and Cinebench to evaluate a CPU performance.
What's more, benchmark tools use pieces of code that are designed to be as platform-agnostic as possible. It is no coincidence that SPEC tests, Geekbench, and Novabench results are in broad agreement WRT to CPU relative scores.

Stotckfish has never proven to be a relevant tool to compare CPU performance, especially CPUs from different architectures. It's not designed to. The same can be said for Cinebench BTW. It is unclear whether the level of optimisation is the same for X86 an ARM (the first ARM version is only months old).

You can find programs that run super slow on M1 compared to X86 CPUs. These are usually open-source programs that have a long history of optimisation for X86 and have only been ported recently to ARM with minimal/no optimisation. Just for compatibility.
In fact, some programs run much slower under ARM native than under Rosetta 2, which shows how much the code lacks ARM optimisations (no SIMD code), while the X86 version have optimisations that Rosetta can take advantage of.
 
Last edited:
  • Like
Reactions: Tagbert

jeanlain

macrumors 68020
Mar 14, 2009
2,462
956
And that is what it's all about; real use performance.
Sure, but when the tools you use have not been tested, the best you can do is look at results form synthetic benchmarks.
I intend to use my next Mac for data analyses using various programs, none of which is used by testers on the web. But I know that my next Mac will run these tools fast, because results from synthetic benchmark tools show that the M1 is among the best CPUs across the board.
 

Apple_Robert

Contributor
Sep 21, 2012
35,666
52,486
In a van down by the river
Sure, but when the tools you use have not been tested, the best you can do is look at results form synthetic benchmarks.
I intend to use my next Mac for data analyses using various programs, none of which is used by testers on the web. But I know that my next Mac will run these tools fast, because results from synthetic benchmark tools show that the M1 is among the best CPUs across the board.
The logical thing to do is get the Mac, test it with your needed apps, and then determine if the Mac will meet your needs. Using the Benchmark doesn't answer any real world usage questions.
 

jeanlain

macrumors 68020
Mar 14, 2009
2,462
956
The logical thing to do is get the Mac, test it with your needed apps, and then determine if the Mac will meet your needs. Using the Benchmark doesn't answer any real world usage questions.
I don't want to go into the hassles of returning a Mac if it doesn't suits me. And I can't test the tools I need in an Apple Store.
Benchmark tools do give a good general idea of what to expect. That's what they're designed for! They consist in a range of "real world" workloads.
 
  • Like
Reactions: Tagbert

mi7chy

macrumors G4
Oct 24, 2014
10,625
11,296
No, the issue is not downplaying unfavorable benchmarks, but the benchmarks must be fair and properly coded for the ARM/M1 platform.

I don't expect the discussion to be at the level of Phoronix but not even the basic questions were asked like is it native app, installed under Homebrew, using standardized configuration template, how to improve results, etc. before quickly downplaying it since results are unfavorable. That's why medication lists not only positive attributes but also side effects so people can decide based on pros vs cons.
 

xraydoc

Contributor
Oct 9, 2005
11,027
5,488
192.168.1.1
And I can't test the tools I need in an Apple Store.
If the benchmark apps would fit on a flash drive, I don't see why you couldn't run them at the Apple Store. They mostly don't care what you do on the demo machines as they re-image them nightly anyway.
 

leman

macrumors Core
Oct 14, 2008
19,521
19,679
Funny how people selectively preach dorkbench when M1 does well but downplay a more relevant real world workload when M1 doesn't. Chess has been a relevant workload and benchmark going back to IBM Deep Blue to current DeepMind AlphaZero that topple all the grandmaster human players.

There was maybe one intelligent response trying to understand what the bottleneck is.

Which is exactly the point. M1 performs excellently in practically all synthetic and real world tests. It performs especially well in workloads that deal with traversing complex data structures and simulations (just like chess). So if someone comes along and claims that M1 is 10 times slower than Intel CPUs on a particular workload, I will immediately dismiss them because either their test is flawed or it’s x86 software that hits worst-case corner cases of Rosetta2.

It’s up to the person producing the claim to show that their methodology makes sense and their result is valid. Then we can try to have a productive discussion. OP didn’t show any interest in being constructive, they just came with an isolated statement that contradicts all we know up to now. So yeah, my immediate reaction is “nice story, who cares?”
 

MarkC426

macrumors 68040
May 14, 2008
3,697
2,097
UK
Agreed.

Have seen some real world tests for my 3D software (Lightwave 3D), using the same scene file.

6 Core i9 MacBook Pro - 3 minutes
12 Core 2013 MacPro - 2 minutes
M1 MacBook Air 8gb - 2 minutes 40 seconds
Also forgot to mention M1 is using Rosetta, which makes the result even better.
 

jeanlain

macrumors 68020
Mar 14, 2009
2,462
956
If the benchmark apps would fit on a flash drive, I don't see why you couldn't run them at the Apple Store. They mostly don't care what you do on the demo machines as they re-image them nightly anyway.
These tools require installing with admin privileges. Anyway, there's no App Store close to where I live, and I have no doubt about the performance the M1 should yield on the tools I use.

As for the OP's claims according to which the M1 is more than ten times slower than the competition, they appear to contradict results reported here: http://talkchess.com/forum3/viewtopic.php?f=2&t=75911&start=50

I tested
Code:
stockfish bench 64 1 20
on my iMac with a core i5 7600k. I get 1560 kN/s, which is in line with results reported above.
The M1 yields 3024 kN/s with the command I used.

The scores of 230000 kN/s that the OP mentions must correspond to 256-core compute nodes or something.
 
Last edited:

Mcckoe

macrumors regular
Jan 15, 2013
170
352
I have A10X chips that are just marginally slower than our 12X chips for 99% of workload. I’m now upgrading to the new M1, and really the only reason is because my youngest daughter; needs an new tablet and our old 10.5” pro is still better than the base line iPad(Promotion). The A10X was way overpowered, and still does everything; A12X was insanely overpowered, and 3 years later still pushes boundaries; M1 is game changing, and evades the boundaries of custom silicon.

I’m sorry a version of the M1 architecture: didn’t perform an older coding language, through two emulators, faster than a chip architecture it was directly designed for.

In real-world testing: the M1, allows almost perfect non-gaming 4K computing. Anything more, will not be needed for 99% of the customer base that falls into this category.
 

Jorbanead

macrumors 65816
Aug 31, 2018
1,209
1,438
before quickly downplaying it since results are unfavorable.
Funny how people selectively preach dorkbench when M1 does well

Most people preach real-world examples because that’s ultimately what matters. Every single major benchmark that is out right now was developed and optimized for X86 hardware. Some of them have better optimization for ARM than others, but Cinebench which claims it can run on ARM for example doesn’t actually take full use of the M1 chip when you look at how much power is consumed compared to say an AMD chip. So the benchmarks are biased.

Sure people here like to quote geekbench, but that’s because it actually correlates to all of the real world testing done more so than other benchmarks that are available.

Edit: it seems OP’s original claims on Stockfish were misleading and not accurate. M1 does very well. http://talkchess.com/forum3/viewtopic.php?f=2&t=75911&start=50#p881883
 
Last edited:

BigMcGuire

Cancelled
Jan 10, 2012
9,832
14,032
These tools require installing with admin privileges. Anyway, there's no App Store close to where I live, and I have no doubt about the performance the M1 should yield on the tools I use.

As for the OP's claims according to which the M1 is more than ten times slower than the competition, they appear to contradict results reported here: http://talkchess.com/forum3/viewtopic.php?f=2&t=75911&start=50

I tested
Code:
stockfish bench 64 1 20
on my iMac with a core i5 7600k. I get 1560 kN/s, which is in line with results reported above.
The M1 yields 3024 kN/s with the command I used.

The scores of 230000 kN/s that the OP mentions must correspond to 256-core compute nodes or something.
Well there you go, very nice. Thread resolved. lol and the OP got the attention they wanted. Thanks for the info on benching :).
 

AppleB

macrumors 65816
Oct 18, 2011
1,156
1,380
The logical thing to do is get the Mac, test it with your needed apps, and then determine if the Mac will meet your needs. Using the Benchmark doesn't answer any real world usage questions.

More truth.

I bought my M1 just expecting Logic Pro to work well because it’s a Apple program. To my surprise almost everything worked and worked well.
It was the quickest transition I ever made from one platform (Intel) to Apple Silicon. It was truly plug and play. I spent more time waiting for a Amazon delivery of a second display.

Pro Tools isn’t even optimized for Apple Silicon but it runs like it is.
 

jeanlain

macrumors 68020
Mar 14, 2009
2,462
956
Which is exactly the point. M1 performs excellently in practically all synthetic and real world tests. It performs especially well in workloads that deal with traversing complex data structures and simulations (just like chess). So if someone comes along and claims that M1 is 10 times slower than Intel CPUs on a particular workload, I will immediately dismiss them because either their test is flawed or it’s x86 software that hits worst-case corner cases of Rosetta2.
In fact, Stockfish appears to perform rather well on the M1, if these results are to be believed (two posters got consistent results). The M1 is 64% faster (single core) than an iMac Pro's xeon, and almost twice faster than my i5 7600k, while it's "just" 65% faster on Geekbench (I get a single-core score of 1041).
The OP must have seen this page, which reports Stockfish scores by CPUs consuming >10 times the power of the M1.

EDIT: my stockfish score of 1560 kN/s seems too low compared to the M1. I may be using a different version from the one tested there. It appears that stockfish and Geekbench give very similar relative scores.

TL,DR: move along. Nothing to see here.
 
Last edited:

BigMcGuire

Cancelled
Jan 10, 2012
9,832
14,032
When I was heavily into chess (Crafty, etc) back on Windows - UCI applications were abundant. I've tried looking for some on Mac and hit dead end after dead end after dead end. I compiled Stockfish for the M1 and really don't want to pay $70+ for Deep Hiarcs.
 

jeanlain

macrumors 68020
Mar 14, 2009
2,462
956
When I was heavily into chess (Crafty, etc) back on Windows - UCI applications were abundant. I've tried looking for some on Mac and hit dead end after dead end after dead end. I compiled Stockfish for the M1 and really don't want to pay $70+ for Deep Hiarcs.
I installed stockfish on my iMac with homebrew. Could you let me know the score you get on the M1 with the command I posted above?
 
  • Like
Reactions: BigMcGuire

BigMcGuire

Cancelled
Jan 10, 2012
9,832
14,032
Looks like the Stockfish app for Mac has an Arm64 executable?

1619297959674.png


I renamed the top 3 just to test it and yep, it's running the stockfish-arm64.
 

BigMcGuire

Cancelled
Jan 10, 2012
9,832
14,032
I installed stockfish on my iMac with homebrew. Could you let me know the score you get on the M1 with the command I posted above?
I'm a Windows programmer by trade - can you tell me how you ran stockfish bench 64 1 20 ?

I type that into terminal and I get zsh: command not found: stockfish. Sorry for being a noob.

I'll try installing it via brew.

That did it, now I can run it in terminal. Hah!
 

snorkelman

Cancelled
Oct 25, 2010
666
155
I don't expect the discussion to be at the level of Phoronix but not even the basic questions were asked like is it native app, installed under Homebrew, using standardized configuration template, how to improve results, etc. before quickly downplaying it since results are unfavorable. That's why medication lists not only positive attributes but also side effects so people can decide based on pros vs cons.

I'm not 'downplaying it because it's results are unfavorable', but because for me and the tasks I put my mac to, it's results are *irrelevant*

I've already bought the machine and satisfied myself that the performance gains over my old machine that the published Geekbench results were suggesting, have indeed translated into gains in real world performance *for the things I do*

As a result of that the M1 could turn in the same performance as a 1990 Harris 286 in this other benchmark, for all it would matter to my workflow or cause me any concern.

Add to that a look at posting history before i first replied suggested the motivation for the OP starting the thread wasn't necessarily due to any genuine concern / wish to understand these results, or even to highlight them to others..
 
  • Like
Reactions: leons
Status
Not open for further replies.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.