What makes ARM superior?

[leman]

Superior or...inferior?

I think this is in most of our heads as a thought. A fear...hidden or not...
Many people in the computer business, still wonder how this 'thing' with arm, could ever be better in computing power, than intel.
Maybe it is called as an 'apple silicon', and not doubt apple will make the best it can do,
but still, it is an arm cpu, no matter what the optimizations would be.

Many friends of mine, advice me to go for an intel mbp as far as I still can,
because arm mbp, at least at the first generations/years,
will be a big downgrade in computing power.

These friends of your would benefit from looking at available facts rather than their stereotypes

We dint know how well Apple ARM Macs will perform. What do we know is that Apples iPhone chip delivers almost the same peak performance as Intel’s newest Tiger Lake core, while consuming 4 times less power. Whatever conclusions you draw from this are yours to make.

 

[Pistol Peto]

Apple has never been concerned with ultimate performance. Their goal has been to offer
a balance between performance and form factor/noise (i.e., performance/watt, since it's TDP that determines the needed size and fan noise).

I would go a little further. Apple's goal is to maintain their margins and increase sales. A great way to do this is own the chip design. The iPhone is faster than any phone. Not only do they make more margin because they own the hardware design/software stack, this control over the chip design and their excellent engineering allows them to bring new features (secure Enclave, machine language, processing for specific tasks that is unrivaled, LIDAR, photo algorithms, etc.) that keep their margins high and increase sales.

I expect the same for the Mac. Excellent engineering will bring increased speed. I expect Mac to have the fastest processors on the planet for desktops. They'll leverage this to bring features that intel based computers will have a hard time competing with because of this!

 

[Joelist]

Superior or...inferior?

I think this is in most of our heads as a thought. A fear...hidden or not...
Many people in the computer business, still wonder how this 'thing' with arm, could ever be better in computing power, than intel.
Maybe it is called as an 'apple silicon', and not doubt apple will make the best it can do,
but still, it is an arm cpu, no matter what the optimizations would be.

Many friends of mine, advice me to go for an intel mbp as far as I still can,
because arm mbp, at least at the first generations/years,
will be a big downgrade in computing power.

First thing is to understand what Apple Silicon is; and it is NOT a standard ARM. In fact the only ARM thing about it is the use of the instruction set. Everything else especially the microarchitecture (which is what REALLY determines performance) is a 100% Apple design and does not resemble Cortex at all. If anything, it resembles the Intel Core 2 microarchitecture more than anything else - and seeing as Apple aggressively poached design talent from Intel (including Johnny Srouji from Intel Israel) this is not shocking. Apple pretty obviously went after the design braintrust that came up with Core 2.

 

[ChrisA]

I now a little bit about processor technology and have read up on x86 and ARM architecture but I still do not really understand what makes ARM so superior to Intel or x86 technology in general that has people believing the the new ARM Macs will be much better and faster than Intel based macs.

If you have a problem that be be decomposed to many parallel tasks then you can take advantage of many ARM cores. If the problem is not decomposable then you will be better off using an Intel core.

But with Macs, most users don't need fast cores. That Youtube video will take 10 minutes to watch no matter how fast the CPU is. A very few Mac users do things like transcode media files. Apple is building for "most users" who don't do that.

What I really wonder is the future of the Mac Pro. Can they replace a 12-core Xeon with perhaps 36 ARM cores? Can FCPX use 36 cores? Maybe they drop the Mac Pro and FCPX?

 

[leman]

If you have a problem that be be decomposed to many parallel tasks then you can take advantage of many ARM cores. If the problem is not decomposable then you will be better off using an Intel core.

Doesn't Apply to Apple ARM cores. They are as fast as Intel's in single threaded performance. With the potential of being substantially faster (we just don't know, since all we saw from Apple was limited to 5 watts per core).

 

[vigilant]

First thing is to understand what Apple Silicon is; and it is NOT a standard ARM. In fact the only ARM thing about it is the use of the instruction set. Everything else especially the microarchitecture (which is what REALLY determines performance) is a 100% Apple design and does not resemble Cortex at all. If anything, it resembles the Intel Core 2 microarchitecture more than anything else - and seeing as Apple aggressively poached design talent from Intel (including Johnny Srouji from Intel Israel) this is not shocking. Apple pretty obviously went after the design braintrust that came up with Core 2.

Thats interesting. Core 2 was beautiful, do you have a source on this?

I’m not saying you are wrong. You may have information that we can’t see, but if there is something you can share that’d be fantastic.

 

[StellarVixen]

Single core performance still leaves a lot to be desired. But we will see.

 

[vigilant]

Superior or...inferior?

I think this is in most of our heads as a thought. A fear...hidden or not...
Many people in the computer business, still wonder how this 'thing' with arm, could ever be better in computing power, than intel.
Maybe it is called as an 'apple silicon', and not doubt apple will make the best it can do,
but still, it is an arm cpu, no matter what the optimizations would be.

Many friends of mine, advice me to go for an intel mbp as far as I still can,
because arm mbp, at least at the first generations/years,
will be a big downgrade in computing power.

I question this logical whole heartedly.

I work in the industry and don’t have direct first hand knowledge. What I do have is many people saying that Apple isn’t the only one going all in on ARM.

Obviously Nvidia bought ARM, and that can only mean more chips coming from Nvidia directly. But there is definitely reason to believe that other “non-Apple” players are slowly prepping to go all in on ARM as well.

Stick with Intel as long as possible, that depends on the workload. If you have a workload that uses the imaginary “hyper threading” sure you may get some sort of performance boost occasionally from something being multithreaded. But that imaginary extra thread isn’t consistent. In fact, I’ve seen things that show that under ideal conditions you can expect anywhere from a 20% boost to a 30% boost. Across 6 chips that isn’t that impressive, and at least to me implies that conditions need to be very favorable to count on it.

If your on a Mac, and you are writing instruction set specific code, that was a mistake. If you wrote towards the Accelerate framework, I believe theres been 3 different revisions that Apples made to their processors on AS that take that into account. I’d assume that they are probably covering MMX, AVX, and possibly AVX-512. For AVX-512, I think thats probably really niche considering the trade offs that Intel put into place.

Any good developer should be following best practices. We are a few years down the road of Apple stating what those best practices are.

Developers, maybe unfairly, live or die on their ability to listen to best practices.

I know I’m going to hear from someone about how they need to run Windows XP because software that manages a 20 year old video surveillance won’t rune will try to burn me down. It’s happened. My answer is “buy stuff from a company that has a development roadmap.”

I think in the first year, for the general public things are going to run fine. Best I can tell, Apple is creating dynamically recompiled binaries based off of new code in X64 when it’s referenced for the first time.

The people that are going to hurt the most are people, who bought software from sloppy developers, that have ignored years of Apple saying what should be done in the form of best practices. Apple really isn’t subtle when it comes to direction.

Intels x86/x64 is woefully filled with compromises. They’ve done an excellent job of sweeping under the rug the legacy crap they are forced to support, and between that and their inability to even pronounce 7nm should tell you where the problems are.

 

[leman]

Single core performance still leaves a lot to be desired. But we will see.

Single core performance essentially matches Intel’s at 1/4 or lower power consumption. If Apple can deliver a comparable level of performance at 5 watts where Intel needs 20 watts in their most efficient 10nm designs, what kind of single core performance can we expect once Apple is not limited by a phone form factor?

 

[Joelist]

The references are out there but take a little hunting. Here is one:

Apple's A7 Cyclone CPU detailed: A desktop class chip that has more in common with Haswell than Krait - ExtremeTech

Some six months after Apple shocked the world with its 64-bit A7 SoC, which appeared in the iPhone 5S and then the iPad Air, we finally have some hard details on the Cyclone CPU's architecture. It seems almost every tech writer was wrong about the A7: The CPU is not just a gradual evolution of...

www.extremetech.com

And another:

Apple's Cyclone Microarchitecture Detailed

www.anandtech.com

These are speaking of A7 (Cyclone). Apple WAS using Cortex designs until the A6 when they debuted their own design (Swift). This was also when their performance and PPW skyrocketed. And what happened then? Big wide cores with out of order execution, super accurate branch prediction and a host of other elements right out of Core 2. And Johnny Srouji (who heads the Apple Silicon unit at Apple) was one of the developers of Core 2 when at Intel.

 

[theorist9]

Doesn't Apply to Apple ARM cores. They are as fast as Intel's in single threaded performance....

Single core performance essentially matches Intel’s at 1/4 or lower power consumption.

That may be true, and we're of course all hoping that the AS chips in future Macs significantly outperform their Intel counterparts.

But I will continue to beat the drum that we should be framing it something like: "Preliminary benchmarks suggest Apple's current single core performance matches Intel's at 1/4 or lower power consumption", rather than speaking as if we know this is the case, since (a) these comparisons rely on synthetic benchmarks, rather than the real-world apps we actually use, and (b) they are comparing benchmark performance by AS on iOS with benchmark performance by Intel on MacOS. Thus they introduce two confounding variables relative to the comparison we actually want to make.

I.e., what we have is:

iOS version of benchmark on AS running iOS vs. MacOS version of benchmark on Intel running MacOS

When what we want is:

AS-optimized MacOS app on AS running MacOS vs. Intel-optimized MacOS app on Intel running MacOS (for a wide range of apps, from a wide range of independent sources).

 

[aleni]

For consumers:

Much higher performance difference on year to year basis without depending on intel to do the job. More optimization across software and hardware, much less power consumption performance/watt, and you can expect separate modules to increase the performance of the mac pro.

for apple: more control of their ecosystem, more profit for the company.

 

[leman]

But I will continue to beat the drum that we should be framing it something like: "Preliminary benchmarks suggest Apple's current single core performance matches Intel's at 1/4 or lower power consumption", rather than speaking as if we know this is the case, since (a) these comparisons rely on synthetic benchmarks, rather than the real-world apps we actually use, and (b) they are comparing benchmark performance by AS on iOS with benchmark performance by Intel on MacOS. Thus they introduce two confounding variables relative to the comparison we actually want to make.

What I wrote is a reflection of Anandtech articles who rely on SPEC CPU2006 benchmark suite. SPEC2006 are not synthetic benchmarks, they are drawn from real-world applications and they measure how fast can the CPU subsystem deal with some common work-intensive tasks. The benchmarks are written in a way that avoids I/O or system calls, to take peripheral hardware and OS out of the equation as much as it is possible. They also avoid use of hardware-specific accelerators to make sure that the comparisons are "fair". SPEC benchmarks are an accepted and trusted industry-standard tool for comparing performance of CPUs — they are used to quantify the performance of professional workstations and servers. Actually, the biggest source of uncertainty for comparing results of these benchmarks is the compiler — if you use different compilers on different platforms, they might optimize the code differently, thus not comparing what they should be comparing. To control for this, Anandtech used clang compiler of matching versions all all the platforms.

What you say makes a lot of sense. The end user should always look at the performance of the specific software they are interested in, for any target platform. But doing performance comparisons using real-world apps is extremely complicated. You never know what exactly are you comparing. Not to mention that real-world apps do more than just processing — they do complex I/O, wait for user input and other very complicated things. In your scenario, you would be testing an amalgam of implementation characteristics + system performance. This does not necessarily give you an insight into the capabilities of the hardware itself. Carefully designed benchmarks like SPEC do. This is why I think, that for the purpose of looking at the CPU architecture, SPEC is the way to go. They won't give us the best idea about the final system performance, but they pain a very clear picture about the CPU, the cache and the memory.

Looking at these threads, I am seeing a lot of comments like "Apple Silicon Macs will be faster because they will contain custom accelerators". I do not agree with this sentiment. Custom accelerators are the icing on the cake. They will definitely enable new applications, and they will definitely be a big part in the value proposition. But Apple Silicon Macs won't be fast just because of them. They will be fast because Apple currently has one of the best-performing CPU microarchitectures on the planet. We know that these CPUs are very fast on themselves. The only unknown is how they will behave when the ultra-mobile restrictions of iPhones and iPads are lifted.

 

[theorist9]

What I wrote is a reflection of Anandtech articles who rely on SPEC CPU2006 benchmark suite. SPEC2006 are not synthetic benchmarks, they are drawn from real-world applications and they measure how fast can the CPU subsystem deal with some common work-intensive tasks. The benchmarks are written in a way that avoids I/O or system calls, to take peripheral hardware and OS out of the equation as much as it is possible. They also avoid use of hardware-specific accelerators to make sure that the comparisons are "fair". SPEC benchmarks are an accepted and trusted industry-standard tool for comparing performance of CPUs — they are used to quantify the performance of professional workstations and servers. Actually, the biggest source of uncertainty for comparing results of these benchmarks is the compiler — if you use different compilers on different platforms, they might optimize the code differently, thus not comparing what they should be comparing. To control for this, Anandtech used clang compiler of matching versions all all the platforms.

What you say makes a lot of sense. The end user should always look at the performance of the specific software they are interested in, for any target platform. But doing performance comparisons using real-world apps is extremely complicated. You never know what exactly are you comparing. Not to mention that real-world apps do more than just processing — they do complex I/O, wait for user input and other very complicated things. In your scenario, you would be testing an amalgam of implementation characteristics + system performance. This does not necessarily give you an insight into the capabilities of the hardware itself. Carefully designed benchmarks like SPEC do. This is why I think, that for the purpose of looking at the CPU architecture, SPEC is the way to go. They won't give us the best idea about the final system performance, but they pain a very clear picture about the CPU, the cache and the memory.

Looking at these threads, I am seeing a lot of comments like "Apple Silicon Macs will be faster because they will contain custom accelerators". I do not agree with this sentiment. Custom accelerators are the icing on the cake. They will definitely enable new applications, and they will definitely be a big part in the value proposition. But Apple Silicon Macs won't be fast just because of them. They will be fast because Apple currently has one of the best-performing CPU microarchitectures on the planet. We know that these CPUs are very fast on themselves. The only unknown is how they will behave when the ultra-mobile restrictions of iPhones and iPads are lifted.

Thanks for the correction that SPEC 2006 is not synthetic, but instead combines multiple real-world applications. But still, it is a benchmark, since it is SPEC that selects those applications. And it is now 14 years old. It may or may not correlate well with CPU performance on modern CPUs and applications.

CERN, for instance, found that SPEC06 showed good correlation to its actual workloads in 2009, but that was no longer the case in 2019. Further, SPEC CPU 2017 also didn't correlate well with its current workloads. [https://indico.cern.ch/event/773049...931/3200035/20191104-CHEP2019-BMK-AV-v005.pdf]

Hence it is more accurate to restrict our statements to what has actually been found — that single-core performance of current AS on a 2006 CPU benchmark run in iOS is comparable to what's obtained with that benchmark for Intel on MacOS, at 1/4 or lower power consumption.

Again, I'm not saying your version of the statement isn't true. I'm saying we won't have enough information to make such a statement, in the completely assured and unqualified way you did, until we have AS running MacOS released into the wild (and a wide range of apps are updated and optimized to run on AS). Until then, some degree of uncertainty needs to be recognized.

And how was Anandtech able to control for the difference between iOS and MacOS?

 

[Woochoo]

People that don't want it will make it virtually impossible to compare x86 and ARM, there will always be an excuse for why it isn't a fair comparision (be it the OS, the software/app optimization, the binary translation, etc).

In the end what really matters is if the ARM Macs are going to allow users to make their stuff faster than they could before with Intel, for less energy and heat. If that happens which I'm sure of, it will be a big win already. We just need to wait a month.

 

[adib]

Until there is an ARM device running macOS any comparisons are going to be flawed.

Technically, there is. Performance is somewhere between 13" and 16" MBP.

Look at the bottom set of numbers – "Developer Transition Kit - A12Z (Native)".

Source.

 

[adib]

It's not as widespread as x86-based designs in terms of traditional computing. Apple will be among the first to offer an ARM-based notebook and desktop solution. There are some ARM machines out there but most are power-hungry servers.

Technically "among the first" ARM desktop was Acorn Archimedes, circa 1987.

The more modern ones would be Microsoft Surface RT, circa 2012.

 

[poorcody]

People that don't want it will make it virtually impossible to compare x86 and ARM, there will always be an excuse for why it isn't a fair comparision (be it the OS, the software/app optimization, the binary translation, etc).

One interesting thing to develop from this transition will be the first time one can make a close comparison of x86 and ARM (or at least Apple Silicon), since Big Sur will run "the same" OS and Apps on the latest processors from both camps.

 

[leman]

Thanks for the correction that SPEC 2006 is not synthetic, but instead combines multiple real-world applications. But still, it is a benchmark, since it is SPEC that selects those applications. And it is now 14 years old. It may or may not correlate well with CPU performance on modern CPUs and applications.

It is definitely true that SPEC doesn't represent every use case. It is also trie that it's aged.. but it's a god approximation. Looking at it's benchmark suite, it covers cases such as symbolic data transformation, graph manipulation, random number generation, pointer chasing etc. — all of these are still very much relevant today. I would argue that it is still provides reasonable estimates of what CPU can do — especially when you look at the individual benchmark scores (as it gives you idea how the CPU behaves with different types of workloads).

CERN, for instance, found that SPEC06 showed good correlation to its actual workloads in 2009, but that was no longer the case in 2019. Further, SPEC CPU 2017 also didn't correlate well with its current workloads. [https://indico.cern.ch/event/773049...931/3200035/20191104-CHEP2019-BMK-AV-v005.pdf]

CERN is speaking here about HEP workloads. SPEC is definitely not HEP, it only focuses on the CPU subsystem.

Hence it is more accurate to restrict our statements to what has actually been found — that single-core performance of current AS on a 2006 CPU benchmark run in iOS is comparable to what's obtained with that benchmark for Intel on MacOS, at 1/4 or lower power consumption.

Again, I'm not saying your version of the statement isn't true. I'm saying we won't have enough information to make such a statement, in the completely assured and unqualified way you did, until we have AS running MacOS released into the wild (and a wide range of apps are updated and optimized to run on AS). Until then, some degree of uncertainty needs to be recognized.

I don't think we disagree My statement tend to sound stronger than my believes are — call it occupational disease if you wish. All I am saying is that independent evidence — from many different sources (SPEC, Geekbench, third-party tests and blogs) — seem to point a similar system. Bu of course all this has to be taken with a grain of salt until we have the chance to study the real deal. And of course, there are still a lot of open questions. What will the final peak performance be? What core config would Apple ship? etc.

By the way, I read an interesting piece somewhere (can't find it now unfortunately) arguing that Apple will have smaller batteries in the upcoming iPhones (iPad Air is already marginally smaller), so that the mobile A14 is optimized for power instead for performance. It was theorized that the Mac version of A14 should be significantly faster.

And how was Anandtech able to control for the difference between iOS and MacOS?

There is not much difference to be honest... the main one is that iOS does not have a backing store for virtual memory (swap). But system software wise, iOS and macOS are much closer than say, macOS and Windows or Linux, since iOS and macOS run on the same kernel. It's all OS X anyway. Also, iOS and macOS use the same compiler, so one can pretty much eliminate any noise from that part.

Regardless, OS should play a little role, since these benchmarks do not rely on system-provided services (no I/O, no threading etc.). The only system-dependent bit is memory allocators, but I would assume they deal with it independently.

 

[Joelist]

If I had to blind guess what the Mac family will look like it probably will have the same microarchitecture as A14 but the Power/Performance Core counts will be higher, more cache memory and probably some new SOC blocks we have not yet seen. For exampole, A14 has 2 high performance and 4 efficiency cores. I think our Mac version will sport 6 high performance and 6 efficiency cores.

 

[Stene]

I now a little bit about processor technology and have read up on x86 and ARM architecture but I still do not really understand what makes ARM so superior to Intel or x86 technology in general that has people believing the the new ARM Macs will be much better and faster than Intel based macs.

I understand that the advantages of ARM are power efficiency and the ability to have many more cores but isn’t Intel still better in raw power in multi threaded operations?
Will ARM at first be a replacement for intels mobile processors which are arguably already worse in many ways than an A12Z or A13 or will they also be able to create a processor than can beat i9 and even Xeon processors?
Can we really expect a „night and day“ difference?


By the way - just yesterday, it was announced that the fastest supercomputer of the world is now ARM based. it uses ARM processors made by Fujitsu:https://www.arm.com/company/news/2020/06/powering-the-fastest-supercomputer
Fits perfectly to Apples announcement.

Apple is licensing and using the ARM instruction set and uses that to create a custom-made system on a chip with unique characteristics - highly optimised GPU in combination with a large number of specialised sub-processors that speeds up demanding processing, sub-processors that are highly integrated with OS software used by close to all applications. Examples of these types of specialised sub-processors are video compression, encryption, mathematical processing, matrix operations etc. This is why you cannot really compare the ASi system on a chip with a standard processor from Intel or AMD.

 

[leman]

Apple is licensing and using the ARM instruction set and uses that to create a custom-made system on a chip with unique characteristics - highly optimised GPU in combination with a large number of specialised sub-processors that speeds up demanding processing, sub-processors that are highly integrated with OS software used by close to all applications. Examples of these types of specialised sub-processors are video compression, encryption, mathematical processing, matrix operations etc. This is why you cannot really compare the ASi system on a chip with a standard processor from Intel or AMD.

I don’t think you are giving enough justice to Intel and AMD here. Intel’s CPUs also contain a fast GPU, a built-in AI accelerator, wide vector unit, matrix operations, encryption, video encoder, I/O controllers, integrated WiFI controller etc... Apple SoCs might contain more specialized processors and their ML accelerators are much faster, but fundamentally, the principal differences between these systems is minor.

And all these custom systems have little to do with CPU itself. A system might have best ML acceleration in the world, but it’s utility is going to be limited if it struggles with basic tasks. Apple CPUs are custom-designed, sophisticated devices that offer very high performance at very low power draw. This is their key advantage to an average user (be it a home user or a professional).

 

[cool11]

Technically "among the first" ARM desktop was Acorn Archimedes, circa 1987.

The more modern ones would be Microsoft Surface RT, circa 2012.

I do remember this since 80's. Indeed.

 

[theorist9]

By the way, I read an interesting piece somewhere (can't find it now unfortunately) arguing that Apple will have smaller batteries in the upcoming iPhones (iPad Air is already marginally smaller), so that the mobile A14 is optimized for power instead for performance.

From what I've read, most iPhone users are happy with the speed, but would like more battery life; so it makes sense they would do this.

It was theorized that the Mac version of A14 should be significantly faster.

I would agree that whatever they put into the Macs has to be faster -- even with the same chip, the thermals are better. But I think AS's scaling behavior at higher power levels is still an open question (at least for us, the public, who don't have access to Apple's internal testing)—namely how the performance/power ratio changes, and how high it can be scaled before that ratio gets too low.

There is not much difference to be honest... the main one is that iOS does not have a backing store for virtual memory (swap). But system software wise, iOS and macOS are much closer than say, macOS and Windows or Linux, since iOS and macOS run on the same kernel. It's all OS X anyway. Also, iOS and macOS use the same compiler, so one can pretty much eliminate any noise from that part.

Regardless, OS should play a little role, since these benchmarks do not rely on system-provided services (no I/O, no threading etc.). The only system-dependent bit is memory allocators, but I would assume they deal with it independently.

What you've written should apply to all cross-platform comparisons, regardless of benchmark. I.e., you're not only arguing that going from iOS to MacOS wouldn't matter for SPEC, you're arguing it wouldn't matter for other benchmarks as well, like Geekbench. But, if so, why did Linus Torvalds, in 2019, specifically caution against trying to extrapolate from iOS GB performance to desktop GB performance? [See: https://www.realworldtech.com/forum/?threadid=185109&curpostid=185132 ] Conversely, if Torvalds is right, and the OS does matter for GB, why wouldn't it matter for other benchmarks as well, like SPEC?

 

[leman]

I would agree that whatever they put into the Macs has to be faster -- even with the same chip, the thermals are better. But I think AS's scaling behavior at higher power levels is still an open question (at least for us, the public, who don't have access to Apple's internal testing)—namely how the performance/power ratio changes, and how high it can be scaled before that ratio gets too low.

Yes, definitely, that is the most important topic on which we have no data. Anandtech tests of A12 show that power consumption raises rapidly towards the end of the frequency range, extrapolating the curve further suggests that the chip doesn’t have much space to grow. Of course, a lot could have changed in two generations and one node shrink. Let’s see what A14 can do.

What you've written should apply to all cross-platform comparisons, regardless of benchmark. I.e., you're not only arguing that going from iOS to MacOS wouldn't matter for SPEC, you're arguing it wouldn't matter for other benchmarks as well, like Geekbench. But, if so, why did Linus Torvalds, in 2019, specifically caution against trying to extrapolate from iOS GB performance to desktop GB performance? [See: https://www.realworldtech.com/forum/?threadid=185109&curpostid=185132 ] Conversely, if Torvalds is right, and the OS does matter for GB, why wouldn't it matter for other benchmarks as well, like SPEC?

First of all, let me be clear that Linus is definitely much more experienced than I ever hope to be, so you should definitely listen to what he has to say. Here is my take on these things:

In order to make sure that the benchmark is fair, we need to establish a definition of fairness. For this discussion (CPU performance), I propose something along the lines of running the same workload on different systems by utilizing the general-purpose computational hardware. This presupposes a number of things. First, one needs to ensure that the workload is the same. For this, it is sufficient to use the same code, translated into well-optimized idiomatic target ISA using the same set of optimizations. This means using the same compiler with the same settings. At the same time, you want to keep the OS and other devices out of the way. This means no system calls, no I/O. You will need memory management - so you grab large blocks from the OS and use a custom allocator. Finally, no custom acceleration hardware. If you are using the fancy video encoder to test the encode performance, you are not testing the CPU. That would be a different benchmark.

As far as my limited understanding goes, SPEC takes care if these things. Geekbench - not nessesarily. For example, they use SQLite3 for some tests, a highly complex software that will have some platform-specific code in it etc. I am not up to date on how Geekbench treats these matters these days though.

Finally, there will always be some difference that is difficult or impossible to factor out. For example, macOS will likely have something else running in the bs kgriund, be it indexing or backing up or something else - it can’t dedicate as many resources to the active app as iOS can. Regarding what Linus said in the post you quoted specifically, I’m not sure I agree though. Yes, iOS has a 16kb page, which is long overdue in desktop (luckily Apple brings it to Mac now), and it will cause differences in memory access performance - but I would rank it under the CPU subsystem differences, so I think it’s fair to compare across. As to phones tend to have simpler libraries... I don’t think it’s the case for iOS. It’s full-featured OS X. The code is open-source anyway...