Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Silicon in Sciences
- Thread starter HiddenPaul
- Start date
- Sort by reaction score
OK, thanks. When I have a chance I'll rerun the benchmarks on my iMac (needed to create a new baseline), and then PM you to arrange to send the files to you.
Last edited:
I finally had a chance to rerun my Mathematica (MMA) benchmarks on v. 13.2.
To start, I did a comparison of MMA 13.0.1 (with Mac OS 12.4) to MMA 13.2 (with MacOS 12.6.2) on my 2019 i9 iMac (i9-9900; more details in signature line). The only hardware difference is that I had 32 GB RAM installed in the former case and 128 GB in the latter, but since this benchmark uses only a few GB RAM that should have no effect. The results show that MMA did indeed improve in performance somewhat, but the changes were very much task-dependent. E.g., on one integration there was an 87% improvement, while on others there was no improvement at all:
But the question to which you wanted an answer is whether the performance gap between a 9th-generation Intel i9 and an M1-generation Apple Silicon remains disappointingly small. [Based on their relative GB 5 SC scores, we'd expect to see an average ~35% improvement if Mathematica were as well-optimized for AS as it is for Intel. Instead, the improvement averaged about half that on my symbolic and graphing tasks, and the M1 was 50% slower on my image processing task.]
Here are two results on MMA 13.2, one from @casperes1996 who kindly agreed to run this a second time, and from @leman, who kindly agreed to run one on his machine for comparison
With MMA 13.2:
casperes1996's results
Machine Details 16" M1 Max MacBook Pro, 24-core GPU, 32 GB RAM, running MacOS 13.2, on battery, undisturbed
leman's results
Machine Details: 16" M1 Max MacBook Pro, 32 GB RAM, running MacOS 13.2, plugged in on high-power mode.
For comparison, here are the results with MMA 13.0.1:
From the above, you can see that, while MMA 13.2 performs better than MMA 13.0.1 on some tasks, the performance gap between a 9th-gen Intel Core i9 and an Apple M1 Max remains essentially unchanged.
Caspares1996's battery results are essentially the same as Leman's plugged-in results on high power mode, showing those don't have an effect on performance.
Last edited:
Hello theorist9,
Thank you (and casperes1996 and leman) very much for running your benchmarks with Mathematica 13.2 (Intel Core i9 vs. Apple M1 Max)! This data will be very helpful as I contemplate my next computer upgrade.
Currently, my main workhorse is a 2013 Mac Pro (see my signature below). However, I will soon need to get it replaced for my computational needs. Like you, I use the Mac Pro for scientific endeavors....in particular, mathematics research (Ramsey theory), and not for video/audio work. So my main concerns are CPU, multiple cores, RAM and SSD storage.
Some quick general questions....Are you happy with your current Mac Pro setup for your computational needs? Is there anything that you would do different? Any general recommendations/advice?
All the best,
richmlow
Hi richmlow. I don't know the extent to which these general comments will be applicable to your needs but, FWIW:
I don't use a Mac Pro, I use a 2019 i9 iMac. It works reasonably well for my needs. My only complaints are:
(1) I do experience delays and spinning beachballs, so I'd like higher SC speeds. Perhaps I'll consider upgrading when the M3 Studio comes out. It's a question of how much I'd have to pay relative to the performance improvement I'd get.
(2) While I like the quality of the iMac's 27" Retina monitor, I often create spreadsheets with my data that are too large to view comfortably on a 27" screen, so I'd love to have larger Retina-class display, say 32"/6k, 36"/7k, or 43"/8k. Ideally, I'd like to have three of those! But those will all have to wait until I upgrade to a Studio, since my iMac is limited to 2 x 4k externals.
If you do get an Apple external display, I'll caution you that the nano-texture version's anti-reflective properties are very strong (great if you've got bad reflections), but at the cost of a noticeable (for me) loss in sharpness compared to the glossy, as well as a distractingly sparkly white background. Thus make sure you compare the two at an Apple retailer before making a purchase. I can control the reflections in my office, so I would opt for the glossy myself. The upcoming 32" Dell 6k is matte only, but I suspect its matte coating is less "strong" than Apple's nano-texture finish, providing both less of the benenfit and less of the weaknesses. Also, if you do go for a three-display setup, you can sacrifice some pixel density on the side displays; i.e., the main thing is that your central monitor is "Retina".
Finally, starting 6 months ago I stopped using spinning disks for backups. I got a 4 TB Kingston XS2000 SSD for Time Machine, and a 2 TB Kingston XS2000 for Carbon Copy Cloner, and I'm very happy with them. [You just need to upgrade their cables; the ones included are cheap and cause disconnections. I recommend the Cable Matters certified cables.]
Last edited:
Hi theorist9,
Thanks for the additional comments and pointers!
For now, I'll wait for Apple's supposed announcement of the Apple Silicon Mac Pro and then take it from there.
When I get my upgrade, I'll update this thread/post.
All the best,
richmlow
Hi theorist9,
Thank you for updating your benchmark runs with @leman's results.
All the best,
richmlow
Hi all,
Now that Apple has announced updated ASi Mac minis and Powerbooks, I still eagerly await Apple's announcement of ASi Mac Pro.
While the recent M2 Pro/Max updated machines are nice, the RAM restriction is my concern.
My next Mac purchase will be mainly used for mathematics research. So, I need a reasonable number of cores and considerable RAM.
What does everybody else think about Apple's recent upgrades?
All the best,
richmlow
Now that Apple has announced updated ASi Mac minis and Powerbooks, I still eagerly await Apple's announcement of ASi Mac Pro.
While the recent M2 Pro/Max updated machines are nice, the RAM restriction is my concern.
My next Mac purchase will be mainly used for mathematics research. So, I need a reasonable number of cores and considerable RAM.
What does everybody else think about Apple's recent upgrades?
All the best,
richmlow
Last edited:
Thanks for this.
If you're looking for a followup project (hah!) something that remains very unclear is the performance of Apple Silicon for
- Virtual Machines
- "non-trivial" parallel code
For Virtual Machines the issues of interest are the same sorts of issues as on the x86 side
- how much slowdowns? under what conditions?
- how much do TLB operations cost?
- are there weirdnesses in IO performance?
- how good is the VM performance isolation (ie can one VM slow down another?)
For parallel code interesting issues include
- cost of locks (of various types, including remote atomics)
- cost of synchronization (eg if a thread in P-cluster 1 has to synchronize or engage in a cache-coherency operation with either P-cluster 2 or E-cluster)
Also whether these have been materially improved in M2. (My gut feeling is that these are the sorts of things that might have worked on M1, but were sub-optimal because they were not pushed on iPhone/iPad, and M2 was a chance, even though not much seems to have happened, for these sorts of technical mac-specific issues to have been improved).
Apple just doubled the number of AMX blocks on the M2 Pro/Max, which boosts the FP64 compute power from ~0.50 TFLOPS to ~1.10 TFLOPS. Definitely better for linear algebra.
Thanks for that!
I think it's clear that MMA still has a lot of optimization ahead of it on AS. On the other hand, pretty much everyone at Wolfram uses a MacBook as their preferred device, so I expect this will happen as fast as humanly possible!
Obvious issues I saw when I looked at this closely (which was some time ago!) were
- bignum performance was terrible. If Wolfram are using GMP (which the legalities say they are) and they are using the LATEST bignum this should improve. The most recent GNU bignum on ARM is not perfect (there is at least one unnecessary instruction in the core assembly) but it's a whole lot better than it used to be. BUT it makes no use of an Apple private instruction that provides for 53b->106b integer multiplication (similar to an AVX512 instruction) which could boost it if GNU were to write code using that.
Apple provides some bignum support via Accelerate vBigNum which may well use that instruction -- but the API may not fit well into the rest of MMA :-(
- stuff that would be routed to vector-parallel functions on Intel (think of eg Tan[ large FP array ]) are usually not routed to such functions on AS. Once again this is basically an issue of the appropriate functions being written between Apple, Wolfram and probably LLVM and GNU for various pieces.
- even then, at least at first, those functions will probably route to NEON. Better would be if they routed to AMX (AMX is growing to be essentially an AVX512 equivalent) but the AMX side is very much a work in progress for Apple...
- similarly things like BLAS/LAPACK (optimized up the wazoo in MKL) are a lot more generic on the ARM side, and Wolfram seems not to be using Apple's AMX-optimized versions. I expect that in time Apple and Wolfram (and a few obvious other clients like Matlab) will figure out a way to get access to AMX that works for all parties while honoring Apple's concerns (primarily that AMX changes a lot in each new CPU); but neither side probably has discussing this as a high priority until more basic issues are sorted out by each side.
- similarly I saw that (not sure why!) MMA was a lot more reluctant to use multiple cores for cases where you x86 would use such behind the scenes (eg for large matrix manipulation). Maybe this reflects crazy-optimized MKL LAPACK vs barely-optimized ARMv8 open source LAPACK?
- and of course there's all the more specialized stuff. Much of what MMA does could probably be routed to good effect to the GPU and even the NPU. (Including some non-obvious stuff, like requests to generate large arrays of random numbers). But that requires time...
So that's background. Most of it seems not relevant to your symbolic tests, though did some of them perhaps hit bignums in a serious way? What may be likely is that no-one at Wolfram has yet even tried serious AS optimization of the core symbolic engine. For example there are many ways to build a hash table, and Wolfram is likely using whatever parameters were optimized for Intel without even considering whether changing the hash parameters (eg to better match the cache sizes and page sizes of AS, or the fact that many more integer operations can be performed simultaneously, so more complex hashes can be considered). There may well be 15 or 20% improvement possible in things like that. Hell, the two specialized mechanisms added to the A15/M2 to speed up interpreters (most obviously JS, but also Objective-C dispatch) could be used by any interpreter, and my guess is that somewhere in the guts of MMA is an interpreter...
Generally I'd say that if Apple's first, lowest-end core can match Intel's highest end core, in spite of all the above, we're doing OK! Everything takes time (on Apple's HW side, on their SW side, on Wolfram's side, on the open source libraries side)!
So it seems that, to summarize:
- the CPU matches your requirements in some cases (in which case complaining about the GPU seems silly)
- did you try routing your code to AMX to get substantially *better* CPU performance?
- your complaints seem more about the state of GPU libraries. Which is, on the one hand, a legit complaint if you just want code to work. But is not a legit complaint as to technical direction, especially if the CPU is just as good.
You seem willing to jump through crazy hoops on the CUDA/SPIR-V side, but not to be willing to make any effort on the Metal or Apple AMX side.
Obviously it's your life and your time, but it seems strange to complain "the Apple code I put one hour of effort into doesn't work as well as the CUDA code I put two years of effort into! What gives???"
I'd say if you can wait do so.
My analysis (FWIW!) is that A14/M1 were designed as planned and on track.
A15 was designed as planned, as an energy optimized version of A14, giving iPhone what it needed more than performance. Perhaps M2 (and just M2) was planned as well.
Meanwhile the plan was for A16 and "desktop" M2 to be on N3, A16 in Sept 2022, M2 Pro and Max coming out now. These (A16', M2' designs) would make full use of N3's increased transistors to implement IPC improvements (and probably various ISA tweaks like the newest ARM instructions [except SVE, that's different discussion...])
BUT
covid comes along and throws the plans into disarray. Both Apple and TSMC have to run slower than planned (eg TSMC can't just fly people between Taiwan, the US, Japan, and Europe every week to solve problems! You do what you can with zoom, but it's slower).
So at some point Apple realizes the A16' and M2' designs (ie the N3 based designs) won't ship in time. Mad scramble! OK, we "recompile" the A15 to N4 and take what we can in terms of process improvements. And we accept that the M2 Pro/Max will be a disappointing A15 N5 design, not the hoped for kick-ass N3 designs. Oh well.
To add insult to injury, the M2 Pro/Max were probable targeted for about three months ago, but then we get unrest in China so we have to delay the products until that's sorted out!
I don't think this is special pleading. Look at the schedules: Compare M1 track vs M2 track.
Sept 2020 A14
Nov 2020 M1
Oct 2021 M1 Pro Max
March 2022 M1 Ultra
Sept 2021 A15
June 2022 M2
Jan 2023 M2 Pro Max
Sept 2022 A16 (on N4)
To me this looks like a schedule that's constantly trying to delay things to see which of two options (plan A or a fallback plan B) are feasible, not a long-term strategy playing out as designed.
Which is all a long way of saying that since A14/M1 Apple has basically been forced into two generations of what looks like stagnation (A15 designed for low energy, which is less immediately interesting on Mac, though ultimately valuable; A16 as a response to covid-delayed N3.) But once we get back on track with N3 and the CPU+SoC designed for it, I think we will all be pleasantly surprised...
But it remains a huge question when that will be -- how long can you wait?
One option is Apple announce "M3" (ie the N3 design) for the Mac Pro and related devices (Studio, iMac) at WWDC. This has the advantage that it looks good, and in a sense those are the devices that "want" the best silicon right now, not iPhone.
But iPhone is a nice place to test-drive silicon features that are only relevant to the very high end because if they are find to be problematic, you can set a chicken bit to not use them, and try again with the next design (M#) and hopefully get it right by Pro and Max. And iPhone is still king of the cash heap, so it gets max attention.
On the THIRD hand, the N3 delay has meant that Apple could start testing the A17 (the one that should have been the A16) almost a year earlier than usual (in tiny volumes on N3 risk production)... So this round of pretesting on iPhone chip before the M and M Pro/Max could already have happened...
I personally am waiting for WWDC before buying. Sure the M2 mini's look nice, but I suspect for what I care about (mostly MMA) the N3 designs will be substantially nicer – and it's at least plausible we will see them at WWDC.
Hello name99,
As I look over the specifications of the current Mac Minis and Mac Studios, I too am leaning towards waiting for Apple's announcement of the ASi Mac Pro. As tempting as the current computers are, the RAM limitation is really holding me back.
The main purpose of this next purchase will be for math research. I envision mainly using Mathematica, Python, Java and various SAT-solvers on it. For example, I was running a custom-designed Mathematica program/calculation on my 2013 Mac Pro (3.7GHz Quad-core Intel Xeon E5, 64GB RAM, macOS Mojave 10.14.6). It took 4 hours and consumed 64GB (towards the end of the run).
I would like my next desktop Mac system to have a minimum of 192GB RAM and 16 cores.
All the best,
richmlow
Hello all,
Slightly off-topic.....
I like this thread. It's nice to see that there are other workflows (e.g. scientific research) which exist and use Apple computers! Typically in these forums (and even in Apple marketing), everybody always seems to be blathering about video, as if there is nothing more to the matter!
It's unfortunate that Apple doesn't highlight the use of Apple hardware in the STEM disciplines. Years ago, that was not the case. At the time, Apple had xGrid, parallel computing, etc. They even had a sub-section "Apple and Science" on their website!
All the best,
richmlow
Slightly off-topic.....
I like this thread. It's nice to see that there are other workflows (e.g. scientific research) which exist and use Apple computers! Typically in these forums (and even in Apple marketing), everybody always seems to be blathering about video, as if there is nothing more to the matter!
It's unfortunate that Apple doesn't highlight the use of Apple hardware in the STEM disciplines. Years ago, that was not the case. At the time, Apple had xGrid, parallel computing, etc. They even had a sub-section "Apple and Science" on their website!
All the best,
richmlow
Most likely, IMHO, due to revenues from STEM are not consistent and/or attractive eough for Apple.
I think you misunderstand what Philip is doing. He tested both AMX and the GPU. His work on the GPU side is both to evaluate its feasibility for double-precision calculation as well as explore possibilities for porting CUDA code.
Can third parties use Apple's AMX? Is it still undocumented?
BLAS support for M1 ARM64 via Apple's Accelerate · Issue #42312 · JuliaLang/julia
The default BLAS Julia uses is OpenBLAS. Apple's M1 has proprietary dedicated matrix hardware that is only accessible via Apple's Accelerate BLAS implementation. That proprietary interface can prov...
github.com
Tuning for Apple M1 AMX2 coprocessor · Issue #3789 · OpenMathLib/OpenBLAS
I know that there is already a similar discussion for the general tuning on Apple M1 chips (see #2814) but I wanted to revive this topic a little bit (focusing on Apple's AMX2 extension). I guess, ...
github.com
GitHub - corsix/amx: Apple AMX Instruction Set
Apple AMX Instruction Set. Contribute to corsix/amx development by creating an account on GitHub.
github.com
AFAIK, only way to use it is to run on macOS via a Xcode compiled binary calling macOS APIs.
I recently benchmarked Accelerate vs. OpenBLAS. For most operations, Accelerate is faster because it uses AMX. But for complex eigendecomposition (needed for quantum chemistry), OpenBLAS is 2x faster. The number of FLOPS is also so low, that you wouldn't need AMX. At least for the non-complex eigendecomposition, Apple gets >50 FP64 GFLOPS while on single-core. That says they're using AMX but not parallelizing. OpenBLAS gets slightly better performance with multithreading.
For complex eigendecomposition, you have to un-interleave data before feeding to the AMX. This might not prevent reaching maximum FLOPS for ZGEMM, but requires more threads. Accelerate uses 4 threads for DGEMM (haven't checked ZGEMM), but doesn't multithread its AMX in ZHEEVD. Again, this may not be problematic. Apple gets 34 GFLOPS using one core, OpenBLAS gets 68 GFLOPS using 2-4 cores. DFT simulations can perform multiple eigendecompositions in parallel so you might call into Accelerate ZGEMM from 8 threads. OpenBLAS might do 2-4 eigendecompositions in parallel, therefore slower overall. I haven't benchmarked that yet.
[Data Attached]
For SGEMM and DGEMM, calling into Accelerate from multiple threads wrecked performance. If it's the same with eigendecomposition, I could get better performance using the GPU. If it doesn't (meaning CPU achieves ~300 GFLOPS), even emulated double-single won't provide much speedup (<550 GFLOPS). Mind quantum chemistry does 100% of calculation in FP64, MD is a different story.
I've seen mentions on MATLAB forums that you can direct it to a different BLAS library. In that case, you can change OpenBLAS to Accelerate.
I'm not sure if it's the same thing, but it looks like you can change the BLAS library without recompiling.
GitHub - JuliaLinearAlgebra/libblastrampoline: Using PLT trampolines to provide a BLAS and LAPACK demuxing library.
Using PLT trampolines to provide a BLAS and LAPACK demuxing library. - JuliaLinearAlgebra/libblastrampoline
github.com
You may be interested in this post.
Welcome to AMD
AMD delivers leadership high-performance and adaptive computing solutions to advance data center AI, AI PCs, intelligent edge devices, gaming, & beyond.
Last edited:
My FLOPS formula was off by a factor of 2. Apple got up to 156 GFLOPS on a single core for DSYEVD, within the ~200 GFLOPS limit if one core utilized 2 AMX blocks. OpenBLAS also reached 169 GFLOPS, consistent with using 4 cores worth of SIMD units. Using all 8 cores for a single problem harms performance because of cache incoherence.
I think ZHEEVD was optimized for being energy-efficient. One core could only utilize one AMX block because swizzling interleaved complex takes several instructions. Accelerate achieved 88 GFLOPS/core, while OpenBLAS 41 GFLOPS/core. It may also be more scalable when running multiple ZHEEVDs in parallel, compared with OpenBLAS optimizing single-ZHEEVD performance at all costs. I'll make a better table to visualize this:
| Processor | FP64 GFLOPS |
| P-Core | 51.6 |
| AMX Block | 103.2 |
| Accelerate | OpenBLAS | |
| GFLOPS (1 DSYEVD) | 156 | 169 |
| P-Cores Used | 1 | 4 |
| Clock Frequency | 3.228 GHz | 3.036 GHz |
| AMX Blocks Used | 2 | 0 |
| ALU Utilization | 76% | 87% |
| Max Concurrent DSYEVD's | 4 | 2 |
| Clock w/ Concurrent DSYEVD | 3.132 GHz | 3.036 GHz |
| Max Theoretical GFLOPS | 605 | 338 |
| Hypothesis 1 | Accelerate | OpenBLAS |
| GFLOPS (1 ZHEEVD) | 88 | 162 |
| P-Cores Used | 1 | 4 |
| Clock Frequency | 3.228 GHz | 3.036 GHz |
| AMX Blocks Used | 1 | 0 |
| ALU Utilization | 85% | 83% |
| Max Concurrent ZHEEVD's | 8 | 2 |
| Clock w/ Concurrent ZHEEVD | 3.036 GHz | 3.036 GHz |
| Max Theoretical GFLOPS | 662 | 324 |
Perhaps with an over-balanced P-Core:AMX ratio (1:1), not every instruction is spent dispatching stuff to AMX. There might be room to put additional calculations onto the NEON units, like what OpenBLAS does. I really need to make a benchmark and test this hypothesis.
| Hypothesis 2 | Accelerate | OpenBLAS |
| GFLOPS (1 ZHEEVD) | 88 | 162 |
| P-Cores Used | 1 | 4 |
| Clock Frequency | 3.228 GHz | 3.036 GHz |
| AMX Blocks Used | 2 | 0 |
| ALU Utilization | 43% | 83% |
| Max Concurrent ZHEEVD's | 4 | 1 |
| Clock w/ Concurrent ZHEEVD | 3.132 GHz | 3.036 GHz |
| Max Theoretical GFLOPS | 342 | 324 |
Another note, Accelerate's GFLOPS on ZHEEVD outperformed DSYEVD for small matrices (~100) by 3x. The crossover happened at n=256 where ZHEEVD stopped improving and DSYEVD skyrocketed. Also for such small matrices, Accelerate and OpenBLAS are the same. Likely because they're single-core and Accelerate can't utilize AMX yet. Between n=128 and n=192, GFLOPS exceeds 50. That means Accelerate starts using AMX, and OpenBLAS starts using multiple cores.