What is that Neural Engine doing anyway?

[dotnet]

I’ve been a happy user of an M1 Mac Mini for a few months now. It is part of my dedicated stereo system. I run apps like Audirvana and Roon on it, upsampling to 768kHz or DSD256 as required, as well as convolution software for room correction. As expected, the Mini never breaks a sweat.

Looking at the chip temperatures as reported by iStat Menus, I’m a bit puzzled by what I see:

It looks like the CPU cores are the coolest part of the chip (about ambient temperature – this is Australia in summer, mind you). I can understand the GPU being a bit warmer: even though I’m running the Mini headless with remote screen sharing only I do have a dummy HDMI thingie plugged in for better remote screen performance. So far so good, and as expected.

What I don’t get is the neural engine temperature. Why would that be any warmer than the actual working parts of the chip? What is the neural engine doing? This Mac isn’t exposed to a flurry of online or iCloud or any sort of interactive stuff, its only purpose is playing music. I’m fairly certain that neither Roon nor Audirvana use machine learning, so what else is going on there?

This is not a serious concern, of course, but I’d really like to know what the NE is doing in this case, and also what its big mission is overall, since Apple puts to much emphasis on it.

 

[T'hain Esh Kelch]

it is trying to compute why it matters to you. Schrödingers neural engine!

 

[Toutou]

Both the Neural Engine sensor and the Image Signal Processor always show 30 °C on my machine (Air), and never change. I guess the sensors just don't work or need different signal processing.

 

[StoneJack]

I’ve been a happy user of an M1 Mac Mini for a few months now. It is part of my dedicated stereo system. I run apps like Audirvana and Roon on it, upsampling to 768kHz or DSD256 as required, as well as convolution software for room correction. As expected, the Mini never breaks a sweat.

Looking at the chip temperatures as reported by iStat Menus, I’m a bit puzzled by what I see:

View attachment 1964776

It looks like the CPU cores are the coolest part of the chip (about ambient temperature – this is Australia in summer, mind you). I can understand the GPU being a bit warmer: even though I’m running the Mini headless with remote screen sharing only I do have a dummy HDMI thingie plugged in for better remote screen performance. So far so good, and as expected.

What I don’t get is the neural engine temperature. Why would that be any warmer than the actual working parts of the chip? What is the neural engine doing? This Mac isn’t exposed to a flurry of online or iCloud or any sort of interactive stuff, its only purpose is playing music. I’m fairly certain that neither Roon nor Audirvana use machine learning, so what else is going on there?

This is not a serious concern, of course, but I’d really like to know what the NE is doing in this case, and also what its big mission is overall, since Apple puts to much emphasis on it.

computational photo/video work.

 

[Mr.Blacky]

What does it do for you, if you (think to) know the temperature of cores, engines, CPUs or GPUs? Is there any benefit to this?

 

[satcomer]

What does it do for you, if you (think to) know the temperature of cores, engines, CPUs or GPUs? Is there any benefit to this?

Yes when you are using layers and heavy photoshopping! Video makers also get better times on video crunching too!

 

[Mr.Blacky]

Yes when you are using layers and heavy photoshopping! Video makers also get better times on video crunching too!

So by seeing the (allegedly) temperatures of cores and what not, it makes certain tasks faster? ?

 

[dotnet]

Both the Neural Engine sensor and the Image Signal Processor always show 30 °C on my machine (Air), and never change. I guess the sensors just don't work or need different signal processing.

Yes, that could be the explanation. The NE temperature appears to be a solid 30 degrees over time, so either the sensor isn’t working or there is a bug in iStat Menus.

 

[throAU]

Likely doing things like:

AI pattern matching on photos
AI on battery charge behaviour
AI on application usage patterns


etc.

 

[satcomer]

So by seeing the (allegedly) temperatures of cores and what not, it makes certain tasks faster? ?

Think of it it's a replacement of the chip that video crushes liked just couple of years ago!

 

[darngooddesign]

So by seeing the (allegedly) temperatures of cores and what not, it makes certain tasks faster? ?

It can.

When I know the cores are thermally throttling, and therefore slowing down a render, I can manually set the fans to max speed, which will stop them from throttling and the render is completed in less time.

 

[now i see it]

It’s building the beloved “For You Album” in the Photos App

 

[MysticCow]

Maybe the Neural Engine is...wait for it...there's a punch line coming...THINKING!

Someone had to say it...

 

[Reggaenald]

Likely doing things like:

AI pattern matching on photos
AI on battery charge behaviour
AI on application usage patterns


etc.

Unlikely on a machine hosting no photo library.
Or having a battery.
Most likely it’s just displayed this way because iStat isn’t the most reliable.

 

[Reggaenald]

It can.

When I know the cores are thermally throttling, and therefore slowing down a render, I can manually set the fans to max speed, which will stop them from throttling and the render is completed in less time.

For that I use custom fan rules, which I highly recommend everyone to do, especially with Intel systems.

 

[yitwail]

Yes, that could be the explanation. The NE temperature appears to be a solid 30 degrees over time, so either the sensor isn’t working or there is a bug in iStat Menus.

An astute observation. In my MBA, both NE and ISP are always 30C, and for that matter, both GPU temps are almost always at 30C, except for occasional tiny spikes of less than 1 degree. Maybe someone can ask iStat Menus support about this.

 

[mr_roboto]

What I don’t get is the neural engine temperature. Why would that be any warmer than the actual working parts of the chip? What is the neural engine doing?

It's probably doing nothing. You should look at power rather than temperature, as temperature data is convoluted with other nearby things. Silicon is a decent thermal conductor, so if there's anything hot nearby, it'll heat up the neural engine too. (This will be especially true if they didn't bury the neural engine temperature sensor in the middle of the neural engine, and instead placed it on the border where it neighbors something else.)

Try running Terminal.app and using this command: sudo powermetrics -i 1000 -s cpu_power . You should see a line like "ANE Power: 0 mW". That's the number to look at. If it's 0, the ANE (Apple Neural Engine) is completely idle and depowered.

Both the Neural Engine sensor and the Image Signal Processor always show 30 °C on my machine (Air), and never change. I guess the sensors just don't work or need different signal processing.

See above re: neighbor influence. Since you're on an Air, you can actually make the ANE wake up and do stuff: just run FaceTime. Apple uses the Neural Engine to process and enhance the video stream coming from the FaceTime camera in the notebooks. @dotnet won't be able to do that on a Mini, though. I'm not sure what built in software will actually use the ANE other than FaceTime.

 

[Spock]

It is plotting...

 

[turbineseaplane]

I’ve always wondered about the neural engine also

Maybe it's "reticulating splines"?

 

[anthony13]

ai, neural engine, all these terms have always made me wonder if its just enticing sales pitches or is it actually using repeating patterns of some sort to make 'educated' guesses? I mean I'm no where near informed enough to understand, but I see the op's point.

 

[darngooddesign]

In the same way that the GPU is optimized for certain calculations, the same with true for the NE.

 

[phobos]

It's so funny. I just posted a video about the power of the neural engine cores on my youtube channel.
One of the things it's perfect for, is photogrammetry (creating a 3d object out of photographs). The performance gains are quite incredible. In this specific task even the low end Macbook Air was faster than my 10 core iMac Pro.

If you're a 3D user you will absolutely appreciate this extra power. But even if you're not I think we will get to see more and more functions getting advantage of the neural engine cores.

 

[yitwail]

ai, neural engine, all these terms have always made me wonder if its just enticing sales pitches or is it actually using repeating patterns of some sort to make 'educated' guesses? I mean I'm no where near informed enough to understand, but I see the op's point.

I know a couple of the things it does is touch ID fingerprint recognition and Optimized battery charging (ie. not always charging to 100%). If you have macOS Monterey, it also does Live Text. I'm sure there are other applications, but those I came up with off the top of my head.

 

[dotnet]

Try running Terminal.app and using this command: sudo powermetrics -i 1000 -s cpu_power . You should see a line like "ANE Power: 0 mW". That's the number to look at. If it's 0, the ANE (Apple Neural Engine) is completely idle and depowered.

Ah yes, the ANE is using 0mW consistently:

E-Cluster Power: 55 mW
P-Cluster Power: 18 mW
ANE Power: 0 mW
DRAM Power: 22 mW
CPU Power: 73 mW
GPU Power: 3 mW
Package Power: 103 mW

So it’s simply a read-out error. Whether the on-chip sensors or iStat Menus are to blame has yet to be determined.

 

[vddobrev]

I’ve been a happy user of an M1 Mac Mini for a few months now. It is part of my dedicated stereo system. I run apps like Audirvana and Roon on it, upsampling to 768kHz or DSD256 as required, as well as convolution software for room correction. As expected, the Mini never breaks a sweat.
...

Unrelated to your question, but I am interested to find out more about your Hi-Fi components. How do you output from the Mac mini to your gear, and what is the system components?