Memory Limit Overload M2

[straightryder]

I have currently moved over to Mac from PC so I am still fairly green with Apple/ MBP's. I'm really enjoying the new switch with all the work to do date 👍

Today I was working on a project in AE (it's a very simple one at that) not too big and the files are very minimal.... My memory gets jacked up to almost 90% and AE just grinds to a halt. I'll provide screen shots to let some experts checkout the current settings/ preferences.

I'm working in a basic 1920x1080 Comp. No 3D animation going on atm. I'm working off external Samsung 990Pro 2TB nvme and all the Cache files are stored on it as well. My internal HD is 4TB and only 500 GB used on it.

96 Gigs is just getting flushed out within 10min of working on this project in AE - Can someone tell me what I am doing wrong?

Thank you, and everyones help is very much appreciated. This forum has helped out a lot in the past.

 

[herbert7265]

Can you post also a detail screenshot of the memory usage by processes / program? You can get it either from iStat Menu or Activity monitor?

In your preferences you allow 84 GB for the Adobe products. Considering that also other processes and programs (system) utilize the memory the shown memory usage could make sense. Have you done a test and assigned less memory to Adobe programs and then observed a change?

Last but not least, maybe totally off topic: I used Norton Antivirus in the past on my Mac and at one point in time, for whatever reason, it utilized nearly 100% of memory and CPU performance in one of the accounts on the machine. Nothing helped to solve that topic, so I finally switched to BD and since then I never ever observed such a weird behavior. Therefore, it could be helpful to see the detailed memory usage by process / program in you case.

Herbert

 

[straightryder]

Can you post also a detail screenshot of the memory usage by processes / program? You can get it either from iStat Menu or Activity monitor?

In your preferences you allow 84 GB for the Adobe products. Considering that also other processes and programs (system) utilize the memory the shown memory usage could make sense. Have you done a test and assigned less memory to Adobe programs and then observed a change?

Last but not least, maybe totally off topic: I used Norton Antivirus in the past on my Mac and at one point in time, for whatever reason, it utilized nearly 100% of memory and CPU performance in one of the accounts on the machine. Nothing helped to solve that topic, so I finally switched to BD and since then I never ever observed such a weird behavior. Therefore, it could be helpful to see the detailed memory usage by process / program in you case.

Herbert

I've disabled most of what wasn't needed to run in the background and this I what I have left over.

I updated my AE Preferences also Ram is still skyrocketing to 78%

 

[Herbert123]

Did you try this?

In AE 2022/2023 go to 'composition' < 'preview' and turn of the 'Cache Frames when Idle' function.

Others have reported similar problems with AE filling up memory like there is no tomorrow - on both Mac and Windows.

Reverting to AE2020 seems to fix it as well.

 

[straightryder]

Did you try this?

In AE 2022/2023 go to 'composition' < 'preview' and turn of the 'Cache Frames when Idle' function.

Others have reported similar problems with AE filling up memory like there is no tomorrow - on both Mac and Windows.

Reverting to AE2020 seems to fix it as well.

I'll try that right now.

thanks brb

 

[Analog Kid]

I'm not an After Effects user, but just from the symptoms you're showing I can't figure out what would cause a slowdown.

You've allocated 84GB to Adobe, and 87GB of your memory is used. That would suggest to me that Adobe preallocates the RAM. No swap is used, or has even been created, and CPU load is quite low.

Again, not something I'm familiar with, but memory doesn't look like a constraint here.

 

[straightryder]

Did you try this?

In AE 2022/2023 go to 'composition' < 'preview' and turn of the 'Cache Frames when Idle' function.

Others have reported similar problems with AE filling up memory like there is no tomorrow - on both Mac and Windows.

Reverting to AE2020 seems to fix it as well.

After unchecking the 'Cache Frames when idle' function - The Ram usage now hovers around 71%

 

[straightryder]

I'm not an After Effects user, but just from the symptoms you're showing I can't figure out what would cause a slowdown.

You've allocated 84GB to Adobe, and 87GB of your memory is used. That would suggest to me that Adobe preallocates the RAM. No swap is used, or has even been created, and CPU load is quite low.

Again, not something I'm familiar with, but memory doesn't look like a constraint here.

I'm blown away at what's causing this .... the Mac is pretty brand new and so is the nvme drive im working off of.

 

[Analog Kid]

Why are you using an external drive for project and cache files and not the much faster internal drive?

 

[herbert7265]

For what you show now the behavior of your computer looks normal.

You allow 64 GB of memory for the Adobe products and AE uses a little more than 63 GB. The other approx. 8 GB up to your total shown usage is quite a normal value, I see the same in my MacBook Pro M2 Max w/ 64 GB of memory.

Why AE utilizes the full amount of memory allocated is beyond my knowledge, I am not using and don’t really know that program. But using PS and LrC on my computer I recognize that also these Adobe programs use whatever memory is allowed / available.

Considering now that you just utilize 72 GB out of your 96 GB available, with a really low memory pressure and no swap at all, I would not be concerned at all. What sense does it make to have memory available when finally not using it?

You may play around a bit with the value of memory you allow for the Adobe programs considering
- how much memory do you need for other processes and programs under your normal working conditions (multitasking)
- at what point you may see a slow down of the AE performance
and then set a value that would enable normal working conditions and keep the memory pressure overall still in the green area.

Herbert

 

[chabig]

Memory Pressure is also quite low. I'm with Analog Kid in that the memory doesn't look like it's constraining anything. I'm going to suggest that having your cache files on the external SSD is the reason you're getting an AE slowdown. I assume the rest of the system is responsive at the same time. But I do note that you didn't specify how the Samsung 990Pro is connected.

 

[Analog Kid]

Why AE utilizes the full amount of memory allocated is beyond my knowledge

The fact that it's asking how much it can use, in a preference, suggests that Adobe feels they can handle memory allocation more efficiently than the OS because they know the memory pool usage patterns. Adobe does the same with swap-- they don't use system swap, they create a scratch file on disk and manage it themselves.

 

[straightryder]

Why are you using an external drive for project and cache files and not the much faster internal drive?

I switched from PC to Mac a few months ago and the person who is a Mac user told me to work off a fast external drive such as an nvme to reduce wear and tear on the drives lifespan... so I got the Samsung 990pro inside of a Acasis enclosure connected usb-c.

 

[Allen_Wentz]

After unchecking the 'Cache Frames when idle' function - The Ram usage now hovers around 71%

Nothing whatsoever wrong with 71%. As long as there is some free space it means that the fast Apple UMA RAM is unconstrained and allowing optimal operation. But IMO intentionally working from an external drive when you have 4 TB fast internal makes no sense. We buy hardware to use it, not to sequester it so it does not wear out.

I do recommend keeping HDDs/SSDs ~ half full for optimal operation long term but not to the extent of moving active work off of internal SSDs.

 

[straightryder]

Nothing whatsoever wrong with 71%. As long as there is some free space it means that the fast Apple UMA RAM is unconstrained and allowing optimal operation.

I'm watching it and it drops to 65 - 60 - then 70% fluctuating around there.

 

[straightryder]

Nothing whatsoever wrong with 71%. As long as there is some free space it means that the fast Apple UMA RAM is unconstrained and allowing optimal operation. But IMO intentionally working from an external drive when you have 4 TB fast internal makes no sense. We buy hardware to use it, not to sequester it so it does not wear out.

This guy beat it into my head that the constant read and write of the media cache files will quickly reduce the lifespan/ tbw of the internal drive...

 

[Allen_Wentz]

The fact that it's asking how much it can use, in a preference, suggests that Adobe feels they can handle memory allocation more efficiently than the OS because they know the memory pool usage patterns. Adobe does the same with swap-- they don't use system swap, they create a scratch file on disk and manage it themselves.

IMO not "Adobe feels they can handle memory allocation more efficiently than the OS," but rather Adobe simply wants to sequester resources for the benefit of Adobe apps at the detriment of other apps/OS that might want to access those same resources.

 

[Analog Kid]

I switched from PC to Mac a few months ago and the person who is a Mac user told me to work off a fast external drive such as an nvme to reduce wear and tear on the drives lifespan... so I got the Samsung 990pro inside of a Acasis enclosure connected usb-c.

This guy beat it into my head that the constant read and write of the media cache files will quickly reduce the lifespan/ tbw of the internal drive...

You bought a 16" MBP Max with 96GB of RAM and a 4TB drive. No sense wasting that investment so you don't have to replace your SSD in 10 years...

If performance is what you're after, use the tool you have to the best of its ability. People get weirdly precious about SSDs and batteries in ways that don't make sense in these contexts. Care for your tools, sure, but use them. Maybe someone here has a story about an SSD wearing out, but it's certainly not common-- and if it does fail it's because you used it to death which only goes to show that you got your money out of it.

 

[Allen_Wentz]

This guy beat it into my head that the constant read and write of the media cache files will quickly reduce the lifespan/ tbw of the internal drive...

I lack the expertise to ruminate on SSD tech at a high level, but as long as a drive is not overfilled IMO intentionally working from an external drive when you have 4 TB fast internal less than ~85% filled makes no sense. Net speed internally should be much faster than externally connected, and we buy hardware to use it, not to sit it on a shelf.

 

[straightryder]

I lack the expertise to ruminate on SSD tech at a high level, but as long as a drive is not overfilled IMO intentionally working from an external drive when you have 4 TB fast internal less than ~85% filled makes no sense. Net speed internally should be much faster than externally connected, and we buy hardware to use it, not to sit it on a shelf.

yeah I hear ya - might just get on that.

 

[straightryder]

You bought a 16" MBP Max with 96GB of RAM and a 4TB drive. No sense wasting that investment so you don't have to replace your SSD in 10 years...

If performance is what you're after, use the tool you have to the best of its ability. People get weirdly precious about SSDs and batteries in ways that don't make sense in these contexts. Care for your tools, sure, but use them. Maybe someone here has a story about an SSD wearing out, but it's certainly not common-- and if it does fail it's because you used it to death which only goes to show that you got your money out of it.

yeah im gonna move it to the internal drive.

 

[Analog Kid]

IMO not "Adobe feels they can handle memory allocation more efficiently than the OS," but rather Adobe simply wants to sequester resources for the benefit of Adobe apps at the detriment of other apps/OS that might want to access those same resources.

Maybe, but I'm not sure there's much they can do to prevent their allocated hoard from getting swapped out if it's under pressure from other apps. The OS will warn the app when memory is getting tight, which I think is when Adobe starts utilizing its own scratch, but I'm not sure they can resist the OS's attempt to swap out.

What they can do is minimize the risk of fragmentation by managing their own pool. That's not uncommon in memory intensive apps. Adobe does it to an extreme, but I think that's an artifact of earlier, leaner times.

And I'm guessing if you've spent this much money for a machine to run Adobe apps, you kind of want Adobe to run at its best even at the expense of other services, so it's probably a reasonable approach for Adobe to take.

 

[Allen_Wentz]

IMO not "Adobe feels they can handle memory allocation more efficiently than the OS," but rather Adobe simply wants to sequester resources for the benefit of Adobe apps at the detriment of other apps/OS that might want to access those same resources.

Maybe, but I'm not sure there's much they can do to prevent their allocation from getting swapped out if it's under pressure from other apps. To OS will warn the app when memory is getting tight, which I think is when Adobe starts utilizing its own scratch, but I'm not sure they can resist the OS's attempt to swap out.

What they can do is minimize the risk of fragmentation by managing their own pool. That's not uncommon in memory intensive apps. Adobe does it to an extreme, but I think that's an artifact of earlier, leaner times.

And I'm guessing if you've spent this much money for a machine to run Adobe apps, you kind of want Adobe to run at its best even at the expense of other services.

Correct, Adobe does it to an extreme as an artifact of earlier, leaner times. In the old days we spent half our time optimizing the size of the Photoshop Scratch Disk to suit what apps we were running. PS with RAM limiting totally bogged down, so yes, we did want Adobe to run at its best even at the expense of other services.

Today the Mac OS does a very, very good job memory managing. And most of us have more (fast UMA) RAM available both in real and relative terms. Today's workflows (at least mine) are much more complex but not more demanding. Twenty years ago and also now I prefer not to have (greedy bastards) Adobe doing the RAM management; I always want to clearly be able to intentionally set any scratch disks if and only if it suits some app-specific workflow.

 

[TechnoMonk]

I switched from PC to Mac a few months ago and the person who is a Mac user told me to work off a fast external drive such as an nvme to reduce wear and tear on the drives lifespan... so I got the Samsung 990pro inside of a Acasis enclosure connected usb-c.


*sorry for the double post

I have MBA 2011 and a 2014 MBP, which has been running as file server, streaming. I write Ton in to the drive and reads, never had any problems. Why was the person concerned about wear and tear? These things easily outlive HDD by years.

 

[smirking]

I switched from PC to Mac a few months ago and the person who is a Mac user told me to work off a fast external drive such as an nvme to reduce wear and tear on the drives lifespan...

Just use your computer the way it was meant to be used. It's gonna be fine. There are too many Mac users who look at their computers like they're jewelry pieces and not tools. They can take a beating.