MP 7,1 Running command 'memory_pressure -l critical' is always failing with segmentation fault

[bxs]

Subject: Running command 'memory_pressure -l critical' is always failing with segmentation fault

I will now and then use the 'memory_pressure -l critical' command to check the system's ability to deal with memory pressure as part of my periodic regression testing after installing software.

See 'man memory_pressure'.

I run this same command on my late 2016 15" rMBP13,3 16GB RAM and it runs without issues.

On my MP7,1 having 384GB RAM it will consistently fail with a segmentation fault after consuming some 80GB of memory.

I will be filing a Problem Report to Apple about this.

The following was executed as normal Admin user and under sudo.

memory_pressure -l critical
The system has 2147483648 (524288 pages with a page size of 4096).
Stats:
Pages free: 95580572
Pages purgeable: 61534
Pages purged: 3773
Swap I/O:
Swapins: 0
Swapouts: 0
Page Q counts:
Pages active: 1135012
Pages inactive: 759120
Pages speculative: 371916
Pages throttled: 0
Pages wired down: 2814716

Compressor Stats:
Pages used by compressor: 0
Pages decompressed: 0
Pages compressed: 0

File I/O:
Pageins: 714824
Pageouts: 0

System-wide memory free percentage: 97%

CMD: Allocating pages to go from level: 1 to level: 4...............................................zsh: segmentation fault memory_pressure -l critical

===========================================================================

bxs@Calwah ~ % sudo memory_pressure -l critical
Password:
The system has 2147483648 (524288 pages with a page size of 4096).
Stats:
Pages free: 95535004
Pages purgeable: 61640
Pages purged: 3785
Swap I/O:
Swapins: 0
Swapouts: 0
Page Q counts:
Pages active: 1155042
Pages inactive: 778804
Pages speculative: 372697
Pages throttled: 0
Pages wired down: 2819816

Compressor Stats:
Pages used by compressor: 0
Pages decompressed: 0
Pages compressed: 0

File I/O:
Pageins: 716690
Pageouts: 0

System-wide memory free percentage: 97%

CMD: Allocating pages to go from level: 1 to level: 4...............................................zsh: segmentation fault sudo memory_pressure -l critical

 

[arock]

Yes, I saw the same crash - also 384GB ram on a 16 core MP 7,1.

 

[DeltaMac]

hmm...
A segmentation fault can mean that an app is attempting to access memory that is not within the allowable range, or in a way that is not allowed (?)
I suspect that the terminal command doesn't know what to do with that much memory.
I might try removing part of your RAM, maybe to leave an amount that you might use in an older Mac, maybe 64, or 128 GB. Try the command with that lower amount of RAM installed. If it runs properly with a smaller amount of RAM, but not with the 384 GB, then you might assume that Apple needs to fix the code for that command.

 

[Schismz]

Same. 28 core, 384GB RAM.

memory_pressure -l critical

The system has 2147483648 (524288 pages with a page size of 4096).

Stats:
Pages free: 70117300
Pages purgeable: 1033364
Pages purged: 734057

Swap I/O:
Swapins: 0
Swapouts: 0

Page Q counts:
Pages active: 13391396
Pages inactive: 11729672
Pages speculative: 1659131
Pages throttled: 0
Pages wired down: 3762260

Compressor Stats:
Pages used by compressor: 0
Pages decompressed: 0
Pages compressed: 0

File I/O:
Pageins: 90056605
Pageouts: 545

System-wide memory free percentage: 96%

CMD: Allocating pages to go from level: 1 to level: 4................................................Segmentation fault: 11

 

[bxs]

hmm...
A segmentation fault can mean that an app is attempting to access memory that is not within the allowable range, or in a way that is not allowed (?)
I suspect that the terminal command doesn't know what to do with that much memory.
I might try removing part of your RAM, maybe to leave an amount that you might use in an older Mac, maybe 64, or 128 GB. Try the command with that lower amount of RAM installed. If it runs properly with a smaller amount of RAM, but not with the 384 GB, then you might assume that Apple needs to fix the code for that command.

Thanks.... it simply needs to be fixed by Apple. Have sent them a PR for their attention.

 

[tommy chen]

there is nothing to fix from apple
macOS does not allow access to protected memory areas

and so my 5.1 shows the same message

 

[bxs]

there is nothing to fix from apple
macOS does not allow access to protected memory areas

and so my 5.1 shows the same message

The memory_pressure simulation command does its job without issues on my late 2016 15" rMBP13,3 with 16 GB RAM, and also on my iMac Pro with 128GB RAM.

This command tests how well the system handles memory allocations when the memory resource becomes scarce and under pressure to keep processes running. When the memory resource becomes scarce the system will begin to send kill 9 signals to some processes to make more RAM available, and to allow the system to survive without getting a real twist in its knickers. The system is designed to handle memory pressure in a way to avoid getting stuck, unresponsive, and to avoid crashing or being dead-locked with no way to get out of the situation.

The whole point of memory_pressure is to verify that the system's memory management is doing its job when the RAM resource becomes under pressure, and the memory_pressure program should not be failing with a segmentation fault. This issue with memory_pressure has arisen in the past and Apple resolved it with a software fix.

My PR to Apple stands.

 

[AidenShaw]

This is one of those funny times when the technology has changed completely, but archaic terminology lives on.

A common example is that we "dial" a number on our cellphones, and we get cellphone plans based on the number of "lines" we need - even though push-button phones became popular in the 1960s and cell phones don't use lines. And notice how often an image of a floppy disk is used as the "save" button.

In the 50s and 60s and early 70s computers used "segments" as a manual way to extend addressing. By the later 70s, "virtual memory" became popular, so the manual remapping of address segments became unnecessary.

But 50 years later, we still get "segmentation faults" that generate "core dumps".

(At least on UNIX-like systems - Windows throws "access violation" errors and creates "dump" files - but that was the terminology VMS used so naturally it continued in NT.)

 

[Snow Tiger]

- even though push-button phones became popular in the 1960s and cell phones don't use lines.

I just looked at my iPhone and , yup , it's connected to my Mac with an USB line ...

Imagine if Tesla had been a co-founder of Apple . Wireless powered devices of all sorts , big and small .

 

[AidenShaw]

I just looked at my iPhone and , yup , it's connected to my Mac with an USB line ...

Does your voice traffic go over the USB cable, or via radios?

 

[Snow Tiger]

Does your voice traffic go over the USB cable, or via radios?

Hmmm ... radio ? Technically , yes , as it's set to WiFi .

And although I've had a strong preference for empty orange cans and high performance data string since my youth , I'm still having trouble finding an appropriate interface adapter for my cell phone .

I blame the lack of industry standards here .

Hopefully , the Data String Alliance will do something about that soon .

 

[ibuick]

This problem is due to a program bug in memory_pressure.

In a short word, the program writer for the memory_pressure doesn't take the Memory for App Used which bigger than 64GB into consideration. This not due to the physical memory size, just the Memory that App can use.

In its source code, first of all, it contains a definition

#define MAX_RANGE_SIZE    64 * 1024 * 1024 * 1024ULL

This defines the max space this tool can allocate, 64GB. So, if your system's free memory (the free memory can be used by a user process. can be calculated by total memory - system wired - other processes used - total memory * critical level percentage) is less than 64GB, this program can run without issue. But, if your system has more usable memory than this hardcoded 64GB limit, the segmentation fault comes out.

range_start_addr = mmap(NULL, MAX_RANGE_SIZE, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, 0, 0);

if (page_op == PAGE_OP_ALLOC) {

            if (tool_mode == TOOL_MODE_FOR_PERCENT && USE_WIRED_PAGES_FOR_PERCENT_MODE) {
                error = mlock(range_current_addr, size);
                if (error == -1) {
                    perror("Failed to lock memory!");
                    exit(-1);
                }

                memset(range_current_addr, 0xFF, size);
                range_current_addr += size;

            } else {

                pthread_mutex_lock(&reference_pages_mutex);
                while (start_allocing_pages == 0) {
                    pthread_mutex_unlock(&reference_pages_mutex);
                    sleep(1);
                    pthread_mutex_lock(&reference_pages_mutex);
                }
                pthread_mutex_unlock(&reference_pages_mutex);

                for (i=0; i < num_pages; i++) {

                    if (reached_or_bypassed_desired_result()) {
                        //printf("stopped faulting after %d pages\n", i);
                        break;
                    }

                    memcpy(range_current_addr, random_data, PAGE_SIZE);
                    range_current_addr += PAGE_SIZE;
                }

                pthread_mutex_lock(&reference_pages_mutex);
                start_referencing_pages = 1;
                pthread_cond_signal(&reference_pages_condvar);
                pthread_mutex_unlock(&reference_pages_mutex);
            }

these code snippets show the core issue, the program continuously write (memcpy) data to the allocated 64GB memory space without check the boundary. This always causes segmentation fault.

You can try to build a custom version of the memory_pressure to do the test on Mac Pro 7,1.

 

[bxs]

Thank you very much for confirming to me that the Apple's memory_pressure code has a bug, or at least needs to be changed to deal with RAM sizes beyond 64 GB.

It's a shame the memory_pressure code was not made to check for the MAX_RANGE_SIZE environment variable to override its internal definition.

I did grab the source code for Apple's memory_pressure and modified it and ran my own version to validate your information..... and it worked..... so THANK YOU.

 

[AidenShaw]

Thank you very much for confirming to me that the Apple's memory_pressure code has a bug, or at least needs to be changed to deal with RAM sizes beyond 64 GB.

By definition, it is a bug. If it can't handle supported configurations - it is a bug.

A gray area would be a trash can with 128 GiB of RAM. Since the trash can never supported more than 64 GiB of RAM, memory_pressure would not show a bug on any supported system. We all know, however, that 128 GiB works in a trash can. Unless you run memory_pressure.
[automerge]1582936870[/automerge]

MAX_RANGE_SIZE environment variable

Is that a common environment variable, or a system variable created when the system sizes memory at boot time?

 

[bxs]

By definition, it is a bug. If it can't handle supported configurations - it is a bug.

A gray area would be a trash can with 128 GiB of RAM. Since the trash can never supported more than 64 GiB of RAM, memory_pressure would not show a bug on any supported system. We all know, however, that 128 GiB works in a trash can. Unless you run memory_pressure.
[automerge]1582936870[/automerge]

Is that a common environment variable, or a system variable created when the system sizes memory at boot time?

We crossed paths.... see my last posting where I ran a modified memory_pressure program with success using most of my 384 GB of RAM.
[automerge]1582938387[/automerge]

By definition, it is a bug. If it can't handle supported configurations - it is a bug.

A gray area would be a trash can with 128 GiB of RAM. Since the trash can never supported more than 64 GiB of RAM, memory_pressure would not show a bug on any supported system. We all know, however, that 128 GiB works in a trash can. Unless you run memory_pressure.
[automerge]1582936870[/automerge]

Is that a common environment variable, or a system variable created when the system sizes memory at boot time?

The MAX_RANGE_SIZE is a #define in the memory_pressure's source code, as ibuick stated in a previous post.

 

[bxs]

With my modified memory_pressure program I tested memory over subscription to check Apple's ability to deal with this case, and it did very well.

 

[ibuick]

Thank you very much for confirming to me that the Apple's memory_pressure code has a bug, or at least needs to be changed to deal with RAM sizes beyond 64 GB.

It's a shame the memory_pressure code was not made to check for the MAX_RANGE_SIZE environment variable to override its internal definition.

I did grab the source code for Apple's memory_pressure and modified it and ran my own version to validate your information..... and it worked..... so THANK YOU.

Happy to see that.

It's 6am in China when I wrote down the last post, I'm so sleepy then, cannot modify even single line of code . 😂😂😂

 

[ibuick]

I build a dynamically allocation version of the memory_pressure , then you can use it to test on Mac Pro

 

[bxs]

I build a dynamically allocation version of the memory_pressure , then you can use it to test on Mac Pro

Thanks.... but I modified the memory_pressure's source code directly, recompiled and loaded it, and it now runs for my case on MP7,1 with 384 GB RAM.

 

[AidenShaw]

The MAX_RANGE_SIZE is a #define in the memory_pressure's source code, as @ibuick stated in a previous post.

You called it an environment variable....

 

[bxs]

You called it an environment variable....

What I was trying to convey was that if the memory_pressure code was designed to look for an environment variable MAX_RANGE_SIZE being defined and to use its value instead of the hard coded one in the code, this would allow the memory_pressure program to be used without resorting to changing the actual code and recompiling it.

Such as executing the following.... for example, allowing memory_pressure to deal with a Mac having 1.5 TB RAM. (1500 x 1024 x 1024)

export MAX_RANGE_SIZE=1572864000
memory_pressure -l critical

 

[Snow Tiger]

What I was trying to convey was that if the memory_pressure code was designed to look for an environment variable MAX_RANGE_SIZE being defined and to use its value instead of the hard coded one in the code, this would allow the memory_pressure program to be used without resorting to changing the actual code and recompiling it.

Such as executing the following.... for example, allowing memory_pressure to deal with a Mac having 1.5 TB RAM. (1500 x 1024 x 1024)

export MAX_RANGE_SIZE=1572864000
memory_pressure -l critical

Thank you . I may just need to use this today ...

 

[bxs]

Thank you . I may just need to use this today ...

Note that memory_pressure program as delivered by Apple at this time does not look for an exported MAX_RANGE_SIZE variable to override its internal value for MAX_RANGE_SIZE.

 

[AidenShaw]

Note that memory_pressure program as delivered by Apple at this time does not look for an exported MAX_RANGE_SIZE variable to override its internal value for MAX_RANGE_SIZE.

Isn't there a system call to get installed memory on the machine? There must be - system profiler shows installed RAM size.

 

[bxs]

Isn't there a system call to get installed memory on the machine? There must be - system profiler shows installed RAM size.

Yes I agree, and actually thought that was already being used in the memory_pressure code, but unfortunately not. Maybe Apple or the Apple software person thought otherwise for some reason. It has been a long-standing lessen for coders that using hard coded values is inappropriate and can lead to issues down the road. Long ago assembly language coders loved to use 'jump to from current instruction pointer +2' only to find later than a subsequent coder placed in more instructions immediately after the 'jump instruction' and all 'hell' broke loose.