sudo lsof -i no longer returns most processes after updating to Sequoia

[humpbacktwale]

Sometimes I use sudo lsof -i to see what connections various applications and daemons are currently making. Immediately after updating from Sonoma to Sequoia, as in after the restart, the terminal command no longer returns the vast majority of processes, only a subset of them. At first I though it was just that several processes had changed after the update, or were not running yet as it was only after an update. However, after connecting to my wifi network, I can observe the usual background processes in the network tab of activity monitor sending and receiving data. However, when running sudo lsof -i, none of them appear. In fact the only process I was able to get the connection information of was firefox. No other processes, whether they are running under my account, root or one of the various other built in ones, returns anything at all.

Given this happened directly after the update, I have to assume it's related. But I have no idea what is actually causing it or how to fix it.

 

[humpbacktwale]

I've been wondering if a reset of the terminal would work, but I can't figure out what would cause a large number of processes to no longer be listed, but some are, directly after an update.

 

[bogdanw]

Have you read the manual for lsof in Sequoia? (man lsof in Terminal)
Maybe something has changed from Sonoma to Sequoia.
"-i
selects the listing of files any of whose Internet address matches the address specified in i. If no address is specified, this option selects the listing of all Internet and x.25 (HP-UX) network files.
If -i4 or -i6 is specified with no following address, only files of the indicated IP version, IPv4 or IPv6, are displayed. (An IPv6 specification may be used only if the dialects supports IPv6, as indicated by ``[46]'' and ``IPv[46]'' in lsof's -h or -? output.)
Sequentially specifying -i4, followed by -i6 is the same as specifying -i, and vice-versa. Specifying -i4, or -i6 after -i is the same as specifying -i4 or -i6 by itself.
Multiple addresses (up to a limit of 100) may be specified with multiple -i options. (A port number or service name range is counted as one address.) They are joined in a single ORed set before participating in AND option selection."

 

[humpbacktwale]

"-i selects the listing of files any of whose Internet address matches the address specified in i. If no address is specified, this option selects the listing of all Internet and x.25 (HP-UX) network files."

This is the command I am using, and the traffic for all the other processes should fall under this. I mean firefox is, so I don't see how all the other apple processes could be doing anything different. If you run the command, what are you seeing, in comparison to what activity monitors network tab shows?

 

[bogdanw]

I'm not familiar with the lsof command, I use Netiquette to monitor connections. https://objective-see.org/products/netiquette.html

 

[humpbacktwale]

Fair enough, but I would still like to find out some possible cause for this, as it is very odd.

 

[JoeSilver]

I'm new to Mac OS, but I can confirm that sudo lsof -i doesn't show all the connections.
If I just watch a video on Youtube with Safari and run a

sudo lsof -i | grep -i Safari

there's no match

while Firefox shows up.
Maybe some sort of lockdown has been added which prevents lsof (even when run as administrator) to see everything?

But then, why lsof and not, for example, tcpdump?

 

[svenmany]

I see that netstat shows certain established connections that aren't shown with lsof. This could be quite an obstacle for me if I can't figure out which processes are not available. I see many processes with lsof, but Safari isn't one.

 

[Bigwaff]

I'm new to Mac OS, but I can confirm that sudo lsof -i doesn't show all the connections.
If I just watch a video on Youtube with Safari and run a

sudo lsof -i | grep -i Safari

there's no match

while Firefox shows up.
Maybe some sort of lockdown has been added which prevents lsof (even when run as administrator) to see everything?

But then, why lsof and not, for example, tcpdump?

lsof will truncate process name. Try sudo lsof -i +c0 | grep -i Safari or sudo lsof -i +c0 | grep -i WebKit instead.

 

[JoeSilver]

lsof will truncate process name. Try sudo lsof -i +c0 | grep -i Safari or sudo lsof -i +c0 | grep -i WebKit instead.

No, it doesn't work. "Safari" and "WebKit" are shorter words than "airportd", "launchd" and "firefox", which are shown instead.

 

[JoeSilver]

I'm intrigued.
It looks like the websites are listed in the Process Name column of the Network tab of the Activity Monitor
By inspecting the the open file and ports of the process "https://www.bbc.co.uk" (from where I'm streaming a video), via the Activity Monitor interface, I can see in the last line a Unix socket:

->0x226e9a17091057d8

This appears many times on the right hand side of the output of sudo lsof -U

I'm not an expert of Unix domain sockets, but I wonder if all these web connections are channelled through this socket.

Also notice that the actual process which is transferring a lot of data, according to Activity Monitor, is "Safari Networking". I have lsofed its pid

sudo lsof -p 2546

I see the same socket in the output, but still don't understand why this internet connection is so obfuscated (if compared for example to a Firefox process).

 

[Bigwaff]

No, it doesn't work. "Safari" and "WebKit" are shorter words than "airportd", "launchd" and "firefox", which are shown instead.

On my Mac, lsof return “com.apple….” by default. Add the +c0 arg to get full name which include “Safari” or “WebKit”. So either my Sequioa 15.4 different than yours or you didn’t try before posting response.

 

[gilby101]

If I just watch a video on Youtube with Safari and run a

sudo lsof -i | grep -i Safari

there's no match

On my Mac, lsof return “com.apple….” by default. Add the +c0 arg to get full name which include “Safari” or “WebKit”. So either my Sequioa 15.4 different than yours or you didn’t try before posting response.

The process name in lsof is not the same as in Activity Monitor. The +c0 changes the name (for some processes) but not the com.apple.WebKit.Networking to Safari.

I use PortsMonitor which makes a better stab at using the Activity Monitor process names - e.g. Safari Networking is reported in the same way as AM, not com.apple.WebKit.Networking as in lsof.

Ports Monitor is also way faster, auto updates and has the features that make it worth while having.

But sudo lsof does indeed report all ports.

 

[JoeSilver]

On my Mac, lsof return “com.apple….” by default. Add the +c0 arg to get full name which include “Safari” or “WebKit”. So either my Sequioa 15.4 different than yours or you didn’t try before posting response.

The command that you suggested:

sudo lsof -i +c0 | grep -i Safari

and also the other one:

sudo lsof -i +c0 | grep -i WebKit

do not show any match at all.
The reason being that:

+c w defines the maximum number of initial characters of the name, supplied by the UNIX
dialect, of the UNIX command associated with a process to be printed in the COMMAND
column. (The lsof default is nine.)

Note that many UNIX dialects do not supply all command name characters to lsof in the
files and structures from which lsof obtains command name. Often dialects limit the
number of characters supplied in those sources. For example, Linux 2.4.27 and Solaris
9 both limit command name length to 16 characters.
If w is zero ('0'), all command characters supplied to lsof by the UNIX dialect will
be printed.

If w is less than the length of the column title, ``COMMAND'', it will be raised to
that length.

My understanding is that this option applies only together with the option -c c
which:

selects the listing of files for processes executing the command that begins with the characters of c

I could also have used a regex directly in c.

However trying to select the command is a bit fiddly because for example

sudo lsof -c Safari

does show Safari's processes, .i.e., it is case sensitive.

Now, it is true that the main process, the one in the top row of my screenshot which is responsible for all the network traffic, seems associated to the command com.apple.WebKit.Networking.

However, what the OP is observing, is that all these Safari processes are not showing any TCP connection, no matter how you run lsof, even without flags. Try:

sudo lsof | grep TCP

no Safari processes.

Now try with Firefox and you will see loads of them.

 

[svenmany]

I'm not doing any searchings by name. I simply connect to my local website using Firefox and Safari, then check for those two connections. Only the Firefox connection shows up. I look for the connection by the IP address of my local machine.

 

[gilby101]

However, what the OP is observing, is that all these Safari processes are not showing any TCP connection, no matter how you run lsof, even without flags. Try:

sudo lsof | grep TCP

no Safari processes.

Now try with Firefox and you will see loads of them.

When I look, I see Safari making use of UDP and not TCP. So what you are seeing may be a reflection of the browsers working in different ways. Also that I am using Private Relay.

Here is snippet from PortsMonitor filtered by the term "Safari":

Note the "Process Name" above is the same as given by Activity Monitor, not lsof.

I see the same connections using lsof -i with the process names shown as com.apple. As I see it, lsof is showing the connections - it needs more interpretation/guesswork to understand them.

I really do recommend PortsMonitor.

 

[humpbacktwale]

Ok so this has taken off a little. So to confirm, all of you are seeing that sudo lsof -i is not listing many of the processes with network activity that activity monitor is otherwise showing are making connections?

 

[JoeSilver]

When I look, I see Safari making use of UDP and not TCP. So what you are seeing may be a reflection of the browsers working in different ways. Also that I am using Private Relay.

Here is snippet from PortsMonitor filtered by the term "Safari":
View attachment 2502444
Note the "Process Name" above is the same as given by Activity Monitor, not lsof.

I see the same connections using lsof -i with the process names shown as com.apple. As I see it, lsof is showing the connections - it needs more interpretation/guesswork to understand them.

I really do recommend PortsMonitor.

Surely Safari is working in a different way. Note that Chrome also behaves like Firefox, showing its TCP connection to the web servers. I still don't understand why.

Regarding PortsMonitor. I have downloaded the trial version, and I still can't see Safari's activity (while I can see the other browsers connections when they browse the Internet).

Also, I'm almost certain that it's not possible to use UDP for Web navigation, it's too unreliable.

EDIT:

I was wrong: the new Safari might be using a protocol built on top of UDP:

QUIC - Wikipedia

en.wikipedia.org

 

[humpbacktwale]

There would be no reason to assume that lsof would show open sockets for applications which previously made a connection. Because of that, it's hard for me to confirm that I have this problem. Can you list some particular, standard processes which seem to be omitted from lsof output at the moment they show network activity?

lsof has always shown all the current connections for all applications that are visible in network activity. All daemons and any processes that I am running. Now, after updating, the only one that appears is firefox. For example, cloudd, mdnsresponder, nurlsessiond etc Pretty much any of the daemons that make make connections when you first connect to a network. There's at least a 15 or so that I can recall off the top of my head. Now, regardless of all of them still running, and activity monitor stating they are all making connections, sending and receiving data, only firefox turns up in sudo lsof -i. A few of the other daemons still do, but none of the ip addresses associated with their network activity do.

 

[svenmany]

lsof has always shown all the current connections for all applications that are visible in network activity. All daemons and any processes that I am running. Now, after updating, the only one that appears is firefox. For example, cloudd, mdnsresponder, nurlsessiond etc Pretty much any of the daemons that make make connections when you first connect to a network. There's at least a 15 or so that I can recall off the top of my head. Now, regardless of all of them still running, and activity monitor stating they are all making connections, sending and receiving data, only firefox turns up in sudo lsof -i. A few of the other daemons still do, but none of the ip addresses associated with their network activity do.

Been messing up my posting. I accidentally deleted one when I was trying to fix another. I'll keep my mouse away from the "delete" button in the future.

Originally I was answering you and saying I also saw a problem in that mDNSResponder wasn't showing in lsof. It turns out that I accidentally just omitted the "sudo". With sudo I get:

mDNSRespo   459 _mdnsresponder    6u  IPv4 0xc8703f508bb30bce      0t0    UDP *:5353
mDNSRespo   459 _mdnsresponder    7u  IPv6 0xcd44b7375e38240f      0t0    UDP *:5353
node        581       xxxxxxxx   24u  IPv4 0xa5f51501c52cf7fb      0t0    UDP *:5353
node       1133       xxxxxxxx   24u  IPv4 0x944d3156eb31e7af      0t0    UDP *:5353

So, I'm not seeing the issue yet.

 

[svenmany]

lsof has always shown all the current connections for all applications that are visible in network activity. All daemons and any processes that I am running. Now, after updating, the only one that appears is firefox. For example, cloudd, mdnsresponder, nurlsessiond etc Pretty much any of the daemons that make make connections when you first connect to a network. There's at least a 15 or so that I can recall off the top of my head. Now, regardless of all of them still running, and activity monitor stating they are all making connections, sending and receiving data, only firefox turns up in sudo lsof -i. A few of the other daemons still do, but none of the ip addresses associated with their network activity do.

Activity Monitor doesn't make it easy to figure out which of the listed processes that have some network traffic have open connections (i.e. listening on a UDP or TCP port or having a TCP socket to another endpoint). You have to catch them when they have active traffic or soon after that. For example, during my experiments, Safari dropped its connection 30 seconds after retrieving a page.

My cloudd and nurlsessiond have so little traffic reported in Activity Monitor that I'm unlikely to catch them when they are active.

So as of now (sorry about deleting the posts), I've seen Safari and mDNSResponder show up in lsof output and I'm not aware of any other process that I should be seeing that I'm not.

 

[gilby101]

So to confirm, all of you are seeing that sudo lsof -i is not listing many of the processes with network activity that activity monitor is otherwise showing are making connections?

No. I think I have already made it clear that lsof -i is showing all processes with network activity. This issue is how to interpret the results and connect to a different way of naming processes. I suggest that lsof is a difficult tool to use.

Look for Safari related processes in Activity Monityor, noting the process id. Then feed your lsof -i output into a grep by process id. And look for both TCP and UDP.

 

[humpbacktwale]

Activity Monitor doesn't make it easy to figure out which of the listed processes that have some network traffic have open connections (i.e. listening on a UDP or TCP port or having a TCP socket to another endpoint). You have to catch them when they have active traffic or soon after that. For example, during my experiments, Safari dropped its connection 30 seconds after retrieving a page.

My cloudd and nurlsessiond have so little traffic reported in Activity Monitor that I'm unlikely to catch them when they are active.

So as of now (sorry about deleting the posts), I've seen Safari and mDNSResponder show up in lsof output and I'm not aware of any other process that I should be seeing that I'm not.

mDNSResponder does show up. But ordinarily, after first connecting to a network, you could see the remote connections to a few apple servers from that process, though I will admit the data was small. All I can see are the two it shows for you with no actual connections.

I agree you have to catch them soon after, but before the update, that meant running it just after connecting to a network listed many results. Now there is literally nothing at all apart from FF.

So for example, if you connect to a network, and you run sudo lsof -i, do you literally see no remote connections, despite there clearly being network activity in activity monitor?

 

[svenmany]

mDNSResponder does show up. But ordinarily, after first connecting to a network, you could see the remote connections to a few apple servers from that process, though I will admit the data was small. All I can see are the two it shows for you with no actual connections.

I agree you have to catch them soon after, but before the update, that meant running it just after connecting to a network listed many results. Now there is literally nothing at all apart from FF.

So for example, if you connect to a network, and you run sudo lsof -i, do you literally see no remote connections, despite there clearly being network activity in activity monitor?

After connecting to a network and not doing anything else, I'm not able to see any connection from Apple processes to servers outside of my LAN. The only remote connections I see are from the processes of my own software. Examples are Dropbox and OneDrive.

I do see Apple processes listening and I see some local connections. rapportd has a bunch of IPv6 connections to other local devices (like my phone). Those are clearly local because of the format of the addresses.

I did look in Little Snitch. There is very little activity from mac processes, but there is some. Also, Little Snitch identifies most of it as TCP. So, I am a bit surprised that I see nothing in the lsof output. I am also surprised that i don't see more in Little Snitch.

I did a little experiment. I opened the Contacts app; it connects to Apple's and Fastmail's servers. I saw in lsof's output

networkse   940       xxx   22u  IPv4 0xe5d35561b17437b4      0t0    UDP 10.27.80.27:56240->172.224.17.6:https

That process is owned by me. Perhaps that makes a difference.

But without further investigation, my only reason I think something is hidden is that you see a difference from the previous OS. I trust your memory.

One thing I do notice is that a a number of the Apple processes, which communicate to their servers via https, are using the UDP-based QUIC protocol. Maybe that's new and the reason you no longer see established connections to Apple servers that you used to see. But, that's just a guess.

QUIC - Wikipedia

en.wikipedia.org

 

[buggz]

I don't understand as, sudo lsof -l|grep EST,
works here.