MacOS Monterey breaks port forwarding?

[redheeler]

I'm having a strange issue with port forwarding on two Macs running MacOS Monterey. Passing a port forwarding rule to pfctl such as rdr pass inet proto tcp from any to any port 58750 -> 127.0.0.1 port 58750 works fine in the loopback interface, but trying to connect to it using another Mac on the network results in a connection time out error. This issue doesn't happen with my iMac on MacOS Big Sur, and I can't think of any other reason for it. Doing some research yielded a couple isolated threads with similar issues after updating to Monterey, but no replies.

 

[svenmany]

I'm going to try this later tonight and see what happens for me. Unfortunately, I'm very much a novice; most of my experience and intuition is on the Linux firewall. Just to be sure I understand...

You have some service opening port 58750 only on the loopback interface. Then, when an external request is made to the external interface, the service doesn't receive the traffic. Is that the gist of it?

Have you tried with differing ports? What about explicitly identifying the external interface rather than just "any"?

Maybe I'm just being silly, but tell me...after a redirect, is there a way to make the ruleset be processed again on the result of the redirection? Your rule would cause an infinite loop if so. But I can't find any evidence of such a setting in the PF documentation. I also can't find any evidence that PF itself was recently upgraded and somehow default options have changed.

 

[redheeler]

I'm going to try this later tonight and see what happens for me. Unfortunately, I'm very much a novice; most of my experience and intuition is on the Linux firewall. Just to be sure I understand...

You have some service opening port 58750 only on the loopback interface. Then, when an external request is made to the external interface, the service doesn't receive the traffic. Is that the gist of it?

Have you tried with differing ports? What about explicitly identifying the external interface rather than just "any"?

Maybe I'm just being silly, but tell me...after a redirect, is there a way to make the ruleset be processed again on the result of the redirection? Your rule would cause an infinite loop if so. But I can't find any evidence of such a setting in the PF documentation. I also can't find any evidence that PF itself was recently upgraded and somehow default options have changed.

I believe this is the gist of it, yes. 58750 is exposed on the loopback interface but not on e0. Curiously, the above never caused an infinite loop even on my Big Sur iMac where it worked to make the service accessible on e0.

If there's a bug in the Monterey implementation which is causing an infinite loop, that might've explained the symptom. Unfortunately I've also tried making the forwarded port different from 58750 and specifying e0 and got the same result for computers on the network, a connection would be attempted but time out.

 

[redheeler]

The simplest way to test if port forwarding is completely broken in Monterey would be to launch a simple service (e.g. web server on port 80) and try forwarding that to a different port (e.g. 8080), then making a request to the forwarded port from the loopback (which works) and a computer on the local network. Even with that I seemed to be having the issue.

 

[svenmany]

Still haven't tested - that's tonight.

I know it's crazy but...

PF breaks private relay. My understanding is that private relay is only used in Safari and Mail, but I don't put it past Apple to break something unrelated. Since PF breaks private relay (just having an empty anchor is enough), maybe private relay breaks PF. If you have private relay turned on, could you try with it turned off?

Oh, did you turn on forwarding in the kernel (net.inet.ip.forwarding), or is that something PF does for you automatically? I see on my Monterey computer that it's set to 0. On my BigSur computer it's set to 1. (Though, it's probably something I set on BigSur at some point in the past.)

-- Edit --

I'm just assuming you know about the sysctl command. Let me know if not.

 

[svenmany]

still learning here - getting the same results as you...

I have set the kernel forwarding parameter to 1.

I restricted httpd to 127.0.0.1:8080. I created a rule that directs all traffic to port 1234 forwarded to 127.0.0.1:8080.

From another machine I hit port 1234. I see in the PF logs that the redirect happened successfully. I see nothing the httpd access_log.

I'm still playing with it; I might get lucky and stumble across something.

 

[svenmany]

Well, I decided to have my mac redirect out to another machine so that I could easily see everything with Wireshark. I was able to compare:

First scenario: Windows browser -> to Mac:1234 -> to Windows:8050 (didn't work)

Second scenario: Mac browser -> to Windows:8050 (did work)

On the Windows machine I can compare the incoming TCP SYN (out of my depth so far). Windows does not ACK that in the first scenario. Windows does ACK that in the second scenario. Something fishy in the SYN packet.

 

[svenmany]

I'm afraid I'm of no help.

I have really no idea how return packets are supposed to work. I've seen no mention of this point in the PF user guide's discussion of redirection, so I'm lacking some basic understanding.

Good luck.

 

[svenmany]

Just some final thoughts before I drop it completely. I hope you've already moved on to get help from people who might actually know things

(The TCP SYN thing was a complete red herring. I was having routing/NAT confusion, so ignore that.)

There might be no reason to compare local access to remote access since local access bypasses some of the network stack. So, it is not unreasonable that a bug in the stack is not manifesting when you test the forwarding from right on the local machine.

The problem has nothing to do with kernel setting of net.inet.ip.forwarding. I set it to 0 on my Big Sur computer and the forwarding still worked fine.

The problem has nothing to do with the localhost interface. I did the experiment on Monterey with two ethernet cards. It still failed. I had a server listening on one ethernet card's interface and connected from the outside to the other. The redirection didn't work.

I went a bit deeper to figure out what didn't work. I wrote a simple program that opened a socket on localhost and waited for a connection. I then connected (from an external machine) to the external interface which should have redirected to localhost. My program never saw a connection attempt. So, it has nothing to do with TCP negotiation or anything fancy.

Maybe it's just a bug. It's super easy to reproduce by just comparing Big Sur to Monterey behavior. Sorry I have nothing good to report.

If you figure it out, I'd love to hear.

 

[redheeler]

@svenmany Thanks for your extensive tests and confirming that this is a Monterey-specific issue. Seems there's nothing left to do but file a bug report to Apple. I'm curious if users on the Ventura beta are experiencing the same.