Fix for broken HTTPS / TLS / Certificates in Leopard-Webkit

[wicknix]

So after applying @Wowfunhappy 's squid proxy package on Snow Leopard to help resolve some minor issues, i got the idea to build squid4 via macports on Leopard.

sudo port -v install squid4 +ssl_crtd

It worked! Leopard-Webkit can once again access sites that it had previously not been able to be access. This workaround makes LWK 100% usable again. So, for the brave, you have 2 options: Install macports and build squid4, then use pacifist to extract wowfunhappys squid.conf from the package linked above and edit it point to /opt/local/etc/squid rather than /Library/Squid, OR (i tested this as well), run squid on another machine (i used 10.6 to test this) and connect to it from Leopard. Instructions on how to import your new certificate into your keychain, and how to configure your network settings are available in Wowfunhappy's .dmg linked above.

The easiest route is running squid on another machine (it's faster than waiting 24-ish hours for gcc7, etc to build). For instance, install his Squid installer for 10.6-10.9 and follow the directions and ensure squid is properly running on your 10.6-10.9 machine. Now edit the last 2 lines in /Library/Squid/squid.conf and change "deny" to "allow" and save. Reboot.

Now in Leopard, open system prefs -> network -> ethernet (or wifi depending on how you are connected) -> advanced -> proxies. Now check "secure web proxy (https)" and add in your machine running squid's internal IP address using port 3128 and hit apply.

If you did everything right, LWK will once again be able to access wikipedia and other sites that had previously stopped working.

My plan now is to take an old macmini and toss a barebones Linux on it, set up squid, and run it headless so it's out of the way. This will take the load off of the G5 (which is currently running squid) and i can then connect any other "retro" machine to the macmini running squid if i need to.

Cheers

Edit: If you dont have another machine, or don't want to install macports, here is a prebuilt package that you can manually install. I still recommend using squid on another device and connecting to that, but this may help if you don't have the option. There is also an installer package here for ppc and intel.

 

[Wowfunhappy]

For instance, install his Squid installer for 10.6-10.9 and follow the directions and ensure squid is properly running on your 10.6-10.9 machine. Now edit the last 2 lines in /Library/Squid/squid.conf and change "deny" to "allow" and save. Reboot.

I actually recommend doing this a little differently!

Above the block that allows access to localhost, add the following:

acl localnet src 0.0.0.1-0.255.255.255    # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8        # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10        # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16     # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12        # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16        # RFC 1918 local private network (LAN)
acl localnet src fc00::/7           # RFC 4193 local private network range
acl localnet src fe80::/10          # RFC 4291 link-local (directly plugged) machines
http_access allow localnet

Don't change anything else, including http_access deny all. This allows local IP addresses to access the proxy, but not remote ones.

BTW, I am currently working on updating the Squid package to install on Leopard (literally as I write this—I logged onto MacRumors to kill time while MacPorts compiles stuff). But the separate computer method should be useful on Tiger, Mac OS 9, NEXTSTEP, etc. Not to mention, while Squid is quite lightweight, on very old Macs there may be a performance advantage to running it on a different machine.

 

[wicknix]

Thanks for the heads up. I'll add that to my config.
On a side note, if you relink Dictionary.app to LWK and use it on a machine connected to squid it works with wikipedia again. The formatting isn't the best, but it works again.

 

[Wowfunhappy]

...would you mind filling me in, as someone who doesn't own a PowerPC Mac, what is the deal with Leopard Webkit's HTTPS support anyway? Does it work? (It clearly works enough to load Wikipedia).

The developer managed to do for Leopard what I utterly failed to do on Mavericks—update and recompile Apple's Security Framework. I couldn't get the damn thing to build properly; I don't know why Apple bothers to release code that depends on unreleased code in order to actually build.

Edit: Wait, no, you said Dictionary only works if you relink it _and_ have Squid running. So relinking it doesn't fix HTTPS, it merely forces it to use the system proxy. That's... interesting. I still prefer the approach of patching CFHTTPStream.

Edit2: Oh, except that won't work on Leopard. https://developer.apple.com/documentation/cfnetwork/1426754-cfnetworkcopysystemproxysettings requires 10.6. Well that's dumb.

 

[wicknix]

From what i gather, and i could be completely wrong, but when LWK was last released in 2018 it closely mirrored Safari 11. However i vaguely remember reading something about how not everything was able to be back ported, which i believe to be some of the frameworks and libs needed for full TLS 1.2 and 1.3 support. This was the roadblock going forward, and most likely the reason it was never updated after that. Everything worked fine back in 2018/2019, but as sites started turning off the older ssl, tls 1.0 and 1.1 compatibility, it broke LWK on those sites. So while it does come with 10.12's certificates, the backend now lacks the proper https support to actually use them.

Edit: without squid, LWK could no longer connect to wikipedia.
Edit 2: relinking to LWK brings any app using webkit up to safari 11's capabilities more or less from version 5 shipped with Leopard.

Cheers

 

[Wowfunhappy]

i believe to be some of the frameworks and libs needed for full TLS 1.2 and 1.3 support

Thank you, that makes sense! So it supports the new cipher suites (presumably?), but not TLS 1.2.

Edit: relinking to LWK brings any app using webkit up to safari 11's capabilities more or less from version 5 shipped with Leopard.

That doesn't quite explain what's going on with the Dictionary app though. It doesn't actually matter, it's just bizarre.

The Dictionary app, on every OS, usually ignores the system proxy when querying Wikipedia. You can try this on even e.g. Big Sur by putting some nonsense address into the proxy settings in System Preferences. This will render most apps unable to use the internet, because they're trying to connect via a proxy which doesn't exist. But the Dictionary app will connect to Wikipedia without trouble, because it doesn't use the system proxy at all!

So adding a proxy shouldn't make a difference within the Dictionary app, unless Leopard Webkit was already enough on its own. I'd like to know what's going on there. Probably never will though, oh well! 🤷‍♂️

(More here btw: https://apple.stackexchange.com/que...dictionary-app-doesnt-use-system-proxy/408830. 1110101001 in the comments was the real hero, he figured everything out!)

 

[Wowfunhappy]

Hey, @wicknix, want to help me? (Or anyone else with PPC Leopard and MacPorts experience. )

It turns out, there's no way to get MacPorts to build PPC software without PPC hardware. I thought I'd be able to use Rosetta, but Rosetta is being stupid.

You'd need to install the attached portfile with the +openssl and +ssl_crtd varients. It will complain at the end that files were created outside of MacPorts's directories, which is both okay and exactly what we wanted to happen.

Afterwards, you'd need to collect all the pieces with this mess of commands, and send them to me.

cd /path/to/a/folder/
cp /opt/local/sbin/squid .
cp /Library/Squid/security_file_certgen .
cp /Library/Squid/unlinkd .

mkdir lib
sudo port install dylibbundler
dylibbundler -p '@executable_path/lib' -d lib -b -x unlinkd
dylibbundler -p '@executable_path/lib' -d lib -b -of -x security_file_certgen
dylibbundler -p '@executable_path/lib' -d lib -b -of -x squid

cp /usr/lib/libc++abi.dylib ./lib/
cp /usr/lib/libc++.1.dylib ./lib/
install_name_tool -change /usr/lib/libc++abi.dylib @executable_path/lib/libc++abi.dylib squid
install_name_tool -change /usr/lib/libc++.1.dylib @executable_path/lib/libc++.1.dylib squid
install_name_tool -change /usr/lib/libc++abi.dylib @executable_path/lib/libc++abi.dylib unlinkd
install_name_tool -change /usr/lib/libc++.1.dylib @executable_path/lib/libc++.1.dylib unlinkd
install_name_tool -change /usr/lib/libc++abi.dylib @executable_path/lib/libc++abi.dylib security_file_certgen
install_name_tool -change /usr/lib/libc++.1.dylib @executable_path/lib/libc++.1.dylib security_file_certgen
install_name_tool -change /usr/lib/libc++abi.dylib @executable_path/lib/libc++abi.dylib lib/libc++.1.dylib

Skip the commands for libc++ if that library doesn't exist. I'm not quite sure what the story is with MacPorts's libcxx package; it's currently causing me problems on the Intel side.

My plan is to lipo everything together into a three-arch universal binary (ppc, x86, x86_64).

 

[wicknix]

Done. Grab the files here.

Cheers

 

[Wowfunhappy]

Done. Grab the files here.

Cheers

That was fast, thank you!

 

[Wowfunhappy]

@wicknix Was that the portfile I attached, or the normal one for Squid?

If it's the one I attached, it may not be possible to get PPC Squid to work outside of MacPorts, because it's blatantly ignoring my compilation options...

 

[wicknix]

I didn't see any attachment, so I just built the variants from macports. The paths to find and copy the files were standard macports /opt/local/sbin, /opt/local/libexec/squid, and /opt/local/etc/squid.

 

[Wowfunhappy]

(facepalm) I forgot to actually attach it. I changed the compilation options so Squid stores stuff in /Library/Squid to avoid messing with the MacPorts paths.

 

[wicknix]

Ok. Rebuilt. Files here: http://s000.tinyupload.com/index.php?file_id=08368842764379285886

 

[micahgartman]

I'm not really sure what just transpired here, but THANK YOU!!!!!

 

[Raging Dufus]

I'm not really sure what just transpired here

That's because it was all... in code

I'll see myself out.

 

[Macbookprodude]

Why can't someone just upload the actual executable, so it can just work out of the box ?

 

[Macbookprodude]

Can't new TLS just be placed into the actual APP LWK instead of using a proxy server ?

 

[wicknix]

Why can't someone just upload the actual executable, so it can just work out of the box ?

That's what we're working on.

Can't new TLS just be placed into the actual APP LWK instead of using a proxy server ?

No.

 

[Wowfunhappy]

Okay, can someone (preferably just one person, because I'm not expecting great things) test this on a real PPC Mac running Leopard?

I suspect that it does not work, because it didn't work in Rosetta. In which case, I need to stop being clever with universal binaries and create separate packages for PPC and Intel. I didn't particularly want to do that because universal binaries are cool, but oh well...

Edit: Attachment that didn't actually work has been removed because it didn't actually work.

 

[Wowfunhappy]

Can't new TLS just be placed into the actual APP LWK instead of using a proxy server ?

No. Squid uses OpenSSL, a cross-platform library which is relatively easy to update. By contrast, LWK uses Apple's SecureTransport library, which is integrated into the operating system.

 

[micahgartman]

Okay, can someone (preferably just one person, because I'm not expecting great things) test this on a real PPC Mac running Leopard?

No joy:

 

[Wowfunhappy]

No joy:

Thanks for trying—as expected, really, but good to get confirmation. (I assume Squid isn't running in Activity Monitor.) You should probably go ahead and run the uninstaller for now.

Welp, I guess I'm making separate subpackages for Intel and PPC. Extremely annoying but whatever. The universal binary is making dyld do something really weird.

 

[Wowfunhappy]

Take II. I feel better about this one, because I was able to run it under Rosetta (albeit with awful performance).

Curious to know if you have to restart after installing for Squid to work. I did, with Rosetta—which is odd, because I have a script which should make it unnecessary, and it does work on Intel. But if a restart is required, I'll want to force it in the installer.

Alternately, if everything works perfectly, I'll make a new, proper thread some time tomorrow. Off to ice skating for the evening!

 

[micahgartman]

Curious to know if you have to restart after installing for Squid to work.

Didn't work before and after restarting ¯\_(ツ)_/¯

 

[Wowfunhappy]

:/

Well if it still didn’t work at all I’m not sure what to do, given that it ran in Rosetta.