Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
The bug I'm encountering is that stock XCode LD fails to allocate enough memory to link the WebCore component of TenFourKit. The error message indicates that this is happenening at precisely the 3 GB mark. This is despite there still being plenty of free physical RAM (and virtual memory).

So, I'm assuming this may be a Mac OS X 32-bit memory space limitation. Even though the maximum addressable space for 32-bit should be 4 GB, there is often a general memory limit for programs before this. For instance, in 32-bit Windows programs, such a memory limit occurs at 2 GB for most user-land programs, except those few which are compiled to be "Large address aware".

Since Tiger is supposed to support 64-bit user-land address space and I'm compiling on a G5, I'm assuming the problem is with LD being 32-bit. However, I'm only in the early stages and it could just as easily be a 32-bit dynamic library it's using that's causing the problem or even something completely different. Nonetheless, my current plan is to try and get a 64-bit version of LD working to see if that may fix the problem or improve my workflow.

As for GCC-4.2, I did try to install apple-gcc42 +gpl3 with MacPorts a couple of months ago, however, MacPorts failed to find one of the patches in the repositories. I may try apple-gcc-40 or tinkering around with trying to manually compile apple-gcc-42, but for now I've just been lazy and stuck with the stock version of GCC in Xcode.

BTW, why would you use Tiger on a G5? I mean, “because we can” is still an acceptable answer, but from practical point of view this seems like a lot of unneeded pain with zero benefit (older SDK, a lot of stuff broken, perhaps beyond repair, ppc64 support is incomplete, default toolchain is more or less useless, so one has to bootstrap cctools to build anything at all, commercial software of that period also supported 10.5 better and in newer versions, finally with 10.4 you are on your own, mostly: GCC upstream, while not refusing to support 10.4, stopped testing on it a while ago, AFAIK, MacPorts is dropping 10.4, I gave up on 10.4 long back as well).
 
BTW, why would you use Tiger on a G5? I mean, “because we can” is still an acceptable answer, but from practical point of view this seems like a lot of unneeded pain with zero benefit (older SDK, a lot of stuff broken, perhaps beyond repair, ppc64 support is incomplete, default toolchain is more or less useless, so one has to bootstrap cctools to build anything at all, commercial software of that period also supported 10.5 better and in newer versions, finally with 10.4 you are on your own, mostly: GCC upstream, while not refusing to support 10.4, stopped testing on it a while ago, AFAIK, MacPorts is dropping 10.4, I gave up on 10.4 long back as well).

The main reason has more to do with the reasons I use my PowerPC computer. Except for browsing the web and occasionally watching a video or MS Office, most of the programs I run on my PowerPC G5 are Classic Mac OS games.

While I may be able to dual-boot OS 9 (due to the wonderful work from those who got OS9 to work on G5's) as an alternative, it is still reasonably convenient for me to run Classic Environment in Tiger, since it can handle most of the Mac OS games I want to play and still run the main OS X programs I use on the side without needing to reboot.

It would be a shame to completely drop Tiger support, since MacPorts is a wonderful tool and Tiger is the last OS X to support Classic Environment. If there is a way to back-up or archive all of the port files / distfiles / patches that work for OS X 10.4 to ensure they never disappear, for the few of us who still use Tiger, please let me know.
 
Last edited:
  • Like
Reactions: ChrisCharman
By the way, I have some insight into the problem you are having with llvm and ppc64 and may be able to help. Those errors look like the kind of assembler errors that occur whenever the compiler generates faulty assembly code (i.e. from standard input rather than a file). However, the output log on MacPorts does not display which command is causing those errors to occur.

If you can track down the command that is running in the Makefile just before the series of 'standard input' errors, that would be very helpful. And if it is GCC, then please consider adding "-S" and changing the "-o" destination to something like "-o ~/Desktop/buggy_output.s" and attaching it, so that the assembly code can be examined to figure out what the errors are and what is causing them.
 
The main reason has more to do with the reasons I use my PowerPC computer. Except for browsing the web and occasionally watching a video or MS Office, most of the programs I run on my PowerPC G5 are Classic Mac OS games.

While I may be able to dual-boot OS 9 (due to the wonderful work from those who got OS9 to work on G5's) as an alternative, it is still reasonably convenient for me to run Classic Environment in Tiger, since it can handle most of the Mac OS games I want to play and still run the main OS X programs I use on the side without needing to reboot.

It would be a shame to completely drop Tiger support, since MacPorts is a wonderful tool and Tiger is the last OS X to support Classic Environment. If there is a way to back-up or archive all of the port files / distfiles / patches that work for OS X 10.4 to ensure they never disappear, for the few of us who still use Tiger, please let me know.
There are so many amazing legacy pieces of software lost to time. It’s easy to forget that newer doesn’t always mean better.

You would have to volunteer to maintain ports for tiger or fork macports yourself and maintain it that way as very few if any developers have the time or inclination to support even Leopard these days. I think there is or was a tigerbrew or tiger macports something that exists or existed and there are alternatives to MacPorts and Homebrew but you’d have to research and test this yourself, and probably be prepared to fix a lot of things without support.
 
  • Like
Reactions: GA204
Wholeheartedly agree!

I'm certainly happy to join the Tiger and PowerPC cause where I can. It may be more than I handle to be the sole maintainer for Tiger MacPorts, but I would be happy to rsync and fork a Tiger variant that goes along at its own much slower pace if that's the best route going forward, or join-in with other developers who are willing to continue to back-port software to Tiger. And of course, I'm more than happy to lend a hand to the Leopard-side where I can as well.
 
  • Like
Reactions: ChrisCharman
It would be a shame to completely drop Tiger support, since MacPorts is a wonderful tool and Tiger is the last OS X to support Classic Environment. If there is a way to back-up or archive all of the port files / distfiles / patches that work for OS X 10.4 to ensure they never disappear, for the few of us who still use Tiger, please let me know.

To address this point: I have no control over choices MacPorts upstream makes and stopped committing there since the beginning of this year. The best thing you could do is respond in the related thread on MacPorts Developer mailing list or/and open a ticket on Trac to express interest in having 10.4 supported.
In my opinion, MacPorts in a sense of its upstream stopped supporting legacy systems a while ago (with an exception of legacy-support library), so it is rather a matter of [not] actively deleting what already exists in the base and ports tree. Breakages will keep accumulating, since no one tests ports, unless someone begins building stuff from every update and submitting fixes. (10.5 is still supported, in principle, and most of problems will probably be common for all three systems anyway.)
You may get working ports for powerpc from my fork, but no Tiger there, sorry.

To make a backup, fork MacPorts GH repo and “freeze” (or revert) whatever you may need to the versions compatible with 10.4. There are no pre-built ports for 10.4, so nothing to back up there. I am not sure about distfiles, but in most cases they are sourced externally anyway. You could try rsync from MacPorts archives; since they are accessible publicly, that may work.
 
  • Like
Reactions: ChrisCharman
Wholeheartedly agree!

I'm certainly happy to join the Tiger and PowerPC cause where I can. It may be more than I handle to be the sole maintainer for Tiger MacPorts, but I would be happy to rsync and fork a Tiger variant that goes along at its own much slower pace if that's the best route going forward, or join-in with other developers who are willing to continue to back-port software to Tiger. And of course, I'm more than happy to lend a hand to the Leopard-side where I can as well.

You could take a look at what I have here: https://github.com/macos-powerpc/powerpc-ports
Notice, this is an overlay, so it contains only what is different from MacPorts upstream. Some of the stuff may be helpful for Tiger (since 10.6.8 also uses libstdc++, also has pretty old system toolchain and SDK), though not everything is guaranteed to be compatible.
If you will find it sensible and convenient, I am fine with someone submitting fixes for 10.4 to my fork (as long as that does not break builds on 10.6). Otherwise make yours or cooperate with someone else specializing on Tiger.
 
There are so many amazing legacy pieces of software lost to time. It’s easy to forget that newer doesn’t always mean better.

You would have to volunteer to maintain ports for tiger or fork macports yourself and maintain it that way as very few if any developers have the time or inclination to support even Leopard these days. I think there is or was a tigerbrew or tiger macports something that exists or existed and there are alternatives to MacPorts and Homebrew but you’d have to research and test this yourself, and probably be prepared to fix a lot of things without support.

TigerBrew is alive, I guess, and I had a brief interaction with their maintainers: https://github.com/mistydemeo/tigerbrew/issues/1056

NetBSD’s `pkgsrc` has (had?) some minimal support for Tiger. Here is what they got pre-built (not much): https://ftp.netbsd.org/pub/pkgsrc/misc/nia/tigersrc/packages/All
But that could be a starting point.

There was some discussion with Fink upstream regarding powerpc support too: https://github.com/fink/fink/issues/270
 
By the way, I have some insight into the problem you are having with llvm and ppc64 and may be able to help. Those errors look like the kind of assembler errors that occur whenever the compiler generates faulty assembly code (i.e. from standard input rather than a file). However, the output log on MacPorts does not display which command is causing those errors to occur.

If you can track down the command that is running in the Makefile just before the series of 'standard input' errors, that would be very helpful. And if it is GCC, then please consider adding "-S" and changing the "-o" destination to something like "-o ~/Desktop/buggy_output.s" and attaching it, so that the assembly code can be examined to figure out what the errors are and what is causing them.

Yes `standard input` stuff is usually assembler errors, but the ld error, at least from what I recall from building gcc, looked like `32-bit absolute address out of range` or similar, explicitly referring to bitness.
I will try to find time to give it another attempt.

(The problem is not to get ld64 built for ppc64, but getting gcc built for ppc64.)
 
  • Like
Reactions: ChrisCharman
Yes `standard input` stuff is usually assembler errors, but the ld error, at least from what I recall from building gcc, looked like `32-bit absolute address out of range` or similar, explicitly referring to bitness.
address out of range error reminds me of linking 68K classic Mac code. You have to reorder the objects so that those that call each other are within 16 bits or whatever but I think those were relative addresses. I'm not sure what a 32-bit absolute address would be - unless you have 2 or 4 GB of code or global data...
 
Well, I've made some progress - I've got LD 97 to build in PPC64 using only the stock GCC 4.0 and Xcode in Tiger, and it works!

However, it hasn't quite solved my vm_allocate issue, but I now know why. Apparently, to properly allocate memory in 64-bit address space, you need to use mach_vm_allocate instead of vm_allocate. But, there's only one direct reference, which is in the rebase.c code. So, the next task will be tracking down places where vm_allocate is used indirectly and determine whether it can be changed in the LD code itself or whether I need to recompile one ore more of the DYLD's as 64-bit to fix the out-of-address space bug.

As well, this may not be LD 127, but it's definitely a start and an available option for anyone who wants a 64-bit build right now. I can post my code, build instructions and/or patches once I determine how best to process with addressing the mach_vm_allocate issue.
 
Well, I've made some progress - I've got LD 97 to build in PPC64 using only the stock GCC 4.0 and Xcode in Tiger, and it works!

However, it hasn't quite solved my vm_allocate issue, but I now know why. Apparently, to properly allocate memory in 64-bit address space, you need to use mach_vm_allocate instead of vm_allocate. But, there's only one direct reference, which is in the rebase.c code. So, the next task will be tracking down places where vm_allocate is used indirectly and determine whether it can be changed in the LD code itself or whether I need to recompile one ore more of the DYLD's as 64-bit to fix the out-of-address space bug.

As well, this may not be LD 127, but it's definitely a start and an available option for anyone who wants a 64-bit build right now. I can post my code, build instructions and/or patches once I determine how best to process with addressing the mach_vm_allocate issue.

You may wish to discuss the matter with Iain, since he worked on ld64 specifically and has a much newer ld64 for PowerPC (corresponding to Xcode 9 or 10, as I recall).
ld64 127 was a step back by the way, it has more bugs, especially for ppc64.
 
Attached is the working version of LD64-97.17 (source code + PPC64 binary with PPC32/PPC64 libcrypto dylib already built for convenience). I've labelled it "Version A" to differentiate it from future revisions.

Please let me know if you encounter any bugs in LD (or typos in my post) and I will be happy to try and correct them. Below are my build instructions, but TL;DR - If anyone reading this wants to skip ahead and use LD +ppc64, I've included pre-compiled binaries for convenience, which you can simply use by unzipping the archive and then copying ld to /usr/local/bin and libcrypto-64.dylib to /usr/local/lib.

(and if you want to use my binaries but want to rename them or don't want to put them in /usr/local, you may need to run install_name_tool on libcrypto - e.g. install_name_tool -id /opt/local/lib/libcrypto-64.dylib libcrypto-64.dylib ; install_name_tool -change /usr/local/libcrypto-64.dylib /opt/local/lib/libcrypto-64.dylib ld-ppc64 - where '/opt/local/lib/libcrypto-64.dylib' is whatever name or path you want to move libcrypto).



0. Perl 5.10 or later (Tiger-only pre-requisite)
Before you can install OpenSSL 1.1.1w on Tiger, you need to update Perl to at least 5.10. I'm personally using 5.40 and provide instructions below for those who are also using Tiger:

Bash:
# Download Perl 5.40 or use wget as below:
wget https://www.cpan.org/src/5.0/perl-5.40.1.tar.gz .
tar -zxvf https://www.cpan.org/src/5.0/perl-5.40.1.tar.gz
cd perl-5.40.1

########
# Then patch cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Darwin.pm by removing the following line:
#       $self->{CCFLAGS} .= " -Wno-error=implicit-function-declaration";
# Then patch dist/ExtUtils-CBuilder/lib/ExtUtils/CBuilder/Platform/darwin.pm by removing the following line:
#       $cf->{ccflags} .= ($cf->{ccflags} ? ' ' : '').'-Wno-error=implicit-function-declaration';
########

./Configure -des -Dprefix=/usr/local/
make && make test
sudo make install


1. OpenSSL 1.1.1w pre-requisite
To build LD 97, you need a working PPC64 version of libcrypto.

To compile yourself, you need to first download the source code for OpenSSL 1.1.1w and then apply mpsuzuki's workarounds for OS X 10.3-10.5 - https://github.com/openssl/openssl/issues/19684 , as well as one minor fix to the test suite, described below:

One of the OpenSSL tests which fails on PPC64 (../test/recipes/04-test_bioprint.t) since it uses the 32-bit SIZE_MAX variable in the system's openssl/bio.h instead of the one in its own architecture-independent SIZE_MAX in interals/numbers.h. To fix the test, you need to add the following lines to the file test/bioprinttest.c just before the line #include "internal/numbers.h":

C:
...
#ifdef SIZE_MAX
#undef SIZE_MAX
#endif
#include "internal/numbers.h"
...

Please also note ../test/recipes/80-test_ssl_old.t may fail on Tiger, since the test relies on compiling against your existing OpenSSL installation, which is on Tiger is only 32-bit PowerPC, and you are building 64-bit PowerPC code which can only link against 64-bit PowerPC (or FAT PowerPC) libraries.

To build OpenSSL 1.1.1w yourself, after making the above modifications, the commands I used from the Terminal in the openssl-1-1-1w-64 directory were:

Bash:
cd openssl-1.1.1w-64
./Configure darwin8-ppc64-cc --prefix=/usr/local --openssldir=/usr/local no-asm
make depend && make

(If you are building on Leopard, you may wish to change the prefixes and try "darwin64-ppc-cc" istead of "darwin8-ppc64-cc". You could try without the "no-asm" on Leopard, but I it definitely breaks most of the tests on Tiger if you don't use the "no-asm" for PPC64 builds.)

Now that you are done building, you could now run "sudo make install"; however, the libraries you have just built are only compatible with 64-bit PowerPC programs. If you wish to link or use the libraries with other programs (perhaps existing ones), then I would recommend instead building FAT OpenSSL libraries, which are compatible with both 32-bit and 64-bit PowerPC code. This is easy to do and can be accomplished with the following commands in the Terminal (assuming you are still in your openssl-1-1-1w-64 directory):

Bash:
cd ..
cp -rp openssl-1.1.1w-64 openssl-1.1.1w-32
cd openssl-1.1.1w-32/
make clean
./Configure darwin8-ppc-cc --prefix=/usr/local --openssldir=/usr/local no-asm
make depend && make
cd ..
lipo -create ./openssl-1.1.1w-32/libcrypto.1.1.dylib openssl-1.1.1w-64/libcrypto.1.1.dylib -output libcrypto.1.1.dylib

# Although only libcrypt is required by LD 97, if you want make the other OpenSSL libraries compatible with both PPC32 & PPC64, which is a good idea:
lipo -create openssl-1.1.1w-32/libssl.1.1.dylib openssl-1.1.1w-64/libssl.1.1.dylib -output libssl.1.1.dylib

cp libcrypto.1.1.dylib openssl-1.1.1w-64/
cp libssl.1.1.dylib openssl-1.1.1w-64/

# Engine DYLIB's
lipo -create openssl-1.1.1w-32/engines/capi.dylib openssl-1.1.1w-64/engines/capi.dylib -output capi.dylib
lipo -create openssl-1.1.1w-32/engines/dasync.dylib openssl-1.1.1w-64/engines/dasync.dylib -output dasync.dylib
lipo -create openssl-1.1.1w-32/engines/ossltest.dylib openssl-1.1.1w-64/engines/ossltest.dylib -output ossltest.dylib
lipo -create openssl-1.1.1w-32/engines/padlock.dylib openssl-1.1.1w-64/engines/padlock.dylib -output padlock.dylib

cp capi.dylib openssl-1.1.1w-64/engines/
cp dasync.dylib openssl-1.1.1w-64/engines/
cp ossltest.dylib openssl-1.1.1w-64/engines/
cp padlock.dylib openssl-1.1.1w-64/engines/

# Static libraries
lipo -create openssl-1.1.1w-32/libssl.a openssl-1.1.1w-64/libssl.a -output libssl.a
lipo -create openssl-1.1.1w-32/libcrypto.a openssl-1.1.1w-64/libcyrpto.a -output libcrypto.a
lipo -create openssl-1.1.1w-32/apps/libapps.a openssl-1.1.1w-64/apps/libapps.a -output libapps.a
lipo -create openssl-1.1.1w-32/test/libtestutil.a openssl-1.1.1w-64/test/libtestutil.a -output libtestutil.a

cp libssl.a openssl-1.1.1w-64/
cp libcrypto.a openssl-1.1.1w-64/
cp libapps.a openssl-1.1.1w-64/apps/
cp libtestutil.a openssl-1.1.1w-64/test/

cd openssl-1.1.1w-64/
sudo make install


2. CCTools, DYLD and LibUnwind headers
The ZIP file contains versions of these headers already patched using all of the MacPorts patches. I don't recall making any other changes, but if you want to be sure, you can always diff the include sub-directories (as these are the only ones used by LD 97). However, please note that I've only included the headers and not the full contents of the cctools, dyld and libunwind directories, just to cut down on space and not exceed the MacRumors file upload size limit. Please further note that the MacPorts distfiles no longer has version 5.0.2 of libunwind, but this can easily be found on LLVM's website (the patches and Portfile are still available at the time of this writing).


3. Compiling LD64-97.17
This is where I've made the majority of any changes. I started by applying all of the the LD 97 patches and modifications specified in the Portfile by hand. It needed a couple of minor changes to get it compiling on GCC 4.0 which can be diff'd. However, I discovered that one of the Portfile modifications breaks PPC64 (PPC32 needs some the registers specified as rax, rbx, ..., which PPC64 on Tiger needs specified as __rax, ...) So, I made the code conditionally compile one was for PPC32 and another for PPC64, using #ifdef _ARCH_PPC64.

I haven't tested this on Leopard, but if you notice any bugs, let me know, and I can use #if statements to ensure it compiles and works properly on both systems.

Accidentally along the way, I found a nasty surprise in the Makefile provided by MacPorts, where it uses PREFIX=/usr as the default. Hence, if you don't provide PREFIX for make install, it will clobber your installed /usr/bin/ld and related manpages. So, I changed the default PREFIX to /usr/local to prevent such mishaps for other people.

My build command for LD 97.17 was as follow:

Code:
gnumake PREFIX=/usr/local OTHER_LDFLAGS="-g -arch ppc64" OTHER_LDFLAGS_LD64="/usr/local/lib/libcrypto.1.1.dylib" OTHER_CFLAGS="-g -arch ppc64" OTHER_CXXFLAGS="-g -arch ppc64 -I../cctools-cctools-949.0.1/include -I../libunwind-5.0.2.src/include -I../dyld-dyld-655.1.1/include" && sudo gnumake install PREFIX=/usr/local

Note that in the above example, I've used libcrytpo.1.1.dylib; but, if you prefer to use the version I've already compiled (which I've named libcrypto-64.dylib to avoid potential conflicts with local installs and TigerBrew), just change the above line to:

Code:
gnumake PREFIX=/usr/local OTHER_LDFLAGS="-g -arch ppc64" OTHER_LDFLAGS_LD64="/usr/local/lib/libcrypto-64.dylib" OTHER_CFLAGS="-g -arch ppc64" OTHER_CXXFLAGS="-g -arch ppc64 -I../cctools-cctools-949.0.1/include -I../libunwind-5.0.2.src/include -I../dyld-dyld-655.1.1/include" && sudo gnumake install PREFIX=/usr/local


4. Note about LD64 97.17 on Tiger with GCC 4.0
While LD64 97.17 works well most of the time in the testing I've done (including TenFourKit's WebCore), it does fail with a Bus Error on one test, which is when I've tried to use LD64 91.17 to build itself (!).

However, this error occurs regardless of whether you use a 32-bit PowerPC binary or a 64-bit PowerPC binary version of LD64 97.17 to compile itself (nor whether you attempt to compile LD as 32-bit or 64-bit). So, the bug does not seem to have anything to do with PPC32 vs. PPC64.

As well, I have not tested it on Leopard nor other versions of GCC 4.0 so this may be Tiger or a GCC 4.0 specific bug.

From the cursory debugging I've done, this seems to be an issue in the LD source code where an "Atom" object is generated without creating an "fSection" variable. According to GDB, the crash occurs when the getAddress() method calls fSection->getBaseAddress() on the NULL fSection pointer.

I haven't quite tracked down where in the code this rogue "Atom" object is being generated, but the object LD is processing at the time of the crash is the debug symbol: ___keymgr_dwarf2_register_sections

Otherwise, LD seems to work well on other code I've tested, and it is certainly worth giving LD64 97.17 a try on Leopard (in case the bug is Tiger-only) or if you need to build something which would benefit from having the 64-bit address space (like WebKit).

In the meantime, I will continue to slowly search for an answer to the Atom bug, and of course would appreciate any advice if you may have some insight into the cause of the bug.
 

Attachments

  • 64-bit LD 97.17 - Version A.tar.gz.zip
    5.9 MB · Views: 6
Last edited:
1. OpenSSL 1.1.1w pre-requisite
To build LD 97, you need a working PPC64 version of libcrypto.

To compile yourself, you need to first download the source code for OpenSSL 1.1.1w and then apply mpsuzuki's workarounds for OS X 10.3-10.5 - https://github.com/openssl/openssl/issues/19684

This is a side point, but from the PR it follows that those fixes are only relevant for 10.3–10.4 (but not for 10.5).

Accidentally along the way, I found a nasty surprise in the Makefile provided by MacPorts, where it uses PREFIX=/usr as the default. Hence, if you don't provide PREFIX for make install, it will clobber your installed /usr/bin/ld and related manpages.

That is probably the default by Apple, not by MacPorts. Since MacPorts does set the correct prefix, there is no reason to patch Makefile in addition. (It should not be expected that ports will build correctly outside of MacPorts build system with no changes.)

OTHER_LDFLAGS_LD64="/usr/local/lib/libcrypto.1.1.dylib"

This is no more than my opinion, but FWIW: it is risky to use /usr/local for any libs or headers, since this path is often searched by default. This means that any build later on, whether using MacPorts or not, may opportunistically use those libs or headers without user being aware of that. In practice it may not happen very frequently, but it does happen: I had about a dozen incorrectly linked ports, because I had something installed into /usr/local/lib ages ago and forgot about that. There is a good reason why sane package management systems (MacPorts and pkgsrc) use a dedicated prefix for their stuff.
There may be good reasons to use system prefix in special circumstances, but better make users aware of potential consequences.
 
Last edited:
Bash:
cd openssl-1.1.1w-64
./Configure darwin8-ppc64-cc --prefix=/usr/local --openssldir=/usr/local no-asm
make depend && make

(If you are building on Leopard, you may wish to change the prefixes and try "darwin64-ppc-cc" istead of "darwin8-ppc64-cc". You could try without the "no-asm" on Leopard, but I it definitely breaks most of the tests on Tiger if you don't use the "no-asm" for PPC64 builds.)

Would you be interested in looking into how to fix that? There should not be much difference in assembler as such between 10.4 and 10.5 (no ppc64 in 10.6, so that is off).
I can look into the matter from my side too, though perhaps in about two weeks, since I won’t have access to my hardware from tomorrow for a while.
OpenSSL upstream is very helpful in my experience, and as long as needed fixes are not massive, chances are high it can be merged to the master and perhaps backported to other branches.

P. S. It will be easier to do this on GitHub, perhaps. Please feel free to tag me in your issue (with OpenSSL or your repo) or open an issue in mine.
 
Would you be interested in looking into how to fix that? There should not be much difference in assembler as such between 10.4 and 10.5 (no ppc64 in 10.6, so that is off).
I can look into the matter from my side too, though perhaps in about two weeks, since I won’t have access to my hardware from tomorrow for a while.
OpenSSL upstream is very helpful in my experience, and as long as needed fixes are not massive, chances are high it can be merged to the master and perhaps backported to other branches.

P. S. It will be easier to do this on GitHub, perhaps. Please feel free to tag me in your issue (with OpenSSL or your repo) or open an issue in mine.

Absolutely, and I've made some progress!

I've tracked down and fixed the main bug which caused a bunch of tests to fail with Segmentation Faults. The Perl script crypto/aes/asm/vpaes-ppc.pl was generating assembler code which long branched to the wrong location since it was referencing a local label (Lconsts) in outside of the function in three locations. This is undefined behaviour and easy to fix.

Since Lconsts is the last local label under vpaes_consts, I've added a separate non-local label just after, and changed all of the long branches to point to it. It is worth noting that Lconsts also marks the end of the vpaes_consts section for the Perl script, hence why I chose to add another label rather than rename it.

Attached is my unified diff file, which I intend to submit to the OpenSSL team.

However, there is still one last test that fails due to a different assembler bug, which I am still in the process of tracking down. The affected test is ../test/recipes/15-test_ec.t which runs the executable file test/ectest, whereby the program crashes during iteration 19, at the line ecp_nistz256_point_add+584: "ld r6,64(r17)" because r17 incorrectly gets set to the value 0x2. The stack trace is as follows:

#0 0x00000000004ea248 in ecp_nistz256_point_add ()
#1 0x00000000004ea248 in ecp_nistz256_point_add ()
#2 0x00000000004ac9a4 in EC_POINTs_mul ()
#3 0x0000000000002560 in group_order_tests ()
#4 0x0000000000007cb8 in internal_curve_test_method ()
#5 0x000000000000c488 in run_tests ()
#6 0x000000000000c684 in main ()

Once this last bug is fixed, OpenSSL should hopefully and likely pass all of its tests with the PowerPC 64-bit assembler optimizations enabled.
 

Attachments

  • aes_Lconst_asm_fix.patch.txt
    1 KB · Views: 5
Absolutely, and I've made some progress!

I've tracked down and fixed the main bug which caused a bunch of tests to fail with Segmentation Faults. The Perl script crypto/aes/asm/vpaes-ppc.pl was generating assembler code which long branched to the wrong location since it was referencing a local label (Lconsts) in outside of the function in three locations. This is undefined behaviour and easy to fix.

Since Lconsts is the last local label under vpaes_consts, I've added a separate non-local label just after, and changed all of the long branches to point to it. It is worth noting that Lconsts also marks the end of the vpaes_consts section for the Perl script, hence why I chose to add another label rather than rename it.

Attached is my unified diff file, which I intend to submit to the OpenSSL team.

However, there is still one last test that fails due to a different assembler bug, which I am still in the process of tracking down. The affected test is ../test/recipes/15-test_ec.t which runs the executable file test/ectest, whereby the program crashes during iteration 19, at the line ecp_nistz256_point_add+584: "ld r6,64(r17)" because r17 incorrectly gets set to the value 0x2. The stack trace is as follows:

#0 0x00000000004ea248 in ecp_nistz256_point_add ()
#1 0x00000000004ea248 in ecp_nistz256_point_add ()
#2 0x00000000004ac9a4 in EC_POINTs_mul ()
#3 0x0000000000002560 in group_order_tests ()
#4 0x0000000000007cb8 in internal_curve_test_method ()
#5 0x000000000000c488 in run_tests ()
#6 0x000000000000c684 in main ()

Once this last bug is fixed, OpenSSL should hopefully and likely pass all of its tests with the PowerPC 64-bit assembler optimizations enabled.

Awesome!
Do all 32-bit ppc tests pass by the way? I.e. the issue we have only affects ppc64?
(Perhaps you have seen, but just in case: not too long ago OpenSSL added some VSX code, which cannot compile on pre-ISA 2.06 cpus (i.e. all PowerPC), and while supposedly it should not be used either on ppc or ppc64 on Darwin, that could still be a source of some issues, potentially.)
 
For PPC32 with assembler optimizations enabled, all of the tests (including ../test/recipes/15-test_ec.t) pass except for ../test/recipes/80-test_ssl_old.t which fails on the IPv6 sub-test. This test passes when I've compiled it in the past with assembler optimizations turned off. So, it seems there is another bug to fix.

Regarding ../test/recipes/15-test_ec.t, it seems that the Bus Error bug affects only PPC64.

Considering what you are saying about recent changes to the code, I should mention that I'm working with OpenSSL 1.1.1w at the moment and not the main branch. Nonetheless, the bugs I'm finding still exist in the main branch code. Plus, if we can get a working version of the assembler code in OpenSSL 1.1.1w, then it should be easy to use that as a template to fix or conditionally patch the assembler optimization code in the main branch to get it also working with both 32-bit and 64-bit PowerPC on OS X.
 
For PPC32 with assembler optimizations enabled, all of the tests (including ../test/recipes/15-test_ec.t) pass except for ../test/recipes/80-test_ssl_old.t which fails on the IPv6 sub-test. This test passes when I've compiled it in the past with assembler optimizations turned off. So, it seems there is another bug to fix.

Regarding ../test/recipes/15-test_ec.t, it seems that the Bus Error bug affects only PPC64.

Considering what you are saying about recent changes to the code, I should mention that I'm working with OpenSSL 1.1.1w at the moment and not the main branch. Nonetheless, the bugs I'm finding still exist in the main branch code. Plus, if we can get a working version of the assembler code in OpenSSL 1.1.1w, then it should be easy to use that as a template to fix or conditionally patch the assembler optimization code in the main branch to get it also working with both 32-bit and 64-bit PowerPC on OS X.

BTW, how long does it take to run the test suite, roughly? (For the reference, to distinguish between something actually taking time to complete and being frozen.)
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.