llvm and ld64 for ppc64 on Leopard

[barracuda156]

I dunno if this info is gonna be of value for anyone, but just in case: if you want to build software on Leopard for ppc64 with Macports or otherwise, you will likely bump into a problem with llvm, which is required for ld64 (native Xcode one won't work for new stuff) and has to be of the same architecture (so you can't just skip arch-check).

llvm-3.3 and llvm-3.4 fail to build for ppc+ppc64 and even just ppc64. Anyone interested, see: https://trac.macports.org/ticket/64760

#64253 (llvm-3.3 and llvm-3.4 fail for ppc+ppc64) – MacPorts

trac.macports.org

I spend several days on that, went past ranlib errors, but it still failed.

However what I found is that llvm-3.7 (and also llvm-3.8) builds for ppc64 easily with no tweaks to settings or code. Then it allows to rebuild ld64-97 as +llvm37 (again, a standard variant, no need to tweak port file) and then rebuild ld64 for ppc64. In result you get ppc64 linker.

The caveat: llvm-3.7 does not build with gcc-4.2. So you will need to first build llvm-3.4 as ppc32, then ld64 as ppc32, then build gcc6 or gcc7 as universal, then build llvm-3.7 as ppc64, then rebuild ld64.
(Or if you know a better way, please advise me.)

 

[alex_free]

Somewhat relevant, I just built gcc6 on Intel Tiger for my new SM64EXCreator.app without any issues. Last time I did this maybe 7 months ago for PowerPC it just worked so something changed since then.

 

[barracuda156]

Somewhat relevant, I just built gcc6 on Intel Tiger for my new SM64EXCreator.app without any issues. Last time I did this maybe 7 months ago for PowerPC it just worked so something changed since then.

I don't think there is much use of having llvm and ld64 built as ppc64 on Tiger tbh. Any machine capable of running ppc64 binaries will be better off with Leopard.

 

[alex_free]

I don't think there is much use of having llvm and ld64 built as ppc64 on Tiger tbh. Any machine capable of running ppc64 binaries will be better off with Leopard.

Kencu recently merged his tigerports and leopardports forks into macports AFAIK which is super nice!

 

[barracuda156]

Kencu recently merged his tigerports and leopardports forks into macports AFAIK which is super nice!

From what I understood he was asked to remove them. I don't think his fixed were merged, at least not all of those.
See: https://trac.macports.org/ticket/64655#comment:3

 

[barracuda156]

Quick update: llvm-5.0 builds for ppc64, but portfile lacks universal variant. I will try to add it.

P. S. Looks like llvm-3.7 and llvm-3.8 are in the same boat: they are built for ppc64, but universal missing. Apparently it just got ignored.

 

[unilock]

Did you get anywhere with this? I'm trying to compile TenFourFox on Leopard 10.5.8 (ppc64), but I run into the same Failed to build llvm-3.3: command execution failed.

I'm following the instructions exactly as described at HowToBuildFPR, so it's possible I'm doing something very wrong (as that's written for Tiger 10.4, asks for GCC 4.8, etc.)

(I'm actually trying to build ccache first, but that also requires llvm-3.3.)

 

[barracuda156]

Did you get anywhere with this? I'm trying to compile TenFourFox on Leopard 10.5.8 (ppc64), but I run into the same Failed to build llvm-3.3: command execution failed`.

Honestly I forgot and cannot check atm (away from my PPC hardware for a couple of weeks to come). You could try adding this to portfile and see if it works:

PortGroup muniversal 1.0

You may also try muniversal 1.1, if the first one fails.

Keep in mind, you can always open a ticket for anything Leopard-related at: https://trac.macports.org (check earlier opened tickets first).

However, according to TFF developer, ppc64 build is not supported. So I am unsure if it is even feasible. You may try asking him on GitHub (tag me too in such a case, @barracuda156).
From what I understand, you would also need to build gcc48 (yeah, an ancient one), which again may not build as universal (but likely will build as ppc64).

 

[unilock]

(Sorry, I edited my reply immediately after posting it, as I hit "Post reply" too early )

I'm keeping track of the ticket you've already filed: https://trac.macports.org/ticket/64253

What does PortGroup muniversal 1.0 do? Would I put it in the llvm-3.3 portfile specifically? (I've rarely touched MacPorts until now.)

 

[barracuda156]

(Sorry, I edited my reply immediately after posting it, as I hit "Post reply" too early )

I'm keeping track of the ticket you've already filed: https://trac.macports.org/ticket/64253

What does PortGroup muniversal 1.0 do? Would I put it in the llvm-3.3 portfile specifically? (I've rarely touched MacPorts before now.)

muniversal PG does magic to build universal ports LOL

Generally speaking, you would require it in cases when non-Apple gcc is used for a build. (Because only Apple gcc42 can take multiple arch flags at once, any later gcc needs to build archs separately and the lipo them together.)

Check some portfile examples, where muniversal is been used. You just add that line to portfile heading, usually that’s enough (though not always, in some cases more fancy code is needed to be added).

To be clear, I am not sure you can build llvm33 as universal. Please update us here, and if it fails, we can try doing something about it. I wanna add also that the latest revision on llvm5–llvm7 changed something which makes PPC build fail (I did not try fixing that yet, but at least a standard procedure failed for me). You may want to go to GitHub history and pick an earlier portfile version (from 2021). Those ones have built for me with no issues.

What you will need:
1. Open portfile and comment out/remove libcxx part for darwin9. (It will fail to build, as of now, though I have a fix for libcxx for PPC, but it is untested.)
2. Either add ppc/ppc64 into enabled archs in portfile or use a command line like: sudo port -v install llvm-5.0 supported_archs="ppc ppc64" build_arch="ppc64" configure.compiler=macports-gcc-10. You need at least gcc6 to build those, itself built as universal (!).

 

[unilock]

muniversal PG does magic to build universal ports LOL

Generally speaking, you would require it in cases when non-Apple gcc is used for a build. (Because only Apple gcc42 can take multiple arch flags at once, any later gcc needs to build archs separately and the lipo them together.)

Check some portfile examples, where muniversal is been used. You just add that line to portfile heading, usually that’s enough (though not always, in some cases more fancy code is needed to be added).

To be clear, I am not sure you can build llvm33 as universal. Please update us here, and if it fails, we can try doing something about it. I wanna add also that the latest revision on llvm5–llvm7 changed something which makes PPC build fail (I did not try fixing that yet, but at least a standard procedure failed for me). You may want to go to GitHub history and pick an earlier portfile version (from 2021). Those ones have built for me with no issues.

What you will need:
1. Open portfile and comment out/remove libcxx part for darwin9. (It will fail to build, as of now, though I have a fix for libcxx for PPC, but it is untested.)
2. Either add ppc/ppc64 into enabled archs in portfile or use a command line like: sudo port -v install llvm-5.0 supported_archs="ppc ppc64" build_arch="ppc64" configure.compiler=macports-gcc-10. You need at least gcc6 to build those, itself built as universal (!).

Ah, I don't think I'm building llvm-3.3 as universal - at least, I haven't explicitly specified the +universal option. (Actually, that's going to cause issues when building 104fx for PPC < G5, won't it...)

Here are the changes I've made to the default macports.conf, if at all relevant:

build_arch = ppc64
buildfromsource = always
configureccache = yes
universal_archs = ppc ppc64

(I'm not able to get the full file at the moment.)

You mention building with a compiler other than Apple's - might the problem be that I don't have any compiler other than Apple's gcc-4.2 installed?
On that note, do you know whether 104fx compile with a compiler other than gcc-4.8 and/or a version of llvm other than 3.3?

 

[barracuda156]

Ah, I don't think I'm building llvm-3.3 as universal - at least, I haven't explicitly specified the +universal option. (Actually, that's going to cause issues when building 104fx for PPC < G5, won't it...)

Here are the changes I've made to the default macports.conf, if at all relevant:

build_arch = ppc64
buildfromsource = always
configureccache = yes
universal_archs = ppc ppc64

(I'm not able to get the full file at the moment.)

You mention building with a compiler other than Apple's - might the problem be that I don't have any compiler other than Apple's gcc-4.2 installed?
On that note, do you know whether 104fx compile with a compiler other than gcc-4.8 and/or a version of llvm other than 3.3?

1. I cannot say much re TFF, since I have not yet fixed its portfile for PPC. Here is an Intel version from @kencu: https://github.com/kencu/tenfourfox-macports
From what I know, it does require a specific version of gcc (4.8), and newer ones won’t build it. Not tested personally though.

2. If you don’t have a G5 machine, don’t build for ppc64, G4 cannot run these.

3. I would recommend to build everything for ppc (ppc32) first, since some stuff is/may be broken for ppc64 (including TFF itself). ppc32 should be smooth, you can just go with sudo port -v install, Macports gonna take care of the rest.

4. Having anything as universal won’t create problems. If a) you have G5 machine and b) ports in question do build as universal, it is better to build them as universal, since then you can build dependencies for either arch or as universal too. Having said that, you may have a hard time building everything as universal. In particular, Pythons are broken (I fixed them, but it is not in Macports), LLVM will be hard, many ports require some manual adjustments.
I also have no idea if gcc48 builds as universal, you will have to experiment here.

5. I don’t think you need this: configureccache = yes, unless you added that for a reason.

6. For compilers, in a case of Leopard you should be generally good using a default selection. If a port does not ask for a specific gcc, then it gonna build with an Apple one. This is not always true for building universal.
However, llvm-5.0 is not officially supported on Leopard, as well as gcc10–gcc11, so to get these you do need manual tweaks.

 

[unilock]

Sorry for the long absence. Life caught up with me.

After not touching my Power Mac G5 running Mac OS X 10.5.8 for about 2 years, I'm trying to get MacPorts to compile stuff for ppc64 again. Unfortunately, I'm still running into this issue with llvm-3.3 not building successfully.

Forgive my ignorance, but what should my next steps be? Can I only compile software for ppc32 due to this issue?
Ideally I'd like to get everything working with ppc64, since I'm under the impression that there are performance benefits there. Though I'm not sure how true that is ;p

@barracuda156

 

[barracuda156]

Sorry for the long absence. Life caught up with me.

After not touching my Power Mac G5 running Mac OS X 10.5.8 for about 2 years, I'm trying to get MacPorts to compile stuff for ppc64 again. Unfortunately, I'm still running into this issue with llvm-3.3 not building successfully.

Forgive my ignorance, but what should my next steps be? Can I only compile software for ppc32 due to this issue?
Ideally I'd like to get everything working with ppc64, since I'm under the impression that there are performance benefits there. Though I'm not sure how true that is ;p

@barracuda156

Well, I have been pointing out that issue for quite a while, but nobody in MacPorts cares and I could not quickly fix it, and no time to dive deep into this (

I do not have a solution which will work for sure. However, what I tried to do was not a default setup, but getting gcc14 to build for ppc64. So going with gcc7 for the time-being might work, but ld64 port needs a change (AFAIR, it is ld64 which pulls in llvm).

What can be tried:
1. Install ld64-97 +xcode (custom variant, which symlinks Xcode ld).
2. Install ld64-97 and ld64 for ppc (32-bit), use it to link ppc64 stuff. Here you will need to add to portfiles an argument to prevent checking an arch of ld64 (see gcc10-bootstrap or clang-11-bootstrap portfiles as example). And install like this: sudo port -v -n install ld64-97 build.arch="ppc".
3. You can build llvm for ppc64, but you need at least llvm5, which needs C++11 compiler. What worked for me was to build gcc10-bootstrap as +universal (ppc+ppc64), then use that to build llvm5 or llvm7 (universal or just ppc64). Then rebuild ld64 against the new llvm.
Rebuilding can be done like:
sudo port -v -n upgrade —enforce-variants ld64-97 -xcode +llvm5 build.arch="ppc64" (as an example).
This is a long pathway, but it works. What failed for me was to build gcc14 for ppc64 (with any version of ld and cctools which I tried).

 

[barracuda156]

Sorry for the long absence. Life caught up with me.

After not touching my Power Mac G5 running Mac OS X 10.5.8 for about 2 years, I'm trying to get MacPorts to compile stuff for ppc64 again. Unfortunately, I'm still running into this issue with llvm-3.3 not building successfully.

Forgive my ignorance, but what should my next steps be? Can I only compile software for ppc32 due to this issue?
Ideally I'd like to get everything working with ppc64, since I'm under the impression that there are performance benefits there. Though I'm not sure how true that is ;p

@barracuda156

P. S. By the way, it may be useful to open a ticket on Trac (or write in an existing opened one) on the matter, so at least it is obvious that not only myself wants this fixed but some other ppl too.

 

[unilock]

1. Install ld64-97 +xcode (custom variant, which symlinks Xcode ld).

I don't see an +xcode variant of ld64-97...? (https://ports.macports.org/port/ld64-97/details/)

2. Install ld64-97 and ld64 for ppc (32-bit), use it to link ppc64 stuff. Here you will need to add to portfiles an argument to prevent checking an arch of ld64 (see gcc10-bootstrap or clang-11-bootstrap portfiles as example).

So I'd have to edit the portfile of every (ppc64) port I wish to install? Sounds unwieldy...

3. You can build llvm for ppc64, but you need at least llvm5, which needs C++11 compiler. What worked for me was to build gcc10-bootstrap as +universal (ppc+ppc64), then use that to build llvm5 or llvm7 (universal or just ppc64). Then rebuild ld64 against the new llvm.

So... the steps there would be:

1. sudo port install gcc10-boostrap +universal
2. (edit llvm-5.0 portfile to use gcc10-bootstrap)
3. sudo port install llvm-5.0
4. sudo port install ld64-97 +llvm50

...correct?

P. S. By the way, it may be useful to open a ticket on Trac (or write in an existing opened one) on the matter, so at least it is obvious that not only myself wants this fixed but some other ppl too.

I left a comment on https://trac.macports.org/ticket/64253 ^u^

 

[barracuda156]

I don't see an +xcode variant of ld64-97...? (https://ports.macports.org/port/ld64-97/details/)



So I'd have to edit the portfile of every (ppc64) port I wish to install? Sounds unwieldy...



So... the steps there would be:

1. sudo port install gcc10-boostrap +universal
2. (edit llvm-5.0 portfile to use gcc10-bootstrap)
3. sudo port install llvm-5.0
4. sudo port install ld64-97 +llvm50

...correct?


I left a comment on https://trac.macports.org/ticket/64253 ^u^

Sorry, it is a separate port: https://ports.macports.org/port/ld64-xcode
So, install that, then install/rebuild ld64 with +ld64_xcode (yes, the variant has underscore): https://ports.macports.org/port/ld64/details

For llvm, well, I haven’t built llvm-5.0 in ages, but it will probably build.
Take this: https://github.com/barracuda156/pow...53134345f1/lang/llvm-powerpc/Portfile#L52-L72
Use it with llvm-5.0.

Or use my llvm-powerpc (= 7.1) with corresponding changes to ld64 and cctools.

powerpc-ports/devel/cctools at b304ec96c0e7a8557fa0cf9b867b7353134345f1 · barracuda156/powerpc-ports

The MacPorts ports tree for macOS PowerPC [Unofficial] - barracuda156/powerpc-ports

github.com

powerpc-ports/devel/ld64 at b304ec96c0e7a8557fa0cf9b867b7353134345f1 · barracuda156/powerpc-ports

The MacPorts ports tree for macOS PowerPC [Unofficial] - barracuda156/powerpc-ports

github.com

powerpc-ports/lang/llvm-powerpc at b304ec96c0e7a8557fa0cf9b867b7353134345f1 · barracuda156/powerpc-ports

The MacPorts ports tree for macOS PowerPC [Unofficial] - barracuda156/powerpc-ports

github.com

Modify as needed, this is pretty much experimental stuff, though I use these as my defaults on 10.6 ppc.

P. S. To build LLVM for ppc+ppc64 you will also need to add `PortGroup muniversal 1.0`. Notice, this has not been tested for my ports at all, since I built it either for ppc (on 10.6) or for ppc64 (on 10.5).

 

[unilock]

@barracuda156 The following seems to have gotten ld64 installed!

sudo port install ld64-xcode
sudo port install ld64 +ld64_xcode

However, I'm now running into an error when trying to build libgcc7 (a dependency of something else):

ld: bl out of range (-17201188 max is +/-16M) from tree_function_versioning...
collect2: ld returned exit 1 status

Do I still need to install ld64-97 specifically?

EDIT: By the way, I don't really have any intention of building +universal packages (unless necessary); I only really want ppc64 working at the end of the day.

 

[barracuda156]

@barracuda156 The following seems to have gotten ld64 installed!

sudo port install ld64-xcode
sudo port install ld64 +ld64_xcode

However, I'm now running into an error when trying to build libgcc7 (a dependency of something else):

ld: bl out of range (-17201188 max is +/-16M) from tree_function_versioning...
collect2: ld returned exit 1 status

Do I still need to install ld64-97 specifically?

EDIT: By the way, I don't really have any intention of building +universal packages (unless necessary); I only really want ppc64 working at the end of the day.

Well, that looks like errors I kept getting. See, for example: https://github.com/macports/macports-ports/pull/23402#issuecomment-2050591349

I think ld64-xcode never worked for me for ppc64 (though in theory it should) and the only ld64 which actually worked before was with llvm5. (I did not have llvm7 back then at all.)
The problem is that I was unable to reproduce the successful build of gcc13–14 for ppc64 recently.

It is interesting that gcc7 also fails now in a similar manner.

It looks like something has broken badly in MacPorts, but I do not know what. And I do not have a backup of my old 10.5 install where ppc64 worked.

No ready solution, sorry (I also would like to have it). What can be tried (with no guarantee):
– tweak gcc portfile to use Xcode tools directly instead of those from MacPorts prefix (there are settings in the portfile for that);
– use ld64-97 with standard llvm, built for ppc32 (I referred to that above);
– get llvm-powerpc built for ppc64, try make that work to build gcc (a similar procedure worked for me 2 years back but not now);
– just use gcc10-bootstrap as the main compiler (needs minor tweaks, likely gonna work, but resulting setup will be very different from what MacPorts assumes).

Bring attention to the issue, maybe try tagging kencu there on GitHub thread.

 

[barracuda156]

I dunno if this info is gonna be of value for anyone, but just in case: if you want to build software on Leopard for ppc64 with Macports or otherwise, you will likely bump into a problem with llvm, which is required for ld64 (native Xcode one won't work for new stuff) and has to be of the same architecture (so you can't just skip arch-check).

llvm-3.3 and llvm-3.4 fail to build for ppc+ppc64 and even just ppc64. Anyone interested, see: https://trac.macports.org/ticket/64760

#64253 (llvm-3.3 and llvm-3.4 fail for ppc+ppc64) – MacPorts

trac.macports.org

I spend several days on that, went past ranlib errors, but it still failed.

However what I found is that llvm-3.7 (and also llvm-3.8) builds for ppc64 easily with no tweaks to settings or code. Then it allows to rebuild ld64-97 as +llvm37 (again, a standard variant, no need to tweak port file) and then rebuild ld64 for ppc64. In result you get ppc64 linker.

The caveat: llvm-3.7 does not build with gcc-4.2. So you will need to first build llvm-3.4 as ppc32, then ld64 as ppc32, then build gcc6 or gcc7 as universal, then build llvm-3.7 as ppc64, then rebuild ld64.
(Or if you know a better way, please advise me.)

Okay, there is apparently a record of what worked for me back then (I switched to llvm5, but yes, I tried earlier ones). Perhaps try that.

 

[unilock]

I guess that would be:

1. sudo port install llvm-3.4 build.arch="ppc"
2. sudo port install ld64-97 build.arch="ppc"
3. sudo port install gcc7 +universal
4. sudo port install llvm-3.7
5. sudo port -n upgrade --enforce-variants ld64-97 build.arch="ppc64"

Correct?

 

[barracuda156]

I guess that would be:

1. sudo port install llvm-3.4 build.arch="ppc"
2. sudo port install ld64-97 build.arch="ppc"
3. sudo port install gcc7 +universal
4. sudo port install llvm-3.7
5. sudo port -n upgrade --enforce-variants ld64-97 build.arch="ppc64"

Correct?

I would do it a bit differently. Start with setting macports.conf to build for ppc and universal_archs to ppc ppc64. Then in variants.conf set +universal.

This way by default you get universal builds (I know you do need that, but you can disable it later). Have fundamental ports as universal, it doesn’t hurt.

Then install all dependencies of llvm34 and ld64-97 (aside of llvm34) as universal (everything should build up to that point).

Then install ld64 as
sudo port -v -n install ld64 -universal

That should install remaining stuff as ppc-only, leaving universal ones as-they-are.

Then try building gcc7 as
sudo port -v -n install gcc7

You may need a patch for python27 to build it for ppc+ppc64. I will look that up once I am back to the machine with Leopard.

llvm-3.7 or higher will pull in clang by default and perhaps do not support any ppc (I mean, per portfile). So you need to force a correct build. Also see a note below.

Your command for rebuilding ld64-97 does not enforce any variant by the way. `build_arch` is not a variant.

P. S. Going this way, you end up with ld64 depending on libgcc (via llvm-3.7). This is a circular dependency, which is a very fragile set-up. It will work as long as you don’t touch that, but it prone to failure.
I think it is safer and more correct to use gcc10-bootstrap to build llvm, since that won’t create a dependency cycle.

 

[unilock]

I managed to build gcc10-bootstrap for ppc64 overnight, but it doesn't look like I can use it to compile llvm-7.0 directly, since it (or one of its dependencies) depends on ld64-97, which in turn requires LLVM in the first place...

Would my best bet be trying to use your llvm-powerpc port at this point?

Alternatively, for going the "circular dependency" route, I would need to:

1. Compile all of llvm-3.3 and ld64-97's dependencies as +universal
2. Install those two packages for ppc
3. Install gcc7 as +universal
4. Install llvm-3.7 for ppc64
5. Re-install ld64-97 for ppc64

Correct?

(as an aside, it's sudo port install <package> build_arch="ppc", not sudo port install <package> build.arch="ppc")

This is all very confusing 😅

 

[barracuda156]

I managed to build gcc10-bootstrap for ppc64 overnight, but it doesn't look like I can use it to compile llvm-7.0 directly, since it (or one of its dependencies) depends on ld64-97, which in turn requires LLVM in the first place...

Would my best bet be trying to use your llvm-powerpc port at this point?

Alternatively, for going the "circular dependency" route, I would need to:

1. Compile all of llvm-3.3 and ld64-97's dependencies as +universal
2. Install those two packages for ppc
3. Install gcc7 as +universal
4. Install llvm-3.7 for ppc64
5. Re-install ld64-97 for ppc64

Correct?

(as an aside, it's sudo port install <package> build_arch="ppc", not sudo port install <package> build.arch="ppc")

This is all very confusing 😅

You do not have to use specifically my llvm port, but you could make similar modifications to the way llvm-5.0 is built, for example.
llvm-powerpc minimizes dependencies (approach is not unique or novel: it was originally introduced by @catap in clang-11-bootstrap and gcc10-bootstrap ports, I just adapted it to cases of libcxx and llvm) and uses gcc10-bootstrap to build. So at least it should make the whole process a bit shorter.

Sorry for not offering you exact instructions, I simply do not have a complete solution myself. It seems at the moment that ld64 built for ppc64 does not work correctly. Or something in MacPorts setup prevents it from working correctly. But we need to discover a way to make it work, and hopefully a reproducible one.

What I can suggest is do things “interactively”: if you try doing something, inform us what works and what fails, and maybe we can figure out together what to do.

I will try to check if I have old portfiles and maybe some notes for 10.5.

P. S. gcc10-bootstrap was substantially changed after its revision 0, and I am not sure changes were desirable. However, both versions failed to build gcc14 for me last time I tried.

 

[barracuda156]

AFAIR, what I had working on 10.5 was:
1. gcc10-bootstrap +universal (revision 0, not later ones).
2. cctools and ld64-97 both universal, built against universal llvm-5.0.
3. gcc11 +universal, which could compile code for ppc and ppc64 both.

I used gcc11 to build llvm-5.0, which was a suboptimal choice: I should rather have used gcc10-bootstrap.

By the way, gcc7 will not work correctly for ppc64 even if you manage to build it. For example, it could not build icu for ppc64, which meant nothing much will build, since no libxml2 it result, etc.
You should really aim at getting gcc10+. Not necessary the latest gcc, but not lower than 10.