The Browser Marks: Gauging Performance for PPC Browsers

[Doq]

The Browser Marks
Straightforward performance metrics for PPC OS X web browsers​

TL;DR

Not all browsers are created equal and you should pick a browser that fits your ideal balance of performance and compatibility, but if you want the browser with the best performance, compatibility be damned, then your pick is Aquafox. But there's more to it than that.

What is this?

The Browser Marks is a side project by yours truly, to present as quantitatively as possible how well each web browser in the OS X PowerPC space performs. There are quite a few of them for such a niche platform, and new and experienced users alike may struggle with which one is right for them. This is an aid in helping users make that decision.

Realistically, all of these browsers will perform well enough in rendering plain HTML and CSS; thus The Browser Marks test JavaScript performance, as that comprises almost all of the modern web.

To accomplish this, independent variable-free testing is performed on many classes of machine on browsers as they came-- replicating as best as possible the scenario in which a user will simply download and launch a browser.

The testing scenarios

For each browser to pass through the testing to make it on The Browser Marks, the browser is tested on at least one (all three if possible) of the following test suites:

• Speedometer 2.1
• Octane 2.0
• Sunspider 1.0.2

Furthermore, each test gets three initial runs. On occasion, and for each major browser release, each browser is then ran one or two more times for sanity.

Each browser's tests are conducted on three machines, representing top, high, and mid-end PowerPC machines:

• Implacable - A1117 Power Mac G5 Quad, 2.5GHz, Leopard
• Shiratsuyu - A1139 PowerBook G4, 1.67GHz, Tiger
• Amagi - A1002 eMac, 1.0GHz, Tiger

There are no G3s listed, because I only have one G3, and it's the lowest end G3 ever made. It scored 0.7 in Speedometer. Also, I would not recommend a G3 on the modern web, unless you are a masochist, no matter how good the G3 is.

For browsers with generation-specific builds (G3, G4/e, G5), the correct build for each machine is used. This is only applicable for Aquafox and TenFourFox, as other browsers only provide a single generic build.

The results

The results are presented in a table, with each browser's average score in each benchmark. Alongside this, a quantitative score is presented, which is the average performance ratio of each test to a baseline, which is also included on the table.

Finally, a spreadsheet of all results are available on placeholder.directory for those that want raw numbers. While RetroZilla is included in the full results, it is not included in this table because of poor overall compatibility making it unsuitable for the modern web anyway.

Aquafox, 1.0.0
generation specific builds available
Overall: 131.65 (7/9 tests)

Spoiler: table

Machine	Score	Speedometer		Octane		Sunspider
Implacable	275.65	7.16	179.08	5278	422.21	902.5	277.00
Shiratsuyu	96.31	2.64	66.00	1617	129.39	2389.8	104.61
Amagi	85.95*	DNF (test stalled)		DNF (browser hung)		2908.6	85.95

Arctic Fox, 38.9
Overall: 105.23 (7/9 tests)

Spoiler: table

Machine	Score	Speedometer		Octane		Sunspider
Implacable	131.57	9.09	227.17	1511	120.88	3014.2	82.94
Shiratsuyu	123.16*	3.72	93.08	DNF (errors on zlib test)		1534.1	163.00
Amagi	71.91*	2.10	52.58	DNF (errors on zlib test)		2542.2	98.34

InterWebPPC, RR4
Overall: 100.46

Spoiler: table

Machine	Score	Speedometer		Octane		Sunspider
Implacable	119.76	7.70	192.58	1457	116.56	3267.2	76.52
Shiratsuyu	102.90	2.74	68.42	1588	127.07	1994.8	125.33
Amagi	82.28*	DNF (test stalled)		947	75.76	2797.8	89.36

TenFourFox, FPR32 SPR5
generation specific builds available
Overall: 119.19 (8⅓/9 tests)

Spoiler: table

Machine	Score	Speedometer		Octane		Sunspider
Implacable	272.24	7.71	192.67	5257	420.56	1004.0	249.00
Shiratsuyu	94.43	2.61	65.25	1674	133.92	2594.4	96.36
Amagi	65.87*	1.70 (1 test)	42.50	985	78.83	2930.7	85.31

Do not take the results of these tests as gospel when choosing your browser; there is more to a browser selection than JavaScript performance.

 

[Doq]

Interestingly, after I saw the runaway numbers of the browsers with G5 builds, I decided to put the G5 build of InterWebPPC RR3 through the Marks as well, and I can conclude that my prior assumption that generation-specific builds don't matter is not entirely true. Even though there is noticeable worse performance in Speedometer, IWPPC-G5 runs away in Octane and Sunspider just like the other G5 builds.

InterWebPPC, RR3 (G5 build)

Machine	Score	Speedometer		Octane		Sunspider
Implacable	262.7	6.71	167.83	5187	414.96	960.4	260.30

 

[thewireless]

While optimizing Aquafox last summer, I tried to use the Speedometer 2.1 benchmark to measure its performance, but I encountered significant inconsistencies that were quite frustrating. At times, Aquafox performed exceptionally well, even leaving both InterWebPPC RR4 and TenFourFox 20240505 far behind. However, there were also instances where it lagged significantly behind the other two.

Although the results from three consecutive runs often fell within a similar range, giving a false sense of consistency, re-running the benchmark at a later time could yield completely different results. I've concluded that these results should be taken with a grain of salt, especially when comparing one browser to another. There are simply too many unknown variables that can affect the outcomes of these tests.

I have since stopped using Speedometer, but today I conducted several runs to illustrate my point. I want to emphasize that I'm not trying to prove that Aquafox is faster than InterWebPPC or stock TenFourFox. My main point is that these tests provide an indication of performance at a very specific moment in time, and three runs are not sufficient to accurately measure their overall performance.

Below are the results of the Speedometer 2.1 tests I conducted today on my 2.3 GHz Dual-Core Power Mac G5, running Mac OS X 10.4.11. I have not calculated any averages, and the results are presented in the same order in which the tests were performed. Each browser was started with a new profile before running the test three times in a row.

Spoiler: Round one

Aquafox, 1.0.0:
- 7.32
- 7.02
- 7.03

InterWebPPC, RR4:
- 6.86
- 6.55
- 6.58

Arctic Fox PPC, 38.9:
- 8.25
- 8.22
- 8.37

TenFourFox, FPR32 SPR5:
- 6.23
- 5.90
- 6.11

TenFourFox, 20240505-FPR32:
- 6.32
- 5.94
- 6.24

Spoiler: Round two

Aquafox, 1.0.0:
- 5.88
- 5.65
- 5.48

InterWebPPC, RR4:
- 6.08
- 5.83
- 5.59

Arctic Fox PPC, 38.9:
- 7.97
- 7.83
- 7.99

TenFourFox, FPR32 SPR5:
- 5.66
- 5.62
- 5.51

TenFourFox, 20240505-FPR32:
- 6.13
- 5.80
- 5.69

As you can see, there is quite a difference, especially for Aquafox, and the results almost consistently worsen. I'm not sure what the cause of this is; perhaps foxes just get tired after taking too many tests, or maybe a reboot is required to obtain more reliable results.

I'd say the reason you've had InterWebPPC RR3 G5 score lower than InterWebPPC RR4 in Speedometer can most likely be attributed to this phenomenon. I would like to see you conduct more testing to create a larger dataset so that we can obtain more reliable results, but I also understand that this takes a lot of time for little to no reward.

Regardless, I'm happy and honored to have contributed to the PowerPC browser landscape, but I couldn't have done it without some of you. I genuinely hope to see something come from the 10.5 branch of Arctic Fox, or perhaps a Leopard WebKit revival; it would be great to have another updated browser to accompany Aquafox.

 

[Doq]

You are right: Speedometer isn't really a reliable bench on older machines. I have included it mainly on the grounds of there not being very many benchmarks that work on older machines to begin with, and even Speedometer itself has issues the further back you go, evident by the number of stalled runs on Amagi. Octane and Sunspider seem to be more consistent, so I may weight those results heavier.

I've concluded that these results should be taken with a grain of salt, especially when comparing one browser to another. There are simply too many unknown variables that can affect the outcomes of these tests.

I have tried to eliminate as many variables as possible, primarily through using mostly-stock configurations in not just browser but OS as well-- the key exception being Shiratsuyu representing a well-used system. Along with that, occasional retests for sanity and with new releases will at least attempt to maintain order in the chaos.

Also there are multiple classes of browser for different use cases. Maybe a heavy modern browser isn't required; for that RetroZilla or even Links2 can suffice. Arctic Fox provides a good middle ground between performance and compatibility. There is so much that goes into the web experience; this is just one of them.

I gotta dig out the Macs again anyway now that Aquafox 2 is out.

 

[thewireless]

Perhaps tests that don't focus too much on JavaScript performance could be considered, such as AnTuTu's HTML5 Test, which I assume focuses on HTML5. Another option is Basemark Web 3.0; I'm not entirely sure what it does, but it shows CSS and HTML5 conformance with a percentage score.
These two ran to some extent on my G5. I'm not exactly familiar with them or browser benchmarks in general, but I'm sure there are other suitable benchmarks.

Arctic Fox provides a good middle ground between performance and compatibility.

What exactly does this entail? I'm not really familiar with AF PPC (or any Palemoon-based browser), as I've mostly used Safari, Firefox, and TenFourFox on Mac, (and just Firefox on other platforms). However, when I attempted to download something from SourceForge, it wouldn't let me, while any TFF-based browser would. So, my experience in terms of compatibility hasn't been very positive.

 

[Doq]

What exactly does this entail?

It's more observation than factual statement, based on the anecdote that older browsers are generally considered faster but less compatible with modern web standards. On one end, the TenFourFox descendants push forward in supporting as much of the modern web as possible, at the cost of being more resource intensive. On the other end are older or less feature-rich browsers like Links2 or RetroZilla, sacrificing compatibility for being lighter and snappier.

Arctic Fox, from my own experience, is somewhere in the middle. You can get a good chunk of normal websites to work, but heavier or more exotic sites tend to fall apart in some way.

However, when I attempted to download something from SourceForge, it wouldn't let me, while any TFF-based browser would.

I have the exact inverse problem! Arctic Fox will load a SourceForge page no problem whereas TFF-based browsers would trip a rendering error and the page looks broken and is functionally unusable.

 

[Alubook133]

Regardless, I'm happy and honored to have contributed to the PowerPC browser landscape, but I couldn't have done it without some of you. I genuinely hope to see something come from the 10.5 branch of Arctic Fox, or perhaps a Leopard WebKit revival; it would be great to have another updated browser to accompany Aquafox.

IIUC from some readings in this forum, the Leopard Webkit does use GPU acceleration contrary to all the Mozilla/Firefox originated browsers. So yeah this would be great, would most probably dramatically improve performance especially on the modern web (provided such Webkit-based browser would be compiled for the right OS (i.e. up to 10.5.8 max) and the "right" graphics card (the NVidia GeForce FX Go5200 (AGP) for my machine thank you, selfish as I am ^^).

 

[repairedCheese]

To be honest, I'd be curious how Wayfarer running under MorphOS compares to those other browsers, both in compatibility and performance. I know it's kind of a big ask, but they are asking for money for MorphOS, so if it's no better, that would be kind of funny.

 

[Doq]

I probably would have to source a MorphOS license to make that happen, and also take Shiratsuyu apart every time to test as I'm not aware of MorphOS playing nice in dual-boot let alone seven-boot. (Implacable has an incompatible GPU)

 

[ervus]

If these benchmarks could be run from disk without a network connection, that should remove the speed of the internet connection as a variable.

 

[Jubadub]

To be honest, I'd be curious how Wayfarer running under MorphOS compares to those other browsers, both in compatibility and performance. I know it's kind of a big ask, but they are asking for money for MorphOS, so if it's no better, that would be kind of funny.

Keep expectations about anything MorphOS low. I'd be surprised if Wayfarer fared any better than the current OS X PPC browsers, if the rest of the state of the OS is any indicator. Note: I do have (collaterally) a MorphOS license.

 

[wicknix]

Keep expectations about anything MorphOS low. I'd be surprised if Wayfarer fared any better than the current OS X PPC browsers, if the rest of the state of the OS is any indicator. Note: I do have (collaterally) a MorphOS license.

Wayfarer is based on current WebKitGTK. It’s 10x better than an old version of firefox 45. While not the fastest it will play YT in browser without a hiccup, and it renders most of the modern web properly. It’s basically the equivalent of Safari 18 but on much slower hardware. Don’t expect it to be super speedy on a 20 year old G4, but it’ll get you places TFF and it’s spinoffs wont.

This video is a few years old now, but still shows wayfarers speed. Not slow at all.

 

[Alubook133]

@wicknix, thanks for the heads-up on MorphOS. Not planning to install it, but still on the verge of installing a Linux distro that might exhibit similar good results in terms of web browsing, just unclear as to *which* distro to install on a 2004 PBG4 w/ GeForce GPU (AGP).

Clearly Wayfarer makes a huge difference in MorphOS in terms of web browsing and YT support, no wonder why : it's based on a late-2023 Apple Webkit. And that's what would be great to see implemented for our good old G4 PowerPC Mac's running Tiger or Leopard (or Linux for that matter)...

Hmm... maybe I should give MorphOS a try, and if it does deliver on my machine and makes it a lot more usable (and that's a big "if"), I might consider getting a license (though I don't find its pricing strategy super attractive, euphemism inside).

 

[wicknix]

If you install Debian 12 or Fienix you'll be able to use sealion and brass monkey which use the same backend as pale moon.

As for morphos, I don't think it'll run on your machine as it requires a Radeon gfx card.

 

[Alubook133]

Ah, I didn't know that about MorphOS, so that's a definite no-go apparently... too bad, still I'll have a look at the dedicated thread, there's often collateral info of interest hidden in those threads.

As for Linux distros, interesting tip, however isn't Arctic Fox also built upon Pale Moon ? (which btw I do use in Win11 since it supports the VLC web plugin required by some of my web apps). Because I've tried AF, that's good AF so to speak but so far doesn't seem to be any better than say Aquafox or Interweb as far as modern web sites or YT support is concerned - I might be wrong about this though, but that's what I've seen so far on my machine.

Again from what I understand it's browsers that are built upon Webkit that seem to exhibit the best performance/usability trade-off, and I don't know that Pale Moon falls in that category. Or does it ?

 

[Doq]

I don't know that Pale Moon falls in that category. Or does it ?

Correct. Pale Moon is a Firefox derivative, though one that had forked away from Firefox years and years ago. That's also why it looks like Firefox 20 instead of Firefox 45 like TFF and its derivatives.

 

[wicknix]

Arctic Fox is derived from the now defunct Pale Moon 27 which got killed off in 2018, which was a fork of FF38 with some additions, so it’s actually a bit older than TFF. Riccardo has been patching it up, but it’s only at like version 41/42 at this point and is still lagging behind TFF.

Current Pale Moon was a hard fork from FF52, but with many improvements and additions since. It’s pretty much on par with FF78 - FF102 these days. SeaLion and BrassMonkey which are built upon UXP (Pale Moon’s rendering engine) fair the same, and are honestly quite fast on PPC Linux.

 

[wicknix]

Again from what I understand it's browsers that are built upon Webkit that seem to exhibit the best performance/usability trade-off, and I don't know that Pale Moon falls in that category. Or does it ?

Safari and leopard-webkit were fast on PPC OSX. However, any webkit browser i’ve ever used or built for PPC Linux are dog slow and/or are crash prone compared to the FF derived browsers. Different OS’s, different network stacks, different UI toolkits, etc all play a part in how well (or bad) an application will perform.

 

[repairedCheese]

Safari and leopard-webkit were fast on PPC OSX. However, any webkit browser i’ve ever used or built for PPC Linux are dog slow and/or are crash prone compared to the FF derived browsers. Different OS’s, different network stacks, different UI toolkits, etc all play a part in how well (or bad) an application will perform.

And this is what made me curious about Wayfarer in the first place. I knew it was a big ask, but considering it's among the last few 32-bit PPC web browsers standing, it's at least worth thinking about, even if we can't port it to anything else.

 

[wicknix]

And this is what made me curious about Wayfarer in the first place. I knew it was a big ask, but considering it's among the last few 32-bit PPC web browsers standing, it's at least worth thinking about, even if we can't port it to anything else.

The good news is that Wayfarer is stable and relatively fast. I mean it is a modern webkit browser engine on ancient PPC hardware. The bad news for porting it to PPC OSX is that even MorphOS has current development tools just like PPC Linux does. Macports only gets us so far on PPC OSX. Cocoa is outdated, the frameworks are outdated, secure transport (ssl/tls) is outdated, etc.

If the sole developer of Wayfarer had some help, it *could* be possible to get it working on PPC OSX with a native Cocoa UI, but in my years of porting browsers, other than Riccardo with Arctic Fox, not a single person stepped up to lend a hand. Not even a single PR (pull request) on my many github browser projects. So sadly, i don’t see this ever happening. If you want to browse the “modern web” on your PPC, i suggest installing Linux or MorphOS. Otherwise enjoy PPC OSX for what it is… nostalgia.

 

[Alubook133]

Safari and leopard-webkit were fast on PPC OSX. However, any webkit browser i’ve ever used or built for PPC Linux are dog slow and/or are crash prone compared to the FF derived browsers. Different OS’s, different network stacks, different UI toolkits, etc all play a part in how well (or bad) an application will perform.

and that's a little disappointing to me, with all the developments going on around browsers for PPC based machines I was hoping to find one that's performing above all others in terms of supporting the modern web (with javascript CSS and all bells and whistles) while providing a reasonably fluid experience. Well, in all fairness browsers like Aquafox or Artic Fox (non-Webkit based) are relatively close to reaching that goal, which already is a great achievement.

And this is what made me curious about Wayfarer in the first place. I knew it was a big ask, but considering it's among the last few 32-bit PPC web browsers standing, it's at least worth thinking about, even if we can't port it to anything else.

yeah, it's a real bummer, but porting Wayfarer really looks like too much of a task (esp. considering @wicknix post above ). I for one would definitely be willing to help such a development on either OSX or Linux, though my contribution would be limited to testing on my single machine (and maybe help dev's formulate things in a "layman's manner" so that they can concentrate on debugging, building, compiling and such and less worry about communicating to the "outside world").

p.s. being in a cheese lovers country, find your pseudo real funny ^^

 

[Jubadub]

Ah, I didn't know that about MorphOS, so that's a definite no-go apparently...

Like I said, "keep expectations about anything MorphOS low". Don't get me started on FireWire bootability, general FW support, various SSD/HDD issues in particular on those being "too big for it", and so on (like the GPU support issue you saw), or it not using multiple processors, let alone multiple cores. And that's just the hardware-side, the OS experience is IMO even worse. Not a good UI, and certainly not a good UX (again, IMO).

That being said, I'm surprised Wayfarer, and all the other ported GNU/Linux tools, were ported as far, and as close to current, as they were. Thanks for all those details @wicknix.

By the way, I have to say, if a current internet browser is your no. 1 desire in an OS, it makes one wonder if you actually care much about using the OS itself. Sounds like an ordinary ChromeBook or even a mere cellphone would suffice?

 

[Alubook133]

Well, I already have a smartphone, a tablet and a (Win11) mini PC, so obviously browsing the web is no issue.

No, my N°1 priority here was to revive my old precious great looking Alu Powerbook, which I did : maxed out RAM, replaced the (dead anyway) 60GB HDD with a 256GB SSD, install OS X Tiger and Sorbet Leopard on it (with Linux to come next). Then to make use of it and enjoy it as much as possible : internet browsing for the most part, movie watching, music, etc... and maybe a bit of coding later on to taylor the OS to my liking (esp. since both battery are dead I need to make sure things start automatically at reboot, the time is properly set, etc... - actually I think I've seen "hibernate" in Sorbet as a possibility, need to check it again).
You make me think now... over the years I've dev'd a series of web apps in Win11, with some adjustments I think many of them might run properly on my PBG4 (javascript allowing), just need to install PHP on it (hmm wonder which version will do) and find a good code editor (I use Notepad++ in Win11).

So the topic du jour is : usability of a 20-years old machine within today's challenging h/w and s/w environment.

 

[wicknix]

Not a good UI, and certainly not a good UX (again, IMO).

If you are old enough to have grown up with an Amiga you wouldn’t think that. If you are a young buck that grew up with Windows or OS X, then i can see your point. For us older folks, MorphOS’s Ambient desktop mimics Amiga’s WorkBench and feels 100% natural.

 

[repairedCheese]

By the way, I have to say, if a current internet browser is your no. 1 desire in an OS, it makes one wonder if you actually care much about using the OS itself. Sounds like an ordinary ChromeBook or even a mere cellphone would suffice?

The main source of software for for my old PPC Macs tends to be the Macintosh Garden, which is a website. But why stop there? The reason an up-to-date browser is so important is because, basically no other computer task has changed in the last 20 years. Almost all the other tools are still fit for purpose, many having been for much, much longer, and if I was trying to do this on a Windows XP system with a Pentium 4 it would be doable.