With my shiny new MacBook in hand (2.0 GHz) I thought it’d be fun to put the machine through its paces and see just what it can do compared to Windows and with some additional upgrades to maximize its performance.
I ran these tests over the course of a week and it took a lot of time. Feedback or criticism is welcomed and if you have an idea on how to improve, I’m interested as well. I intend to do a follow up that focuses solely on MacBook gaming performance.
Xbench
With only a handful of testing suites available for the Mac, Xbench has received a fair amount of press. This was my first time using Xbench, version 1.2.
Each test was run three times and the results averaged to yield the typical result for that configuration.
Xbench, however, appears to have several problems that cause one to question its validity. First, the program has not been updated in sometime and there have (apparently) been some compiler changes in OS X in the interim.
The results of the tests in some cases, in particular the User Interface Test, vary wildly, so much so that they can skew an overall test result by a large margin. The entire Disk Test is suspect, as slower 5400rpm drives consistently outperform 7200rpm drives, for example.
In short, I recommend using the Xbench numbers as a comparison point but to treat their results with suspicion.
Cinebench
Developed by Maxon, Cinebench is a graphics rendering benchmark program available for both Mac and Windows. This makes it handy to draw cross-platform comparisons on the same machine. Version 9.5 was used.
Each test was run thee times and the results averaged to yield the typical result for that configuration. Cinebench results have a very small degree of variance and remain consistent; they appear to be an accurate benchmark of CPU and GPU rendering capabilities. Cinebench does not measure disk or I/O performance at all.
Photoshop Test
This test is taken from the Retouchartists website and was run in Adobe Photoshop CS2, version 9.0.1. Again, because of the cross-platform nature of Photoshop this test can be run on both OS X and Windows.
Testing was measured once from the clicking of the “Start” dialog box until the “Stop” dialog box. The human margin for error is no worse than 1 second in either direction.
The test performs a number of graphic effects, resizing, and filters. This test appears to be a good measure of CPU and memory performance.
Temperature Measurements
All temperatures were measured using the CoreDuoTemp program, available online. The deep idle temperature was after 20 minutes of no activity, sleep mode disabled. The idle temperature was with periodic use, checking mail, etc, but nothing that really strained the CPU at all. The peak temperature was derived after putting a load on both cores using the standard “yes > /dev/null” command in Terminal twice (once for each core). The peak temperature was the highest temperature observed at any point during testing. The sustained load temperature was the highest average temperature noted, with the machine running under full load for at least 10 minutes.
I couldn’t find a Windows XP program that would measure temperature for the MacBook, or an easy way to put both cores under a full load.
The Results
In doing these tests I identified a number of baseline assumptions that I wanted to address to determine the “real world” results from. We’ve been told that more RAM, for example, increases OS X performance; I wanted to see by how much.
OS X RAM Performance
The MacBooks (and most Macs) ship with a paltry 512MB of RAM. In this day and age, that’s a lackluster amount by anyone’s standards. Much to Apple’s credit, OS X still runs very well on 512MB of RAM. However, what gains can be seen by increasing the amount of available system memory?
As evidenced in the chart, overall system performance increased by over 17% when the amount of memory was increased to 1GB. Photoshop rendering time was cut in half, no doubt an indicator of how memory-hungry Rosetta is, especially with Photoshop. Cinebench saw no meaningful gains in adding additional RAM. Four seconds were shaved off of boot time.
Increasing the RAM further to 1.25GB saw an approximate 10% increase in overall performance while our Photoshop rendering times shaved off another minute-and-a-half. Cinebench, again, saw no performance improvement.
Going to 2GB of RAM, according to Xbench, yielded a performance decrease of 10%, thanks to the ever-fluctuating User Interface Test. Boot time shaved off a few more seconds and another 20 seconds was removed from the Photoshop rendering test. Cinebench saw a slight improvement in rendering speed, no doubt due to dual channel memory (discussed below).
Recommendation: At a bare minimum, increase your MacBook’s RAM from 512MB to 1GB. If you intend to do any heavy graphics work or pro applications, consider more memory to make these tasks run faster, but do not expect overall system performance to increase much, if at all.
Dual Channel Memory Performance
Something we’ve been told is that using two pairs of the same memory size will allow the Macs to use dual-channel mode, increasing performance. But by how much?
Of the configurations tested, only two – 512MB and 2GB – used two like-paired memory sticks and were running in dual channel mode. Of these two configurations the Xbench Memory Test – one of the few consistent tests – did show a memory performance improvement between 5 and 10%. Overall system performance, however, was minimal, except in rendering in Cinebench: the dual-channel 2GB configuration saw a 15% in OpenGL software rendering and a 35% improvement in OpenGL hardware rendering.
Peak temperatures for the dual-channel configuration also increased by 2° C, however.
Recommendation: Dual-channel, in day-to-day use, provides little tangible or visible benefit. If you can easily and affordably configure your system with two matched pairs, do so, but don’t expect any large gains.
5400 RPM vs 7200 RPM Hard Drive Performance
This test has vexed me more than any other.
First, the Disk Test provided by Xbench gives, at best, strange results. With different drives and different memory configurations, the results are all over the map.
In nearly every case, the stock 5400 rpm drive that came with the MacBook (a Seagate ST96812AS) outperformed other user-submitted Xbench test results with 7200 rpm drives. Tapping the online database of Xbench results against “stock MacBooks” from Apple Stores, shows results that vary quite a bit. The 80GB black MacBook drive, for example, seems to perform quite well.
To add insult to injury, the Hitachi 7200 rpm drive that I received and tested (0A25016) was bested by the stock Seagate at every turn. The unit’s performance was poor, so much so that the system locked up installing OS X on the fresh drive. Disk test results were, in a word, abysmal.
The replacement Seagate drive (ST910021AS) fared better, though.
Eventually, in order to move away from Xbench’s questionable results, I had to rely on a Windows benchmarking program – Drive Speed Checker – to provide me with more accurate results. Those are included on a second chart.
All test results were run three times (sans the Hitachi) and averaged for their overall speed result.
Comparing the results of Xbench’s Disk Test and Drive Speed Checker it is clear that in 90%+ of most cases, a typical user will see little to no benefit in using a 7200 rpm drive. There may be a bit of a placebo effect in play here (“it seems snappier”) but the results do not bear this out. In fact my challenge to those who have 7200 rpm drives who claim their systems are faster is PROVE IT.
In the tests I ran the stock Seagate drive equals or beats the Seagate and Hitachi 7200 rpm drives. In only the Overall Read Speed test do the 7200 rpm drives show any clear margin of victory, but even then the difference is minimal at best.
One item that I am loathe to bring up is that perhaps there is a problem with the SATA interface of the MacBook that stunts the performance of 7200 rpm drives? Something to consider. Keep in mind that Drive Speed Checker is a Windows program, meaning that if there is a problem, it is at the hardware level and not with OS X.
Recommendation: Do not spend the additional money for a 7200 rpm drive. You will see little to no benefit. The Hitachi Travelstar drive in particular seems like a bad match for the MacBook; stick to the Seagate ST910021AS.
OS X vs Windows XP Performance
With the addition of Boot Camp it is now possible to benchmark Window applications on Mac hardware, allowing for some interesting comparisons. In the interest of time, only the 2GB configuration was tested.
In Cinebench, Windows performance was on par with OS X, but slightly worse. In OpenGL software rendering, Windows dropped by 18% but kept pace in the OpenGL hardware rendering.
The most surprising test, however, was the Photoshop test. Running a Windows XP native version of Photoshop CS2 (9.0.1), the test took 1 minute and 31 seconds longer than the same test in OS X, running the non-Universal binary version of Photoshop under Rosetta. I ran this test no less than three times with identical results.
Recommendation: There’s really no recommendation to be had here, only to dispel the myth that non-Universal Adobe applications under OS X run significantly slower than under Windows.
I ran these tests over the course of a week and it took a lot of time. Feedback or criticism is welcomed and if you have an idea on how to improve, I’m interested as well. I intend to do a follow up that focuses solely on MacBook gaming performance.
Xbench
With only a handful of testing suites available for the Mac, Xbench has received a fair amount of press. This was my first time using Xbench, version 1.2.
Each test was run three times and the results averaged to yield the typical result for that configuration.
Xbench, however, appears to have several problems that cause one to question its validity. First, the program has not been updated in sometime and there have (apparently) been some compiler changes in OS X in the interim.
The results of the tests in some cases, in particular the User Interface Test, vary wildly, so much so that they can skew an overall test result by a large margin. The entire Disk Test is suspect, as slower 5400rpm drives consistently outperform 7200rpm drives, for example.
In short, I recommend using the Xbench numbers as a comparison point but to treat their results with suspicion.
Cinebench
Developed by Maxon, Cinebench is a graphics rendering benchmark program available for both Mac and Windows. This makes it handy to draw cross-platform comparisons on the same machine. Version 9.5 was used.
Each test was run thee times and the results averaged to yield the typical result for that configuration. Cinebench results have a very small degree of variance and remain consistent; they appear to be an accurate benchmark of CPU and GPU rendering capabilities. Cinebench does not measure disk or I/O performance at all.
Photoshop Test
This test is taken from the Retouchartists website and was run in Adobe Photoshop CS2, version 9.0.1. Again, because of the cross-platform nature of Photoshop this test can be run on both OS X and Windows.
Testing was measured once from the clicking of the “Start” dialog box until the “Stop” dialog box. The human margin for error is no worse than 1 second in either direction.
The test performs a number of graphic effects, resizing, and filters. This test appears to be a good measure of CPU and memory performance.
Temperature Measurements
All temperatures were measured using the CoreDuoTemp program, available online. The deep idle temperature was after 20 minutes of no activity, sleep mode disabled. The idle temperature was with periodic use, checking mail, etc, but nothing that really strained the CPU at all. The peak temperature was derived after putting a load on both cores using the standard “yes > /dev/null” command in Terminal twice (once for each core). The peak temperature was the highest temperature observed at any point during testing. The sustained load temperature was the highest average temperature noted, with the machine running under full load for at least 10 minutes.
I couldn’t find a Windows XP program that would measure temperature for the MacBook, or an easy way to put both cores under a full load.
The Results
In doing these tests I identified a number of baseline assumptions that I wanted to address to determine the “real world” results from. We’ve been told that more RAM, for example, increases OS X performance; I wanted to see by how much.
OS X RAM Performance
The MacBooks (and most Macs) ship with a paltry 512MB of RAM. In this day and age, that’s a lackluster amount by anyone’s standards. Much to Apple’s credit, OS X still runs very well on 512MB of RAM. However, what gains can be seen by increasing the amount of available system memory?
As evidenced in the chart, overall system performance increased by over 17% when the amount of memory was increased to 1GB. Photoshop rendering time was cut in half, no doubt an indicator of how memory-hungry Rosetta is, especially with Photoshop. Cinebench saw no meaningful gains in adding additional RAM. Four seconds were shaved off of boot time.
Increasing the RAM further to 1.25GB saw an approximate 10% increase in overall performance while our Photoshop rendering times shaved off another minute-and-a-half. Cinebench, again, saw no performance improvement.
Going to 2GB of RAM, according to Xbench, yielded a performance decrease of 10%, thanks to the ever-fluctuating User Interface Test. Boot time shaved off a few more seconds and another 20 seconds was removed from the Photoshop rendering test. Cinebench saw a slight improvement in rendering speed, no doubt due to dual channel memory (discussed below).
Recommendation: At a bare minimum, increase your MacBook’s RAM from 512MB to 1GB. If you intend to do any heavy graphics work or pro applications, consider more memory to make these tasks run faster, but do not expect overall system performance to increase much, if at all.
Dual Channel Memory Performance
Something we’ve been told is that using two pairs of the same memory size will allow the Macs to use dual-channel mode, increasing performance. But by how much?
Of the configurations tested, only two – 512MB and 2GB – used two like-paired memory sticks and were running in dual channel mode. Of these two configurations the Xbench Memory Test – one of the few consistent tests – did show a memory performance improvement between 5 and 10%. Overall system performance, however, was minimal, except in rendering in Cinebench: the dual-channel 2GB configuration saw a 15% in OpenGL software rendering and a 35% improvement in OpenGL hardware rendering.
Peak temperatures for the dual-channel configuration also increased by 2° C, however.
Recommendation: Dual-channel, in day-to-day use, provides little tangible or visible benefit. If you can easily and affordably configure your system with two matched pairs, do so, but don’t expect any large gains.
5400 RPM vs 7200 RPM Hard Drive Performance
This test has vexed me more than any other.
First, the Disk Test provided by Xbench gives, at best, strange results. With different drives and different memory configurations, the results are all over the map.
In nearly every case, the stock 5400 rpm drive that came with the MacBook (a Seagate ST96812AS) outperformed other user-submitted Xbench test results with 7200 rpm drives. Tapping the online database of Xbench results against “stock MacBooks” from Apple Stores, shows results that vary quite a bit. The 80GB black MacBook drive, for example, seems to perform quite well.
To add insult to injury, the Hitachi 7200 rpm drive that I received and tested (0A25016) was bested by the stock Seagate at every turn. The unit’s performance was poor, so much so that the system locked up installing OS X on the fresh drive. Disk test results were, in a word, abysmal.
The replacement Seagate drive (ST910021AS) fared better, though.
Eventually, in order to move away from Xbench’s questionable results, I had to rely on a Windows benchmarking program – Drive Speed Checker – to provide me with more accurate results. Those are included on a second chart.
All test results were run three times (sans the Hitachi) and averaged for their overall speed result.
Comparing the results of Xbench’s Disk Test and Drive Speed Checker it is clear that in 90%+ of most cases, a typical user will see little to no benefit in using a 7200 rpm drive. There may be a bit of a placebo effect in play here (“it seems snappier”) but the results do not bear this out. In fact my challenge to those who have 7200 rpm drives who claim their systems are faster is PROVE IT.
In the tests I ran the stock Seagate drive equals or beats the Seagate and Hitachi 7200 rpm drives. In only the Overall Read Speed test do the 7200 rpm drives show any clear margin of victory, but even then the difference is minimal at best.
One item that I am loathe to bring up is that perhaps there is a problem with the SATA interface of the MacBook that stunts the performance of 7200 rpm drives? Something to consider. Keep in mind that Drive Speed Checker is a Windows program, meaning that if there is a problem, it is at the hardware level and not with OS X.
Recommendation: Do not spend the additional money for a 7200 rpm drive. You will see little to no benefit. The Hitachi Travelstar drive in particular seems like a bad match for the MacBook; stick to the Seagate ST910021AS.
OS X vs Windows XP Performance
With the addition of Boot Camp it is now possible to benchmark Window applications on Mac hardware, allowing for some interesting comparisons. In the interest of time, only the 2GB configuration was tested.
In Cinebench, Windows performance was on par with OS X, but slightly worse. In OpenGL software rendering, Windows dropped by 18% but kept pace in the OpenGL hardware rendering.
The most surprising test, however, was the Photoshop test. Running a Windows XP native version of Photoshop CS2 (9.0.1), the test took 1 minute and 31 seconds longer than the same test in OS X, running the non-Universal binary version of Photoshop under Rosetta. I ran this test no less than three times with identical results.
Recommendation: There’s really no recommendation to be had here, only to dispel the myth that non-Universal Adobe applications under OS X run significantly slower than under Windows.
