Macbook pro inflated benchmarks
I've tried to bring this up to apple, but was never responded to on any avenue, I can't even get a proper set of their internal benchmark data to properly prove it.
I work in a performance test lab where we deal with comparative evaluations, and more often I have to explain simple statistics to our clients. It all started when I noticed that apple always seems to claim its "2x faster" than its old gen (
http://www.apple.com/macbookpro/performance.html)
So heres what I put to them
I would just like to inform Apple that they have been misrepresenting the improvements offered by their new generation MacBook Pro. For example, In the 15-inch comparison, let us assume that the Baseline metric for the previous generation is a score of 100, and the N.Nx metrics for the various benchmarks are are reported as the multiple of the baseline score (if not, then they are not represented accurately), such that the data is
Modo 501 : 220
Cinebench 11 : 200
FC Studio 3 : 190
Mathematica 8 : 190
Aperture 3.1.1 : 160
Baseline : 100
Being in the benchmarking business I understand that, on first glance, you would take the Modo score, divide it over the baseline, and say that the new generation is 2.2x faster, or as it is stated "Up to 2x faster". This is incorrect. By performing the above calculation you are not solving for the factor of improvement OVER the previous generation, but instead the improvement factor OF the previous generation. The correct statement is: "Up to 2x the performance of the previous generation."
To calculate the "times faster" letting N = new and P = previous, the equation is ((N-P)/P), taking the 220 score and the 100 score, we find ((220-100)/100) --> 120/100 --> 1.2, meaning the accurate statement for the website to reflect is "Up to 1.2x faster..."
What i find interesting is that other sites do benchmark testing and don't find it odd that theyre only seeing a 35% improvement when apple is claiming that it should be up to 200%..