macworld MBA review is nowhere close to being accurate

[cckwokchun]

First of all, according to Macworld and its benchmark test.
the 2009 1.86Ghz beats the 2009 2.16Ghz in CPU test.......

what the hell........seriously........

and then the new 2010 1.86Ghz MBA double the score of the Mid 2009 1.86Ghz MBA while they both use the exact same model of CPU: SL9400
They said the thermal protection slow down the last gen MBA when its heat up, then why don't u consider this in test and make it fair.



On the other hand, Xbench produce a much much more accurate result, it clearly shows the new and old 13 inch score the same mark in CPU test.....and the 11 inch MBA is 30% slower.


THIS IS WHY XBENCH ONLY HAS $35 DONATION THIS MONTH. AND MACWORLD IS SUCH A BUSINESS.

go figure

 

[barbro]

I'm sorry you don't understand the macworld methodology, but it is valid I assure you. The processor is only a very small amount of the speed of many tasks - they were trying to approximate real world usage, rather than raw processing speed.

I fear we are repeating this conversation in many other threads too...

 

[Dammit Cubs]

These tests score vary on what you to compare. If cpu is a bigger factor in the score, the obviously you will see both be closer than normal.

I think the a very interesting score is when reviewers use PCMARK by doing a windows boot. This measures the overall performance and you will notice that overall..its not bad. ....not bad at all.

 

[Hellhammer]

That is not totally incorrect. Those CPUs support SpeedStep that means that they usually operate below their advertised frequency to use less power (this is the de-throttling thing that Scottsdale has been talking about). Thus SL9400 may be faster than SL9600 is some cases. As the old MBA's ran pretty hot, I would assume that SL9600 never or very rarely ran at full 2.13GHz. I've heard people running tests on their own MBAs and sometimes SL9400 won the SL9600, not sure what app was used though.

I admit that it doesn't make any sense.

 

[cckwokchun]

I'm sorry you don't understand the macworld methodology, but it is valid I assure you. The processor is only a very small amount of the speed of many tasks - they were trying to approximate real world usage, rather than raw processing speed.

I fear we are repeating this conversation in many other threads too...

These tests score vary on what you to compare. If cpu is a bigger factor in the score, the obviously you will see both be closer than normal.

I think the a very interesting score is when reviewers use PCMARK by doing a windows boot. This measures the overall performance and you will notice that overall..its not bad. ....not bad at all.

How do you explain the same generation 2009, 1.86Ghz beats 2.16Ghz in pure CPU test?

Not to mention the 2.16Ghz model has a SSD.

I can answer that for you, because the thermal protection system. They already know the problem, they just don't make the whole test fair and accurate.

 

[cckwokchun]

That is not totally incorrect. Those CPUs support SpeedStep that means that they usually operate below their advertised frequency to use less power (this is the de-throttling thing that Scottsdale has been talking about). Thus SL9400 may be faster than SL9600 is some cases. As the old MBA's ran pretty hot, I would assume that SL9600 never or very rarely ran at full 2.13GHz. I've heard people running tests on their own MBAs and sometimes SL9400 won the SL9600, not sure what app was used though.

I admit that it doesn't make any sense.

You are correct. Most Apple products suffer from the heat because the slim design.

And heat is a major issue when it comes to performance

Therefore, the test should be done in a fair environment (such as temp). This is the problem I want to point out.

 

[admanimal]

You are correct. Most Apple products suffer from the heat because the slim design.

And heat is a major issue when it comes to performance

Therefore, the test should be done in a fair environment (such as temp). This is the problem I want to point out.

There would be no point in doing a test in an artificial environment created to make sure both hardware configurations could run at their maximum speed. In that case, you could just look at the numbers (CPU, bus speed, etc.) and have your answer. The whole point of the Macworld benchmarks is to see which is faster under normal usage conditions, i.e. when the computer is just sitting on a desk in a room with normal temperatures and left to manage its own performance characteristics.

 

[misterbig]

AnandTech's benchmarks also show the 2010 13" MBA as faster than the 2009 13" MBP in most tests despite a lower clock speed. His review is very detailed and I suggest reading it in full to understand the reason.

 

[cckwokchun]

There would be no point in doing a test in an artificial environment created to make sure both hardware configurations could run at their maximum speed. In that case, you could just look at the numbers (CPU, bus speed, etc.) and have your answer. The whole point of the Macworld benchmarks is to see which is faster under normal usage conditions, i.e. when the computer is just sitting on a desk in a room with normal temperatures and left to manage its own performance characteristics.

Normal usage condition?

The benchmark itself is NOT normal usage conditions already. Do you convert 200 songs everyday? No.

you can make a normal usage condition benchmark, but there would be no point.

 

[Hellhammer]

Normal usage condition?

The benchmark itself is NOT normal usage conditions already. Do you convert 200 songs everyday? No.

you can make a normal usage condition benchmark, but there would be no point.

MacWorld's benchmarks are as normal as they can be. You can't really benchmark web page load time or how well Adium and Mail run

Benches like iMovie import&export, iPhoto import, PS CS5, HandBrake, Pages etc are something that the end-user may also use and where the specs of your computer can be important. GeekBench and xBench are synthetic and thus do not tell the real world performance.

 

[stewie1]

The way I understood it, the 2009 MBA they benchmarked was the spinning hard drive version, not the SSD. That would account for a huge part of the difference. How much, I don't know. I own a rev b 1.86 SSD and I would love to know how it stacks up against the new one, but I can't find that info anywhere.

 

[cckwokchun]

AnandTech's benchmarks also show the 2010 13" MBA as faster than the 2009 13" MBP in most tests despite a lower clock speed. His review is very detailed and I suggest reading it in full to understand the reason.

AnandTech review is better in the explanation of this.

The thermal protection system is the what the problem is.
2010 13" MBA CPU is a lower voltage one, that's why it performs better under the heat.

That's what I guess...at the first place.

 

[Hellhammer]

AnandTech review is better in the explanation of this.

The thermal protection system is the what the problem is.
2010 13" MBA CPU is a lower voltage one, that's why it performs better under the heat.

That's what I guess...at the first place.

The CPUs are exactly the same, SL9400 (1.86GHz) and SL9600 (2.13GHz). Both have TDP of 17W.

The only difference in CPU that I can figure out is the cooling. If 2010 MBA's cooling is better, it can run at higher clock speed. As older MBAs ran very hot, the CPUs were pretty much always de-throttled and thus they are slower in the tests. With better cooling, the SL9600 can run at 2.13GHz when needed but not if it's already running too hot.

 

[cckwokchun]

I just use it when I travel......prolly just web browsing most of the time.
They should be all the same in everyday tasks (I mean surfing, typing document etc.)

But just want to let you guys know, if u don't use MBA as workstation (i m sure most ppl don't)......
Buying a Rev C. while they are still in the market is also a considerable good choice because :

1. price, you can get a brand new 2009 13" for as low as $900 to $950 USD now.

2. curving edge design. its a nicer touch altho lacking of 1 extra USB port.

3. Illuminated Keyboard which the 2010 version remove this features.

4. Bigger screen for a lower price than a 11". 11" is too netbook-like (i know its not a netbook)

 

[cckwokchun]

The CPUs are exactly the same, SL9400 (1.86GHz) and SL9600 (2.13GHz). Both have TDP of 17W.

The only difference in CPU that I can figure out is the cooling. If 2010 MBA's cooling is better, it can run at higher clock speed. As older MBAs ran very hot, the CPUs were pretty much always de-throttled and thus they are slower in the tests. With better cooling, the SL9600 can run at 2.13GHz when needed but not if it's already running too hot.

Its interesting that altho the model number are the same, the 2010 version is lower watt (compared to 2008 rev. b) according to:
http://www.anandtech.com/show/3991/apples-2010-macbook-air-11-13inch-reviewed/6

and probably stronger fan, because it says the 2010 13" is noisy.

 

[ct2k7]

From what I've heard, the fan is the same used in the mac minis. I've heard mac mini fans, and they're not loud, just a whoosh.

 

[Hellhammer]

Its interesting that altho the model number are the same, the 2010 version is lower watt (compared to 2008 rev. b) according to:
http://www.anandtech.com/show/3991/apples-2010-macbook-air-11-13inch-reviewed/6

and probably stronger fan, because it says the 2010 13" is noisy.

2010 MBA's CPU is drawing 11.39W while 2008 MBA's is drawing 7.59W, so 2010 is actually using more power.

The reason is what I said above and what Anand said in his article.

The 2008 machine is throttling back to reduce overall temperature while the 2010 system keeps going.

2008 MBA's cooling is worse thus the CPU cannot run at full speed without overheating. The CPU is wise enough to clock itself down when reaching critical temperatures.

As 2010 MBA has better cooling, it can run at higher speed and thus uses more power.

 

[KPOM]

First of all, according to Macworld and its benchmark test.
the 2009 1.86Ghz beats the 2009 2.16Ghz in CPU test.......

what the hell........seriously........

go figure

They were surprised last year at that result, and attributed it to processor throttling from heat. I tend to agree with that, given how just about everyone is reporting much higher CPU benchmarks from the new models, even though they are using the same or slower processors than before.

 

[bcaslis]

From what I've heard, the fan is the same used in the mac minis. I've heard mac mini fans, and they're not loud, just a whoosh.

I agree, the fan isn't as noisy as some are saying and it can't normally be heard.

So is the OP going to edit the thread title since he was wrong, not Macworld?

 

[cckwokchun]

2010 MBA's CPU is drawing 11.39W while 2008 MBA's is drawing 7.59W, so 2010 is actually using more power.

The reason is what I said above and what Anand said in his article.

2008 MBA's cooling is worse thus the CPU cannot run at full speed without overheating. The CPU is wise enough to clock itself down when reaching critical temperatures.

As 2010 MBA has better cooling, it can run at higher speed and thus uses more power.

right.

I guess the better cooling is a result of better fan, or its might be hotter than the rev. b because the review says both CPU are at about the same temp before throttling.

form factor didn't change that much.

In practice I found the 2008 13-inch MBA launched applications quicker (short bursts of full clock speed), but after prolonged use or completing CPU intensive tasks it was tough to tell apart from the new 11-inch.

Ah.....Old generation is faster before it gets heat up. In benchmark test, of course, it runs @ full speed for more than 10 min (that's not normal usage condition)

No wonder in Xbench, old gen scores higher mark because Xbench is a 30 sec test.

Thumbs up for AnandTech, that's how people write a good review.

not like macworld. bad bad crap

 

[bcaslis]

...
Ah.....Old generation is faster before it gets heat up. In benchmark test, of course, it runs @ full speed for more than 10 min (that's not normal usage condition)

No wonder in Xbench, old gen scores higher mark because Xbench is a 30 sec test.

Thumbs up for AnandTech, that's how people write a good review.

not like macworld. bad bad crap

Simply because you didn't understand design impacts and testing methodologies, doesn't make a review bad. You are incorrect, the Macworld review is in fact accurate.

 

[cckwokchun]

Simply because you didn't understand design impacts and testing methodologies, doesn't make a review bad. You are incorrect, the Macworld review is in fact accurate.

nah....its not about the methodologies.
Anand use the same method as macworld, I know wat macworld uses before I posted this.
but why I still say Anand is much better?

read wat AnandTech says regarding this issue:

This is something I'm going to have to devise tests for and pay more attention to in future reviews. Sneaky, Steve, sneaky.

That's wat a review should be. You have to devise tests and pay more attention to details!!!

Not just go out and say MBA is competing MBP already.
That only helps Apple to promot new products, not help consumers to buy the right thing.

 

[ct2k7]

nah....its not about the methodologies. I know wat macworld uses before I posted this.
but why I still say Anand is much better?

read wat AnandTech says:



That's wat a review should be. You have to devise tests and pay more attention to details!!!

Not just go out and say MBA is competing MBP already.
That only helps Apple to promot new products, not help consumers to buy the right thing.

NO. It's just clock throttling. AnandTech wants to know more about it. He's not going to devise tests based solely on that platform. Reviews can be metricless. Macworld's review is valid.

Next you're going to tell me that a 1.86GHz CPU will always outperform a 2.13GHz unit.

 

[bcaslis]

nah....its not about the methodologies.
Anand use the same method as macworld, I know wat macworld uses before I posted this.
but why I still say Anand is much better?

read wat AnandTech says regarding this issue:



That's wat a review should be. You have to devise tests and pay more attention to details!!!

Not just go out and say MBA is competing MBP already.
That only helps Apple to promot new products, not help consumers to buy the right thing.

Lame response. You were wrong, you didn't understand how the designs are different and you didn't understand the different testing methodologies. Just another newbie without the courage to admit they were wrong.

 

[darngooddesign]

You are correct. Most Apple products suffer from the heat because the slim design.

And heat is a major issue when it comes to performance

Therefore, the test should be done in a fair environment (such as temp). This is the problem I want to point out.

Of course that is fair. If the old one slows down under load due to heat and the new one doesn't then the new one is faster under load. Its completely relevant to people who use their machines heavily. If you don't then feel free to disregard that particular test and pay attention to the tests that mirror your usage patterns.