Why do people care about benchmarks?

[Michael Goff]

It seems, at least to me, that we have gotten to a point where they have no point. Not only that, but between the Bay Trail thing and the S4 thing, I'm starting to even get tired of reading the word benchmark.

Companies have made them pointless by playing to them.

So why do we still give them credence?

 

[mathi-vadhanan]

So why do we still give them credence?

Becuase, it is objective?

 

[Michael Goff]

Becuase, it is objective?

Really?

It is objective when we catch the S4 optimizing to benchmarks, and when we end up catching a certain other benchmark favoring x86 over ARM?

How is it objective when a device can do better on it simply by throwing a couple of lines into the code and get a better score?

 

[ChrisTX]

Bragging rights. People feel better about their choice in mobile phone if it benchmarks higher.

 

[Vegastouch]

It seems, at least to me, that we have gotten to a point where they have no point. Not only that, but between the Bay Trail thing and the S4 thing, I'm starting to even get tired of reading the word benchmark.

Companies have made them pointless by playing to them.

So why do we still give them credence?

I don't care a bit about benchmarks. Only how it performs for me.

 

[Dontazemebro]

Benchmark scores are secondary in my opinion, I normally only run them when I first get a phone.. A slow device is a slow device but it's always nice to see how your new phone stacks up against the competition.

 

[spinedoc77]

Bragging rights. People feel better about their choice in mobile phone if it benchmarks higher.

Pretty much this.

 

[The Robot Cow]

It's all about bragging rights. Same thing with quad-core processors. I forgot which Moto X article i read talked about how very little android actually uses all 4 cores.
I think benchmarks had a point back in the early Froyo/Gingerbread days.

 

[skratch77]

Really?

It is objective when we catch the S4 optimizing to benchmarks, and when we end up catching a certain other benchmark favoring x86 over ARM?

How is it objective when a device can do better on it simply by throwing a couple of lines into the code and get a better score?

95% of the gs4s use the snapdragon soc and only the special Korean LTE version got the octo core to run at its rated clocks 1.8 vs 1.6 for the unlocked 3g octo core only when being benchmarked so how is this bad for benchmarking and how did the gs4 optimize anything ?

The Korean version hit its rated clock speeds when being benched and only hit 1.6 for everyday use woppdedooo for the 5 people in the world that have that special version gs4

 

[strausd]

Benchmarks still give a fair amount of information as to how a device will perform. Snapdragon 600 benchmarks higher than the S4 Pro in the Nexus 4. From benchmarks we get a decent idea as to how much faster it will be. Same thing with the Snapdragon 800. Sure, many people will say "the S4 Pro is plenty fast for me." Thats all fine and well, but benchmarks still provide a little information as to how much faster CPUs can perform, even if it won't be night and day difference in average consumer use.

 

[narutoninjakid]

Bragging rights. People feel better about their choice in mobile phone if it benchmarks higher.

This makes one able to compare and bash phones easily my phone has a higher benchmark so it automatically better lol

 

[animalx]

Benchmarks still give a fair amount of information as to how a device will perform. Snapdragon 600 benchmarks higher than the S4 Pro in the Nexus 4. From benchmarks we get a decent idea as to how much faster it will be. Same thing with the Snapdragon 800. Sure, many people will say "the S4 Pro is plenty fast for me." Thats all fine and well, but benchmarks still provide a little information as to how much faster CPUs can perform, even if it won't be night and day difference in average consumer use.

I think what the OP is trying to say is we've reached a bit of a plateau now. Even mid-range phones have very capable processors, and can provide a smooth experience. At this point, a benchmark is nothing more than a score on a piece of paper. It isn't indicative of one experience being better than another. At this point, hardware is the major differentiating factor anymore, it's software.

 

[Abazigal]

How well a phone actually works in real life is something people can comment on only after weeks, maybe even months of extensive playtesting.

In the meantime, benchmarks is the next best thing. It's not foolproof, nor will it give you the whole picture, but it's better than nothing.

 

[animalx]

How well a phone actually works in real life is something people can comment on only after weeks, maybe even months of extensive playtesting.

In the meantime, benchmarks is the next best thing. It's not foolproof, nor will it give you the whole picture, but it's better than nothing.

I'd go as far as saying they are pretty much worthless at this point. You'd only be concerning yourself with benchmarks with a high-end phone, and let's just be honest, any high-end phone is going to have hardware that will fly. That said, a benchmark isn't going to really provide you any useful information. Every high-end phone nowadays has very capable hardware in it, therefore a benchmark really is rather useless now.

 

[MRU]

Synthetic or not, some folks do like to feel or know that their 2013 handset is better than their previous 2011/12 handset.

Benchmarks give folks a cosy glow when they see them higher even if they are not fully representational of user experience in the real world.



--------- however;

What I find perculiar about benchmarks is the way some folks spend an age optimising their kernel and modding roms - Overclocking this, under-volting that - so that they have or feel they have the quickest handset for bragging rights - and yet if challenged once about the legitimacy of those benchmarks, or something comes along and bests it score wise - they suddenly are the first to scream that the 'benchmarks don't matter '..... Its a bizarre Phenominium I've seen on numerous websites or in user comment sections of reviews of handsets....

 

[Apple fanboy]

It's like buying a car. People always quote the 0-60 time and the top speed.

When was the last time you went from 0-60 in 3.8 seconds, or maxed out the top speed?

Of course when it comes to cars the worst lie from the manufacture is the mpg.

Doesn't matter how careful you drive, you can never hit that (bogus) number!

 

[watchthisspace]

Benchmarks for smartphones don't really tickle my fancy. Especially with iOS and Windows Phone 8 being well mated with certain hardware sets. I understand the benchmarks give objective indicators of a phone's performance, ignoring companies who sadly skew results.

When it comes to desktop CPU and GPU benchmarks, I take massive interest.

 

[MRU]

When was the last time you went from 0-60 in 3.8 seconds, or maxed out the top speed?

I drive a Fiat Punto... I can't get 0-5 in 3.8 seconds !!!

 

[The-Real-Deal82]

Bragging rights. People feel better about their choice in mobile phone if it benchmarks higher.

I think those who are really into tech feel this way and its something I have noticed a lot reading on here. Quite a few people, myself included, couldn't really give a damn as long as the phone is nice to use and is responsive however. I think it is a case of, whatever floats their boat.

 

[Michael Goff]

95% of the gs4s use the snapdragon soc and only the special Korean LTE version got the octo core to run at its rated clocks 1.8 vs 1.6 for the unlocked 3g octo core only when being benchmarked so how is this bad for benchmarking and how did the gs4 optimize anything ?

The Korean version hit its rated clock speeds when being benched and only hit 1.6 for everyday use woppdedooo for the 5 people in the world that have that special version gs4

And when they move all their phones to Exynos?

It's all about bragging rights. Same thing with quad-core processors. I forgot which Moto X article i read talked about how very little android actually uses all 4 cores.
I think benchmarks had a point back in the early Froyo/Gingerbread days.

They did, didn't they?

I didn't know about the quad core thing.

Bragging rights. People feel better about their choice in mobile phone if it benchmarks higher.

That is probably the truth.

I think what the OP is trying to say is we've reached a bit of a plateau now. Even mid-range phones have very capable processors, and can provide a smooth experience. At this point, a benchmark is nothing more than a score on a piece of paper. It isn't indicative of one experience being better than another. At this point, hardware is the major differentiating factor anymore, it's software.

We have a winner.

 

[strausd]

I think what the OP is trying to say is we've reached a bit of a plateau now. Even mid-range phones have very capable processors, and can provide a smooth experience. At this point, a benchmark is nothing more than a score on a piece of paper. It isn't indicative of one experience being better than another. At this point, hardware is the major differentiating factor anymore, it's software.

We are nearing a plateau, but not quite there yet. But I think that mainly has to do with skins manufacturers use on their phones rather than incapable hardware.

 

[appleisking]

Becuase, it is objective?

No never. An iPhone 5 with dual core and 1 gb with RAM runs better than nearly all the android devices on there. The Nexus 4 still runs better than the gs4 even with weaker hardware because no bloatwiz. The software ruins the objectivity.

 

[otismotive77]

benchmarks show you the capabilities, the power, the speed of the processor. if there were no benchmark score comparisons how'd we know the difference between the last year's processor and this year's processor.

 

[Michael Goff]

benchmarks show you the capabilities, the power, the speed of the processor. if there were no benchmark score comparisons how'd we know the difference between the last year's processor and this year's processor.

Does the speed matter if the software isn't taking advantage of it?

 

[T5BRICK]

I think benchmarks are good for an approximation of what the hardware is capable of. To give folks some semblance of an "apples to apples" comparison. Beyond that, I don't really think they're useful.