Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Schranke

macrumors 6502a
Apr 3, 2010
974
1,072
Copenhagen, Denmark
Well I am working on a testing guide for the forum users, any inputs are appreciated
There are a bunch of test you could do. Here are some:

CineBench - especially used for seeing CPU performance in rendering
I would recommend doing the following from idle:
1 run to get a base score, 5 consecutive to get a idea on how it performs when the CPU is warm
Another interesting test would be see the difference when using fan control software set to 100% (app 6000rpm and 5500rpm (left/right)) in the same scenario.

Then there is Geekbench, which also is a great test and much more used. The great thing here is that there is a big online archive with all test date from the free version, so comparing performance on multiple identical devices are possible.
Here the same runs can be done, but seems a little much unless you have a high difference in the CineBench test.

also Prime95 which should hit the CPU hard, but I have not tried that one myself yet, so I am not entirely sure on how it works and how to compare. I will do it later, but if some have more info it would be nice to hear.
Thanks

For utilities you will need
Intel Power Gadget, this is an amazing tool to see power consumption, clock speed, temperature and utilisation of the CPU.
Some kind of fan control software, my favourite is smcFanControl, last updated in 2016 but I have not encountered any problems with it on a 2017 MBP, so my guess is that it will work fine on the 2018 model.
Noting ambient temperature will also be good as to give an idea on environmental impact.

If anyone have some resources on rendering test in FCPX or its likes that is free to use, we could have a more comparative test across machines as we are testing on identical work mass on different hardware, giving us an easy comparison.

EDIT, note, most fans control software can show current RPM and will allow system to go higher (if possible) then what you set it to if needed
EDIT 2, updated with @matt_k Prime95 suggestion from here https://forums.macrumors.com/thread...nce-throttling-testing.2128150/#post-26260011
Thread can be found here
 
  • Like
Reactions: anshuvorty

CodeJoy

macrumors 6502
Apr 3, 2018
400
592
I really don't think this is same situation as the i9. The frequency here doesn't seem to get below base frequency, so it works somewhat as designed (oscillating between base and boost frequency), whereas the i9 gets throttled below the base frequency.
Agreed. The i5 situation is pretty much as can be expected. Not a great result maybe, but fair. The i9 however, I've said from the start that there's no reason for anyone to get it. It's not that you're way overpaying for a little bit extra performance, it's effectively donating money to Apple without getting anything in return at all. If anyone wants to do that, there must be better ways.
 
  • Like
Reactions: unobtainium

ibtj

macrumors newbie
Sep 26, 2017
13
29
graz, austria
Agreed. The i5 situation is pretty much as can be expected. Not a great result maybe, but fair. The i9 however, I've said from the start that there's no reason for anyone to get it. It's not that you're way overpaying for a little bit extra performance, it's effectively donating money to Apple without getting anything in return at all. If anyone wants to do that, there must be better ways.

guys, i really think you're letting apple off the hook much too easily here. delivering 20-30% less perfomance than competitor's machines running the same cpu is unacccetable given the premium in price.

there might be a lack of innovation at intel right now, but apple must have been informed about the thermal envelope of 8th gen chips well in advance, so not adjusting the cooling solution is highly negligent.
 
  • Like
Reactions: The Mercurian

CodeJoy

macrumors 6502
Apr 3, 2018
400
592
guys, i really think you're letting apple off the hook much too easily here. delivering 20-30% less perfomance than competitor's machines running the same cpu is unacccetable given the premium in price.

there might be a lack of innovation at intel right now, but apple must have been informed about the thermal envelope of 8th gen chips well in advance, so not adjusting the cooling solution is highly negligent.
If it's unacceptable then you just don't buy their product. How is there more to it than that? Exactly what else do you suggest as "the hook"? Are you planning to sue them for negligence? Would you like to shame them publicly? Do you think any of that is going to have any possible impact other than wasting your time?
 
  • Like
Reactions: 88Keys

ibtj

macrumors newbie
Sep 26, 2017
13
29
graz, austria
If it's unacceptable then you just don't buy their product. How is there more to it than that? Exactly what else do you suggest as "the hook"? Are you planning to sue them for negligence? Would you like to shame them publicly? Do you think any of that is going to have any possible impact other than wasting your time?

bad decisions in (thermal) design must be openly criticized, since "just don't buy their product" would effectively mean switching ecosystems, which is highly bothersome and not something i'm particulary inclined to do.

other than that i don't quite understand your position here. just don't buy and shut up does not seem like a reasonable approach to me, especially in a forum for discussing hardware.
 
  • Like
Reactions: swerve147

matt_k

macrumors member
Jul 18, 2018
32
16
guys, i really think you're letting apple off the hook much too easily here. delivering 20-30% less perfomance than competitor's machines running the same cpu is unacccetable given the premium in price.

there might be a lack of innovation at intel right now, but apple must have been informed about the thermal envelope of 8th gen chips well in advance, so not adjusting the cooling solution is highly negligent.

Those are however results of comparison with different cpu microarchitectures. I admit that it's quite unusual that coffee lake performs worse than kaby lake, but there may be factors that are not immediately obvious. I.e. it is possible that there are spectre/meltdown mitigations in coffee lake that prevent certain speculative executions which may have significant effects on synthetic benchmarks.

Btw, the Thinkpad T480s scores 143 in cinebench Single CPU / 715 multi CPU. That's a factor of five, which is quite strange for 4 a core CPU.
 

Lennyvalentin

macrumors 65816
Apr 25, 2011
1,431
794
My guess is that really fixing this would require a redesign of the MBPs cooling system and likely the chassis.
Laptops that can run unthrottled are typically substantially thicker than the MBP; the ASUS ROG Strix Scar II for example is roughly 11mm thicker. That gives a lot more room for vents and a (comparatively) big cooler and big blower fans.

It's the super thin chassis that creates this problem.

Can undervolting (and possibly underclocking) be done on MBP's?
Underclocking isn't really desireable; the CPU will underclock itself if it gets too hot. Undervolting might be possible, if someone reverse engineered how the on-board voltage regulators work. If the voltage regulators are controlled through the system management processor that one would have to be reverse engineered too, except it's been incorporated into the T2 coprocessor, so probably fairly locked down and user inaccessible.

Might be difficult. Macs aren't really built for tweaking... Then again, most brand name computers aren't.

If it's unacceptable then you just don't buy their product. How is there more to it than that?
Except Apple claims large performance BOOSTS for the higher core count MBPs (4-core 13"; 6-core 15") on their website. You shouldn't have to research a product to find out if it is deceptive or not before buying; it should go without saying that you should not be deceived in the first place!
 

DeanL

macrumors 65816
May 29, 2014
1,360
1,296
London
Owner of a 2018 13" 2.7 GHz i7 16 GB RAM 1 TB SSD MacBook Pro running the Mojave β here: after running CineBench 4 times in a row, I see no evidence of "heavy" throttling (see screenshot). The first test throttles as the fans do not kick in but after the fans do kick it, throttling is mitigated. (The big drops are in between tests)
https://imgur.com/a/XXtPJNg
 

CodeJoy

macrumors 6502
Apr 3, 2018
400
592
Except Apple claims large performance BOOSTS for the higher core count MBPs (4-core 13"; 6-core 15") on their website. You shouldn't have to research a product to find out if it is deceptive or not before buying; it should go without saying that you should not be deceived in the first place!
While I agree that in a perfect world, product information from manufacturers should genuinely and honestly reflect the user experience, that's not really the world we live in. All product marketing is angled, not just in computing but in everything. On Apple's page I see some claims about extra cores, certainly fully true. Then there is a list of specific benchmark tests in a handful of apps. I have not seen any reviewer who even attempted to validate these claims. They seem reasonable to me, though of course angled. (Please let me know if there's something else that Apple claims that I've missed) Instead, what reviewers do is run cinebench and geekbench because that's easy. For the most part that doesn't show anything meaningful, though there are indeed more serious reviewers out there also. And then users will make purchase decisions based on Apple's angled claims, and reviewers mostly meaningless benchmarks, and then try to extrapolate to their own use case which may or may not apply. Some end up satisfied and some do not. Those dissatisfied will typically try to blame anyone other than themselves, because that's just what people do. It's certainly an unfortunate situation, but I don't see that Apple is doing anything particularly deceptive compared to other manufacturers in this regard.
[doublepost=1531958491][/doublepost]
Owner of a 2018 13" 2.7 GHz i7 16 GB RAM 1 TB SSD MacBook Pro running the Mojave β here: after running CineBench 4 times in a row, I see no evidence of "heavy" throttling (see screenshot). The first test throttles as the fans do not kick in but after the fans do kick it, throttling is mitigated. (The big drops are in between tests)
It's certainly throttling since you're consistently hitting 100 degrees. But that's not the interesting thing really, because any laptop in this form factor is going to do the same. What's interesting is how much this impacts performance. For this, you'd ideally want to run a ~10s multicore test cold vs hot, and compare performance between the two. Still want hot caches though, so probably hard to do right. Or run a longer real world test with the CPU being adequately cooled vs one where it's just room temperature. Also probably hard to do (though D2D did this). But that's when you can start saying something meaningful about throttling.
 
  • Like
Reactions: macintoshmac

NBAasDOGG

Suspended
May 27, 2017
644
1,534
Netherlands
This is not an engineering flaw, it’s a stupid choose Apple made in their BIOS setting.

Have you ever wondered why MacBooks never make noise? That’s simply because the fans don’t turn on until de CPU reaches 83°C. And if they do turn on, it will not go higher that 30% fan-speed, even if it hits 100°C....
Louis Rossmann investigates this flaw few weeks ago, take a look:

They probably fix it in the next update!
 
  • Like
Reactions: jeremiah256

DeanL

macrumors 65816
May 29, 2014
1,360
1,296
London
While I agree that in a perfect world, product information from manufacturers should genuinely and honestly reflect the user experience, that's not really the world we live in. All product marketing is angled, not just in computing but in everything. On Apple's page I see some claims about extra cores, certainly fully true. Then there is a list of specific benchmark tests in a handful of apps. I have not seen any reviewer who even attempted to validate these claims. They seem reasonable to me, though of course angled. (Please let me know if there's something else that Apple claims that I've missed) Instead, what reviewers do is run cinebench and geekbench because that's easy. For the most part that doesn't show anything meaningful, though there are indeed more serious reviewers out there also. And then users will make purchase decisions based on Apple's angled claims, and reviewers mostly meaningless benchmarks, and then try to extrapolate to their own use case which may or may not apply. Some end up satisfied and some do not. Those dissatisfied will typically try to blame anyone other than themselves, because that's just what people do. It's certainly an unfortunate situation, but I don't see that Apple is doing anything particularly deceptive compared to other manufacturers in this regard.
[doublepost=1531958491][/doublepost]
It's certainly throttling since you're consistently hitting 100 degrees. But that's not the interesting thing really, because any laptop in this form factor is going to do the same. What's interesting is how much this impacts performance. For this, you'd ideally want to run a ~10s multicore test cold vs hot, and compare performance between the two. Still want hot caches though, so probably hard to do right. Or run a longer real world test with the CPU being adequately cooled vs one where it's just room temperature. Also probably hard to do (though D2D did this). But that's when you can start saying something meaningful about throttling.


I did other tests and it's actually bad: https://imgur.com/a/aXmPJYi
W
hat's interesting is that the fans weren't even at their max
 

CodeJoy

macrumors 6502
Apr 3, 2018
400
592
This is not an engineering flaw, it’s a stupid choose Apple made in their BIOS setting.

Have you ever wondered why MacBooks never make noise? That’s simply because the fans don’t turn on until de CPU reaches 83°C. And if they do turn on, it will not go higher that 30% fan-speed, even if it hits 100°C....
Louis Rossmann investigates this flaw few weeks ago, take a look:

They probably fix it in the next update!
Apparently the app SMC Fan Control can be used to change fan profiles, at least on older models.
[doublepost=1531959484][/doublepost]
I did other tests and it's actually bad: https://imgur.com/a/aXmPJYi
What's interesting is that the fans weren't even at their max
Looks like it. Have you tried changing the fan profile?
 

DeanL

macrumors 65816
May 29, 2014
1,360
1,296
London
Apparently the app SMC Fan Control can be used to change fan profiles, at least on older models.
[doublepost=1531959484][/doublepost]
Looks like it. Have you tried changing the fan profile?
It doesn't work on new models neither does iStatMenus
 

matt_k

macrumors member
Jul 18, 2018
32
16
I did other tests and it's actually bad: https://imgur.com/a/aXmPJYi
What's interesting is that the fans weren't even at their max

Yeah, that's pretty heavy throttling and it just confirms my opinion that i7 in 13" is waste of money. You can give Macs Fan Control a Try. I'm not sure if it works for MacBook Pro 2018 (given the T2), but it'd be very good to know if it does and if more aggressive fan control helps with throttling.
 

CodeJoy

macrumors 6502
Apr 3, 2018
400
592
It doesn't work on new models neither does iStatMenus
Ok.
[doublepost=1531960067][/doublepost]
Yeah, that's pretty heavy throttling and it just confirms my opinion that i7 in 13" is waste of money.
Yeah, it doesn't gain you anything. Same with the 15", the lowest spec i7 is probably just as fast as the other CPUs. With previous models you could argue that you got 10-20% extra even though you paid way too much for it. But with infinite money you could still gain from the upgrade. But this year it's essentially paying more and getting 0%. Not really worth it for anyone. Sadly. Same with the 555 vs 560.
 

DeanL

macrumors 65816
May 29, 2014
1,360
1,296
London
Yeah, that's pretty heavy throttling and it just confirms my opinion that i7 in 13" is waste of money. You can give Macs Fan Control a Try. I'm not sure if it works for MacBook Pro 2018 (given the T2), but it'd be very good to know if it does and if more aggressive fan control helps with throttling.
That worked. And the throttling is less worse
Screen Shot 2018-07-18 at 20.26.27.png
 

matt_k

macrumors member
Jul 18, 2018
32
16
That worked. And the throttling is less worse View attachment 771423

That does indeed seem better, maybe in future updates Apple will make fans more aggressive out of box. That said, it still goes below the base frequency, which is what caused all the uproar with i9. And the temperature is downright scary.

It's called Torture Test for a reason, but my 2016 i5 @ 2.9GHz can sustain 3.1GHz flat at 84C without breaking a sweat. So, no magic here I guess. No bending physics, thermal solution designed for dual core is clearly struggling with quad. I ordered one with i5 but now I have serious doubts about the longevity of this setup. Svalt D2 for the rescue?
 

DeanL

macrumors 65816
May 29, 2014
1,360
1,296
London
That does indeed seem better, maybe in future updates Apple will make fans more aggressive out of box. That said, it still goes below the base frequency, which is what caused all the uproar with i9. And the temperature is downright scary.

It's called Torture Test for a reason, but my 2016 i5 @ 2.9GHz can sustain 3.1GHz flat at 84C without breaking a sweat. So, no magic here I guess. No bending physics, thermal solution designed for dual core is clearly struggling with quad. I ordered one with i5 but now I have serious doubts about the longevity of this setup. Svalt D2 for the rescue?

I hope Apple will come out and be like "haha sorry, we forgot to toggle this switch. Here's an update that fixes everything."
 
  • Like
Reactions: Blue Sun

joelhinch

macrumors 6502
Oct 2, 2012
382
764
I too am interested to see the difference between the i5 and i7 on the 13". Upgrading to the i7 is a significant price increase.
At the moment if the i7 is going to be throttled I'm thinking the;
i5/16GB/512GB config - thoughts?

Planning for the machine to get me 4+ years of use, light trans-coding/Photoshop and multiple virtual machines.

If the i7 is throttled, I wonder what the performance difference would be and if it is similar to the i7 vs i9 on the 15"
 

CodeJoy

macrumors 6502
Apr 3, 2018
400
592
I too am interested to see the difference between the i5 and i7 on the 13". Upgrading to the i7 is a significant price increase.
At the moment if the i7 is going to be throttled I'm thinking the;
i5/16GB/512GB config - thoughts?
My thoughts on that is that it's probably the second best value 2018 MBP you can get. If you can get by with 256GB, you can put the $200 you save towards a 1TB external SSD or a 4TB external hard drive plus whatever dongles you need. Thanks to USB-C this still ends up being very fast storage. If you think that would work for you, then that's probably the most cost effective thing to do. Otherwise the 512GB option isn't too bad, and it's not like $200 is the end of the world.
 

joelhinch

macrumors 6502
Oct 2, 2012
382
764
My thoughts on that is that it's probably the second best value 2018 MBP you can get. If you can get by with 256GB, you can put the $200 you save towards a 1TB external SSD or a 4TB external hard drive plus whatever dongles you need. Thanks to USB-C this still ends up being very fast storage. If you think that would work for you, then that's probably the most cost effective thing to do. Otherwise the 512GB option isn't too bad, and it's not like $200 is the end of the world.

Thanks CodeJoy.. I actually do have a 1TB T5 but I had the 256GB last time and always found myself making room, so the 512GB I'm sold on this time.

If anyone sees any comparison/reviews come up comparing the 13" i5 and i7 please post here! :)
Doesn't look like there are many on YouTube and such just yet.
 
  • Like
Reactions: macintoshmac

CodeJoy

macrumors 6502
Apr 3, 2018
400
592
Thanks CodeJoy.. I actually do have a 1TB T5 but I had the 256GB last time and always found myself making room, so the 512GB I'm sold on this time.

If anyone sees any comparison/reviews come up comparing the 13" i5 and i7 please post here! :)
Doesn't look like there are many on YouTube and such just yet.
Then I think you may have found the sweet spot :)

If you're really keen on the i7, then I would recommend to wait two weeks to see how it's actually doing in your expected workloads. The i5 and i7 are effectively the same chip, the i7 just being clocked slightly higher. You should expect to see a 15-20% gain in performance. If you see that from reviews, then you can at least make a valid argument for it. You'll be way overpaying for the privilege imo, but then again it's just money.
 

Closingracer

macrumors 601
Jul 13, 2010
4,317
1,849
Hmm.. looks like Apple will need to have redesigned cooling to allow for better heat dissipation.


No they also need to make it thicker as well. Their quest to make everything smaller hurts it as well. I guess this is why they still are using LPDDR3 Ram in the 13 MacBook pros.
[doublepost=1531975130][/doublepost]
jerry wrote in 13 above:
"These new CPUs run hot doing hard tasks like video rendering. The impact on these long tasks is a 40+% increase in time to render. And this is unacceptable since these these are "pro" systems designed for pro user doing things like creating videos and rendering them for publication to youtube or corporate presentations."

I would think that a "pro user" doing heavy-duty video editing and rendering would not normally work on such projects on a laptop -- but rather on a desktop machine (Mac or PC) appropriately configured for such work.

Having said that, the thermal/throttling issues we see with the MacBook Pro line are inherently there due to design compromises made to achieve the all-important Apple goal of "thinness"...



These machines haven't been for "pro" users for a while now. TBH besides a persons preference of OS and Final cut pro generally speaking a Windows machine can probably do it better. There is a video by Austin Evans and while he was comparing the 2018 15 i9 to a 2016 P70. Dollar for dollar a P52 has the same cpu with a better GPU but with a slower HDD and 8GB of RAM compared to the base level 15 inch. Comparing to the 13 inch MacBook Pro the differences is even bigger. You'll get better GPU performance with similar CPU specs and for less money.


The P52 is $1,749.99
2018 15 inch $2,399.99


Even comparing the P52 to the lowest base 2018 13 the price is $1,799.99



 
  • Like
Reactions: macintoshmac

Poki

macrumors 65816
Mar 21, 2012
1,318
903
Will it still be faster than the Core 2 Duo in my 2009 MacBook Pro? Yes, it will. :D

Jokes aside, that situation seems pretty bad in stress tests. However, I think in a lot of different real world scenarios, like photo editing, where you only need the full speed a few seconds at a time, I think the performance advantage of these new models over the older ones will be very noticeable.

It's just with task which continuously use the CPU where the throttling is going to be an issue, like rendering tasks or even exporting a lot of photos at once. Still pretty bad, but at least it won't impact most of my workflow.
 
  • Like
Reactions: macintoshmac
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.