Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Observations & Discussion on Apple Silicon Graphics Performance with MetalFX Upscaling
- Thread starter Malus120
- Start date
- Sort by reaction score
Yep. I believe they’re stored in an AIR (Apple Intermediate Representation) file and then macOS itself will modify them as necessary for the current OS and hardware.
Textures are stored as mipmaps that get gradually blurrier, so a 2k x 2k texture will have a 1k x 1k version and a 512x512, 256x256 and so on all the way down to 1x1 which will be the average of all pixels in the source image.
These "blurrier" low res versions are good because if you draw a polygon that is only 1 pixel on screen, and you use the 1x1 mipmap, that's going to look nice and stable from frame to frame - as opposed to just plucking a random pixel from the 2k texture. For example if you have a black and white checkerboard the 1x1 mipmap would always be gray, while sampling random pixels from the high-res texture would make the texture flip from pure black to pure white randomly and look terrible.
The problem is that when you're rendering at 1.5k but are going to upscale to 3k you need to go up a mipmap detail level - your 1x1 triangle is going to be shown as a 2x2 triangle after upscaling, so instead of using the 1x1 mipmap you actually want the 2x2 mipmap.
It looks to me like the Quality mode has forgotten to offset the mips by 1 level so it's essentially using blurred versions of all the textures.
If I’m not mistaken that’s how the traditional Metal shader library works. Basically compile to IR and the driver will do final compile to GPU binary at pipeline creation (Apple AIR is basically a wrapper over LLVM IR and similar on spirit to SPIR-V). But there seem to be some new precompiled shader feature where you actually ship the shader on binary form from the start. Haven’t looked at it yet.
Yeah, I think the only difference is that the AIR can now be embedded inside a .metallib and loaded in-situ with the PSO.
But yeah I also haven't looked super deep into it because building a binary asset pipeline is pretty low down on the list of fun things to do with Metal.
Will MetalFX help for low resolutions like FullHD or 1600x900? Which resolution it uses for upscaling to 1600x900 (in case of RE8 f.e.)?
Yeah. Looks to be so, not that it will make much difference @60 fps+.
Still. Max quality 3k @ 60-75 fps+ should be very doable, no need for DLSS style upscaling.
It is 2022.
If I am not mistaken MFX Quality is half resolution (maybe it is 66%, not sure if Apple went with the same scale the others did here) and MFX Performance is quarter resolution. @jmho seems pretty sure the Quality setting in RE8 is broken as it produces an overy soft/blurry final output.
I don't think that MetalFX has any notions of "quality" or "performance" presets. The application simply chooses which resolution to use for the input and the output textures. So whatever these settings are that is something the game developer must have come up with.
There are two types of MetalFX scaling. Spatial and Temporal. Spatial is the "dumb" upscaler that is basically just doing a fancy image filter to get more detail from a static image. Spatial can upscale from pretty much any resolution to any resolution.
Temporal is a fancy system that is essentially trying to render the 1/4th of the scene per frame with different offsets and then try to piece those together at the end to make a higher resolution image. The source image can be no smaller than 50% of the target image.
Looking at the MetalFX sample, temporal is actually significantly blurrier than spatial so maybe that's just the way it is.
Guessing I'd say RE:8's quality mode is temporal, and performance is spatial.
Given that the sample starts at 50% scaling, that's probably what RE:8 is using for both, but who knows.
Temporal is a fancy system that is essentially trying to render the 1/4th of the scene per frame with different offsets and then try to piece those together at the end to make a higher resolution image. The source image can be no smaller than 50% of the target image.
Looking at the MetalFX sample, temporal is actually significantly blurrier than spatial so maybe that's just the way it is.
Guessing I'd say RE:8's quality mode is temporal, and performance is spatial.
Given that the sample starts at 50% scaling, that's probably what RE:8 is using for both, but who knows.
Regarding upscaling tech in general and MetalFX in particular... personally I don't usually put much value in the ability to run at higher resolutions or higher frame rates, but the neat thing about temporal upscaling is that it's also an anti-aliasing technique and can help with more fluid gameplay. So you get quite a lot of benefits at the same time while also simplifying the pipeline.
My concern with upscalers is if the source resolution is "too low" you move the bottleneck from the GPU back to the CPU, so utilization drops which just seems like a waste to me.
Yeah in PC land Spatial tends to be blurrier than Temporal, so I have no clue what Apple is doing for that to be flipped.
From my RE8 FullHD experience today it seems that MFX Quality setting blurs image a bit too much as well as makes textures noticeably lower res. It seems to me that to gain more FPS it's better to lower resolution than use MFX. Also, I noticed that with MFX on there were many micro freezes, 0.1-2s. on 8GB RAM machine, at least in the village. I just was pleasantly surprised when I turned the MFX off and the freezes seemingly almost disappeared. But perhaps it was just a coincidence, further testing needed.
Looks like the Digital Foundry video about MetalFX is up
Still an issue on Apple M1 Mac's. Village has hader compilation judder on Mac and PC. Only consoles are from this.
Yeah, it's an issue because they're unnecessarily compiling all the shaders at run time as if it was a PC.
You can compile shaders for Mac at build time, just like you can on console. The Digital Foundry video even mentions this.
Yes but if the devs don't follow that then what's the point. The ports will be from PC and devs don't make a ground up just for the Mac do they?
Well that is an interesting take. MFX Perf is spatial upscaling while quality is temporal. Not sure I am a fan of Capcom just going quarter resolution for the upscaling (in both modes).
