I somewhat agree, though I think the perception of stainless steel as premium has deeper roots than just Apple's marketing. With fine watches, tools and other pricey items, the aluminium option is usually cheaper than stainless steel, if aluminium is offered at all. Rolexes are stainless steel, aluminium is for soda cans and tin foil - that is what people have seen for decades. I think Apple has actually done a lot to elevate the prestige of aluminium, for better and worse: The material is too good to be regarded as disposable, but it does have weaknesses that make it less suitable on items like watches (bends and dings more easily, discolorations from skin contact unless coated), and those are relevant for phones as well.
But as you say, my phones also spend their lives in cases and I do not really care about the material hidden inside. I would love a titanium phone, though, and that one I would use it caseless: light, strong and virtually corrosion-proof. Scratches easily, but so easily that I regard scratches as a signature surface feature of titanium.