you can roll back only if the model had ever supported it at some point. If the machine was released with a version of OS X, it can be roll backed to that version but not any before it due to drivers
Not possible to run it natively on any Retina or current model iMac due to missing kexts needed to run. On an older iMac with El Capitan installed, you can roll back.
It is certainly a bummer that you can't use any version of OS that you want on your machine. This is why I had to buy the MP to run Yosemite in my recording studio. Apple should come out with a way that you can at least get newer drivers for an older OS. Sounds simple enough, but apparently not. Plus I am certain Apple wants to keep people moving forward down the line of their OS chain.