Cool.OK So
1- plug in the TM Backup Drive with Sierra into the Mac.
2- Turn on your Mac.
3- Holt the ALT key.
4- Choose to boot from the Time Machine Backup - THIS IS IMPORTANT - Not Recovery partition!
5- After that, when it boots, go to disk utility and wipe out the high sierra partition and make a new HFS+
This is how I did it for me.
But if you managed to wipe already everything then you will be able for sure to restore.
Cheers!
So because my TM backups are on a NAS i couldn't find a way to boot from them, so in my frustration, i just erased the drive and restarted the laptop (not in any normal mode).
I left it for maybe 20 min, and when I came back to it, it had the option to restore from the internet!
So I ended up restoring it back to Mountain Lion, and am going to try restoring from my NAS TM backup tonight. I'll let you know how it goes