I tried formatting it and restoring it from a backup from before the update to 17 to get back to iOS 16, but unfortunately to no success.
By 'formatting it' you mean restoring correct? You can't actually 'format' an iPhone like a hard drive or SSD or memory card. Apple won't let you, even if you were jailbroken.
Invisible to the user, an iPhone has two partitions. The first is the System partition and the second is the User partition. System files are stored on the System partition and when you do a restore, the device restores to that partition. Only the device/Apple can access the System partition when not jailbroken. A jailbreak can give you access, but can't change anything as a user.
The User partition is where apps and data are stored, including all the settings in the Settings app. It's only
that stuff that gets backed up in a backup to a computer, and only user data to iCloud (no actual apps). No system files exist in your backups.
So, when you do a 'formatting it' you are not restoring to a previous version of iOS because that is NOT what's in the backup.
Lastly, Apple has what is known as digital signing for iOS. As long as Apple keeps signing a version of iOS then you can actually choose on the device (or using an IPSW) to restore back to a previous iOS version.
But Apple doesn't keep signing older versions of iOS for very long. Your device will check and if Apple is not signing it, you cannot restore to an older version. Apple stops signing because they WANT you to update/upgrade. So they make it very difficult, if not impossible, to roll back.
Is there any way to get back to the previous system?
As others have stated, and for the reason I gave above, no. No way back.