Download the IPSW file to your computer and do the "shift click" update method in iTunes. It will update (not restore) to the lower iOS. Only thing you may lose are message conversations.
How to downgrade back using the update method
Step 1: Back up your iOS device.
Step 2: Download the latest IPSW file to your computer.
Step 3: Connect your iOS device to your computer via USB.
Step 4: Launch iTunes and open the Summary page for your iOS device.
Step 5: Hold the Option ( ⌥ ) key and click the Check for Update button. (Windows users will need to hold Shift instead of Option).
Step 6: Select the IPSW file downloaded in step 2 and click Open.
Your iOS device will now be downgraded. This method lets you downgrade while keeping most app data, Home screen layout settings, etc.
I just did this yesterday to "update" to 10.0.3 from 10.1 beta 4. But I'm going back to the beta.