I got rid of the "1" by wiping the iPhone, and setting it up as a new iPhone.
Then I had to wipe it again and restore it to an old backup in order to get everything back.
Nobody knows how it happened, but I seem to have downloaded the 5.0.1 twice, and the one that did not install remained in storage and kept triggering the Settings App to show the "1" as an update waiting.
Apple do not understand how I got the update twice.
The only thing really unusual about my iPhone is that I use a locked SIM. Don't know what that would have to do with it.