The short answer: Live with unorganized apps or restore and re-jailbreak.
This happened to me a couple of times, when I did my first few jailbreaks. After having to do a restore and re-jailbreak a couple times, I've started to just take it easy. Don't spend 20 mins. installing a dozen apps from cydia and the app store, even though we all know it's very addictive(that's when things would go wrong for me). That definitely includes rebooting after installing stuff. I'm not saying baby the phone and don't use it, just try not to do too much at once. After the second time it happened to me, I tried to just take it easy, and I haven't had the problem since.
I've heard that it is due to adding corrupted files, but I've added pretty much the same files every time I jailbreak.
Good luck!