How much RAM do you have? Usually the system will be much smoother if you do fresh-install rather than upgrade. If you have 'only' 4 GB, then better stay in SL, unless your system will be slowing down later due to insufficient memory & paging it out to the disk (although it's fresh-install system).
Thanks Isamilis, I have 8gb ram and did a fresh install (erased and formatted the disk prior to installing ML).
Also, in ML, Safari is a huge memory hog.