It can't be done yet. I made some progress on it, but we'll need to get the libpurple build process updated for 64 bit before I can work on the rest of Adium.
Valid points. But at the moment, using 64 bit apps is the thing causing the loading of 32 bit and 64 bit libraries. If and when Apple start shipping the vast majority of their software (which, lets be honest, 95% of people use on their computer regularly) as 64 bit then it may become a more valid reason, but when you need Safari, Mail and iTunes open pretty much all the time and they are 32 bit then it becomes more of an Apple issue.
I'll wait till 10.6 before I start worrying about 64 bit apps (except the ones that really need it).
Adium typically has a long enough release cycle that 10.6 may well be out sometime around the same time as Adium 1.4 is. Planning for the future is a good thing