Turning off Faces seemed to solve any performance issues I had, especially when I imported about 600 pictures from a wedding reception - talk about a nightmare to analyze. And since I only knew about 5 people from it I wouldn't be able to tag anyone anyways - FEATURE CANCELLED.
I've only created new libraries thus far, two with >1000 pictures each, and Aperture is moving along just fine. Yes, I know, very very small libraries but nevertheless, I'm doing fine. Granted, I do have 10GB RAM....
I do web design (Coding, photoshop, a few browsers) and Aperture and have been up for 5 days now sitting on 13MB in my swap of 256MB, so no memory leak here.