I've been using the new builds since release. The first builds of Yosemite, and iOS were bad enough that I almost did a wipe.
The current builds though are actually pretty good so far. If you aren't doing anything specific that requires you to use them though I would reiterate to avoid for the time being.
Yosemite is stable right now, but it is slower in certain areas then Mavericks. Same goes for iOS 8.
If you do decide to install any of the builds, please have good backups, and just plan on having to do a restore at some point. It's inevitable.
I've had to do restores back to the latest stable build from Apple at least 1 time for Mavericks, iOS 6, iOS 7, Mountain Lion, and probably before then. Not always in the first few builds, but there seems to always be just one that is so bad you can't do anything with it in the mix.