Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
I've had no problems, but I run a pretty clean system.
How many third party addons have you installed?
 
If you feel unsafe to update, make a Time Machine backup, and update. If anything goes terribly wrong, restore from Backup, done.
To be honest, I suggest you hold for a while. I think 10.11.4 makes El capitan slower than before. But my system is upgraded from Yosemite, through many beta El Capitan until 10.11.4. Yours may be much cleaner than mine.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.