What Mac are you using? to be honest I haven't had fully stable CoreAudio for playback or recording since ML 10.8
Some say the USB2/3 ports on some iMac cause an issue
Some say the energy-saving low power modes and app-nap introduced since Mavericks change the buffer size the audio device is expecting from the CoreAudio driver and it causes clipping or skipping and bit-crushing
Some say our legacy hardware just won't work
Some say if you have a Fusion drive you are more likely to suffer CoreAudio hiccups
...Apple remains quiet (except for the admission they no longer plan on development of the Firewire CoreAudio component)
BTW have you updated the drivers
http://www.apogeedigital.com/support/duet-2