Precisely what? Visual voicemail and push messages do not work when DATA ROAMING is set to "OFF".Precisely.
As you pointed out, there is ALWAYS an international charge for voicemail (or even a call you ignored that didn't leave voicemail), unless you have the carrier disable voicemail before you leave. That is nothing specific to AT&T, the iPhone, or visual voicemail.
If you're paranoid, reset your usage on the iPhone right before you leave so that data shows all zeros. Watch the usage and it'll be very apparent if any data is used after that.