You mean, like in Jet's case, perhaps the webmail works fine because HTTP connections through the ISP are working fine but the POP/IMAP connections, which use different ports / different routers / etc don't work fine?
But it seems like he's not saying that mail takes a long time to download, but that it comes after a delay.
Jet, Do you ever get exclamation point icons next to your inboxes in mail.app? It could be that it is experiencing failed attempts to get at the POP/IMAP server (usually it puts an exclamation point next to the inbox but does not always pop up a warning or error).