There can be a number of reasons for this. First of all, check if you have an internet connection (pretty obvious but worth stating). Check your WIFI and 3G/EDGE/GPRS settings. If you're on 3G/EDGE/GPRS try doing a network settings reset, which usually fixes the problems.
Also sometimes the imap.gmail servers simply timeout for a while (happens occasionally). They usually start working soon after that...