I think I know what you're talking about, and I've found that behavior to be caused by one of three things:
1) Most often, the optical sensor just gets a bit confused, and needs to be sort of reset; to do this, just pick the mouse up off the surface for a couple seconds, then put it back down. This almost always works for me.
2) Could be dust confusing the optical sensor--just turn it over and blow on it a little. If it's really dirty for some reason (never seen that happen personally), you might have to clean it with a q-tip or something.
3) As the above poster already said, could be a pattern on your mousepad confusing the sensor. Some mice are more prone to this than others (Apple's mice are pretty good, but my newish Logitech is great), but try it on a differen surface and see if that helps.
Here's a really weird one--I have a matte black mousepad (one of those gel-filled ones), and although it works perfectly if you use it "right side up", if you try to use it with the mouse at a slight angle to the grain of the fabric (say, 10 degrees or more off of straight) it starts getting VERY confused. For a long time I thought it was dirt, until I realized that twisting the mousepad fixed it. That was with an Apple mouse; my Logitech doesn't seem to have nearly as much of a problem with it.
