You're using the wrong tool - clone stamp is poorly named, but what it is designed to do is continuously sample from a relative area and merge into your current position.
Therefore as you move your brush, the area from which it is sampling will also move.
Try the patch tool - i think that may do what you are trying to do.
Cheers