It shouldn't damage the speakers to use full volume.
However, the audio will sound more pleasant and understandable at a lower volume setting, as the small speakers in a phone have a lot of distortion at high output levels. Unless you are in a high noise environment where you really need the high volume to hear the phone, it's a good habit to use lower levels.
When I want high volume, I use headphones - then you can get high volume and low distortion at the same time! And, if there are others around you, they won't be annoyed.