First, make sure that you download all the updates.
Then, be aware that what you see in your list on Apple TV does not diffrentiate between streamed or synced content. I think a good practice that a lot of people do is to sync just a few small items to your Apple TV (say 1 short movie, a few songs and some photos) Stream the rest. This will also leave you more room on your Apple TV for movie rentals, purchases etc.
If you Sync a lot of content from your Computer to TV, it will take a long time.