That is a little confusing. Do you mean you can play the music on your computer when it isn't connected to the internet but you can't play it on the ATV? I have two Apple TVs and use a Mini as an iTunes server (just sits in a cabinet runnning iTunes 24/7). I can access all my music (purchased and ripped) via the Computers icon on the Apple TV's. So something is clearly wrong with your setup.
If my computer is not connected to the internet, then most things just don't work on my AppleTV's. Evidently it needs to be able to connect to Apple's server to make sure you are authorized to access your iTunes library. So for starters, make sure that your computer is connected to the internet.
If that doesn't help, then perhaps your purchases really aren't stored on your computer. The suggestion about creating playlists doesn't make sense to me. I can access all my music through the AppleTV3 computers icon without doing this.
One possibility is that your iTunes library on your computer could be corrupted. This can cause all kinds of problems and can be tricky to fix. One thing that can cause a library to become corrupted is if you store it on an external device and start iTunes without having that device connected. This is because the index to the library is stored on your computer's internal drive. So if it can't find the files on the external drive bad things happen. To prevent this problem, you need to install both the library and the library database on the external drive.