Did you read the instruction manual that came with the computer? It has a nice little illustrated diagram in there.
But it's quite easy:
1) Get something pointy, but not too sharp so you don't scratch it.
2) Use your pointy object to push in the little round button on the bottom of the remote (the thin end opposite the part where the signal comes out).
3) That will cause the battery tray to come popping out. Within is a standard coin-sized battery. Take it to Radio Shack or wherever, and buy the exact same battery (there's a number on it somewhere to reference) to replace it.
4) Put the new battery in, and you're done.
[edit: woah, two different responses while I was typing mine... too slow!]