Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
You don't need to install the software, even on Windows.

Plug your Mac into the router and use a browser to go to 192.168.1.1.

The manual should list the username/password, which'll be something like "[blank]/admin" or "admin/admin".

You can then do everything via the browser.
 
Advice: Change the login name / password if you plan on doing wireless.

Also strongly advise turning on WEP and/or MAC filtering to keep your network private. Otherwise you're dropping your pants to the neighborhood, in a bad way.

http://computer.howstuffworks.com/home-network3.htm
http://computer.howstuffworks.com/router.htm
http://en.wikipedia.org/wiki/MAC_filtering
http://en.wikipedia.org/wiki/Wired_Equivalent_Privacy
http://en.wikipedia.org/wiki/Wifi
http://en.wikipedia.org/wiki/Linksys_WRT54G_series
 
Just to be clear - plug the Mac into the router via an ethernet cable, or you should be able to browse via Airport to the router, which is likely set up with a default "linksys" SSID.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.