Is there a graphics card installed? If so, it'll likely take DVI as its output. DVI is both forwards-compatible with HDMI with an adapter and if it's DVI-I (has the plus-pin instead of the minus-pin) also VGA. If not, install one. Make sure it's compatible with OpenFirmware/Mac OS X first or it won't show a picture.
With the exception of the Late 2005s, G5 towers will take a regular C13 power connector, same as on your modern desktop.
You can use USB keyboards and mice just like on a modern Mac, same with Ethernet.