just randomly guessing things to check:
1) Are the two machines on the same network? (should be, otherwise, you shouldn't have been able to even try.)
2) Were did both machines have "home sharing" enabled for the same iTunes account? (I know that I have two, but I used the same account on both machines.) I don't see a way to turn it off, and I don't see a way to activate more than one account.
3) In Terminal.app:
> telnet <ip address or bonjour name> 3689
does it say: "Connected to <ip address>"?
this verifies that the port IS actually open.