I wonder what settings should I use in handbrake (on modern pc) so video would be playable in default video player
If you plan to run converted TV or film on a 2005 iBook G4, probably stick to preset General > Fast 720p30 mp4 (not mkv) and take care to verify the framerate FPS setting under the “Video” tab is set to “same as source” (as this is usually going to be either 23.97, 24, or 25). This ought to get you to a place where QuickTime, VLC, Mplayer, or CorePlayer can handle things just fine.
(Also keep in mind I use HandBrake 1.5.1 presently, not 1.6.0.)