The Japanese are with you and they have a history with swords as well
It is actually the indirect reason as daft as it sounds though more directly related to the drivers of horse drawn vehicles.
As the majority are right hand dominant people, that led to them sitting on the right to give plenty of space for their whip hands and so at the time motor vehicles became popular, that was just the way it was done and so it stayed.
Today, two thirds of the world drive on the right side. Whilst the elite nations drive on the correct side...