I leave them alone. For example, If I have a b-side that was only available on a CD single, then the album for that song would be the name of the a-side single. For example:
Artist: Pearl Jam
B-Side Song: Alone
B-Side Album: Go (Single)
I try to keep my library as accurate as possible.
EDIT: I don't like dup songs either, so if I have the a-side on the proper album, then I delete the a-side from the single "album". For groups that I have a bunch of b-sides, I have a bunch of albums under that artist. I like it that way because then I can have the art for each single that I have at least 1 track from. It does make cover flow a bit cluttered for certain artists, but I rarely use cover flow!