I think I just figured out why Siri is having problems with my relations. I just discovered that the one's who's middle name is included in their contact name fail but the one's that are just a first name and surname work. So strange.
Confirmed. When I manually delete the loved one's middle name but keep their first and surname, it now works.