It's great to hear your experience. Thank you!
I've been using both Yosemite and Mavericks (dual-boot system) everyday since last June on both my MacBook Air and Mac mini. They do not have Retina or HiDPI displays.
For myself, I'm 16-years-old, and my eyesight is 5.1(L), 5.0(R). So I should have no issue on my eyes.
When I work for more than one hour under Apple-designed-Yosemite apps such as Finder or iWork, I experience eye-strain.
I think there are four contribution factors here:
1) Helvetica Neue
This is partly resolved by a utility I made with Alex, LucidaGrandeYosemite, but even with this, Yosemite still causes eye strain for me, which led me to discover the next issue.
2) The overuse of white color
Mavericks had a lot of light-grey to dark-grey gradient throughout the system, most noticeably applications' frameworks. However, Yosemite replaces them with much whiter gradient, and they cause eye-sour during darker environment (Dark mode do not help with these as they only change the color of Menu bar and Dock). Certain app also has textures stripped away, which previously served as resting area for my eyes and now they're completely white.
I always think simple design should be ones with a clear, simple logic instead of being visually empty. (Just to me,) it feels Yosemite tries to be simple with the latter, not the former.
3) Text on sidebar are a tad shallower to accommodate the visual style
A lot of text throughout the system are shallower, shifting from dark grey to slightly lighter grey. This is most pronounced in sidebar (or as Apple calls them, "Source Lists"), and turn on "Increase Contrast" will partly revert the text darkness to the previous behavior, but rest of the UI looks pretty bad.
4) Buttons bleeding into each other
Buttons are even lighter white, and they do not have clear separators. The white-ness of such button shape seems awkward, as they neither separate themselves from the title bar, nor distinguish themselves as individual buttons. But at the same time, they make my eyes painful because of the stark contrast of color within the title bar. Again, turning on "Increase Contrast" will partly make things somehow better (and even remind me of OS 9!), but rest of the UI looks pretty bad.
5) Poor sub-pixel rendering for selected file under Finder and Safari tab names.
This is not a personal preference. It's a bug (and has been marked as a duplicate, which means I'm not imagining this), where when file name is high-lighted in Finder, the blue overlay makes the text above have really bad sub-pixel rendering.
There are probably also other reasons, but for now, that's my best guess on why it stresses my eyes. Using a Retina/HiDPI display seems to migrate these issues, but as a 15-year-old, I don't have money for those...