The swipe to navigate is WONDERFUL. Have you considered that perhaps yours is broken?
I think this is more of a preference than fact. I dislike the endless swiping when browsing libraries. And I have two Siri remotes, one on a ATV4 and one for ATV4k, and they both behave the same way.
it is way too easy to swipe past what your intended place to stop.
It is especially annoying when putting in passwords, or entering something into a search.
It is SO much better than the aluminum remote! Particularly when using swipe to scrub.
Scrubbing is pretty much the only thing that is better about the touch pad imo. The D-pad has been around for a long time, and I doubt it will disappear anytime soon.
If you think anything about Apple TV/Siri remote is bad, try working a Roku.
I don't own a Roku, but a few days ago, I purchased a new TV that had Roku, and have been mostly impressed by it. The biggest thing missing from it is the gaming library that tvOS has. I am not replacing my ATVs with Rokus, but I will probably buy less of them in the future for our secondary TVs, and just use the built-in Roku if it has it.
As for the remote, it is the TV's remote, not necessarily a Roku remote, but it is fine, as I much prefer D-pad over endless swiping on a touch pad.
Another point about the Roku, I downloaded a Roku remote app and it is much better for navigating and app selection than the Siri remote and the ATV remote app imo.