Based on your system specs using Quicksync, you are not going to get any higher quality unless you either go with the fastest Mac Mini 2018 and CPU encoding or accept a slightly lesser quality with Quicksync. You can change the settings in VideoProc to re-encode to a higher quality file using just the CPU rather than Quicksync, but that may double or even quadruple your encoding time. Your Macbook Air wasn't designed for this type of work unfortunately as it is only a dual core.
Which is why I have a Mac Pro with a lot of cores and threads to do quality encoding for my DV final output if I need to use it with another project, but I think the best economical approach for you and that is you are looking for the best quality output consistently for all your tapes would be to use a professional video conversion service. Not all are alike. You need to find one that also works with movie studios or other professional videographers and they do charge quite a bit more than normal services and the reason is that, they have professional commercial grade equipment that will preserve your footage as close to original as possible. Whereas some other services may use prosumer grade equipment that is slightly better than your deck now and offer quality that is slightly better.
In terms of file output, these services will offer you either in non-compressed form or higher quality compressed form. Prices do start to go up when you want to achieve the same quality as your DV clips and that is because, you start to need to use much faster computer and high grade equipment to capture your tapes. You can certainly improve when using better software on your Mac, but just judging from what I am seeing in your response, I think you are looking for a professional quality commercial grade service in which your current gear won't be able to deliver.