As someone with a similar setup and amount of music, I can tell you the setup process will unavoidably take a lot time. Even if you disable the album artwork download you'll be dealing with iTunes analyzing each track for gapless playback. You can stop the process manually, but each time you relaunch iTunes it will start reanalyzing. Unlike album artwork, there is no setting to disable this process.
The extreme slowdown you were seeing was iTunes trying to analyze gapless playback and process the artwork at the same time. I suggest starting this before you go to bed/work/school and hopefully it will be done when you wake/return. This is a one-time process, so you'll only need to do it once.