Adobe applications can be weird, and there are many things that can slow them down, so if it's another factor, your benefit from the cpu will be limited. In this case the concern is because the OP is dealing with larger files than before. I'm willing to bet Lightroom would gain the most there from a faster cpu. With photoshop it always ends up being a bunch of testing for me. Certain palettes take up more ram and overall resources. Thumbnails can bog things down if you're using a lot of layers. Setting the memory allocated to PS too high can starve the rest of the system. It's just not easy to offer generic advice without seeing when and how it slows down.
It might be an issue of ram or settings, but I haven't seen how much you're running at once or the real behavior of the machine. You only mentioned that it slows down since you went to larger files. I don't know anything about if you're running both at once or layers used and general workflow. You never mentioned what you're shooting (I was curious).