I'm actually having a similar issue. I have a rMBP and a MacMini, basically sitting next to each other, both on the same 5Ghz network, and both using FileZilla. Connecting to an SFTP server I use, the MBP gets 1-3MB/sec while the MacMini gets 8MB/sec (more if I have multiple transfers going).
When I use Speedtest.net, my MacMini pins the needle at 120Mbps while the MacBook bounces around, ranging from 40Mbps to 80Mbps. Both are using the same Speedtest server in my area. I understand the Speedtests are not 100% realiable, but the SFTPing speeds on the same server is what bothers me most.
Only difference is that the Mini is on Mavericks and my MB is Yosemite. Tried removing/remaking the network profile and I even installed the latest 10.1.2 beta. It's baffling me. Tried two different routers (Netgear r7000 and Asus TM-AC1900) . Tried connecting to 2.4. Tried changing the channels, no difference. Not using QoS of any sort.