Is there any specific guide you followed?
I've also been meaning to try Ubuntu on my Nexus 4 but again, been too lazy![]()
The CyanogenMod Wiki has build guides for all supported devices. Go to the device page and click "how to build for this phone" and it'll give you all the steps.