You can do, as long as you delete, and NOT select "delete from iCloud".
You would also lose them all if you cancel your iTunes Match subscription, to they decide to drop it.
I do not think that the iTunes Match take up is huge, so I think it might go the way of Ping.