I found a solution that I think works--give it a try, anyway. I think the folder is still showing up because there's old data still being stored in iCloud by the app.
If so, here's how to get rid of it. On one of your iOS devices, go to iCloud -> Storage -> Manage Storage. Under Documents & Data, select Show All. Find the app you want to remove, select it, hit Edit, and then Delete All. That will remove any old data that the app stored in iCloud, and should remove the app from the list. It may take several minutes for the change to be reflected on your Mac, but eventually, it should go away there as well.
I'm having the opposite problem: there are a number of apps that I know have data in iCloud, and that can retrieve that data, but that aren't showing up in iCloud Drive. I guess the developer has to enable showing an iCloud Drive folder?