I just tried it a couple of times on my mid 2009 macbook pro. I didn't have any rules before so that may factor in here. Created a rule. Closed mail. Waited a few. Still there on open. Tried a restart to see if it was like the notification center one, but it's still there.
Maybe it was an error when you installed yosemite? That's about all I can think of.