Some of my "Smart Mailboxes" listed in the left panel of Mail's Message Viewer window (Mail's main window) sometimes show a small but non-zero unread count, even though all of the messages in the folder were, in fact, read days ago. E.g.:
Right now I have one smart mailbox showing 2 unread messages when everything has been read. Your 41 is a bit extreme.
Spurious unread message counts is one of Apple Mail's idiosyncrasies for as long as I can remember - and not just for smart mailboxes. Either tear your hair out finding the cause or live with it.