Try disconnecting every mouse, trackpad, or other cursor-positioning input device. Sometimes they can generate tiny "move" signals that fool the OS into thinking that input is happening. This will prevent screen-saver or sleep. The signals might not be enough to move the cursor, but they can still prevent sleep.
If the problem continues after they've all been disconnected, make sure you don't accidentally have a wireless input device paired, because it might also be generating spurious inputs. A typical example is a Bluetooth mouse kept in a backpack, that's accidentally turned on.
If it still continues, please post a list of all the USB input devices listed in the System Info.app under the USB pane. There's also a way to get the list using a command-line tool in a Terminal window, but I'd need to look up the details. It's simpler if you expand the list of USB devices and post a screenshot of it.
If you have Bluetooth devices, there's also a Bluetooth pane in System Info.app.