I never noticed my iPad Mini 4 doing that.
I have a fancy theory:
You are maybe watching videos in a pitch black environment and you're holding the iPad so close to your face that, when a white object is displayed on the screen, the light emitted by the object on the screen bounces off your face and tricks the environment brightness sensor of the iPad Mini into believing that you've moved to a brighter environment, therefore leading to an automatic increase in screen brightness.
If this theory is correct, then you should stop noticing this "problem" if you turn OFF the automatic brightness feature in the display settings on your iPad.
I have a fancy theory:
You are maybe watching videos in a pitch black environment and you're holding the iPad so close to your face that, when a white object is displayed on the screen, the light emitted by the object on the screen bounces off your face and tricks the environment brightness sensor of the iPad Mini into believing that you've moved to a brighter environment, therefore leading to an automatic increase in screen brightness.
If this theory is correct, then you should stop noticing this "problem" if you turn OFF the automatic brightness feature in the display settings on your iPad.