Of course. Whenever you play the video fullscreen, it's being upscaled to the resolution of whatever you're playing it on. If you enlarge the window (or ANY video window, for that matter), it's upscaled to whatever the resolution of the window.
And, I might add, comparing the Apple DVD Player app and a Toshiba DVD player that does internal upscaling, the Mac does a DRASTICALLY better job of it. In fact, even my flat panel TV does a better job upscaling than the Toshiba player, but the Mac makes them both look crude in comparison.
The upscaling algorythms are *significantly* more sophisticated (as are the deinterlacing ones, in the case of interlaced source material), and take advantage of a whole lot more processing power available in a computer than your average consumer DVD player or not-outrageously-expensive TV.