As others mentioned, the problem is that the 10.5.4 install DVD doesn't have the necessary files for the new mini.
The best way to do it is to install Leopard Server to an external hard drive. Then, download the 10.5.7 Leopard Server Combo update from Apple and install that to the external drive. Finally, clone the external to the Mac mini. Starting up will walk you thru the config process.
Someone mentioned that you can upgrade to Leopard Server from Leopard Client, but we've seen quite a few errors doing this. The external drive router is the safest.
Hope that helps.