Wow, the amount of misinformation in this thread is pretty astounding.
I'm about 99% certain I have this figured out, based on the keynote demo and the info posted on various blogs since. Basically, iPhone OS 4 no longer quits an app as soon as the user puts it into the background. This is that fast app switching feature, which nobody seems to understand is related to multitasking, but I'm pretty sure it is. What iPhone OS 4 does instead of quitting a background app is, it suspends it. This means the app is still loaded. This is why you can instantly switch back to it.
Here's where things diverge from desktop-style multitasking, though: the scheduler doesn't give background apps any CPU time by default. The scheduler only gives background apps CPU time if they explicitly ask for it using one of the relevant APIs, and then only as needed to do the kind of background processing necessary to the task the app has told the system it wants to perform.
So, what happens then depends on what the app has told the OS about what it wants. In some modes, the app gets suspended, but in response to certain events, like the user's location changing, it automatically gets unsuspended, and the OS notifies it of the location change and gives it some CPU time to execute whatever code it wants run to deal with that change.
In some other modes, like with background audio, the app just plain runs in the background, at least from what I can see. You could, as far as I can tell, just declare your app plays background audio even if it doesn't, and have it do whatever sort of processing it wanted to in the background. There's nothing technically preventing this. Though you'd be cruising for an App Store rejection, I'm sure.
And then there's the "I need more time" mode, in which apps that would otherwise quit can tell the OS they need extra time to finish something up. In this case, the app can do whatever it likes -- until it runs out of time, at which point the OS will ask it to quit, and then kill it if it doesn't. According to the article on Engadget (IIRC) apps can ask for up to five minutes.
So, there's quite a lot of "real multitasking" in here. Developers just aren't allowed to use it for whatever they want for arbitrarily long periods of time, because Apple believes that would kill the battery and result in sluggish performance.
And all of these restrictions are deliberate, not due to the underlying technical implementation of the OS, which means that as the hardware gets more powerful, Apple can lift any of them that it's appropriate to lift at any time.