Frankly, I'd never read that document, as the apps I've written don't use a lot of memory and so it hasn't been a concern. Now that I have, I can see that the problem is largely one of poorly-written applications. Everything needed is already there in the CURRENT iPhone OS. No need to wait for 4.0.
Apple's handling of read-only code pages is a departure from the convention in *nix systems, which typically do NOT support re-loading of code pages from the executable file. Instead, iPhone OS is more similar to how Microsoft Windows does things. In traditional *nix systems, there is no distinction between code and data pages in handing virtual memory - BOTH are written-out to backing store when pages are discarded, and code is never re-loaded from the original file. (One positive side-effect of this is that *nix systems are easy to hot-update, as there isn't the concern about changing an executable file while the application is running as there is under Windows.)
So, we can really cast-away much concern about executable size. Pages of code will be paged-in as needed during execution. This will slow things down, but there's no reason for apps to "run out of memory" for code execution.
But without backing store for data pages, applications have to be careful about how much working storage they use, and how they use it. I suspect that a lot of apps are being pretty dumb about how they use memory.
Somebody asked above about resources - bitmaps, etc. I suspect that one of the problems right now is that application authors are not using the virtual memory system effectively. One technique that can be used is to map resources files into memory using mmap. The file can be read-only OR read-write. So, there's full read-write virtual memory WITH backing store! It just isn't *automatic* backing store.
I'd bet that 99% of iPhone apps aren't doing this, but instead are allocating some memory and reading the resource into that memory. You have to be careful about just what format the file data is in - it does you no good if some kind of format conversion is needed on the data in the file. (Even if so, you still could read from or map a file that has data in one format, and write the modified data to another memory-mapped file that contains the transformed data...)
http://www.alexcurylo.com/blog/2009/05/14/tip-iphone-virtual-memory/
From the link above:
“you can use mmap when running on the iPhone just the same way as you use it when not running on the iPhone”. So man mmap is your friend.
Steve Jobs wants developers to RTFM. I suspect (hope) that they are going to be brutal with developers of background apps to make sure they are making use of virtual memory effectively and not over-using memory resources. I wouldn't be surprised to see them start cracking-down on sloppy non-background apps, as well.
