There is another way to look at your zero page-filling issue, and one you might try to get Apple to adopt. Rather than zero-ing a page on demand (as is done currently) the kernel maintains a list of pre-zeroed pages. This list is refreshed by some kernel-thread that's running every so often, and that takes the pages it want's to pre-zero from say the free-list (or a new free-but-dirty) list. Now the work still has to be done, but with luck it'll be done during some dead time while we're waiting on a disk IO or whatever.
A good suggestion and a suggestion that would work. I had asked the same thing Jim Magee off list:
In the back of my mind lurks that factoid, that Mac OS X has a queue of zeroed RAM pages. If the queue is full, then probably a page fault can be serviced in 4-5 us. When it becomes exhausted, like my test program will do it will take 15 us. Correct ? How big is that queue ? And at what rate does it refill ?
The answer was:
Nope. Nothing is done ahead in this regard. Other Mach implementations experimented with this (having the idle loop zero pages ahead instead of doing "nothing"). But it tended to play hell with caches and/or power-management (we _really_ put the processor to sleep in idle).
I would suspect the cache is more of an issue, since the system oughta quiet down after enough pages have been zeroed. Since cleaning a page does invalidate a sizeable part of the cache, this would be quite detrimental to running code I would think.
If I get a little more idle time, I might wrap this up and make Parts I to VI a bona fide "Mulle" article. :)