C# .NET 3D Frameworks - Part I
C# .NET 3D Frameworks - Part II
C# .NET 3D Frameworks - Part III

Since I figured out that I didn't want a wrappered engine for development. I set about writing a simple demo application in .NET using C# and "DirectX For Managed Code" directly. Why did I do this ? Because I fucking never learn!

Here on this blog I went on a long diatribe how engine wrappers get invariably abandoned, then I assume because it's not really a third party engine but a library from big Microsoft and since I envisioned MS was still in the process of going to all .NET, Virtual Machine bla bla.... which has all these advantages like... um... garbage collection... which you really want, except if you want to sell your software... . So my gut expectation was, this would not be abandoned so quickly. Direct3D showed up in my Assembly Browser in Visual Studio and googling around I found a good start project to learn from with: D3DControls

After a few days of diddling around with it, I had to notice that the MeshBuilder class is nowhere to be found, although I really need it. Now after quite some research it turned out:

The current DirectX SDK (March 2008) and the current DirectX Runtime installer both install the Managed DX 1.1 assemblies, but August 2007 was the last version of the DirectX SDK that included support for developing Managed DX 1.1 applications. In fact, the last time we updated MDX 1.1 was April 2005. It has been deprecated for some time.

Currently I am pondering if I can repeat my mistake by going to XNA or if I wise up and go native.

#include <stdio.h>

main()
{
  long long  i, j;

  for( i = -1; i <= 1; i++)
     for( j = -1; j <= 1; j++)
         printf( "%lld/%lld=%lld %lld%%%lld=%lld\n", 
               i, j, i/j, i, j, i%j); 
  return( 0);
} 

-1/-1=1 -1%-1=0
-1/0=0 -1%0=0
-1/1=-1 -1%1=0
0/-1=0 0%-1=0
0/0=0 0%0=0
0/1=0 0%1=0
1/-1=-1 1%-1=0
1/0=0 1%0=0
1/1=1 1%1=0

replacing long long with int, and %lld with %d yields

-1/-1=1 -1%-1=0
-1/0=0 -1%0=-1
-1/1=-1 -1%1=0
0/-1=0 0%-1=0
0/0=0 0%0=0
0/1=0 0%1=0
1/-1=-1 1%-1=0
1/0=0 1%0=1
1/1=1 1%1=0


Of course division and modulo by zero are undefined... yet...

                              

                              
                           

Until last week I recommended Mac OS X Server as a viable and ultimately more cost effective way to manage a server than lets say Linux. I used to run my shop exclusively on a multitude of Linuxes and got sick of it. Most of my servers are Mac OS X Mac minis and my mail server is hosted on a Mac OS X Server Mac mini.
The advantages of Mac OS X Server to me were, that I could manage Mail easily with a GUI front end and that I didn't have to bother with package and OS updates manually, because I trusted Apple to get that part right. That was worth the €500 to me.

Why it was ultimately a near fatal mistake was something I didn't consider at the time. But let's tell the story first.

First mistake, innocent enough

About three weeks ago or so I decided to perform a softwareupdate on my Mac OS X Server from 10.5.5 to 10.5.6. I am running a plain vanilla setup: email server and iCal server and nothing else, configured with Server-Admin.

Something became incredibly bugged in 10.5.6 as many of my emails, especially those with attachments didn't get mailed out. The bad thing was I didn't notice that. Talk about buzzword bingo, but email is indeed "mission-critical" Alarm bells did ring, when I got an email from my biggest customer saying:

I worry that you may have a second thought about working with us on
the project. The agreement is finalized, ready to be printed out...

Plz to send mai emails ?!

So I took a look online on the email server and had to notice a sizable out-queue with some of my mails in there. While inspecting the machine it became apparent, that the machine was incredibly slow. And sometimes it even froze for a couple of minutes.
A look at the logs or at top didn't reveal anything problematic. The primary suspected was the hard-disk, but neither smartctl nor Disk Utility indicated any problems. Because I trusted Mac OS X Server like the naive fool I am, I deduced it was a hardware defect and bought a new Mac mini as a replacement. The most likely theory being, that a fan was busted, though the Mini didn't get all that hot, but it kind of fit the "freezing" symptoms.

New hardware to the rescue ? Fail!

This turned out to be a bad mistake and something one should really think about, if one is running Mac OS X Server.
I placed the old Mac mini in Firewire Target Disk Mode (yay for that feature!) and copied my 10.5.6 installation to the new Mac mini. That worked beautifully, but the problems remained. So my guess that my problems were due to faulty hardware was most certainly wrong.

The new Mac mini came with Mac OS X 10.5.6 (client) and as it turned out it really needs it. My Mac OS X Server DVD is 10.5, so I can't make a clean install with it. I kept my old setup on the new Mac mini and installed 10.5 and 10.5.5 in order on the old Mini, then I recreated my setup and restoried data from the new Mac mini (now in Target Disk Mode). Another cheer for Firewire Target Disk Mode. When everything was working fine, I reversed the roles and restored the setup to the new Mac mini to at lease enjoy the better performance of that machine.

Unfortunately, that new Mac mini really needs 10.5.6 and not 10.5.5. It wouldn't boot.

In conclusion the conclusion

The critical flaw of Mac OS X Server is, that you don't get an installable updated version of it for new hardware. If the old Mac mini would have been really dead, there would have been no reasonably quick way of getting that Mac mini working again with Mac OS X Server short of throwing another €500 into Apple's mouth.

My conclusion is, that Mac OS X Server (not Mac OS X) is not a viable server platform for me anymore.

Looking closer, more wrinkles

Talking to other people about my problems revealed, that I am far from the only one struggling with Mac OS X Server currently, though the problems are diverse. Apple has become apparently fairly sloppy in quality assurance (see for instance here and here) and looking at the debug output of my email program, where there appears to be basic unfixed incompatibilities between amavisd and the system ps, I am not even convinced anymore that the packaged software is properly tested before release.

I can't quite put my finger on it, but I think there is a bug in PPC NSInvocation, when the method to be invoked is performSelector:withObject: as I had sporadic crashes in _NSI3(). I suspect a register is clobbered there somewhere.

I had a hellish time getting a particular piece of software to run.Wading knee-deep through disassembled code stumbled upon the following in -[NSInvocation invoke]

...
0x90157310 <+144>: li      r3,5
0x90157314 <+148>: bl      0x901383ec <_CFExecutableLinkedOnOrAfter>
0x90157318 <+152>: cmpwi   cr7,r3,0
0x9015731c <+156>: bne-    cr7,0x90157358 <+216>

0x90157320 <+160>: lwz     r0,24(r27)
0x90157324 <+164>: andi.   r2,r0,512
0x90157328 <+168>: bne-    0x90157334 <+180>
0x9015732c <+172>: lwz     r4,8(r30)
0x90157330 <+176>: b       0x90157338 <+184>
0x90157334 <+180>: lwz     r4,4(r30)
0x90157338 <+184>: addi    r1,r1,96
0x9015733c <+188>: lwz     r5,16(r30)
0x90157340 <+192>: mr      r3,r27
0x90157344 <+196>: lwz     r0,8(r1)
0x90157348 <+200>: li      r6,1
0x9015734c <+204>: lmw     r26,-24(r1)
0x90157350 <+208>: mtlr    r0
0x90157354 <+212>: b       0x901565e8 <__NSI3>

0x90157358 <+216>: addi    r1,r1,96
0x9015735c <+220>: lwz     r0,8(r1)
0x90157360 <+224>: lmw     r26,-24(r1)
0x90157364 <+228>: mtlr    r0
0x90157368 <+232>: blr

The mysterious _CFExecutableLinkedOnOrAfter

Now my problem definitely happened in _NSI3 and this code path is selected based on _CFExecutableLinkedOnOrAfter on which Google finds for example this entry on an Ars Techica board:

Most frameworks don't have such huge binary compatibility issues that they need new versions. Then again, Mac OS X has _CFExecutableLinkedOnOrAfter() that system APIs use to determine if an incorrect 10.4 behaviour should be maintained on 10.5. As soon as they compile/link on 10.5 using the 10.5SDK, they are saying they no longer want the 10.4 legacy behaviour.

Here's where it gets strange (i.e. complicated). My test application works when it is compiled as a 10.5 tool, that links to a 10.4 framework.
But when I compile the tool against the 10.4 SDK and set Deployment target to 10.4, it crashes. Basically the other way as I would expect.

What _CFExecutableLinkedOnOrAfter does

Which begs the question, what is _CFExecutableLinkedOnOrAfter doing after all ? Searching with Google yields this a little outdated version http://www.gnu-darwin.org/www001/src/DarwinSourceArchive/expanded/CF/CF-368/Base.subproj/CFUtilities.c

#define checkLibrary(LIBNAME, VERSIONFIELD) \
    {int vers = CFGetExecutableLinkedLibraryVersion(LIBNAME).primaryVersion; \
     if ((vers != 0xFFFF) && (versionInfo[version].VERSIONFIELD != 0xFFFF) && ((version == 0) || \
(versionInfo[version-1].VERSIONFIELD < versionInfo[version].VERSIONFIELD))) \
return (results[version] = ((vers < versionInfo[version].VERSIONFIELD) ? false : true)); }


CF_EXPORT Boolean _CFExecutableLinkedOnOrAfter(CFSystemVersion version) {
    // The numbers in the below table should be the numbers for any version of the framework
    // in the release.
    // When adding new entries to this table for a new build train, it's simplest to use the 
    // versions of the first new versions of projects submitted to the new train. These can 
    // later be updated. One thing to watch for is that software updates
    // for the previous release do not increase numbers beyond the number used for the next release!
    // For a given train, don't ever use the last versions submitted to the previous train! 
    // (This to assure room for software updates.)
    // If versions are the same as previous release, use 0xFFFF; this will assure the answer 
    // is a conservative NO.
    // NOTE: Also update the CFM check below, perhaps to the previous release... (???)
    static const struct {
        uint16_t libSystemVersion;
        uint16_t cocoaVersion;
        uint16_t appkitVersion;
        uint16_t fouVersion;
        uint16_t cfVersion;
        uint16_t carbonVersion;
        uint16_t applicationServicesVersion;
        uint16_t coreServicesVersion;
        uint16_t iokitVersion;
    } versionInfo[] = {
	{50, 5, 577, 397, 196, 113, 16, 9, 52},		/* CFSystemVersionCheetah (used the last versions) */
	{55, 7, 620, 425, 226, 122, 16, 10, 67},	/* CFSystemVersionPuma (used the last versions) */
        {56, 8, 631, 431, 232, 122, 17, 11, 73},	/* CFSystemVersionJaguar */
        {67, 9, 704, 481, 281, 126, 19, 16, 159},	/* CFSystemVersionPanther */
        {73, 10, 750, 505, 305, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF},	/* CFSystemVersionTiger */
        {0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF},	/* CFSystemVersionChablis */
    };
    static char results[CFSystemVersionMax] = {-2, -2, -2, -2, -2, -2};	/* We cache the results per-release; there are only a few of these... */
    if (version >= CFSystemVersionMax) return false;	/* Actually, we don't know the answer, and something scary is going on */
    if (results[version] != -2) return results[version];

    if (_CFIsCFM()) {
        results[version] = (version <= CFSystemVersionJaguar) ? true : false;
        return results[version];
    }
    
    checkLibrary("System", libSystemVersion);	// Pretty much everyone links with this
    checkLibrary("Cocoa", cocoaVersion);
    checkLibrary("AppKit", appkitVersion);
    checkLibrary("Foundation", fouVersion);
    checkLibrary("CoreFoundation", cfVersion);
    checkLibrary("Carbon", carbonVersion);
    checkLibrary("ApplicationServices", applicationServicesVersion);
    checkLibrary("CoreServices", coreServicesVersion);
    checkLibrary("IOKit", iokitVersion);
    
    /* If not found, then simply return NO to indicate earlier --- compatibility by default, unfortunately */
    return false;
}

_CFExecutableLinkedOnOrAfter scans the linked shared libraries by name until one matches. It extracts the version compares it with it's internal table and then it's all set.

Apart from my problem, It's obvious, that this can only work in the very simplest circumstances, where you can recompile everything in your app, that is not from Apple and you don't expect to load third-party NSBundles. Technically I think _CFExecutableLinkedOnOrAfter is a pretty poor idea.

No solution but a workaround

So it looks like it _CFExecutableLinkedOnOrAfter is not really my problem. I fixed my problem, my moving performSelector:withObject: "outside" of the NSInvocation. In case _NSI3() is doing some stuff, that should only be done in 10.5, as I suspect it does, then -[NSInvocation invoke] should rather read:

0x9015731c <+156>: beq-    cr7,0x90157358 <+216>