objc_msgSend, the strange code revealed

I can't take much credit for this. The reason d'etre for the new objc_msgSend code was discovered by Dietmar Planitzer.

Because of the NDA requirement, I can't spell the solution out, but people with access to Tiger will be able to find this hidden feature. Also it's more fun this way. Here goes:

First look at Foundation, you should have a bash shell for the copy/pasting of this stuff:

otool -L /System/Library/Frameworks/Foundation.framework/Foundation | 
tail -3 | 
head -1 | 
awk '{ print $1 }' 

just notice that it's there. You don't need to do anything with it.
Now we extract a new compiler option from the libc library, which helpfully provides it:

NEWFLAG=`strings /usr/lib/libobjc.dylib | 
grep fobjc | 
awk '{ print $6 }' | 
cut -d , -f 1`
echo "NEWFLAG=$NEWFLAG"

Now make yourself a small test file:

cat > x.m << EOF
#import <Foundation/Foundation.h>

main()
{
   NSObject   *p;

   printf( "retain: %lx (%lX %s)\n",      
         (long) [NSObject instanceMethodForSelector:@selector( retain)],      
         (long) @selector( retain),
         (long) @selector( retain));
   printf( "autorelease: %lx (%lX %s)\n", 
        (long) [NSObject instanceMethodForSelector:@selector( autorelease)], 
        (long) @selector( autorelease),
        (char *) @selector( autorelease));
   printf( "release: %lx (%lX %s)\n",     
        (long) [NSObject instanceMethodForSelector:@selector( release)],     
        (long)  @selector( release),
        (char *) @selector( release));
}
EOF

compile and run it.

gcc -o x1 x.m -framework Foundation ; ./x1

There shouldn't be anything unexpected about the output. Now lets use the new compiler option

gcc $NEWFLAG -g -o x2 x.m -framework Foundation ; ./x2

Foundation should print out a dead give away in the start, but also look at the other values that are output.

Finally to round it off, examine with gdb the instructions of interest:

cat > gdb.batch << EOF
set prompt 
fb objc_msgSend
run
x/3i \$pc+4
quit
EOF

( cat gdb.batch | gdb -q x2 | tail -3) 2> /dev/null

Ok you need to know a little PPC assembler, but it shouldn't be too difficult to understand what an XOR together with a compare do.
Hint: $r4 is usually the selector address, and r11 is a scratch register.