Yes, it does. I now have two partitions: One running the experimental Gentoo ppc64 port, and another one running the overly-conservative Debian woody ppc32. The plan is to boot into Gentoo, and run publicly-accessible production services within the Debian woody chroot.
So how did I make it? Well, I gave up on the idea that the usual installation process of any distribution would work. So instead of trying to fix up whatever goes wrong in the installation scripts, I just escaped to a shell ASAP, run mac-fdisk, mkfs.ext3, extracted the stage3.tar.gz and did the rest of the Gentoo install.
Debian was then installed using the convenient debootstrap tool.
One of the major remaining questions is however: Does the Apple XServe Hardware give you anything similar to Sun boxes, where you could just send break over the serial line and get into OpenFirmware? This is very convenient for remotely resetting machines without any local 'reset-staff' present.
After some chatting with Benjamin Herrenschmidt, apparently nobody is working on getting fan rpm/speed/temperature control implemented on the XServe so far. Well, as it's a rack-mounted machine sitting in some hosting center I don't really care about the noise anyway.
More interestingly, the Apple KeyLargo2 based machines have a Hardware Watchdog. Driver Source code is available within the public part of the Darwin kernel, so it should be easy to implement a Linux driver for this. Maybe I'll find some time to dive into this.