The most relaxing flight of my life

The day started early for me. Taking the local train to the airport at 4:09am. Then some annoying KLM ground staff who told me to only bring one piece of hand luggage next time. If you know me, then you certainly know that I find nothing more annoying than people with large carry-on luggage. All I had was a laptop bag and a very small camera bag. No backpack, no trolley. I started a discussion with that staff member, indicating to him that he should read his own regulations before trying to lecture me. Neither laptops nor cameras are allowed in the checked-in baggage, and usually they do not even count as 'piece of hand baggage' but are allowed in addition to that piece. I'll make sure to re-check with KLMs current regulations and file a complaint with KLM. Given my last trouble with the KLM flight to Bangalore last December, I have good contacts to their customer care department now.

Anyway, the situation drastically improved in the Amsterdam - Sao Paulo flight. A Boeing 777-200 with the latest in-flight entertainment system: 111 full-length cinema movies, lots of TV shows (not that I care about them) plus individual music playlists.

It comes even better: I had a whole three seat row for my own. After watching "Memoirs of a Geisha" (I already knew the two Bollywood movies they had), I had a very comfortable sleep. Next time I woke up was already in Brazilian airspace. The only time I had that much space was when going to Israel, but that flight was short enough to not care about that extra comfort.

So in the end it wasn't of much use that I just bought a second battery for my notebook computer ;) Anyway, I'm sure my next, less relaxing long-haul flight will not be too distant. boots on EZX phones

I've finally managed to get a 2.6.x kernel running on the Motorola A780 and E680. Apparently the problems I encountered are part of 2.6.14 (which was current mainline when I started the port). After merging my patches into, everything suddenly worked fine ;)

So what I've got now:

  • kernel booting on both A780 and E680
  • USB host controller towards Neptune BP working
  • USB device controller partially working
  • MTD support for all flash partitions
  • SD/MMC support on E680 (TransFlash on A780 not working yet)
  • Framebuffer working on both models, with nice 4x6 tiny font

The main obstacle now is that TransFlash on the A780 is not working yet. The A780 is actually more important than the E680. For some strange reason, all the response bytes from the TF card appear to be zero (at least that's what the response FIFO of the PXA27x embedded SD/MMC controller reports). I've already tried a lot, but am a bit clueless after many hours of trial and error :(

State of OpenEZX 2.6.x kernel development

During my two days of EZX phone hacking, I've made significant progress. Probably the most important discovery was how to get a serial console on the USB plug, enabling other people to do further kernel development without physically modifying the phone - but it's still a long way to go.

A current list of TODO's:

  • find out why kernel doesn't boot with CONFIG_IWMMXT
  • find out why E680 SD/MMC works, but not A780 TransFlash
  • debug and fix pxa27x_udc in order to provide usbnet (nfsroot!)
  • debug and fix mtd support in order to be able to access system flash
  • port and cleanup video and sound drivers
  • port Motorola-specific SSP/SPI drivers into 2.6.x generic SPI stack
  • port all the driver specific dpm bits from Motorola's 2.4.20 to 2.6.x
  • clean up the already working keypad drivers
  • finish re-implementation of mux_cli and grpsv modules
  • look into re-implementing the proprietary flash fs drivers, though I don't think that is particularly important, we could run our code 100% on SD/TransFlash
  • create a modified bootloader that allows for multi-boot configurations. It could actually include SD/TF support for booting kernels from there.
  • check how the other (later) Motorola Linux smartphones differ and merge their device-specific code into our 2.6.x kernel tree
  • last, but not least, we need to do something about userspace. I'm not a GUI guy at all, and I haven't yet thoroughly investigated all the existing projects like OPIE, etc. I'm sure once the hardware support is there, some more GUI-savvy people will do something in that area, though.

So why am I stating this here? Because it's up to _you_ to help and take care of one of these tasks if we want to see the dream of having a fully-free software E680/A780 before they get phased out ;)

Running a serial console on the A780

After about half a day of trial and error (which was related to a totally different problem, as it turned out), I now have a 2.4.20-based kernel with working serial console for my A780. Unfortunately the console requires soldering four wires onto test pads of the PCB - something that I achieved with 0.1mm diameter magnet wire (Kupferlackdraht for you Germans). The magnet wires are thin enough to get them through the TransFlash slot to the outside, without having to modify the case.

If you're interested in a bootup log captured from the STUART, check this one.

The rest of the day was spent debugging why my (still 2.6.14 based) kernel doesn't want to boot on the machine. As it turns out, booting stops somewhere in the early initialization after head.S has called decompress_kernel(). Debugging this problem has also caused me to actually write some ARM assembly code. For years I'm reading and debugging ARM code, but I've never actually written ARM asm from scratch. So my assembly code now prints one character for every stage of the booting process (ABCDEFGHI) and then stops. At the time the C code should print its first character, the device is already gone. So maybe something with the setup of the registers according to C calling convention, or setup of stack/heap is erroneous.

Interestingly, that startup code has not really changed all that much from 2.4.20 (which runs) and my 2.6.14 based kernel.

It's not unlikely that I'm [again] hunting a totally different problem. I'll probably merge my patches into 2.6.17-rc1 and see whether that works...

Virtualization and IPv6

I am seriously disappointed by both vserver and OpenVZ to not support IPv6 networking. What kind of attitude is this? This is the year 2006. Projects like the servers have native IPv6 connectivity at their hosting provider for a number of years now.

Now I'm investigating consolidation of some of the machines, and none of the ever-so-hyped lightweight virtualization solutions even supports it.

My journey into virtualization therefore ends before it even has begun.

Now you might wonder: Why is he complaining so loudly, rather than implementing it on his own? Because I think it's an almost unforgivable design mistake if you develop any new networking solution that is limited to IPv4 only. Also, OpenVZ is backed by a number of companies, and considering the amount of media fuzz they create, there should be a ton of developers working on it.

Now you might ask: Why not use Xen? Because I don't want simulated virtual machines, but rather some separation of privileges and namespaces. Simulating a whole machine seems a bit odd to me for that purpose.

I'll look at virtualization again when they support virtual Ethernet devices. Capable of running IPv4, IPv6, 802.2LLC and all the other nice protocols that we have in the Linux network stack.

SD Card Association releases 'Simplified specification'

If you're reading this blog for a while, attended one of my recent presentations or are dealing with SD Card support in GPL licensed Free Software, then you probably know about the never-ending issues around that subject.

For considerable time, the SD Card Association appeared to have regulations in place that basically prevented any SD card driver to be GPL licensed, because of NDA's and other stuff. I can only imagine that this was because of their DRM scheme [which I've never seen anyone using, btw]. Instead of creating separate standards, one openly documented for the basics, and one for the crypto function, the NDA covered all of it.

Nonetheless, various projects have meanwhile developed their own GPL licensed SD card support, since it is reasonably close to MMC (which is publicly documented), and the many proprietary drivers that can be re-engineered.

Very recently (it seems April 03, 2006), the SD Card Association has published a set of "Simplified" standards. You can get the documents from their homepage. There's one for SD Card Physical Layer, one for the SD Card Host Controller, and two related to SDIO. Obviously this publication is a couple of years too late. But let's try to be positive and look forward and not to the past.

So it seems their policy seems to change at least a bit. A very welcome move, although for my taste there is still one document missing: One that describes the actual commands on top of the physical layer.

Obtaining Asian Motorola EZX phones in Europe

A couple of days ago, I was looking for a way to obtain Motorola Linux Smartphones in Europe, i.e. those plenty of models that are not officially sold anywhere but China and other areas of Asia.

I've now found a suitable importer specialized in importing Asian phones into the European market. In case you're interested, feel free to contact me for more details. The phones range between EUR 180 and EUR 300, but there's a minimum order of five phones. I'll probably be ordering around early may.

Photography at Vienna central cemetery

Apart from OSCON/LINBIT related stuff, I've also actually taken a couple of hundred pictures (both digital and analog b/w) at Vienna's central cemetery, Europe's largest cemetery with more than 3 million people being buried there.

I yet have to develop the films, and look into the 3+GB digital data. However, I'm very confident that given the good light conditions and the amount of time I spent at the cemetery, there should definitely be some really good pics...

Returning from OSCON Vienna

I've just returned from my tree day trip to OSCON Vienna. LINBIT took great care of me during my stay, and I enjoyed it quite a lot.

The most obscure thing encountered during that trip was the word Liftkarmiesen. Austrians have all these (to us Germans) ancient and strange words in their variant of German. Anyway, it even took three native Austrians until somebody actually knew what it was ;) If you're now curious, try to research it on your own. It's related to curtains ;)

Looking for Motorola Linux phones

Since right now I only have E680i and A780, I would be interested in a way to obtain E896, A1200, A910, A768, A760, A732, A728 as well as ROKR E2. Most of them seem to be mainly sold in China / Taiwan.

If anybody knows a good source (importer in Europe) or some other way how to get these phones in the western half of the world, let me know.

Direct import would also be possible, but I'd need to know a serious exporter in .cn/.tw in order to do so. Any suggestions welcome. launched

In the tradition of my main project netfilter (which has a, I've now also opened a planet site for the OpenEZX project at

This should give users the ability to stay up to date with current developments in the Motorola Linux smartphone hacking community.

If you know of any feeds that I should add to this planet, please let me know

Booting kernels on A780 / E680

It's a bit strange that I still have so much difficulty running my own kernel on the phone. As it appears, there are some subtle hardware (or bootloader?) version differences that made me struggle for so long.

Two out of my three A780, and my one E680 don't boot any self-compiled kernels but rather just crash. The third A780 however boots them just fine.

Obviously, as Murphy's law indicates, the phone it works on is my 'production' phone, i.e. the one I use for my day-by-day phone needs.

We really have to get to the bottom of what's going on here. Also, if there really are differences, then Motorola has to publish the kernel source for both versions under the obligations of the GPL. So far they have only released a single version.

Half a day in the darkroom

I've spent the better part of this day first setting up my darkroom (in the bathroom), and then working in it. It's been quite some time (must be almost two years) since I last found time to produce my own b/w prints, but now the time has come. Used about 50 sheets of 24x30cm PE paper only today ;)

I'll probably continue with some more (postcard sized) prints tomorrow and later next week. There's certainly a backlog of a couple of hundred images that I have on film but not on paper yet. downtime - moving and updating servers

I've spent the whole Monday in the hosting center where, and most of my other projects are hosted. The main reasons for this visit were:

  • do kernel updates on two boxes that are known to be difficult with new kernels
  • move all five machines to a new rack, the old one is too crowded (no space for new machines, too hot)
  • add yet another new box (, which makes the number of machines now six

As usual, Murphy's law applied, so about everything that could go wrong went wrong. And, confirming Murphy's law, the most important machine ( had the longest downtime, something close to 9 hours.

This was mainly due to the last Gentoo update overriding my custom-modified yaboot boot script (for using the serial port, this is a headless XServe cluster node) with the default one, which wants to use the non-existent framebuffer.

That combined with the fact that KDUMP-capable kernels can't be booted from OpenFirmware (why isn't this indicated in the menuconfig help???) and thus the new default boot kernel couldn't be booted from yaboot.

That day I've tried about anything, from attaching a powerbook with bootable cd in firewire target mode to booting yaboot via tftp (which fails to load yaboot.conf via tftp *sigh*).

Now I've learned my lesson: chattr +i on yaboot.conf and the modified boot script for serial console.

Meeting up with Armijn Hemel

During my short trip to Amsterdam, I had a chance to meet with Armijn for a couple of hours. It's always good to meet people face-to-face when you're working with them a lot, especially on delicate issues such as GPL enforcement.

We've decided on how to optimize our work-flow and how to improve internal documentation of the individual cases. The usual thing when you're used to working on something alone (i.e. knowing everything off your head) as opposed to other people getting involved, etc.

Anyway, I'm extremely pleased that somebody is helping me out. There's also another friend of mine who's starting to get involved in the project, mainly on technical issues such as verification of the source code offered by the various (formerly?) infringing entities.

NLUUG Linux meet

During my short visit to Amsterdam, I was invited to speak at a small NLUUG event. I presented on recent, current and future netfilter/iptables development, and the presentation was very well received. Unfortunately I didn't have time to listen to the other two lectures, since I had a meeting scheduled with Armijn Hemel, the person who's currently helping me the most with

Downloading and executing your own code in RAM of EZX phones

In the last two days I've written a small program that allows you to utilize part of the built-in firmware update mechanism of the Motorola EZX phones. In fact, what it does is to download an arbitrary (max 1MB) piece of code from the PC to the phone via USB, and then execute that code on the phone.

On the one hand, this might look like a security hole (but well, nobody really cares about security on mobile phones anyway). On the other hand, this should definitely speed up kernel and driver development within the OpenEZX project, since it basically removes the need to flash the phone for testing of some new code.

Also, once a working driver for the TransFlash slot has been cooked up, it would actually be possible to usb-boot the phone into an OS that mounts its files from TransFlash. This doesn't touch a single bit of flash memory and is therefore ideal for development and probably even something similar to what 'live CD' distributions are to PC systems.

OpenWRT terminates GPL License to SveaSoft

It might not be something new to you at all, but it was new to me, since it happened during my holidays: OpenWRT has sent SveaSoft a note of terminating of rights under the GPL.

I've had SveaSoft on my radar several times, but the whole situation seems to be so messy, and there seems to be a history of different violations with each and every release they made. Also, there seems to be quite some confusion on the whereabouts of the developer[s?], which makes it difficult to find an applicable jurisdiction.

I'm single again

Those of you who know me one a more personal level will find it hard to believe that I'm actually a single again. Especially following up the engagement some two years ago.

After knowing Elisabeth for nine years, having lived together about half that time, it actually feels more like a divorce than 'just' a normal separation / split-up.

I will not make the mistake to state any reasons publicly in this weblog, sorry ;) Let just be said that we both feel very sad, and it was certainly not a lighthearted decision.

There's going to be some rough time ahead, and I'm certainly not in the mood for any kind of serious relationship anytime soon.

Always in motion, the future is.

netfilter do_replace() bug is not remotely exploitable

I don't know how people like securityfocus and and others claim that the recently-discovered and fixed 'do_replace()' bug is remotely exploitable.

In fact, the bug (which was found and fixed by Solar Designer while working for the OpenVZ project) can only happen in a codepath that can be executed by the local root user. Not even a non-root user, neither any remote parties can hit that bug and/or exploit anything.

Returned from vacation in India

Just got back from the airport. Everyone who emailed me: Please keep patient, as I've got some thousands of mails to wade through. Sorry for any inconvenience. I should be back and fully running no later than end of the week.

Invited as keynote speaker to OSCON Vienna

Recently I've been invited to give the keynote at OSCON Vienna (please note that this conference, to the best of my knowledge, has absolutely no relation with the O'Reilley OSCON events).

I'm honored and I'll gladly accept this invitation. AFAIR this is the first time I'll be giving the keynote at any FOSS related conference. The subject was up to me to determine, and I decided about something that is both one of the most important subjects for FOSS today, and well within the subject of the conference: "Kommerz und Community: Schnittstelle zwischen den Welten". It's about the interface between FOSS community and the commercial IT industry.

There are many suboptimalities at this interface. I personally believe that optimization of this interface would greatly benefit FOSS as a whole. Which issues am I talking about? Well, first of all, there are lots of GPL/licensing related issues. But even more importantly, there is the lack of support from the hardware community. As long as hardware vendors will actively hamper FOSS development by not releasing documentation, locking down their products, claiming they "support" Linux with their proprietary binary-only drivers.

For many of these issues, there's a big communication and furthermore cultural problem. That's what I want to address in that keynote.

There's another good point to the OSCON invitation: The trip to Vienna will also help me to improve my bad luck and stupidity while doing photography in Vienna / June 2005.

How to boot your own kernel on the Thecus N2100 - and prove it violates the GPL

My latest candidate for (and hopefully the last before finally leaving for holidays): The Thecus N2100 and N4100 NAS devices.

The Thecus boxes seem nice, at first sight. Apparently somebody recognized the need for a bit more performance, so there's an Intel IOP 80219 with 64bit PCI-X support, DDR400 memory (actually in a socket), an empty miniPCI slot (great!), USB2.0 ports, and SATA (yay). This should definitely be more promising than the usual 33MHz 32bit PCI / IDE / MIPS / SDRAM based smaller NAS boxes. The only thing really lacking with those Intel I/O processors is a hardware crypto unit. Who wants to have unencrypted storage these days?

Looking at the software, the problems start. First, there is no NFS support. iTunes, SMB/CIFS, HTTP, FTP - but no NFS :( Secondly, the web configuration frontend requires flash. Duh! How can you use something as ugly and proprietary as flash for something as simple as a web configuration frontend for an embedded box. God knows.

Anyway, let's get back to the GPL issue. As usual, I cannot make such a claim without verifying it. First of all, the devices (and their firmware updates) ship without a copy of the GPL, any indication that GPL licensed software was used, no written offer and no source code.

But well, where the heck do I know from (and can prove) that they actually run Linux? I won't disclose the reason for my initial hints, since I don't want future vendors of future products to know how they can avoid me ;) But anyway, let's assume I was surprised to see a nmap fingerprint that indicates Linux on the box and now want to go further.

Looking at the firmware update images, they appear to be scrambled / encrypted somehow. At least there is no gzip/bzip2/LZMA/ext3/cramfs/romfs/... signature to be found in them. And even if the firmware updates contain Linux, this doesn't actually prove anything about the software pre-installed on the device.

The running device also doesn't offer any ports apart from the SMB-related ones and http(s). So we're stuck.

This is where I usually take the device apart, carefully analyze it's hardware and go looking for a serial port with my Oscilloscope probe. Unfortunately the PCB of the N2100 didn't seem to have one. It took me some time to figure out that the serial port connector (there's actually a standard 9pin header) is on the SATA backplane rather than on the CPU board ;)

Hooking up a serial console, you can see RedBoot wait for one second and then execute a boot script that loads initrd and kernel, finally executes it. Yay!. Too bad that the actual kernel seems to lack support for a serial console. So all you get is the 'Uncompressing Linux......................................................................................... done, booting the kernel.' line. Together with the firmware scrambling/crypto, this is definitely an attempt to hide the use of GPL licensed software and/or otherwise lock the user out of the device.

Unfortunately hex-dumping the whole memory contents from RedBoot via the serial port, and parsing it on the host side seemed like a rather clumsy - and otherwise unproductive approach to finding proof of GPL licensed software in the device.

Luckily, you can interrupt RedBoot and configure the network device, set up TFTP, cross-compile a kernel for the IOP 80219, and boot that. After some twisting of the .config, I got it to boot without any crashes, and even the RedBoot partition table is correctly recognized and parsed.

So now I'm running Linux on the device, great. But still I can't prove that the device actually ships GPL licensed software in an incompliant way. So all that is missing is a NFS-root capable installation of Debian-arm that we can boot into, and which we can use to read out the mtd partitions.

Oh, and yes. While I appreciate their love for the netfilter project and it's software: There's absolutely no place in a NAS box for having ip_conntrack linked statically into the kernel - unless you voluntarily want to loose performance. At least to my knowledge, performance of NAS devices counts. So, Thecus, in your own interest: disable ip_conntrack in the kernels you ship.