Number of GPL violations still rising

Over the last couple of days I've again verified a number of GPL violations. It's a real pity that those companies still don't get the message.o

It hurts especially, that there are two cases (Netgear, Siemens) where companies with whom we already had a amicable agreement published new devices that again don't comply with the GPL (Netgear WGT634U and Siemens M740-AV). Apparently they don't really care despite the fact they should know better.

Also, we have another number of cases where companies signed an agreement with us, but failed to fulfill that agreement only a couple of months later with exactly the devices mentioned in the agreement.

I'm sick of those cases. What the hell is so difficult to put the source code and the GPL license text on a CD-ROM that has 500MB unused and ships with the device anyway?

Preparing the 21st Chaos Communication Congress

As every year, the Chaos Communication Congress takes place in Berlin, Germany.

For six years, I'm part of the team that takes care of audio and video recording and streaming. Since this year I've become head of the a/v documentation project, I decided to use a 100% Linux based solution instead of the Apple Quicktime stuff that we've had for the last couple of years.

Thanks to the great ffmpeg software, we can even encode four different streams on a off-the-shelf Pentium IV.

Today, I've been with the technicians at the congress center who set up the PA and lighting. This was to make sure everything really reflects our demands, and we have the correct audio signal delivered to the appropriate place, etc.

Setup of the congress will continue over the holidays. Especially the NOC (Network Operations Centre) will have a hard time setting up the internal network for about 3000 attendees, certainly each bringing more than one networked device on average.

ffmpeg is undocumented, ffserver broken

I've been experimenting a lot with ffmpeg and ffserver over the last couple of days. The fact that ffmpeg is very little documented is a pity, but not exactly a problem for someone experienced with free software and C development (use the source, Luke).

However, the ffserver program seems to be horribly broken in a number of ways. Independent of the kind of configuration, it regularly segfaults, glibc complains about double-free's, and valgrind or Electric Fence have numerous complaints.

All information you can find after browsing through mail archives, is that it's apparently broken for a number of years. Maybe I'll spend some time at it and fix it at least partially. So I spent about two days to familiarize myself with the source of libavformat, libavcodec, ffmpeg and ffserver. It's not exactly easy to understand, but I think I now got a good understanding of what's going on where.

Another fundamental insufficiency of ffmpeg seems to be that it cannot put the output of one codec into multiple output files. So let's say I want to encode some MPEG2 video and AC3 audio. This is to be written to a .vob file and at the same time sent as a transport stream over the network. The only way you can achieve this now is to encode the input data twice - which I cannot afford due to CPU limitation.

So I was pondering something like streaming the output over multicast RTP plus running something like rtpdump on the same machine to create the local file.

As a summary, I think it's a pity that there is good encoding software like ffmpeg, and that nobody volunteered yet to fix the remaining issues required to turn it into a good streaming and recording solution.

More and more cases

Today has been a sad day with regard to gpl-violations.org. I just ordered five potentially infringing devices from three different vendors. Apparently the message has not been conveyed to all respective parties yet...

So let's see how they will react if someone actually is in a position to ban their products from all-important pre-christmas sale.

This really sucks. At some point I want to start coding on a day without having to have information in my inbox about yet another gpl violation case.

linux-bangalore 2003
I've just returned from lb/2003, the major linux conference in India. I've had a great time there.
Besides giving two presentations (one about SMP effects in kernel programming and another about the netfilter internals), I've done some travelling to Mysore and Mumbai.
Thanks again to the lb/2003 organizers. They did a great job comforting the speakers in any possible way.

Shopping in Bangalore

Today I went shopping in Bangalore. The first thing I had to learn, is that you need a lot of travel through the heavy traffic in order to get to the respective stores.

Secondly, buying/finding a Sari (Including the blouse and the Petticoat) is not as easy as buying women's clothes in the western world. The choli (blouse) is made-to-measure, and they require more information than the usual under bust / over bust / waist measurements. So I only bought one this time, let's first see how it fits Elisabeth before I buy more items that in the end don't fit.

Getting Hindi learner books (apart from the usual Devanagari alphabet training) in Bangalore turned out to be more difficult than expected. Students tend to get the books from the Schools, and the local language is Kannada. But finally we managed to get them, too.

Finding Bollywood DVD's is obviously the most easy task ;) I got a stack of 8, and I'll probably be buying more of them once I get to Mumbai on Tuesday.

Linux Bangalore is Over

The three-day lb2004 is now over. About 80 presentations from all areas of free software, ranging from hardcore technical subjects to user-experience.

One of the interesting parts was that one developer managed to port the "DotGNU Portable.net" framework to the Simputer in only three days during the conference. Apparently this spawned a lot of media interest.

In the end, the conference went really fine, if it wasn't for the strange rules and regulations of the IISC that tried to undermine the event.

Oh yes, than there is the air condition to which I probably owe catching a cold last year - and this year again :(

Day one of Linux Bangalore 2004

So today lb2004 started, but unfortunately there are lots of problems, some of them really outstandingly ridiculous.

The less problematic issue was that even though the zd1201 driver now works, the access points would not actually get a link to a switch, independent of the kind of cable. So the whole wireless network idea was basically abandoned.

As for Internet access at the conference, there was none. There's not even CDMA reception on top of the roof, and even though that the auditorium is part of the Indian Institute of Sciences there is no connection to the IISC LAN within the complex. Also, they IISC apparently has so little bandwidth, that it's insufficient for their own purpose, let aside connecting some conference.

Then the really interesting thing came up: Because of about 2800 attendees, there was an 500 seat additional auditorium built. Apparently the IISC gave permission to build the auditorium tent on their ground, even charged money for using the ground - but they informed the lb2004 organizers that they were not allowed to use it. They've only given permission to build the auditorium, not to actually use it to give any presentations in there, or even use it only as a lounge.

Believe it or not, it became worse. Someone wanted to fetch food from the catering to the speaker lounge. He was stopped by a security guard, stating that in the room officially designated as speaker lounge by the IISC, there was no food permitted, and a fine would apply if anyone actually tried to do so.

Oh yes, and they suddenly introduced a new rule, active on 1st of December, that as soon as there are more than 25 cars parked on the grounds, another fine would apply.

This is just incredibly ridiculous. This is the Indian Institute of Science, and the conference is held in exactly the same premises for the third time. None of those issues came up in the previous years.

Also, this is the same IISC which boasts himself to have denied an event with Dr. Kalam (India's president and one of the biggest promoters of Free and Open Source in India) at the J.N.Tata Auditorium.

It's very hard to understand that they just want to sabotage that kind of event in any possible way. It makes me feel sick and sad. Somebody should organize a demonstration. Call off half a day and make a 3000 attendee protest in front of the office of the director of the IISC.

Making a broken ZyDAS zd1201 based USB Wireless work

It's amazing what kind of strange and broken USB devices there are. Here at Linux Bangalore, they've got a bunch of 'combo USB WLAN and Flash Disk Sticks' that turned out to be TwinMOS B241 devices. But let's forget about this for a moment and join me on my journey...

They ship with a Linux driver preinstalled onto the flash disk. Unfortunately that driver consists of some hacked wlan-ng driver. For most people who've worked with wlan-ng, they know that it's overly complex, and not really the standard Linux way of doing things.

That modified wlan-ng source code would only build for 2.4.x, the machines here are running Fedora Core 3.

Also, the machines would totally lock up their USB stack as soon as you would enable the WLAN part, even without any driver.

Since the wlan-ng was a modified prism2 USB driver, I though I could somehow merge the changes into the orinoco_usb driver that is in the standard kernel.

After some deeper look, it turned out that the device has no relation with Intersil, and definitely doesn't have a Prism2 chip on the PCB, so my tries to get this working were useless.

Apparently, they didn't even do 'copy+paste', but they did 'edit and forget', i.e. forget about prism2 devices and only support some totally different chipset without actually changing file names or comments in the driver.

So I opened one of the devices and found a AU9254A21-CBS (4 port USB hub), a K9F1G08UOM (the Flash memory for the USB drive), a IC1114-F48LQ (usb storage controller for the flash), and some unknown chip labelled ZyDAS ARM. Also there was a Cyprus semiconductor chip that I though of the EZ-USB controller that connects the alleged prism2 to the USB bus. This fits the driver design, since it has to download some 'bootup code' to the usb device before being able to use it.

After some further analysis, the Cyprus CY62137CV30LL-70BVI turned out to be some SRAM chip, and the ZyDAS ARM the real 802.11 MAC. And luckily, some people are working on a very clean 2.6 style stand-alone driver.

And the driver even worked after just adding the USB device ID to it's list of known devices, at least on little endian platforms.

If the devices specs or documentation would have told us that it is a ZD1201, the driver clearly indicated that it has no relation with prism2 or somebody who wrote the driver actually had a clue how to do this, this would have saved me about four hours of time, at least.

Oh yes, and the usb stack lockup comes from violating the USB specification and only supporting one particular flavour of USB bus enumeration. So nobody actually ever tested it for USB spec compliance, even though there are compliance tests available by the USB forum. *sigh*

Visiting Infosys

Today, the international speakers of LB/2004 were invited to visit the sponsor Infosys, apparently India's largest IT outsourcing company.

They've been growing from 7 to 35,000 engineers very rapidly, and their Bangalore campus is certainly the most luxurious and westernized part of India I've seen so far (not that I've seen much of India either).

Anyway, we were informed about their recent Linux and FOSS related activities, met their internal InfyLUG (Infosys Linux User Group), met one of the seven founders and Andi Kleen gave a lecture about the kernel development process, that was attended by 300 employees and streamed to all the other Infosys campuses.

Leaving for Linux-Bangalore/2004

I'm at the moment packing my suitcase, and I'll be sitting in the plane about 24 hours from now. Do not expect any fast email replies or IRC presence of me before December 9th.

Make CyberJack drivers issue a key-press confirmation beep

This is a very useful feature, especially for blind people. Unfortunately there is no unique way of issuing some beep sound on Linux-based systems, so there needs to be some magic that determines whether running under X11 or not and call the appropriate code for beeping.

Successful TomTom Visit

As indicated before, TomTom B.V. has invited Christian and me to visit them at their offices. Apart from some consulting/training regarding Free Software Licenses and the Free Software Community, they were particularly interested in getting us involved with their Linux kernel related development.

I stressed the fact that it is very important to clean up all the drivers, make them use standard interfaces and eventually get them merged to the mainline kernel. As it seems, they agree and want to contract one or some of the OpenTom developers to do so.

KNF Kongress: Meeting old friends

Today I've given my two gpl related presentations at the annual KNF Kongress. Apparently it helped some people to understand legal requirements of dealing with various free software licenses, which is good.

Also, I was a the OpenTom presentation and could it actually see working with a 2.6.x kernel, sound, framebuffer, USB keyboard, USB CD-ROM and even playing some low-res-movies with mplayer on the console. Great work.

Apart from that, I was just chatting with a lot of people. As a side-note, I've also mentioned the CCCB's current search for a pc-based logic analyzer that either comes with developer documentation or Linux software. People suggested of building the logic analyzer on our own, by using available FPGA's, some SRAM and a USB interface. If you think about it, this actually sounds quite feasible. Now I'll do some research on FPGA's that ship with a free development environment, unlike the proprietary stuff shipped by Altera & Co :(

Never ride trains on weekends

If I'm ever about to travel by train on a weekend, please somebody remind me not to do so. All these crowds trying to find available seats, incredibly busy, delayed trains, ...

Travelling during the week is just so much more convenient.

Two presentations at KNF Kongress coming up

I'll be giving two presentations at the upcoming KNF Kongress 2004, entitled "The GNU GPL Revisited" and "Copyright helps Copyleft".

Also, Christian Daniel from the OpenTom group is going to present on his re-engineering efforts.

If you happen to live in southern Germany, it's probably a good idea to check out the yet small but great KNF Kongress. Looking forward to meeting you there.

I'm pleased to present at Linux Bangalore 2004

Following up my presence last year's Linux Bangalore 2003, I'm very pleased to again be invited to present at this year's incarnation.

Unfortunately I had to shift the main focus of my presentations a bit towards political/legal issues, so there's one presentation about How to interact with the Free Software Community, one about The GPL is not public domain, and for all the tech savvy guys, there's A tour through the Linux 2.6 network stack.

I'm happy to present on those political and legal issues, because I think this is the opportunity to get this kind of knowledge into the Indian IT outsourcing industry, before it is too late (like apparently happened with most of the Taiwanese embedded Linux vendors).

I'm happy to see an increasing number of high profile speakers at Linux Bangalore, and it's now becoming (to the best of my knowledge) a big internationally recognized Linux event.

More work on the REINER SCT CyberJack drivers

I'm not sure if I did mention it on this blog, but I've been contracted by REINER SCT to work on a Linux driver for their CyberJack series of smart card readers for quite some time.

In the last days I've been spending quite an amount of time hunting down user-reported bugs in the driver, which is good. Sometimes it's really surprising to see in what kind of bugs stupid mistakes eventually result.

Also, I've now managed to make the driver work on x86_64, so it's working in little-endian 32 and 64bit, big endian 32bit. I have to test it on my UltraSPARC box to see whether 64bit big endian also works.

Working on lots of Presentation Slides

I didn't even notice it before, but within two weeks I'm now scheduled to give six presentations. Unfortunately, none of them is exactly the same subject on which I've presented before, so the amount of recycling I can do is quite limited.

I've always considered doing slides for a presentation as "necessary evil", but it's OK if you do it once every so often. But preparing six presentations in a row is no fun at all :(

You can follow the progress in the svn repository

I sometimes really feel the need for a secretary... or someone who does boring small jobs like HTML/Postscript conversion of all my presentations, and makes them more conveniently accessible on the net. *sigh*. Sorry guys..

No more time for OpenTom at the moment

Due to an increasing workload, I won't be able to work on the OpenTom project for at least some weeks. I've published the current state of the SD Card driver in my personal directory of the OpenTom Subversion repository. If you want to pick up, feel free. I'll answer questions by email.

wiki.opentom.org online

We've put together some information on our OpenTom efforts at wiki.opentom.org. Feel free to check it out. Additions of content very welcome :)

More hacking on the SD Card driver

re-engineering the SD card stuff turns out to be more time consuming than expected. Not that it's particularly fancy or complicated - just obfuscated. Apparently there are some quite complex data structures involved, that are hard to analyze by looking at the disassembly.

All GPL issues with TomTom B.V. settled

I'm very happy that the GPL issues with TomTom have now all been settled, and despite some early disagreements we're now very happy with the way TomTom has handled this case.

The TomTom GPL page contains the latest source of their 4.42 firmware. Pretty much all of the drivers have been released with their source code (touch-screen, framebuffer, USB device, accelerometer, GPS). Only (obviously) the SD-Card driver is missing in the source and provided as kernel module. This is due to the stupid SD Card Alliance licensing agreement, which basically puts every recipient of the Documentation under an NDA.

So at the moment you have to put all of the OS into the initrd, which is loaded by the bootloader.

We're working on a solution for the card reader, though. At least MMC Card support should be available soon.

The OpenTom Project was founded

Our distributed efforts in opening up the TomTom GO have now found a common home, the opentom.org domain. There's the OpenTom website and the svn.opentom.org subversion server.

There's still a lot under construction, expect more news here in this blog and in the subversion repository.

2.6.10-rc1 kernel for OpenTom

Christian Daniel has managed to get 2.6.10-rc1 running on the TomTom GO. This includes a 2.6.x-rewritten frame buffer driver, USB Host and Device support.

The kernel tree has been made available on svn.opentom.org.

Back blogging again

I had some severe hardware problems during last week, resulting in almost one week of server outage. We had to change power supply, ram, mainboard and cpu in order to get the machine back running again - basically a whole new machine.

Sorry for anybody trying to access www/ftp/.gnumonks.org over that time. Email was not affected, since email is dealt with on a totally different box.

Thanks to my Towersoft friends who took care about the physical repairs of the machine (it's located some 500km from my place).

Chaosradio about Biometric Information in Travel Documents

Yesterday I've participated in a Chaosradio show about the recent international push towards biometrics in travel documents such as passports.

Our focus has been on the flaws of biometric systems, the current plans of the ICAO about MRTD's (Machine Readable Travel Documents), the risks involved and why they are not an applicable tool to prevent terrorist attacks.

If you're interested in listening to a recording of the show, it is available at the usual location, ftp.ccc.de.

GPL Agreement with TomTom B.V.

Two days ago I signed an amicable agreement with TomTom B.V., a Dutch vendor of GPS navigation systems. The press release is as usual at the gpl-violations.org homepage.

According to the agreement, they have a grace period until Oct 30, but apparently they already published some source code.

Unfortunately it's still incomplete to some degree, but I'm looking forward to getting this sorted out.

Also, this source is not enough in order to run your own kernel on the TomTom GO, you will need some information on the firmware image layout and a particular blowfish key. For more details on the internals of the TomTom GO, please see the OpenTom of Christian Daniel.

I'm looking forward to convert to TomTom into a all-in-one car computer, including wardriving (USB WLAN with kismet) support and MP3/Ogg-Player with USB hard drive :) Not to forget bluetooth keyboard support, etc. :)

TomTom and your own kernel

I've started to merge the TomTom specific patches into a plain 2.4.27 kernel. Most of it is quite straight forward, since apparently they backported half of the kernel to 2.4.18-rmk6 (which is what they use as base). I don't really get it why companies still develop new products for 2.4.x, especially for really old version like 2.4.18. In the windows world, nobody still writes windows 3.11 applications, why do they start this kind of crap with Linux? *sigh*

Anyway, I'm thinking about a 2.6.x kernel port at some point, but obviously this is not an important issue on my agenda and I'd rather get some netfilter stuff running first.

Berlinux 2004

Some time ago I was approached if I would be able to give a presentation at Berlinux 2004, Berlin's local incarnation of a Linux conference, organized by the Berlin Linux User Group.

This should be the first contact to any user groups I've had for about five years. I've tried to avoid Linux user groups exactly because of the 'User' part. I have a hard time dealing even with Linux-savoy iptables users, let aside users who need explanation how to install a given Linux distribution or even how to use a file manager.

Unfortunately Berlinux seems to be very user-oriented, too. I arrived about 40 minutes early and am now waiting for a presentation explaining the principles of mounting and the Linux file system layout to finish.

I'm surprised that Berlinux is so small, considering that Berlin is about seven times the size of my old hometown of Nuernberg, and the ALIGN Linux Setup Parties had about the same size.

Oh yes, does the idea trouble you that you know somebody at every international Linux conference, from Bangalore to Ottawa - but at an event in your own hometown you have a hard time finding any person whom you know? That's how I feel. Misplaced, at the wrong event :(

Porting PPTP conntrack/nat helpers to 2.6.x

I've always refused to do the port of the PPTP conntrack/NAT helper I wrote for 2.4.x because there's higher priority items on my agenda.

Apparently it helped, as I was told Mandrake did a port to 2.6.x. I thought that is great news, and I thought it'd take an hour or so to get it merged.

Unfortunately that 'port' was totally incomplete. NAT couldn't have worked at all, and if you sent it a nonlinear TCP packet it would very likely crash your kernel.

In the end I spent the whole afternoon at it, with a resulting patch that is about the same size as the original code :(

The code is now in our subversion repository, I didn't have the time test it so far, so any testing you (yes, you, the reader) might give it would be appreciated.

Another patch submit day.

Today I've submitted hashlimit, CLUSTERIP and CONNMARK to the 2.6.x kernel. After resolving some glitches with CLUSTERIP, DaveM took all three :)

This means we're again one step further submitting stuff from patch-o-matic into mainline, which is always a good thing.

GPL Agreement with Gigabyte Technologies

I've managed to get an amicable agreement with Gigabyte Technologies B.V., yes that's the big worldwide known vendor of Mainboards and other PC equipment :)

The press release is at the gpl-violations.org homepage

I should do more press releases

I'm sorry for that. GPL-enforcement progresses meanwhile. I've been able to obtain amicable agreements with three more vendors (D-Link, Gigabyte, TomTom), and there are two more open / ongoing cases at this point.

Expect more news and even an official press release during next week

Fun with incompetent BMW employees

So during the repairs of my BMW F650's carburetor, I lost the choke plunge. Not a big deal, just a tiny part regulating the fuel/air ratio at engine startup time.

So I picked up the phone and called the spare part department of BMW in Berlin, and told them the exact part I wanted. "Chokekolben" is 100% not possible to be misinterpreted, there is no other part with the same name. So I was told that this part is not available on it's own, but just in a set bundled with the linkage/string that actually attaches to the plunge.

One day later I was called that the part had arrived. It took me about an hour to get to the BMW subsidiary, only to find out that they had ordered the choke string, but it came without plunge.

They showed me the exploded view of the carburetor, and it was very clear that the plunge is sold separately for about EUR 3. I have no idea how one can misunderstand the exploded view and/or the spare part list associated.

After ordering the plunge, I asked them if they made the exploded views available for customers, so they could directly order a particular spare part number in order to avoid such misunderstandings. Apparently they only provide those spare part catalogues to their BMW partners, and they see no way how they could provide me a copy. *sigh*. So I will have to rely on some brain dead spare part sales assistant who has most likely never disassembled that bike ..

Luckily, there's eBay and I found somebody who sold the original BMW spare part catalogue on CD-ROM. What would the world be without eBay.

BMW, this happened about two weeks ago, and I still don't have that spare part.

Yet again more cases coming up

I've authorized my lawyer to act in five more new GPL violation cases. As usual I will not disclose their names until some kind of agreement (or a court order) is in place.

In one of the cases we unfortunately now had to go after a reseller, since the warning notice to the Dutch vendor was unanswered. Apparently the strategy is working, since the German reseller now put pressure on the Dutch vendor, who suddenly now replies to us ;)

Conntrack events for 2.6.x

I've separated out Patrick McHardy's conntrack events from the nfnetlink-ctnetlink patch and ported it to 2.6.x. The patch was posted to netfilter-devel, in case you're interested.

For those of you who don't know what this means: It means that the first part of what is required for a 2.6.x ct_sync port is now done ;)

ct_sync ethereal plugin

While doing some more ct_sync testing/debugging, I found out that for some reason my ctnl_dump program didn't work anymore. Instead of fixing it, and updating it to CTSP (conntrack sync protocol) version 2, I decided to write a plugin for the well-known packet analyzer ethereal.

Due to the nature of the CTSP, it passes arch- endian- and configuration-dependent data structures between master and slave. This means that it is virtually impossible to write a analyzer that will work in any of those combinations.

My plugin now assumes that you use a little-endian 32bit machine with the pptp-conntrack-nat patch applied.

The plugin turned out to provide very useful information, and I was able to fix some issues in ct_sync using it.

No big news this week - I'm in Astaro labs

I'm about to do one week of benchmarking and profiling using an Ixia four-port Gigabit Traffic generator and a Sun Fire v20z dual Opteron box in the Astaro labs. Let's hope I can find some code pieces in the network stack that can be optimized in order to achieve higher performance...

xfrm_user.c doesn't use netlink correctly

If you read the netlink documentation (and look on how existing users such as rtnetlink or ipt_ULOG uses it), then all messages part of a dump have the NLM_F_MULTI flag set, and the dump is terminated with a NLMSG_DONE message.

The code in net/xfrm/xfrm_user.c however dumps those messages without the NLM_F_MULTI flag. I've hacked a first patch, but apparently it doesn't catch all cases.

Motorbike problems

I wanted to take pictures of a recently detonated old building in Berlin. I wanted to go there via motorbike. Unfortunately the bike got some problems: After about 3km from my home, it suddenly stopped and refused to start again. While trying to get it running, I suddenly noticed vast amounts of fuel leaking from the air filter. That's a bad sign, it basically says that somehow the carburetor is getting fuel into the wrong direction.

I went home by public transport (no photos taken), and luckily found a truck rental that was open on Sundays. So I managed to get the bike back home, take everything apart and clean the carburetor. I couldn't find something serious like a worn out fitting... all I found was a minimal amount of dirt.

I'll put the bike pieces back together tomorrow, let's see whether cleaning the dirt actually helped. Jeez, as if I hadn't enough to do already...

Linux Bangalore / 2004

The LB/2004 organizers have officially appointed me as speaker recruiter ;). Apparently they have some trouble in contacting various Linux developers due to over-reactive spam filters (blocking everything from India, heh?).

This means I end up writing emails trying to convince folks such as Alan Cox, Andrea Arcangeli, Russell King, Erik Andersen, Robert Love, ... to attend this wonderful Indian conference.

Did I mention that I'm going to be there this year, too ;)

2.4.x backport of neighbour cache rework

I've finished my 2.4.28 and 2.4.21 backports of our recent neighbour cache re-work (see netdev of last two weeks in case you're interested). 2.4.28 was quite straight-forward, just the missing per-CPU hurt a bit. 2.4.21 was pretty hard, since the neighbour cache apparently changed quite a bit between 2.4.21 and 2.4.28.

But well, it's over now. Thank god :)

Generalized Linux network statistics

While working on the neighbour cache, I introduced some generic neighbour cache statistics. They are done in the core, but exported to userspace for every ncache separately (arp, ndisc, atm_clip, decnet). I used the same techniques and file format as rt_stat.

Martin Josefsson also recently introduced ctstat, the same kind of statistics for ip_conntrack. He did a copy+paste 'port' of the rtstat userspace program. I now also needed four more new copy+paste 'port's. And I couldn't do it. Copy+Paste style ports are what I am fighting in the iptables world for two years, so I certainly don't want to introduce them elsewhere..

The result is what I call lnstat. It's a generalized version of rtstat, it works with neighbour cache, routing cache and conntrack statistics - either separately or all at the same time. It has user-defined formatting (field width) and key selection, as well as some other bells and whistles. Let's hope this gets integrated with iproute2 soon, so people can benefit from it.

I also thought about writing some daemon, but abandoned that idea in favour of writing a ulogd2 plugin for it... this means ulogd2 will be able to log per-packet, per-flow and generic things such as statistics...

First Solaris-based contract in four years

For more than four years, I did 100% linux based work. But apparently there are still people interested in Solaris stuff, since I just got my first solaris based contract in quite some time.

Spent an incredible amount of time getting Solaris 9 installed on my Ultra 5, which was only running Linux before. I never understood how Sun could rectify Solaris being so much slower than Linux on their own hardware ;)

Proceedings of Developer Workshop 2004 online

I finally managed to finish the write-up and markup of the proceedings. They are available in a number of formats at the documentation section of the netfilter home page.

In theory, there could still be lots of semantic markup added, but well, who cares...

pkttables finally making some progress

I've found some time to work on pkttables again. Isn't that great news? If my brain is not completely broken, I've now worked out a RCU-powered way to have full table traversal with a completely lock-less reader path, while providing atomicity either on table- or chain level.

Also, I ripped the "struct nf_attr" and NFA_xx macros from the nfnetlink core, since they get replaced by my vTLV (Versioned TLV) code.

With some luck I'll be able to continue my pkttables work next week

CLUSTERIP is in patch-o-matic-ng

About one year ago I did some work for SuSE in implementing load-balancer-less load-balancing clusters ;) This is achieved by replying to ARP requests with a link-layer multicast address, so all nodes receive all packets. Hashing parts of the ip header now determines whether the packet is to be passed up the stack on a given node.

The result is called the iptables CLUSTERIP target, and I've now finally put it in patch-o-matic-ng, since it was only available in my undocumented public CVS tree so far.

Siemens is violating the Settlement

Siemens is offering the SE-505 firmware on their homepage without any reference to the source code, the GPL, or the GPL text. This is in violation of the signed settlement agreement that I have concluded with them.

The lawyer is already informed, and we'll see what kind of legal options we now have in pushing Siemens [again *sigh*] for GPL compliance.

Reworking the Linux neighbour cache

Since I've lately had some customer issues with regard to neighbour cache overflows, I studied the current code quite a bit. From my point of view, it has a couple of shortcomings.

The general problem goes like this: What do we do, if we're attached to let's say a /16 (formerly 'Class B') network that has a theoretical limit of 65535 neighbours at layer 2, and somebody sends us a single packet for every one of those neighbours. We now start to send ARP requests for all those neighbours, and until those time out (1sec default), thus flooding our neigbour table. The current Linux strategy is to configure a static limit (default: 1024), and as soon as we reach the limit, we start deleting old entries. 'old' entries are those for real hosts to which we've recently had connectivity... We do not expire any of the incomplete neighbour entries in order to avoid ARP-floods.

So if you want to avoid that, you always have to set the gc_thresh3 value to at least the theoretical number of total machines that could be directly reachable at layer 2. While this is not a problem with /16, it suddenly becomes one with /8, or with the extremely large IPv6 prefixes.

The problem is further increased, since the number of hash buckets is very low (static number of 32), and the used hash algorithm apparently has a bad distribution. So either we increase the hash table, increase the number of buckets and improve the hash algorithm, or we change the expiration scheme to also drop incomplete entries. But the current situation is definitely not good.

So I picked up some old 2.4.x patches from Tim Gardner, ported them to 2.6.x and brushed them up. The number of hash buckets is now a kernel boot parameter (if not specified, the hash is dynamically sized, like the TCP syn-queue, fragment queue or ip_conntrack hash). The hashing algorithm now uses a Jenkins hash, just like all other parts of the kernel use, too. The patch is in testing at my machines at the moment, but I think I'll push it soon.

libiptc2 bugfix (upcoming iptables-1.3.0 prerelease)

Since the segfault-bug in my recent re-implementation of libiptc has now been fixed, I think we're about one week before a iptables-1.3.0 prerelease for public beta-testing.

NAPIfied natsemi driver

I've now successfully NAPIfied the second NIC driver: natsemi.c... this was the only remaining driver that I care about, since it is used in the PC Engines WRAP embedded systems that I use as routers/bridges/wlan-gateways.

The result is that I can now get about 34kpps routed on an embedded 266MHz Geode CPU at full 148kpps 64byte single-flow udp flood on the input NIC.

Adding NAPI support to the sungem.c Ethernet driver

Yesterday I implemented NAPI support for the sungem.c driver. This was done because I was annoyed by the fact the my notebook (Apple Powerbook with on-board Gigabit Ethernet) could still be killed by a machine running pktgen and flooding it with some 700 kpps.

After submitting the patch, David Miller pointed out that he has added NAPI support to sungem.c to the bitkeeper tree about four days ago :( So I spend a number of hours in duplicating work that was already there... not that I didn't have other stuff to do.

Well, at least I learned a bit more about Linux NIC drivers..

I'm now facing the task of implementing NAPI for the natsemi.c driver, which is used in the PC Engines boards that I've been using recently as embedded Routers / Firewalls.

Working on the summary / proceedings of the 3rd netfilter developer workshop

Spent a couple of hours putting the notes of the 3rd netfilter developer workshop together in a single file, adding lots of Docbook-XML markup, ...

It's still far from being complete, but I have to finish this ASAP..

Intel e1000 (82546) TX performance

After recent discussions with Robert Olsson at the netfilter workshop, I've decided to investigate a bit further, why the Intel e1000 gigabit MAC's are quite limited when it comes to TX performance and large numbers of pps.

My first assumption was that the in-kernel pktgen.c code might not keep the transmitter busy at all times, resulting in only 760kpps (out of the theoretical maximum of 1480kpps).

So I hacked the e1000 driver to hardcode a refill of the Tx queue with the same skb over and over again. Using a 2048 Tx descriptor ring, I was able to keep the transmitter busy at all times (E1000_ICR_TXQE interrupts).

Unfortunately, I still didn't get more than the 760kpps in this setup (PCI-X, 66MHz, Dual-Opteron 1.4GHz, DDR-333 (PC-2700) RAM. So either we're seeing a limitation of the 82546 chip, or the PCI-X bus / memory latency / whatever.

I'll try the same experiments on a different machine with PCI-X 100 / 133MHz in order to find out what exactly is causing this limit.

netfilter workshop / Linux Kongress 2004

I've not been able to write any articles for this log over the last few days, since I've been busy with the third netfilter developer workshop and Linux-Kongress 2004.

The netfilter workshop went really well, apparently the

Started a new 2.6.x based mini router distribution

I'm in the process of deploying a couple of PC Engines WRAP.1C embedded x86 boards deployed in my apartment. They make neat little playgrounds for Router/NAT/VPN/WLAN/... style appliances.

Unfortunately I didn't find any embedded Linux distribution project that was up to my demands. Apparently they all use age-old kernels (2.4.17 or something ancient like that). And they very rarely come with a decent automatic build system that would allow you to rebuild it from scratch, adding your own patches, ...

So what did I do? I started my own :(. Not that I'm proud of it, but it was necessary. My home VLAN/firewall/PPPoE/NAT/VPN router is now running the very first image of this new distribution I called 'gRouter'.

It's main features are kernel 2.6.8.1, uClibc-0.9.26, busybox-1.00rc3, pppd with in-kernel PPPoE support, quagga, iptables-1.2.11, openvpn-1.6.0, and dropbear for SSH. It all fits in about 8MB of compact FLASH.

The build process is semi-automatic, apart from a few glitches the whole image compiles itself. I stole some of the build magic from the WISP-DIST project (part of LEAF), although this is all quite simple scripting.

After some more cleanups and testing, I plan to release this distribution. Please don't expect any support, or any configuration tools. It will be available for Linux experts who can configure and setup their system from scratch, and want to have the gadgets of the latest software releases.

On the todo list is cross-compilation support (well, since it is uClibc based, it already does cross-libc-compilation), madwifi support, and especially IPsec using the 2.6.x kernel implementation.

Getting the external VGA of my Apple Powerbook (TiBook IV) working

If you've attended one of my presentations during the last 12 months, you will certainly have noticed the poor quality of the slides. Yes, the content and the presentation is poor, too - but I'm mostly referring to the optical quality.

I've already spent at least a whole day in the past in trying to get the external VGA working with Debian/ppc, with little success so far. I really don't care whether the external port mirrors the content of the display, or if it runs in dual head mode.

Today, I spent some three more hours in trail-and-error with the radeon driver of the dri-trunk XFree86. I tried CloneMode, Dual Head, with and without FBMode, and about any other parameter within XF86Config-4.

In the end it turned out that the man page was not up-to-date, and the preferred way to get it running was the so-called MergedFB mode. This wasn't as easy to configure as expected, and I still got lots of 'Signal 11' segfault-style crashes.

The crashes seem to be totally unrelated to my graphics setup. In fact, it crashes when eth0 is not configured yet, but works after the network device is up. Now please somebody step up and explain...

Finishing preparations for upcoming netfilter developer workshop

I've spent a significant amount of time over the last couple of days with the final preparations of the upcoming 3rd netfilter developer workshop. This is the first one where I'm in charge of every tiny bit of the organization, and I hope I got everything right.

The first attendees are scheduled to arrive tomorrow. They might even arrive before me, since I'll be heading the 500km down south tomorrow.

More Allnet Devices contain Linux

I've now successfully proven that the ALL0185A, ALL0186, ALL1297, ALL2100, ALL2110 and ALL6100 devices contain the Linux kernel and are not distributed according to the GPL.

Considering the out-of-court agreement that I have concluded with them earlier this year in ALL0277, I have to say I'm a bit disappointed that this happened again. It should be in their own best interest to distribute within the GPL license terms, and not first try to infringe and wait until somebody complains.

I've contacted them, and they promised to publish the source code and adhere to the license within a short term. Let's see how this continues.

Fujitsu Siemens Corporation not fulfilling amicable agreement

As part of an amicable agreement, Fujitsu Siemens Corporation (FSC) agreed to make a donation to the German Unix Users Group. It came to me as a surprise, that GUUG has not yet received the funds even four months later!

Again, I am very disappointed by the behaviour of the former GPL violators. It should be in their own best interest not to produce any negative publicity.

On VIA's failure to provide adequate Linux support

VIA is definitely one of the most innovative producers of PC-hardware. Their EPIA-series mini-ITX and nano-ITX mainboards are ideal for small appliances, such as firewalls, VPN-gateways, and especially home entertainment platforms such as PVR/DVR applications, DVB-Receivers, DVD/VCD/AVI-players, VideoLan receivers and such.

Just two days ago, VIA made a press release on their new VeXP 3.0 release, a VIA-enhanced fork of xine. To the unfamiliar reader, this press release raises the impression that VIA is really involved with Linux and the Free Software community.

This is just terribly wrong. They do anything but to support GNU/Linux. Comparing this press release with reality, I think VIA's Linux involvement as a whole is nothing more than a PR strategy.

I've recently investigated the "Linux support" they make available for their EPIA platforms. Even from the first glance it was obvious, that VIA just doesn't have any idea on on what it takes to "Support Linux".

All they do is to publish proprietary, pre-compiled kernel frame buffer and XFree86 display drivers for a limited number of particularly old GNU/Linux distributions.

Oh yes, I almost forgot it: They also publish the source to some 'lite' driver which lacks all the functionality needed for hardware-assisted MPEG2 decoding. This is obviously useless, since the whole point of buying a small fan-less board with hardware MPEG acceleration and TV-Out is to use the acceleration.

So their "Linux Support" is so good, that a number of people have to spend days and days in reverse engineering their binary proprietary drivers. You can find more information about the reverse engineering effort. My special thanks are going to Ivor Hewitt for doing all this work.

But wait, wasn't that what the Linux folks usually did with Windows drivers? Welcome to the world of "VIA Linux support", where instead of reverse engineering Windows drivers, we now have to do it with Linux drivers.

If VIA was really interested in providing good GNU/Linux support for their EPIA products, they would

1. write full source code drivers licensed under appropriate Free Software licenses.
2. make those drivers use standard interfaces, the respective project's coding style, contain useful comments.
3. publish those drivers as patches against the latest development version of the respective project (kernel, XFree86, Xine)
4. Work with the respective project maintainers to integrate those patches
5. not have to care about maintaining RPMs for each and every distribution
6. not have to care about porting their drivers to ever-changing API's, since they are included in the respective Free Software projects
7. Provide documentation for their hardware down to the register level, so the Free Software community can continue development extending to features maybe not yet covered by the current driver.

Related Links:

http://lwn.net/Articles/99464/ VIA's original press release

http://www.viavpsd.com/ VIA's EPIA homepage

http://www.viaarena.com/ VIA's support forum and driver downloads

http://www.epiawiki.org/ The comprehensive source of EPIA/Linux related information

http://www.ivor.it/cle266/ The reverse engineered driver page

Video Documentation on 21C3

I've attended a meeting on the subject of providing audio/video documentation at the 21st Chaos Communication Congress. During that meeting, I was appointed as being responsible for this part of the 21C3 conference.

So we want to do on-the-fly encoding of four video signals from DC1394 cameras to DVD-compatible MPEG2, low-resolution MPEG4 for live-streaming, and OGG audio only for live streaming.

I did some preliminary experiments with the available experimental x86_64 assembly patches for ffmpeg, and it turns out that at least theoretically a 1.6GHz AMD64 should have enough power of doing those three encodings at the same time.

Unfortunately the dv1394 device at the moment only supports one encoder mmap() ing the ring buffer of incoming 1394 frames - but that should be fixed pretty easy.

I'll do some more experiments in the next couple of weeks, stay tuned.

Main netfilter.org server has been replaced

Yesterday I finally got around moving almost all netfilter.org services from our old Sun Ultra5 to the new XServe ClusterNode.

Unfortunately there were lots of complications, so I had to stay awake until 5am in order to get all services running again. At least for now, everything seems to run smoothly.

Using a human-based data acquisition plugin

Why buy expensive data acquisition boards, if you can have a cheap human being entering the data on some terminal? No, just kidding.

Anyway, GSPC now has a gpsc_acquire_user.c plugin that retrieves measurement data via a ncurses-based dialog instead of any data acquisition board. This is useful for testing, but also in some real-world cases.

Two hard drives dying in one week

This week already the second hard drive in one of my workstations died.. both times it was the same model: IBM DTLA-307060, produced Nov 2000 in Hungary. If that isn't some coincidence. Maybe they have a built-in 'best before' date :(

So both my main workstations (Dual PIII-733 and a Dual Apple G4-500) were inoperable, isn't that great? The good part is that they've been replaced with silent Samsung SP1213N models, significantly reducing the noise level in my office.

Off-the-shelf multi-port serial cards and Linux

This is now the third time I've bought some PCI serial multi-port card (6 to 8 ports) that claimed to have 'Linux support'. If you then read the document, the vendor bluntly tells you that Linux generally doesn't support more than four ports, so if you have two built-in ports, you can only use two more. I've never read such bullshit anywhere else ;)

So after some minor twiddling, I now submitted a patch adding support for this particular 6port device. Apparently there is either a wide variety of such boards, or almost no Linux users... A couple of years ago I added support for an AFAVLAB 8port serial card, to the Linux serial driver.

I think I now know way too much about the serial driver. Not stopping with those two PCI 8250 based boards, I did lots of serial driver hacking for the XServe G5 and also for my recent ARM embedded work. Let's hope I can again advance to some more exciting work in the future.

Attaching an UW-SCSI hard disk to an embedded ARM922T

No, I'm not doing this for fun, this is part of work. It turned out that nfsroot is a bit of a problem while you're hacking the core network stack (and everything breaks all the time). So I now attached an 18GB UW-SCSI disk to an old aic7xxx controller and plugged this into my ARM development board. Seems to work quite fine, as long as the aic7xxx_old driver is used. The new one apparently calls pci_alloc_consistent from interrupt context ?!?.

News on the GPL Violation Front

It's been some time that I've reported news on the GPL violation side... Thus, no news is good news, one could think. Unfortunately to the contrary, I've been receiving a number of new GPL violation reports, unfortunately none of them containing my copyrighted work - and thus I am now looking for the respective copyright holders in order to get this issue sorted out.

Stay tuned...

Performance of system logging

One of my customers recently had a serious performance issue with one of his installations. Surprisingly, it wasn't even the real applications software itself that had performance issues, but the mechanism used for logging from this application.

So I started to think about the way logging usually works within a Linux-based system.

The server applications can be divided within two groups. One of them logs via syslog(), the other logs directly to it's own files. The logging itself happens synchronously, i.e. blocking the normal code flow until the log line was written. In the case of syslog, it might block because the syslog pipe is full - in case of stand-alone files, the file/io might take some time to complete.

Even in a multi-threaded or forked model of a network server program, this might pose considerable problems with regard to threads waiting for their log i/o to complete.

Syslog itself might not be as bad, especially since the 2.6.x pipe implementation works with only the minimal necessary amount of copying, and supports larger pipe sizes to avoid writer blocking.

Some people however tend to use something like syslogger in order to redirect the log output from programs with no syslog support also into syslog. This means that you have one pipe between your application and syslogger, and another pipe between syslogger and your real syslog daemon.

Comparing this issue with networking is actually not too problematic. In networking, we have packets that are passed from one process to another... with logging it's not a packet but usually one or more lines of text (that is, about 60 to 240 characters per entry).

You don't want to copy this data around and around... and in a lot of installations you'd rather want to use a couple of log lines than to slow down your application just for some statistics that you might collect.

Of course, you don't want to modify any of the existing applications, too - they should just be able to use syslog() calls as usual. OF course you could load a LD_LIBRARY_PRELOAD lib and redirect the syslog() calls, if needed.

So what I came up with, is something like a partially mmap()able pipe. The logging process would log to that pipe like it would with any other file descriptor. Internally, that 'pipe' has a ring buffer of configurable size. The pipe-reader could now mmap() this ring buffer into his address space in order to read the log.

This scheme should have the advantage of not blocking the writer if the pipe is full (it would just wrap around the ring buffer), and it avoids copying the data from some in-kernel pipe buffer into the user-space of the pipe reader.

Did you notice, this now looks perfectly like the DMA ring buffer of your Ethernet device and the Linux softirq handler ;)

Anyway, as I didn't do any vm / vfs hacking in Linux so far, this is not a trivial thing to implement. And I have lots of other work at this point. However, I'd certainly like to investigate the possible performance gains [losses?] of this idea. Comments welcome.

IETF work on NAT behaviour

Apparently some people within the IETF have started a new working group called 'BEHAVE'. It is about the behaviour of NAT devices on the internet, and their inconsistent and incompatible behaviour. The working group aims to give guidelines to ipmlementors, in order to assure interoperability with new applications such as VoIP and peer-to-peer protocols, as well as multicast and others.

Certainly a topic that is in in the main focus of my interest, so I decided this is the right point in time to start participation in the IETF.

For more information about behave, see the mailinglist.

Upcoming Chaosradio episode on software patents

The next Chaosradio radio show will be about the ongoing debade on software patents, especially the recent development within the European Union.

Being part of the anti software patent movement for about 4-5 years now, I am more than happy to help with the radio show on this subject.

The radio show will be on air on Sept 01, 10pm GMT+2. If you understand german, there's a MP3 live stream available on the homepage.

Working on embedded Linux ARM SoC project

While there hasn't been any update on this weblog for quite some time, I've been buried under a lot of work.

One of the most interesting projects is an embedded ARM-based SoC project with special network acceleration hardware. Unfortunately I'm not allowed to talk too much about it at this point, but be assured it is very exciting, and of course runs Linux :)

During development I found it quite comfortable to run the small embedded system with nfsroot mounted from some larger box. The nfsroot contains a debootstrap'ed installation of Debian sarge for ARM.

The main problem for this kind of operation is the limited on-board memory. But I'm tempted to put a 64MB graphics card into one of the PCI slots and hack the Linux kernel to treat this framebuffer as (somewhat slow) RAM :)

Booting from a md raid device on powerpc

Apparently, nobody has ever tried to do this so far, since the mac partition handling code in the Linux kernel had no provisions for enabling auto-detection of md software raid.

I've now written patch for Linux 2.6.8, available at http://gnumonks.org/ftp/pub/patches/linux-2.6.8-mac-autoraid.patch implementing this feature. All you need to do is apply that patch, and make sure your md partitions have the type 'Linux_raid_autodetect' in the mac partition table.

Figured out the fan control on the XServe ClusterNode

I spent the last couple of hours figuring out the missing bits of the fan/thermal control on Apples Dual XServe ClusterNode. Luckily it's very similar to the design Apple used in their Desktop G5 machines, so I can build on the work that Benjamin Herrenschmidt did with his thermal_pm72 driver.

So in case anybody is interested in the technical details: Eight fans are controlled by the FCU (Fan Control Unit), which is attached to a i2c bus of the Apple U3 northbridge.

There are three RPM controlled fans per CPU. The Left CPU (viewing from the front of the machine) has fans #1,2,3. The right CPU: #4,5,6.

The other two fans are not RPM controlled, but just PWM controlled... so instead of setting an RPM, you have to set a pulse-width between 10 and 100%. PWM Fan #1 is located between RPM-fan 3 and 4 (between both CPU's) and it's job is to keep the U3 chip cool. PWM Fan #2 is located behind the PCI-X slots and thus cooling them (too bad in my machine there is no card to be cooled *g*).

Regulating the CPU fans is quite easy, since there is a per-CPU temperature sensor, and also a voltage and current reading, so we can calculate the power consumption of each CPU and tune the fans accordingly.

For the U3 it is a bit more difficult.. I have not yet found a way to get a temperature reading for it, but I'm quite sure there is some temperature sensor somewhere.

As for PCI cards, there is apparently some way to read the power consumption - but of course again undocumented and not reverse engineered yet. As I don't have PCI boards in my box anyway, I personally don't care that much. But I should now stop arguing rationally, since a machine hosted in some rack-space is very unlikely to need fan control at all :)

I'll try to make a somewhat cleaner unified driver for PowerMac7,2 and RackMac3,1 and post a patch in the next couple of days.

I really wonder why Apple is not releasing their FCU driver source code for Darwin... it's really annoying. And I doubt they can claim that it contains any valuable intellectual property that their competitors are not allowed to see ;)

Finally the XServe ClusterNode runs Linux!

Yes, it does. I now have two partitions: One running the experimental Gentoo ppc64 port, and another one running the overly-conservative Debian woody ppc32. The plan is to boot into Gentoo, and run publicly-accessible production services within the Debian woody chroot.

So how did I make it? Well, I gave up on the idea that the usual installation process of any distribution would work. So instead of trying to fix up whatever goes wrong in the installation scripts, I just escaped to a shell ASAP, run mac-fdisk, mkfs.ext3, extracted the stage3.tar.gz and did the rest of the Gentoo install.

Debian was then installed using the convenient debootstrap tool.

One of the major remaining questions is however: Does the Apple XServe Hardware give you anything similar to Sun boxes, where you could just send break over the serial line and get into OpenFirmware? This is very convenient for remotely resetting machines without any local 'reset-staff' present.

After some chatting with Benjamin Herrenschmidt, apparently nobody is working on getting fan rpm/speed/temperature control implemented on the XServe so far. Well, as it's a rack-mounted machine sitting in some hosting center I don't really care about the noise anyway.

More interestingly, the Apple KeyLargo2 based machines have a Hardware Watchdog. Driver Source code is available within the public part of the Darwin kernel, so it should be easy to implement a Linux driver for this. Maybe I'll find some time to dive into this.

IPv6 packet filter benchmarking

It seems like a German university is currently doing feature analysis and benchmarking of IPv6 packet filters. Coincidentally, I'm going to near that university next week anyway, so I'll stop over for a short visit and help them with their ip6tables evaluation setup.

I would be very interested to see some numbers on ip6tables... as we just discovered at the networking conference in Portland, nobody seems to be doing benchmarking / profiling on the Linux IPv6 code so far.

Database Design + Content for GPL-Violations

In order to keep track about the gpl violations that I am encountering myself or that are reported by fellow users, I really need some semi-automatic system to keep track of this.

Being a RDBMS geek in my former life, I designed a SQL-based data model to cope with the individual objects such as vendors, products, product-firmware-versions, violations, settlements, compensations, comments, documents, contracts, ...

It all turned out to be more complex than I thought initially. But I think it was really worth the effort.

This database is for strictly internal use, since there is a lot of confidential information in there. However, according flags indicating the public/private nature of the data records are included in the data model. At some later point I might extract the public information to create some web pages at www.gpl-violations.org.

It's main target is to allow me keep track with what's going on, and also keep track about what has been verified where, if for new upcoming firmware images the source code was made available, if the source was complete, ...

I've already filled in lots of the existing data I have, but it's far from being complete. This needs some more time of filling in data records.

And yes, I built some simple forms using GNU Enterprise Designer and Forms. It's still in 0.x stage, but usable for easy tasks.

Installing Linux on a G5 ClusterNode XServe

Now that I got this decent new dual G5 box, I wanted to install Linux. This turned out to be an extremely difficult job, as apparently nobody has ever tried to install Linux on any of the new XServe G5 Series machines, neither 32bit nor 64bit kernels.

There are a number of challenges:

• No internal IDE or SCSI CD-ROM
• Only serial console
• A very new hardware with little Linux support

First I tried a number of ready-built installation ISO images, including the current sarge Debian-installer image for PPC, and the 32bit and 64bit live images of Gentoo.

The first thing I had to do is to disable autoboot and enable the serial console. Luckily, the box actually ships with a manual that instructs you how to put the OF boot console on the serial port. You have to press the admin (!) Button at the front of the box a magic number of times.

To permanently make the serial console work, use the following OF commands:

> setenv input-device scca
> setenv output-device scca

Next I had to figure out how to boot from the external firewire cdrom.. apparently this depends on your OF device tree and the GUID of your firewire device. On my particular box it works with

> devalias cd /ht/pci@5/firewire@e/node@00d04b3c50090210/sbp-2@c000/disk@0

Using Commands like

> dir cd:,\

I was then able to list files on the CD-ROM. To boot the yaboot loader on a Debian installer cd image, you can use

> boot cd:,\install\yaboot
sbp2:Open ->login?
speed=ffffffff 2 2 load-size=239a4 adler32=a5cf5aa0 

Loading ELF



sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 Config file read, 2907 bytes


sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 \
sbp2:Open ->login?
speed=ffffffff 2 2 Welcome to Debian GNU/Linux sarge!

This is a Debian installation CDROM,
built on 20040729.

The default option is 'install'. For maximum
control, you can use the 'expert' option.

If the system fails to boot at all (the typical
symptom is a white screen which doesn't go away),
use 'install video=ofonly' or 'expert video=ofonly'.

The plain options are for the powerpc family of
processors (from 601 to G4). The *-power3 options
are for IBM Power3 boxes, and the *-power4 options
are for IBM Power4 and Apple G5 boxes. Press the tab
key for a list of options, or type 'help' for help.

************************************
If in doubt, just choose 'install', and if that 
doesn't work, try 'install video=ofonly'.
************************************
Welcome to yaboot version 1.3.12
Enter "help" to get some basic usage information

sbp2:Open ->login?
speed=ffffffff 2 2 boot: 

I tried all of the provided images, with different options - no success. A common option to be used because of the serial port is "console=ttyS0,57600". All I got was:

boot: expert-power4
Please wait, loading kernel...

sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2    Elf32 kernel loaded...
copying OF device tree...done
starting cpu /cpus/PowerPC,G5...failed: 00000000 
Calling quiesce ...

erasing fff06000  of Micron B1 part
flashing fff06000  of Micron B1 part
swapping blocks
DO-QUIESCE finishedreturning 0x01400000 from prom_init

Playing with the Gentoo live cd images didn't bring me any further at all.

I then tried to compile a current 32bit ppc 2.6.8-rc2 kernel by hand (for G5 CPU's). Putting this kernel on the debian installer ISO didn't get me any further. So apparently either the serial port is not working, or the kernel crashes somewhere.

Using a cross-compiler running on my dual G4 PowerMac, I compiled the same 2.6.8-rc2 kernel for ppc64 target platform. Putting this on the debian boot cd helped a lot, I now got it as far as:

boot: expert-g5-64 console=ttyS0,57600
Please wait, loading kernel...

sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2 
sbp2:Open ->login?
speed=ffffffff 2 2    Elf64 kernel loaded...
Looking for displays
OF stdout is    : /ht@0,f2000000/pci@3/mac-io@7/escc@13000/ch-a@13020
Opening displays...
Calling quiesce ...

DO-QUIESCE finishedreturning from prom_init
Found U3 memory controller & host bridge, revision: 53
Mapped at 0xe000000080000000                          
Found a K2 mac-io controller, rev: 96, mapped at 0xe000000080041000
PowerMac motherboard: XServe G5                                    
Starting Linux PPC64 2.6.8-rc1 
-----------------------------------------------------
naca                          = 0xc000000000004000   
naca->pftSize                 = 0x17              
naca->debug_switch            = 0x0 
naca->interrupt_controller    = 0x1
systemcfg                     = 0xc000000000005000
systemcfg->processorCount     = 0x2               
systemcfg->physicalMemorySize = 0x20000000
systemcfg->dCacheL1LineSize   = 0x80      
systemcfg->iCacheL1LineSize   = 0x80
htab_data.htab                = 0xc00000001f800000
htab_data.num_ptegs           = 0x10000           
-----------------------------------------------------
[boot]0100 MM Init                                   
[boot]0100 MM Init Done
idle = native_idle     
Linux version 2.6.8-rc1 (laforge@dathomir) (gcc version 3.4.1) #4 SMP Sat Jul 31 16:12:42 CEST 2004
[boot]0012 Setup Arch
via-pmu: Server Mode is disabled
PMU driver 2 initialized for Core99, firmware: 0c
nvram: Checking bank 0...                        
nvram: gen0=204, gen1=205
nvram: Active bank is: 1 
Adding PCI host bridge /pci@0,f0000000
Found U3-AGP PCI host bridge. Firmware bus number: 240->255
Adding PCI host bridge /ht@0,f2000000                      
Can't get bus-range for /ht@0,f2000000, assume bus 0
U3/HT: hole, 0 end at 9fffffff, 1 start at b0000000 
Found U3-HT PCI host bridge. Firmware bus number: 0->239
Can't get bus-range for /ht@0,f2000000                  
PCI Host 0, io start: fffffffffd800000; io end: fffffffffdffffff
PCI Host 1, io start: 0; io end: 3fffff                         
Top of RAM: 0x20000000, Total RAM: 0x20000000
Memory hole size: 0MB                        
On node 0 totalpages: 131072
  DMA zone: 131072 pages, LIFO batch:16
  Normal zone: 0 pages, LIFO batch:1   
  HighMem zone: 0 pages, LIFO batch:1
[boot]0015 Setup Done                
Built 1 zonelists    
Kernel command line: ro debconf_priority=low devfs=mount,dall init=/linuxrc console=ttyS0,57600
PowerMac using OpenPIC irq controller at 0x80040000
[boot]0020 OpenPic Init                            
OpenPIC Version 1.2 (4 CPUs and 120 IRQ sources) at e000000082ccd000
OpenPIC timer frequency is 25.000000 MHz                            
[boot]0021 OpenPic Timer                
[boot]0022 OpenPic IPI  
[boot]0023 OpenPic Ext
[boot]0024 OpenPic Spurious
[boot]0025 OpenPic Done    
Slave OpenPIC at 0xf8040000 hooked on IRQ 56
[boot]0020 OpenPic U3 Init                  
OpenPIC (U3) Version 1.2  
[boot]0025 OpenPic2 Done
PID hash table entries: 16 (order 4: 256 bytes)
time_init: decrementer frequency = 33.333333 MHz
Console: colour dummy device 80x25              
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)   
Memory: 498688k available (3840k kernel code, 4120k data, 212k init) [c000000000000000,c000000020000000]
Calibrating delay loop... 66.56 BogoMIPS
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
PowerMac SMP probe found 2 cpus                           
Processor 1 found.             
Synchronizing timebase
Got ack               
score 299, offset 1000
score 299, offset 500 
score 299, offset 250
score 299, offset 125
score 299, offset 62 
score 299, offset 31
score 239, offset 15
score -107, offset 7
score 101, offset 11
score -5, offset 9  
score 63, offset 10
score -51, offset 9
Min 9 (score 5), Max 10 (score 87)
Final offset: 9 (61/300)          
Brought up 2 CPUs       
NET: Registered protocol family 16
PCI: Probing PCI hardware         
U3-DART: table not allocated, using direct DMA
PCI: Probing PCI hardware done                
PCI: no pci dn found for dev=0001:04:0f.0 Apple Computer Inc. K2 GMAC (Sun GEM)
PCI: no pci dn found for dev=0001:05:0c.1 PCI device 1166:0240 (ServerWorks)   
SCSI subsystem initialized                                                  
usbcore: registered new driver usbfs
usbcore: registered new driver hub  
nvram_init: Could not find nvram partition for nvram buffered error logging.
rtasd: no RTAS on system                                                    
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
devfs: 2004-01-31 Richard Gooch (rgooch@atnf.csiro.au)   
devfs: boot_options: 0x1                              
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
Initializing Cryptographic API                          
pmac_zilog: 0.6 (Benjamin Herrenschmidt )
ttyS0 at MMIO 0x80013020 (irq = 22) is a Z85c30 ESCC - Serial port 
ttyS1 at MMIO 0x80013000 (irq = 23) is a Z85c30 ESCC - Serial port
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)                                         
sungem.c:v0.98 8/24/03 David S. Miller (davem@redhat.com)

So apparently, there were some issues finding the OpenFirmware dn (distinguished name) for the Ethernet Chips and the ServerWorks chips. I tried to put some printk's into the arch/ppc64/pci_dn.c file to see what's going on. This then led me to the earlier error messages about the U3-DART. After reading some more code, it appeared like the DART is Apple's IOMMU, and it is supposed to be needed only when running with >2GB RAM. My box had 512MB, but I tried to force usage of the DART by putting "iommu=force" into the kernel commandline.

Great, this was apparently the problem, since now I got up to the point where it wanted to mount the root filesystem. I thought I didn't really need an initrd, since the kernel contained all drivers statically linked in. However, Debian installer seems to be running inside initrd only.

First try was just using one of the pre-supplied initrd.gz images. Yes, they have the wrong versions of the modules - but I don't want/need those modules anyway.

Of course this wouldn't work either:

RAMDISK: Compressed image found at block 0                 
Kernel panic: VFS: Unable to mount root fs on unknown-block(0,0)
 <0>Rebooting in 180 seconds..                       

No errror message, nothing. So I thought the problem is with devfs, and I tried passing several different root parameters ('root=/dev/ram', 'root=/dev/rd/0') without any success.

In the end I found out that the structure sizes of the cramfs superblock (include/linux/cram_fs_sb.h) are arch-dependent, so I cannot use an initrd that was built on a ppc32 machine. Unfortunately it is also endian-dependent, and at this time I only have 32bit big endian and 64bit little endian boxes at home.

Next step was to use an ext2 initrd, since reasonable filesystems don't have any strange host/byteorder/wordsize dependencies.

Now it is able to load the initrd, and mount it... although then some other stuff goes terribly wrong. No time yet to investigate this.

Putting multiple SATA drives into a XServe ClusterNode G5

Apple is selling two different models of their Dual G5 XServe: One 'Normal' model, and another 'ClusterNode' Model. They are pretty much the same, but the ClusterNode doesn't have things you usually don't need in a rack-mounted 1U server anyway: CD-ROM and VGA-Card. However, it is also limited to a single hard drive.

I guess Apple's reason is that in a scientific cluster computing environment, the node's local storage is insignificant - whereas on a real server you most likely want multiple (mirrored) drives.

However, the significant price difference (Dual G5 ClusterNode has the same price as the Single G5 XServe) made me ponder buying a ClusterNode and adding another drive.

Fortunately, the hardware is quite similar. It turns out that the Mainboard has three SATA connectors, and the space for the 2nd and 3rd IDE drive was left empty. Also, the Backplane for Apples hotplug drives is not fully assembled - it is missing the connectors for the 2nd and 3rd drive :(

So Putting the drive in place and attaching it via a fixed cable to the SATA connector is no problem at all. However, Power is a slight problem. The whole machine has not a single standard power connector, so my only remaining option was to solder some wires onto the drive backplane PCB. This is ugly, but well.. who cares ;)

I'll put some photos of the modification online soon.

David Miller survived my 13-patch patch-bomb

This is good news, DaveM accepted all the 13 netfilter related patches that I had pending for 2.6.9. The patches included a number of optimizations, the ctstat, connection-based accounting, TCP window tracking, and some conversions to new in-kernel-API (seq_file, module_param).

Now let's hope that 2.6.8 will be released soon and we can start the 2.6.9 cycle...

OLS2004 is over

After holding a BOF on GPL-Violations, and the traditional netfilter/iptables BOF, OLS ended with Andrew Morton's Keynote.

Obviously, there also was the traditional OLS Social Event at the Black Thorn Pub, which I left quite early in order to get some more work done on the ulogd2 flow accounting work.

Final court opinion on Sitecom Appeal released

The court handling the Sitecom appeals case has now released it's final opinion. For those of you who happen to understand legal German, the 20 page document is available as PDF. An English translation will be available soon.

Merging 2.6.8-rc2 changes into patch-o-matic ng

I just started the boring job of merging 2.6.8-rc2 with patch-o-matic-ng... I'm happy that Jozsef, Martin and Patrick did this for the last couple of kernel releases. However, I need to get more into this job again in order to determine which patches still have to be submitted to the mainline kernel...

Expect some pom-ng breakage over the next couple of days...

IPFIX / ulog integration

After some more in-depth study of the IPFIX IETF drafts, I finally started coding. Having written the first dozens of lines, I discovered that on an abstract layer IPFIX doesn't do something too different from my good old ulogd. Ignoring the minor difference that ulogd deals with individual packets and IPFIX with flows, the ulogd_iret_t structure is very similar to what IPFIX templates are trying to describe.

So I now forked a ulogd2 branch off the current ulogd subversion tree and started to reorganize the tree.

For more flexibility, I am going for a stackable plugin infrastructure, where the sysadmin can configure stacks like: ULOG->ulogd_BASE->flow aggregation->IPFIX-over-TCP-export or ctnetlink->IPFIX-over-SMTP-export.

Group Photo of the Kernel Summit

At http://gnumonks.org/static/photos/ks2004/ are the group photos of this year's Kernel Summit. You obviously won't find me on those pictures, since I was behind the camera ;)

First day of OLS

OLS started today (well, it started with the official beer-drinking BOF yesterday night). Like at the kernel summit, there are massive problems with the wireless network, forcing me to operate in offline mode most of the time.

The presenters are apparently all running in slow motion, so I can allocate a small time-slice to listen to them and spend most of the time working on some code (conntrack-accounting/ipfix, qsearch, browsing through Rusty's patches). OLS thus starts more productive than I would have thought ;)

Had lunch with Daniel Phillips, who is now working on clustering infrastructure at RedHat. We detected a general shift from the 'everything is a filesystem' to 'everything is a socket' mentality.

Working towards IPFIX based on conntrack

I've written a patch to add 64bit packet and byte counters for both directions of every ip_conntrack. This should enable a clean and efficient implementation of flow based accounting, when combined with ctnetlink events and a userspace daemon picking up those events.

I need to study the IPFIX (IETF Working Group) specifications in more detail before writing the respective daemon...

The patch is apparently working, you can read the counters via /proc/net/ip_conntrack and also use a modified/extended/updated version of the 'connbytes' match.

Day one of the Kernel Summit

So this was day one of the famous kernel summit. Apart from meeting lots of friends, this basically meant lots of in-depth technical discussions on various subjects.

Most noticeable were long discussions about the deficiencies of the power management API, problems with 3-level-page tables on AMD64, and last but not least: The first-hand technical information from AMD, Intel and IBM on their upcoming CPU generations.

My personal favourite (AMD) will be shipping dual core (not hyper-threading, but two real cores) CPU's by mid 2005. They share the same Hyper-transport and Memory interface, and therefore have to divide I/O Bandwidth between them.

Also had some interesting discussions with Jamal about netfilter performance and the future l3 generalized connection tracking (called nf_conntrack). Maybe I can talk him into attending the netfilter workshop for further discussion of his ideas.

Pattern-matching API in the 2.6.x Kernel

There are various places in the kernel where we need to do some kind of pattern matching on the packet contents. Applications range from connection tracking helpers (looking for FTP PORT command, ...) over the 'string' match to intrusion detection systems.

Two years ago, Phillip