Family - Ties of Blood

Tonight I enjoyed the rare opportunity to watch a Bollywood movie in Berlin, almost at the same time the movie is released in India. We usually only get a hand full of Indian movies every year, to very small cinemas, and about one to two years after they ran in India.

I personally enjoyed the film quite a lot, even though the critics in India seem to be disappointed by it. But well, what do I care about those critics ;)

One advantage of watching Bollywood movies in Germany is that you don't need a ticket reservation, since very few people are interested in those movies anyway. Second, you can actually enjoy a film without some 20 to 30something minutes of advertisements and anti-screener propaganda.

iptables-1.4 branch opened

Since we now have the x_tables kernel side code in the upcoming 2.6.16 series, I'm working on getting iptables-1.4.x done to actually take advantage of the new kernel's abilities.

The main reason why people are interested in this, is to get matches like 'state' and 'conntrack' working for IPv6. Even though 2.6.15 has nf_conntrack and thus state tracking for IPv6, you cannot really use it from ip6tables yet.

The same goes for all native x_tables matches and targets. However, I think we'll also release a new version of iptables-1.3.x just with 'state' and 'conntrack' support, since it gives a more stable foundation for production users than a completely new 1.4.x branch with hundreds of kilobytes of patches.

First GPLv3 draft

As almost every reader of this journal will know, the first GPLv3 draft has been published, and everyone is invited to comment on it.

I obviously already left some comments, though I still want to write up a somewhat larger article on my thoughts on it. This journal entry is not that article ;)

In general, I'm quite relieved. I had somewhat mixed expectations - but almost everything looks quite fine, and there are hardly any issues. I obviously like the DRM countermeasures.

From a gpl enforcement point of view, it is very good to see that the "complete corresponding source code" has been specified in more detail. This should save us from the hassle of ever again starting the discussion (nit-picking) on whether "scripts to control compilation and installation" (GPLv2) really only means scripts, or whether it also covers other methods controlling compilation and installation.

What is a real problem, and I hope this can still be resolved, is the new "60 days" grace period that was introduced. With GPLv2, the right to distribute the software was automatically revoked in the case non-conformant distribution has happened. In the v3 draft, there is a grace period where the rights _may_ be terminated, and only 60 days after being notified by one of the copyright holders.

The intention of it is to take care of "inadvertent violation". As harmless and reasonable as this sounds, this change has the potential to render most of the current enforcement success of gpl-violations.org impossible in the future.

From all the 60+ cases that we've enforced, I cannot tell you one case where the defendant would not claim that the violation was inadvertent. So in reality, inadvertent basically means "we didn't care". However, the whole point of the gpl enforcement exercise is to raise awareness and make them care before it is too late.

The 60 days grace period is not acceptable. On the one hand, we (in Germany) basically loose the ability to apply for preliminary injunctions. PI's are only granted in case of urgency, which translates (depending on the court) to something like 30 days. So if I know for more than 30 days that somebody is infringing on my copyright (and don't get the matter resolved with him in that period of time), then I can't consider this matter as urgent.

The 60 days grace period is also not acceptable, because it would basically reduce the motivation to comply with the license in the first place. So for EvilCorp Inc. it is perfectly possible to design a product using GPL licensed software, not comply with the license, ship the product, wait for a copyright holder to send a notice, make sure that I ship all the remaining in-stock products that do not contain a written offer, GPL text and/or source code in the 60 remaining days, and then start behaving GPL compliant. If such behaviour has no consequences at all, why would anyone behave different in the first place?

Hard disks dying, IBM/Hitachi's packaging guidelines

Now the third hard disk died in the first three weeks of 2006. I hope this doesn't continue throughout the rest of the year. Luckily all important data is mirrored and backed up. Still it is a pity to loose some 200GB of the nice documentaries I recorded from arte using the Linux DVB stack and vdr.

It's been some time since I last RMA'd a IBM/Hitachi hard drive. They still have the same annoying packaging policies, where they only allow rubber foam packaging material and nothing else. Interestingly, all hard drives that I purchase new from one of Europes largest hardware distributros are _never_ packaged like Hitachi Global Storage wants to have them pacakged for RMA.

This means that even if you keep your original packaging, it won't help. Also, for more than five years, the HGST (formerly IBM) web site tells you that ordering packaging material that complies with their packaging instructions will be ``Coming soon''. I wonder what definition of 'soon' they have, considering that their IDE/SATA drives only have three years of warranty ;)

Also, I recently went through my collection of old retired hard drives, most of them too small (<60GB) to do anything useful these days. I found out, that almost all the IBM drives wouldn't spin anymore, even though they were perfectly working when they were taken offline. Since they are dead, I opened them, only to find out that the heads hadn't been parked at all.

For those who don't know: Heads are usually parked when a drive powers down. The idea of this is that you have a special 'rough' surface area where the heads can't stick to the platter (due to cohesion). When I saw my unparked broken drives, I remember reading an article some time ago that IBM allegedly had firmware bugs causing heads not to park accordingly... I would have updated the firmware, if there was any publicly available firmware updates.

Anyways, I still keep the broken drives, just in case I once hit a time where I don't know what else to do and want to learn more about 'contemporary hard drive signal processing'. Unlikely to happen, but you gotta keep your options ;)

userspace conntrack helper code compiles (yet untested)

Finally, both the kernel side (nfnetlink_helper) and the userspace side (libnetfilter_cthelper) code for userspace conntrack helper support is basically finished and compiles. I didn't yet dare to test it, and I'm rather heading off to bed now. Testing will be done tomorrow.

So how is this supposed to work? Well, basically a new nfnetlink subsystem exists, which can (on behalf of an userspace process) create dummy "nf_conntrack_helper" structures inside the kernel. Such a dummy structure has the usual properties (tuple, mask, timeout, etc.) but a dummy expectfn() which only calls NF_QUEUE() to send the packet to userspace. Userspace can then look at the packet, possibly modify it and re-inject it back into the kernel. Since helpers are now processed at a different netfilter hookfn() than the rest of the conntrack code, this actually works.

Now during the reception of such a packet in userspace, the process is likely going to want to create a new expectations. Expectations can already be created by means of libnetfilter_conntrack/nf_conntrack_netlink. However, in order to create the expectation, a number of things are needed. Mainly the tuple(s) of the master conntrack, but also other ancillary data such as ctinfo are sometimes desired. As long as we don't do NAT, the process could derive the tuple from the packet's IP[v6] header, and query nf_conntrack_netlink for the remaining details. However, this is inefficient since we'd add another kernel/userspace round-trip and the associated latency. So instead, I chose to extend nfnetlink_queue a bit, and allow it to have a new queue_mode (NFQ_MODE_PACKET_CT) in which there is a new nested attribute (NFQA_CT) which in turn contains the tuple, id and ctinfo.

Userspace now has all informations to create a new expectation. But wait, what do we do about expectfn()? We use the same magic as with helpfn(): Userspace tells the kernel to which nfnetlink_queue queue_id packets hitting the expectfn() should be sent. The 'minor' difficulty here is that expectfn() is called from the middle of the conntrack code (init_conntrack() actually), and when we get back from the queue (set_verdict or re-inject), then the netfilter hook code would continue at the next hookfn, skipping most of the conntrack code. But we can also return NF_REPEAT in order to call conntrack again. Since our expectation is already confirmed, expectfn() will not be called and it _SHOULD_ somehow just magically work, maybe with some tiny ugly hack here or there.

The NFQA_CT way is still far from being optimal, since we copy the same conntrack tuple for every packet of the control connection to userspace, no matter that this information never changes, and no matter that we actually only need it in those few cases where we want to raise an expectation. So the mid-term plan is to make userspace keep a small copy of selected conntrack state entries. This can be done by sending NEW and DELETE events for all conntracks that have a helper assigned. We could create a new multicast group specifically for this purpose, in order to keep the overhead and memory usage low. Userspace keeps a hash table indexed by ct->id. Packets sent via nfnetlink_queue will therefore only need a single 32bit ID attribute and not the full tuple(s).

Apart from userspace helper code, I've been working on getting some x_tables / nf_conntrack refcounting / dependency issues sorted out. Again another issue where having a couple of dozens of inter-dependant netfilter modules seems to become a major PITA. Sometimes I want to have back the simplicity of a truly monolithic kernel.

x_tables merged mainline

Linus has merged x_tables, even though I introduced some "doesn't build without IPv6 support" breakage that only somebody not into networking would ever detect (hey, would you build a kernel without ipv6?) ;)

Anyway, will try to be more cautious about these issues, as nobody wants to end up with a "your patches break the kernel tree" reputation.

ulogd-2.00beta1 release

Finally, there is a first public beta version of ulogd 2.x

If you use (and like) ulogd-1.x, you should definitely have a look at the 2.x release. Apart from packet-based logging, ulogd-2.x now also support flow-based logging. This means that you can just run this daemon (and a recent 2.6.14/2.6.15 kernel) to log per-connection meta data into text files, syslog, mysql, postgresql, or sqlite3 databases. If you enabled per-connection packet/byte counters in your kernel config, you even get flow-based accounting.

If you're interested, check it out at netfilter.org/projects/ulogd.

Bug-reports welcome. Don't ask for too much documentation at this time, rather contribute some :)

Today marks the first discovery of a ulogd GPL violation

It's actually not really all that important, but today I found the first product that distributes my ulogd program in a GPL incompliant way.

To my biggest surprise, it's not a Firewall/Router/WLAN device, but rather a NAS. Still have to figure out where, how and why they use ulogd on it, but it's there (and no source code [offer]).

x_tables, take 5. nfsim tested.

Today I've posted the (hopefully) final version of x_tables, the in-kernel generalization of {arp,ip,ip6}_tables to netfilter-devel.

After some nfsim hacking, I've been able to add x_tables support to nfsim and have been successfully running the full nfsim testsuite. The testsuite found a single bug (which has been fixed) but otherwise all tests are passed.

Seems like we're going to push x_tables as well as the nf_conntrack port of ctnetlink (nf_conntrack_netlink) for 2.6.16. Also, as I just noticed on kaber's blog, his IPsec patches have made it in time, too. Userspace conntrack helper support is definitely 2.6.17, though.

More drivers moving to the Devicescape stack

I'm happy to see that both the Broadcom and the Realtek driver developers have now ported their drivers to the recently GPL licensed Devicescape 802.11 stack for the Linux kernel. This IMHO shows that a lot of developers still make decisions based on technical merits, rather than on political ("it's not in mainline") fuzz.

Let's hope that there will be a quick transition of the in-kernel stack towards the Devicescape code, and let's hope that we'll see a full-featured HAL-less Atheros driver for it - and all my 802.11 Linux dreams will come true ;)

Holidays in India

For the third year Elisabeth and me have been planning to take holidays in India, unfortunately with no success so far.

But now it's fixed: We've booked our flight tickets for March 2006 just two days ago. We'll be starting in Bangalore, and do some travelling in Karnataka and Kerala.

So please don't expect me to get any productive work done during March this year...

Have to turn down invitation on GPLv3 conference

As you might know, the GNU GPL is currently under review and version 3 is underway. With regard to the GPLv3 process, the FSF will be holding a conference later this January to which I had the honour to be invited.

Since many people have already been wondering why I will not participate: It is not because of the conference or because of the FSF. My previous contacts with the FSF have been very forthcoming and productive, and I would very much like to share my GPL enforcement experience at the GPLv3 conference.

Unfortunately though, the conference will be held in the USA, a country to which I'm not going to travel anymore because I don't want to hand over (and leave) my biometric information (aka fingerprints) in a country that basically has non-existing data protection rights, esp. when it comes to government agencies and foreigners.

In addition to the biometrics issue, there are numerous dangers from the software patent and the DMCA front for people like me who indulge in quite a bit of reverse engineering. In the end, the US just don't sound like a place where I would feel comfortable and/or safe and/or secure in any way.

My best wishes to the GPLv3 conference, I hope they'll have a productive meeting for the future of free software.

22C3 is over

Two days ago, 22C3 was closed. This years incarnation of Europe's largest hacker conference can be seen as a full success. Some 3000 attendees, about 180 lectures, a 10Gigabit Internet Uplink and our own /16.

The video recordings have turned out fine. We've had working WMV live streams, and somewhat intermittently working MPEG2 and MPEG4 live streams, as well as working OGG and MP3 audio streams of all four lecture tracks.

For archival, we have MPEG2Video (5Mbit) as well as the original DV tapes, and a FLAC audio recocrding.

Looking at the tremendous amount of work that went into the A/V recordings, and the fact that I'm involved with the A/V team since seven years, I'm actually thinking about looking for some other area where I can get involved next year.

My two lectures (on OpenEZX and librfid/libmrtd) went fine, even though they both had very little preparation ;)

In the next couple of days I'll be cutting the fourth day of the video recording, and then slowly getting back into netfilter and OpenEZX related development. Oh yes, and I'll also promise more blog updates.

For some strange reason, my git tree seems to have become corrupted over the last two weeks, so I first need to sort this out before getting any reasonable work done.