Finally, there is a hardware vendor who actually releases GPL licensed drivers for embedded technology: Samsung. The subject matter is Samsungs OneNAND flash technology.
It's good to see such a move, and it is greatly appreciated. I hope we see more of those. Thanks, Samsung.
It's not funny anymore. The current rate at which new GPL violations get reported and/or discovered, especially from the appliance/embedded market is really alarming.
For example, I haven't yet seen a single Linux-based NAS product that was even remotely license compliant when first analyzing it. And I'm not only talking about the SoHo NAS boxes with one or two hard disk drives, but even about enterprise storage systems.
On the Enterprise end We're now also Seine carrier grade network equipment such as SONET/SDH switches, metropolitan area Ethernet, DSLAMS and the like.
Also, in some areas of business, competing companies seem to make the same mistake again, rather than learning from their competitor. Some time ago I had to resolve GPL issues with Maxtor Shared Storage drives, when they were first released. Now I found out that Western Digital has similar systems called NetCenter. Ordered one, and it came without GPL license text, written offer or source code.
Finally, there is one good example though. For a very long time, a product that I analyzed was actually GPL compliant. It's good to see that there are a few who get it right, from the beginning: The APC NetBotz family of products. The manual contains a reference to the source code, which can be obtained from ftp://ftp.netbotz.com/gpl/.
Anyway, I need a break (see my holiday related post). Hopefully I'll get back from that trip rested, with lots of energy and an extra portion of patience. This has become more of a burden than I ever thought.
The second and third quarter of this year definitely are the right time to think of a way to incorporate gpl-violations.org as an NGO/non-for-profit. One that can actually pay somebody hunting down those cases, doing the day-by-day work. I have a dream that in some point in the future I can once again concentrate on cool and interesting development, like most other hackers do.
I'm feeling terrible. The second day in a row where I didn't find time to write a single line of code, merge any contributed patches, squash any bugzilla entry. Not even to speak of paid-for work.
While I used to spend about 30% of my time with GPL enforcement related work, it now peaks at about 70% for the last two weeks. This is not a good sign.
So apart from talking to lawyers, proof reading legal paperwork, negotiating with allegedly infringing companies and the like, I now also start having trouble doing test purchases. Not only refuse some retailers to take orders from me, but also if I actually place an order it raises new problems.
The last web store I ordered a test purchase from now asked me for a complete, readable copy of both sides of my ID card. WTF ?!? This is totally against any data protection laws. There is absolutely no requirement for them to know my passport photograph, id card number, size or eye colour. So as a follow-up I had to write an official complaint with the Berlin data protection agency - as if I didn't have any other work to do.
Also, for the last months, I find myself giving about EUR 10k in 0% interest loans to GPL infringing companies. That's the amount of money spent for test purchases that I had to do to confirm GPL violations but which hasn't yet been reimbursed.
About the only positive thing in the course of my work day was producing the Chaosradio Express issue on gpl-violations, which Tim and I did earlier this evening.
Oh, and the best thing that happened today in general, is that the German Federal Constitutional Court has invalidated a recent law that allowed the government to order the military to shoot a passenger plane which was abducted by terrorists. At least some people still have a sane view on human rights.
As you can see from this post by Georg Greve and this one by Werner Koch, a number of high-profile FSFE guys really seem to like the cm4040 driver that I merged recently into Linux 2.6.x
Who would have expected that there apparently was a high demand for using smartcards with GnuPG on notebooks :)
Out of all the embedded network devices that had GPL issues, the Texas Instruments AR7 based devices probably have the worst GPL compliance history I've ever seen. The time has come to properly rant about this.
It's yet unclear whether this is TI's own fault, or just the fault of their OEM/ODM manufacturers. But I'm more than determined to find out.
Anyway, the list of problems with TI AR7 based devices is so incredibly long, that I don't even know where to start.
First of all, re-engineering their devices (for GPL compliance audits and legal action following up to such an audit) is incredibly difficult because they've added LZMA compression to both the kernel image (vmlinux) and squashfs.
Now what's so difficult about this? You might argue that the LZMA algorithm is (L)GPL licensed and publicly available. As is the original kernel source code, and the squashfs code. Also, you might know that numerous individuals have already released patches to add LZMA to kernel boot, initrd and squashfs.
However, there are various methods (with/without LZMA header, with/without p7zip header, etc.), and there simply is no standard on how to build a system from the algorithm.
Getting to the actual infringements. So far I've seen devices that
There are many other potential issues, on whose GPL compatibility (or lack thereof) I do not want to comment at this time, such as their binary only drivers for the DSL chipset, the WLAN driver.
Interestingly, all of the Vendors of TI AR7 based devices with whom I had contact on the GPL issues showed equally little interest into bringing their products into compliance. Now this could all just be a coincidence. But my personal guess is that they just forward whatever questionable policy they get from their upstream chipset and reference software development kit provider: TI.
You might wander about the device manufacturers in question? I'm still a bit hesitant in disclosing names. One of the first companies running into GPL trouble with TI AR7 was D-Link. Another company with anything but the cleanest GPL history on TI AR7 based devices is AVM, who produce the overly popular and widely branded FritzBox devices.
There is another brand that is sold in significant quantities, at least in the German market. We're on the brink of applying for the next gpl-violations.org preliminary injunction, so I won't be able to say any names.
[and now, after some five hours of gpl-violations related device re-engineering before getting up, I'll finally try to find some time go get some breakfast.]
It's already been at some point at the End of 2005, but now I finally got around writing a press release on this subject:
gpl-violations.org has enforced yet another high-profile (at least in the German speaking continental European world) case of a GPL violation. Instead of repeating myself, you might want to read this release or the German version.
My real problem is a lack of time, and it's more than a pity that gpl-violations.org didn't have a press release for nine months - even though those were full of successful enforcement work. I hereby promise to improve my public relations work.
Isn't it a strange coincidence, that a reasonably non-trivial netfilter bug gets the bugzilla ID 404 ?
Well, before I try to build some conspiracy theories about somebody manipulating the bug id number sequence generation of our bugzilla installation, I'd rather concentrate on the real work.
Dave Remien is an excellent bug reporter, so as a maintainer you can actually not expect anything more than his detailed documentation (yes, I know, certificate has expired, too lazy and busy to update it right now, stay tuned). From an outside perspective, it appears like packets get 'stuck' in nfnetlink_queue. In reality, it seems like the kernel is doing everything fine, just the library eats some packets from time to time, meaning that they remain inside the kernel queue and increase it's length (and thus leak memory) one at a time.
The real cause has yet to be discovered, I'm confident that there will be some news tomorrow.
Some time ago, probably in November 2005, somebody on IRC offered me travel sponsorship for FISL 7. Unfortunately I don't keep IRC logs, and neither do I remember who it was.
If you are the person I'm talking about, and you're reading this: Please contact me immediately. I'm about to take care of my travel preparations and need to know whether that sponsorship will actually happen or not.
Thanks a lot!
I've released iptables-1.3.5 earlier today. This will probably mark the last 'new feature' release of the iptables-1.3.x branch.
I'm still working on the initial beta release of iptables-1.4.x, the userspace counter part to what is now known in kernel space as 'x_tables'. Stay tuned.
Out of my four proposed papers at FISL 7.0, three have been accepted. To my big surprise, the paper on gpl-violations.org was turned down. I would rather have dropped one of the other papers than this one :(
Anyway, as indicated before on this blog, I'm more than happy to be able to visit Brazil again.
Today my new Apple Quad-Core 2.5GHz G5 has arrived. Benjamin Herrenschmid's patches for multi-core Apple G5 boxes work like a charm. The only big issue is the thermal management. I looked into fan control / thermal management a bit, but unfortunately I don't have any time to work on this now, since I'm leaving for a one week tonight :(
You might ask yourself why a Linux kernel hacker buys a Quad Core G5 box at this time, now that Apple has started to sell Intel based boxes. However, this is exactly the reason. At this time, this might be almost the last possibility to get a four-way PPC64 SMP box that is available off-the-shelf.
If I want an x86 box, I wouldn't buy an Apple. The sole reason for having bought and used a number of Apple machines during the last six years was because they're mainstream PPC based hardware.
Tonight I enjoyed the rare opportunity to watch a Bollywood movie in Berlin, almost at the same time the movie is released in India. We usually only get a hand full of Indian movies every year, to very small cinemas, and about one to two years after they ran in India.
I personally enjoyed the film quite a lot, even though the critics in India seem to be disappointed by it. But well, what do I care about those critics ;)
One advantage of watching Bollywood movies in Germany is that you don't need a ticket reservation, since very few people are interested in those movies anyway. Second, you can actually enjoy a film without some 20 to 30something minutes of advertisements and anti-screener propaganda.
As almost every reader of this journal will know, the first GPLv3 draft has been published, and everyone is invited to comment on it.
I obviously already left some comments, though I still want to write up a somewhat larger article on my thoughts on it. This journal entry is not that article ;)
In general, I'm quite relieved. I had somewhat mixed expectations - but almost everything looks quite fine, and there are hardly any issues. I obviously like the DRM countermeasures.
From a gpl enforcement point of view, it is very good to see that the "complete corresponding source code" has been specified in more detail. This should save us from the hassle of ever again starting the discussion (nit-picking) on whether "scripts to control compilation and installation" (GPLv2) really only means scripts, or whether it also covers other methods controlling compilation and installation.
What is a real problem, and I hope this can still be resolved, is the new "60 days" grace period that was introduced. With GPLv2, the right to distribute the software was automatically revoked in the case non-conformant distribution has happened. In the v3 draft, there is a grace period where the rights _may_ be terminated, and only 60 days after being notified by one of the copyright holders.
The intention of it is to take care of "inadvertent violation". As harmless and reasonable as this sounds, this change has the potential to render most of the current enforcement success of gpl-violations.org impossible in the future.
From all the 60+ cases that we've enforced, I cannot tell you one case where the defendant would not claim that the violation was inadvertent. So in reality, inadvertent basically means "we didn't care". However, the whole point of the gpl enforcement exercise is to raise awareness and make them care before it is too late.
The 60 days grace period is not acceptable. On the one hand, we (in Germany) basically loose the ability to apply for preliminary injunctions. PI's are only granted in case of urgency, which translates (depending on the court) to something like 30 days. So if I know for more than 30 days that somebody is infringing on my copyright (and don't get the matter resolved with him in that period of time), then I can't consider this matter as urgent.
The 60 days grace period is also not acceptable, because it would basically reduce the motivation to comply with the license in the first place. So for EvilCorp Inc. it is perfectly possible to design a product using GPL licensed software, not comply with the license, ship the product, wait for a copyright holder to send a notice, make sure that I ship all the remaining in-stock products that do not contain a written offer, GPL text and/or source code in the 60 remaining days, and then start behaving GPL compliant. If such behaviour has no consequences at all, why would anyone behave different in the first place?
Since we now have the x_tables kernel side code in the upcoming 2.6.16 series, I'm working on getting iptables-1.4.x done to actually take advantage of the new kernel's abilities.
The main reason why people are interested in this, is to get matches like 'state' and 'conntrack' working for IPv6. Even though 2.6.15 has nf_conntrack and thus state tracking for IPv6, you cannot really use it from ip6tables yet.
The same goes for all native x_tables matches and targets. However, I think we'll also release a new version of iptables-1.3.x just with 'state' and 'conntrack' support, since it gives a more stable foundation for production users than a completely new 1.4.x branch with hundreds of kilobytes of patches.
Now the third hard disk died in the first three weeks of 2006. I hope this doesn't continue throughout the rest of the year. Luckily all important data is mirrored and backed up. Still it is a pity to loose some 200GB of the nice documentaries I recorded from arte using the Linux DVB stack and vdr.
It's been some time since I last RMA'd a IBM/Hitachi hard drive. They still have the same annoying packaging policies, where they only allow rubber foam packaging material and nothing else. Interestingly, all hard drives that I purchase new from one of Europes largest hardware distributros are _never_ packaged like Hitachi Global Storage wants to have them pacakged for RMA.
This means that even if you keep your original packaging, it won't help. Also, for more than five years, the HGST (formerly IBM) web site tells you that ordering packaging material that complies with their packaging instructions will be ``Coming soon''. I wonder what definition of 'soon' they have, considering that their IDE/SATA drives only have three years of warranty ;)
Also, I recently went through my collection of old retired hard drives, most of them too small (<60GB) to do anything useful these days. I found out, that almost all the IBM drives wouldn't spin anymore, even though they were perfectly working when they were taken offline. Since they are dead, I opened them, only to find out that the heads hadn't been parked at all.
For those who don't know: Heads are usually parked when a drive powers down. The idea of this is that you have a special 'rough' surface area where the heads can't stick to the platter (due to cohesion). When I saw my unparked broken drives, I remember reading an article some time ago that IBM allegedly had firmware bugs causing heads not to park accordingly... I would have updated the firmware, if there was any publicly available firmware updates.
Anyways, I still keep the broken drives, just in case I once hit a time where I don't know what else to do and want to learn more about 'contemporary hard drive signal processing'. Unlikely to happen, but you gotta keep your options ;)
Finally, both the kernel side (nfnetlink_helper) and the userspace side (libnetfilter_cthelper) code for userspace conntrack helper support is basically finished and compiles. I didn't yet dare to test it, and I'm rather heading off to bed now. Testing will be done tomorrow.
So how is this supposed to work? Well, basically a new nfnetlink subsystem exists, which can (on behalf of an userspace process) create dummy "nf_conntrack_helper" structures inside the kernel. Such a dummy structure has the usual properties (tuple, mask, timeout, etc.) but a dummy expectfn() which only calls NF_QUEUE() to send the packet to userspace. Userspace can then look at the packet, possibly modify it and re-inject it back into the kernel. Since helpers are now processed at a different netfilter hookfn() than the rest of the conntrack code, this actually works.
Now during the reception of such a packet in userspace, the process is likely going to want to create a new expectations. Expectations can already be created by means of libnetfilter_conntrack/nf_conntrack_netlink. However, in order to create the expectation, a number of things are needed. Mainly the tuple(s) of the master conntrack, but also other ancillary data such as ctinfo are sometimes desired. As long as we don't do NAT, the process could derive the tuple from the packet's IP[v6] header, and query nf_conntrack_netlink for the remaining details. However, this is inefficient since we'd add another kernel/userspace round-trip and the associated latency. So instead, I chose to extend nfnetlink_queue a bit, and allow it to have a new queue_mode (NFQ_MODE_PACKET_CT) in which there is a new nested attribute (NFQA_CT) which in turn contains the tuple, id and ctinfo.
Userspace now has all informations to create a new expectation. But wait, what do we do about expectfn()? We use the same magic as with helpfn(): Userspace tells the kernel to which nfnetlink_queue queue_id packets hitting the expectfn() should be sent. The 'minor' difficulty here is that expectfn() is called from the middle of the conntrack code (init_conntrack() actually), and when we get back from the queue (set_verdict or re-inject), then the netfilter hook code would continue at the next hookfn, skipping most of the conntrack code. But we can also return NF_REPEAT in order to call conntrack again. Since our expectation is already confirmed, expectfn() will not be called and it _SHOULD_ somehow just magically work, maybe with some tiny ugly hack here or there.
The NFQA_CT way is still far from being optimal, since we copy the same conntrack tuple for every packet of the control connection to userspace, no matter that this information never changes, and no matter that we actually only need it in those few cases where we want to raise an expectation. So the mid-term plan is to make userspace keep a small copy of selected conntrack state entries. This can be done by sending NEW and DELETE events for all conntracks that have a helper assigned. We could create a new multicast group specifically for this purpose, in order to keep the overhead and memory usage low. Userspace keeps a hash table indexed by ct->id. Packets sent via nfnetlink_queue will therefore only need a single 32bit ID attribute and not the full tuple(s).
Apart from userspace helper code, I've been working on getting some x_tables / nf_conntrack refcounting / dependency issues sorted out. Again another issue where having a couple of dozens of inter-dependant netfilter modules seems to become a major PITA. Sometimes I want to have back the simplicity of a truly monolithic kernel.
Linus has merged x_tables, even though I introduced some "doesn't build without IPv6 support" breakage that only somebody not into networking would ever detect (hey, would you build a kernel without ipv6?) ;)
Anyway, will try to be more cautious about these issues, as nobody wants to end up with a "your patches break the kernel tree" reputation.
It's actually not really all that important, but today I found the first product that distributes my ulogd program in a GPL incompliant way.
To my biggest surprise, it's not a Firewall/Router/WLAN device, but rather a NAS. Still have to figure out where, how and why they use ulogd on it, but it's there (and no source code [offer]).
Finally, there is a first public beta version of ulogd 2.x
If you use (and like) ulogd-1.x, you should definitely have a look at the 2.x release. Apart from packet-based logging, ulogd-2.x now also support flow-based logging. This means that you can just run this daemon (and a recent 2.6.14/2.6.15 kernel) to log per-connection meta data into text files, syslog, mysql, postgresql, or sqlite3 databases. If you enabled per-connection packet/byte counters in your kernel config, you even get flow-based accounting.
If you're interested, check it out at netfilter.org/projects/ulogd.
Bug-reports welcome. Don't ask for too much documentation at this time, rather contribute some :)
Today I've posted the (hopefully) final version of x_tables, the in-kernel generalization of {arp,ip,ip6}_tables to netfilter-devel.
After some nfsim hacking, I've been able to add x_tables support to nfsim and have been successfully running the full nfsim testsuite. The testsuite found a single bug (which has been fixed) but otherwise all tests are passed.
Seems like we're going to push x_tables as well as the nf_conntrack port of ctnetlink (nf_conntrack_netlink) for 2.6.16. Also, as I just noticed on kaber's blog, his IPsec patches have made it in time, too. Userspace conntrack helper support is definitely 2.6.17, though.
I'm happy to see that both the Broadcom and the Realtek driver developers have now ported their drivers to the recently GPL licensed Devicescape 802.11 stack for the Linux kernel. This IMHO shows that a lot of developers still make decisions based on technical merits, rather than on political ("it's not in mainline") fuzz.
Let's hope that there will be a quick transition of the in-kernel stack towards the Devicescape code, and let's hope that we'll see a full-featured HAL-less Atheros driver for it - and all my 802.11 Linux dreams will come true ;)
As you might know, the GNU GPL is currently under review and version 3 is underway. With regard to the GPLv3 process, the FSF will be holding a conference later this January to which I had the honour to be invited.
Since many people have already been wondering why I will not participate: It is not because of the conference or because of the FSF. My previous contacts with the FSF have been very forthcoming and productive, and I would very much like to share my GPL enforcement experience at the GPLv3 conference.
Unfortunately though, the conference will be held in the USA, a country to which I'm not going to travel anymore because I don't want to hand over (and leave) my biometric information (aka fingerprints) in a country that basically has non-existing data protection rights, esp. when it comes to government agencies and foreigners.
In addition to the biometrics issue, there are numerous dangers from the software patent and the DMCA front for people like me who indulge in quite a bit of reverse engineering. In the end, the US just don't sound like a place where I would feel comfortable and/or safe and/or secure in any way.
My best wishes to the GPLv3 conference, I hope they'll have a productive meeting for the future of free software.
For the third year Elisabeth and me have been planning to take holidays in India, unfortunately with no success so far.
But now it's fixed: We've booked our flight tickets for March 2006 just two days ago. We'll be starting in Bangalore, and do some travelling in Karnataka and Kerala.
So please don't expect me to get any productive work done during March this year...
Two days ago, 22C3 was closed. This years incarnation of Europe's largest hacker conference can be seen as a full success. Some 3000 attendees, about 180 lectures, a 10Gigabit Internet Uplink and our own /16.
The video recordings have turned out fine. We've had working WMV live streams, and somewhat intermittently working MPEG2 and MPEG4 live streams, as well as working OGG and MP3 audio streams of all four lecture tracks.
For archival, we have MPEG2Video (5Mbit) as well as the original DV tapes, and a FLAC audio recocrding.
Looking at the tremendous amount of work that went into the A/V recordings, and the fact that I'm involved with the A/V team since seven years, I'm actually thinking about looking for some other area where I can get involved next year.
My two lectures (on OpenEZX and librfid/libmrtd) went fine, even though they both had very little preparation ;)
In the next couple of days I'll be cutting the fourth day of the video recording, and then slowly getting back into netfilter and OpenEZX related development. Oh yes, and I'll also promise more blog updates.
For some strange reason, my git tree seems to have become corrupted over the last two weeks, so I first need to sort this out before getting any reasonable work done.
The main reason why this blog has been so quite since my return from Bangalore: I'm spending every free minute in preparations for 22C3, the annual Chaos Communication Congress. As usual, my job is to take care of the audio and video recording and streaming.
So for the last days I've been hunting numerous bugs related to this, mainly in ffmpeg, but also radeonfb, vlc, Debian ffmpeg / x264 packages, etc.
I'll be back on track after 22C3 is over. More blog updates then, I promise.