Ulogd 1.20 release

After applying lots of updates that have accumulated in the last months, I've released ulogd-1.20. Changes include dozens of fixes and a new PCAP and SQLITE3 output plugin.

This will probably the last new-feature release for 1.x, since I'm already working on 2.x with included support for flow-based (ct_acct) logging.

Porting patch-o-matic-ng to 2.6.11

Rusty's recent changes to the conntrack/nat helper API in 2.6.11-rcX have rendered all conntrack/nat helpers in pom-ng unusable.

I've created a new svn 2.6.11 pom-ng branch and started porting of all the helpers in there. The opportunity was also good to port all the 2.4.x only helpers to 2.6.x, so we won't have the big gap between 2.4.x and 2.6.x supported helpers.

I expect this to take a couple of days, and even after that, for most protocols I have no opportunity to test (proprietary protocols, proprietary software, ...), so I'll have to rely on your feedback.

Gnuradio / USRP: Software Defined Radio for everyone

As some of you may know, I've recently started to get more into electronics (again). It's been more than seven year since I finished my training as radio communications technician :)

Anyway, I wanted to do some research with regard to passive RFID sniffing, DECT (in)security and other subjects. You can build digital receivers the old-fashioned way: RF, Oscillator, Amplifier, Mixer, IF and Demodulator in hardware. This is what we all know and love ;)

However, recently so-called "software defined radios", a technology that was only available for government services and military (aka big money), are becoming cheaper and cheaper. Software defined radios take the complex IF signal and digitize it with high-speed A/D converters. All demodulation or other further processing can be done by signal processing software on the PC.

To my very big surprise, the Gnuradio project is already providing a very flexible python-scriptable software for doing such processing. Available code for demodulation is still quite limited (e.g. no FM stereo decoding, and only very preliminary NTSC b/w decoding). But well, this is just a matter of time.

What's even more interesting is the USRP (Universal Software Radio Peripheral), basically a USB2-connected FPGA-board with high-speed ADC and DAC's. It's available for less than 500EUR, so I immediately had to buy one. It hasn't yet arrived (shipping from the US), but maybe that's actually better... since experimenting with it will definitely occupy a lot of time that I don't really have :(

Some more ct_sync fixes

The latest bug (endless loop) was caused by one of my last bugfixes. Apparently I introduced an endless loop into a linked list (the nat bysource hash).

Coordination with Free Software Foundation Europe

Finally I've had the opportunity (and the time) to talk to Georg Greve of the Free Software Foundation Europe. It's good to know that they're very supportive of my GPL enforcement efforts, and it seems like we're going to coordinate our efforts at some later point this year.

This comes exactly at the right time, since I really want to get more development done and deal less with those legal issues.. believe me.

SDSL line has arrived

About a week ago the QSC SDSL line was activated. This is great news, and I just cannot describe the amount of difference it makes if you suddenly have eight times the upstream bandwidth.

Another preliminary injunction was granted

About one week ago I had to apply for another preliminary injunction. Unfortunately the respective multi-billion company (name still undisclosed for strategic reasons) refused to sign a declaration to cease and desist before the deadline for obtaining injunctive relief has passed.

The injunction was meanwhile granted, basically banning the company from shipping their product in it's current form. I'm really sad that this happened, since I expect it to harm their business. However, I really see no reason why they couldn't just sign a statement "no, we won't do it again, and we will comply with the GPL from now on".

We're still waiting for their legal staff to get back to us, let's hope they have good news next time.

New development version of grouter (aka linwrap)

Some time ago I started working on a small embedded Linux distribution. You will now ask yourself, why yet another one? Well, any free distribution you can find out there has either not a networking focus strong enough for my demands, or is using horribly outdated software (and especially no 2.6.x kernels).

So I'm now running that distro (still not sure whether I'll finally call it "gnumonks.org router (grouter)" or "Linux Wireless Router Application Platform (LinWRAP)") on three embedded production systems.

It's main features are

  • Linux 2.6.10
  • uClibc 0.9.27
  • busybox 1.00
  • iptables-1.2.11
  • dropbear
  • quagga
  • openvpn
  • iptraf
  • siproxd
  • dhcprelay
  • in-kernel PPPoE
  • fits in less than 15MB of flash

The only hardware supported so far is the PC Engines WRAP embedded x86 platform. More hardware support will be added over time, very likely candidates are IXP42x and probably even some of the Broadcom/ti/intersil consumer access point platforms.

The current state of the distribution can be followed in this svn repository. Please note that there is absolutely zero support or documentation.

Keyframe-accurate mp4 file cutting

I've done some modifications to the mp4clip tool (part of the MPEG4ip software package) to do key frame accurate cutting/clipping of mp4 files. In general it seems to work, but from time to time it corrupts the source (!) files. Need to find time for debugging.

I'll release the patch as soon I consider it to be used safely. Don't want to be responsible for corrupting someones video collection...

Conferences 2005

I'm a bit in planning mood for conferences in the first 6 months of 2005. So far I'm going to visit FOSDEM (Brussels), CLUC (Zagreb), CLT (Chemnitz), LinuxTag (Karlsruhe) and obviously OLS (Ottawa).

If you happen to be at any of those conferences and want netfilter T-Shirts, please contact me beforehand so I can make sure to bring the required sizes and quantities.

Chaosradio 99 - Telekommunikationsueberwachungsverorndung

After about four months, the first Chaosradio radio show that I was participating in. Subject of the show was the telecommunications surveillance act (TKUeV) and the corresponding technical directive. Starting from 1st January 2005, any "provider of telecommunication services" has to provide lawful interception interfaces for government and police authorities.

The big issue is that it isn't only about providers, but about anybody who runs more than 1000 mailboxes on an email server, even if it is non-for-profit.

If you're interested in the full show, you can download it from the usual location on ftp.ccc.de.

Infrequent blog updates

The regular reader of this blog will have noticed the infrequent updates since december last year. There's a relatively easy explanation: lack of time. Or even more detailed: I used to write my blog at the time I went to bed. The data of the blog only existed on my notebook, and the notebook usually is in the bedroom.

However, during the last weeks I regularly don't go to bed before 2am to 5am - a time where my fiance, bound to university day schedule, is already sleeping. This means I cannot write a blog entry from the bed - you get the point.

This is set to change now, since the blog data will be checked into my personal subversion server.

SDSL is coming

After something like three years with asymmetric connectivity (less upstream than downstream), I've finally decided to order a SDSL line again. Even though it means I'll have to afford a 200% increase of ISP charges.

Back in Nuernberg almost ten years ago, I used to have an analogue leased line which ran at mind-blowing 33.600bps. Later I used the same line type with two Pairgain SDSL modems at about 1.5MBps... this is still the line where some of my old systems like coruscant.gnumonks.org, sungate.gnumonks.org and corellia.gnumonks.org are located.

www.gpl-violations.org was down

If it wasn't for some user sending me email about the gpl-violations.org web-server being down, I wouldn't have noticed it. Apparently I made a stupid mistake while adding a new vhost to the apache2 config on that machine that went unnoticed until apache was restarted.

I'm not going into the embarrassing details here, but I would like to reveal that it was related to a new web-page called gpl-devices.org which I am about to launch. Let's see whether I can turn my ideas about it into reality, or if I never find the time, like with other interesting projects :(

Anyway, I'd like to apologize for the downtime. If someone had sent me an email earlier... *sigh*.

Frame Accurate Cutting of MPEG2/MPEG4/OGG

Since I now have the job of cutting (cropping/clipping) the A/V recordings of the more than 200 presentations of 21C3, I've been looking for a number of days for available free software to do GOP / key frame accurate cutting of MPEG2, mp4 and OGG/Vorbis files.

As for OGG/Vorbis, the vorbis-tools package contains a program called vcut, which basically does almost the full job. However, it's a bit clumsy to use, since it always splits a original file into two halves, before and after the cut position. I've modified it a bit in order to accommodate my needs better.

As for combined audio+video containers such as MP4, it becomes a bit more difficult, since you need to find key frames for both audio and video as close as possible to the user-specified cut point.

However, after learning a bit about Apple Quicktime and the MP4 container, plus the help of libmp4v2 from the MPEG4IP package, I was able to create a small tool for key-frame accurate cutting, too.

For MPEG2, there is lve (Linux Video Editor). This program even provides a graphical user interface for navigation through the video, creating clips and a cut&paste interface. Unfortunately the UI is not intrusive in any way, and it even seems to use it's own toolkit. After playing with it for more than 45 minutes, I wasn't able to actually cut a single video using it :(

Since MPEG2 is not a priority at the moment (we need to make .ogg and .mp4 available for download ASAP), I deferred this problem for now.

Maybe at some point I'll find the time to put together all the pieces and create some generic media cutting/clipping/cropping tool for any kind of format. However, judging from the differences of the media formats, there wouldn't be much more common code than parsing the command-line options ;)

Number of GPL violations still rising

Over the last couple of days I've again verified a number of GPL violations. It's a real pity that those companies still don't get the message.o

It hurts especially, that there are two cases (Netgear, Siemens) where companies with whom we already had a amicable agreement published new devices that again don't comply with the GPL (Netgear WGT634U and Siemens M740-AV). Apparently they don't really care despite the fact they should know better.

Also, we have another number of cases where companies signed an agreement with us, but failed to fulfill that agreement only a couple of months later with exactly the devices mentioned in the agreement.

I'm sick of those cases. What the hell is so difficult to put the source code and the GPL license text on a CD-ROM that has 500MB unused and ships with the device anyway?

Preparing the 21st Chaos Communication Congress

As every year, the Chaos Communication Congress takes place in Berlin, Germany.

For six years, I'm part of the team that takes care of audio and video recording and streaming. Since this year I've become head of the a/v documentation project, I decided to use a 100% Linux based solution instead of the Apple Quicktime stuff that we've had for the last couple of years.

Thanks to the great ffmpeg software, we can even encode four different streams on a off-the-shelf Pentium IV.

Today, I've been with the technicians at the congress center who set up the PA and lighting. This was to make sure everything really reflects our demands, and we have the correct audio signal delivered to the appropriate place, etc.

Setup of the congress will continue over the holidays. Especially the NOC (Network Operations Centre) will have a hard time setting up the internal network for about 3000 attendees, certainly each bringing more than one networked device on average.

ffmpeg is undocumented, ffserver broken

I've been experimenting a lot with ffmpeg and ffserver over the last couple of days. The fact that ffmpeg is very little documented is a pity, but not exactly a problem for someone experienced with free software and C development (use the source, Luke).

However, the ffserver program seems to be horribly broken in a number of ways. Independent of the kind of configuration, it regularly segfaults, glibc complains about double-free's, and valgrind or Electric Fence have numerous complaints.

All information you can find after browsing through mail archives, is that it's apparently broken for a number of years. Maybe I'll spend some time at it and fix it at least partially. So I spent about two days to familiarize myself with the source of libavformat, libavcodec, ffmpeg and ffserver. It's not exactly easy to understand, but I think I now got a good understanding of what's going on where.

Another fundamental insufficiency of ffmpeg seems to be that it cannot put the output of one codec into multiple output files. So let's say I want to encode some MPEG2 video and AC3 audio. This is to be written to a .vob file and at the same time sent as a transport stream over the network. The only way you can achieve this now is to encode the input data twice - which I cannot afford due to CPU limitation.

So I was pondering something like streaming the output over multicast RTP plus running something like rtpdump on the same machine to create the local file.

As a summary, I think it's a pity that there is good encoding software like ffmpeg, and that nobody volunteered yet to fix the remaining issues required to turn it into a good streaming and recording solution.