OpenEZX.org project launched

Today I've started a small preliminary homepage about my A780/E680 hacking efforts at openezx.org. This also means that the old a780-hackers@lists.gnumonks.org list was renamed to openezx-devel@lists.gnumonks.org.

Expect no big news for some time, since I'm mostly working on porting/merging all EZX specific stuff into a 2.6.14-rc4 kernel.. a quite big job that will certainly take some time.

Stay tuned.

Adding Mifare Ultralight support to librfid

Since (as opposed to MiFARE Classic) the Philips proprietary MiFARE Ultralight RFID Transponder is actually documented quite well, I've added support for it to librfid. In theory it should work (I've implemented it just like the data sheet says), but unfortunately the transponder doesn't reply to READ/WRITE commands yet :(

The reason for implementing MiFARE ultralight is mainly to have a closer look at the Champions League Tickets from last year, since they are the "beta test" for the Soccer World Championship here in Germany next year.

Restructuring the netfilter.org project homepage

Some years ago, the netfilter project only had the kernel side netfilter/iptables code, and the userspace iptables program. Then we added patch-o-matic(-ng), and more recently there were a number of more sub-projects growing, like ipset, all the nfnetlink-related code, ctnetlink, etc.

Unfortunately the homepage design didn't really cope with the fact that there is now a more hierarchical structure with many sub-projects.

It was always my hope that some "new webmaster" would take care of it. Unfortunately we still don't have a webmaster, so I spent some time on it today. You can see the results at www.netfilter.org.

E680 has arrived

I've managed to obtain a 2nd hand E680 phone, which is based on the same Motorola EZX platform as the A780. The E680 are only sold in Asia, so the device I now have is actually a Chinese model.

Next on the plan for A780/E680 hacking is playing with the JTAG port, and trying to flash a non-OEM non-branded non-chinese firmware into the E680.

Once JTAG is running, I will be trying to port the drivers to a 2.6.14-ish kernel and compile and install that more recent kernel.

A780 batteries/charger dead?

I'm unable to recharge any of my two A780 batteries, at least not via USB. Since I'm travelling, I cannot try with the real power-supply charger. Let's hope I can somehow resolve this, and it isn't really some damage to the phone's built-in charging controller :(

On the A780 hacking front, I've now successfully confirmed that there are indeed JTAG pads on the PCB, both for the PXA270 and for the ARM7TDMI, which is great news.

I also think there is still hope that the USB device port could actually be used as a host port. At least the PXA270 supports various options for OTG. Now the big question is only whether this is compatible with Motorola's overloading of the USB (called Enhanced Mini USB).

net-2.6.15 tree has opened

Since DaveM is on holidays, Acme is now in charge of running the net-2.6.15 tree. I've already submitted nf_conntrack, the ip_conntrack hash table resizing code from Rusty, as well as "revisions" support for {arp,ip6}_tables.

I'm also basically finished with x_tables now. Everything has been merged with a post-nf_conntrack tree, and all the conntrack related matches/targets have been ported to x_tables.

Now I need to do some serious testing (including nfsim), before it can be submitted, too.

Hanging out at 0sec in Bern

0sec 1.0 (the first incarnation of a security conference / hacker meet-up in Berne, Switzerland) has concluded today. Despite spending an enormous amount of time writing new netfilter and librfid code, I've had some interesting discussions and met a number of interesting people.

What I found especially interesting is all the work on syscall proxying that Uberwall are doing. I need to look into that stuff in more detail.

Linux Kongress

After my delayed trip back from Seville, I'm now in Hamburg for Linux Kongress. This turns out to be an extremely busy event, I have two 'regular' presentations, one full-day tutorial, and also have to host a number of sessions as "session chair" on behalf of the organization committee.

This means that there is practically no progress in either the usbdevio fix nor in the current x_tables work. However, I found some time to fix a couple of 14443B related problems in librfid.

Somehow I have the feeling that Linux Kongress has lost some of it's spirit over the last couple of years, which is sad. Especially sad, since the first Linux Kongress 12 years ago was the first time that Linux Kernel hackers have ever met.

Tomorrow I'll be leaving for 0sec in Bern/Switzerland, which I'm looking forward to.

Stuck in Seville

Iberia decided to reschedule my flight without informing me, even though that change was executed more than one month ago. They claim to have informed my travel agent. Not surprisingly, my travel agent claims never to have received such information.

This means that I'm stuck for one more day in Seville, since the next flight is only leaving at 7am tomorrow morning. Since Iberia claims it was not their fault, they're also not willing to cover any accommodation expenses.

Pablo Neira was friendly enough to invite me to stay at his place for the extra night, which means I don't have to fight with Iberia and the travel agent for any expenses.

Unfortunately I was scheduled to travel to Hamburg tomorrow, so I have to alter my train reservation and somehow make sure I'll still be in Hamburg at Linux Kongress for my tutorial.

I'm starting to get sick of those travel irregularities. This means I'm again back to my (old) plan of cutting down the number of conferences next year.

More netfilter work at workshop coding day 1

After having terminated the traditional workshop part, we've today had day 1 of the workshop.netfilter.org hacking sessions.

Despite the different topic, I spent the better part of the day with Michael Bellion and Henrik Nordstrom working out the details of nf-hipac / nfnetlink integration.

Apart from that, there's now a nf_conntrack header cleanup in my git tree, I've ported ebt_[u]log to nf[netlink]_log, fixed some minor Kconfig issues, merged some patches from Yasuyuki and Pablo, and pushed forward a round of fixes and updates to DaveM.

Second day of netfilter workshop

If I would start to write about everything that we discussed or only about the results from the discussions and presentations, I would probably need all night to write this blog entry.

It's been a very productive two days, and I'm looking forward to the hacking session that will happen on the next two days. Some of the TODO items for the hacking session will be:

  • nfnetlink-enabling nf-hipac
  • resolving some header file issues for 2.6.14 / nfnetlink
  • using Gandalf's hashtrie as conntrack hash
  • nfnetlink-enabling ipset
  • using string search api for pattern matching in conntrack helpers
  • completing userspace conntrack helpers using nfnetlink_{queue,conntrack}

Ok, have to stop for now, too much exciting stuff keeping me busy here :(

Heading off to workshop.netfilter.org

Tomorrow morning at 8am, I'll be leaving for workshop.netfilter.org, the annual netfilter developer workshop.

For the first year, we actually have presentations that are intended for sysadmins (aka 'users'). I'm missing the first day of this user event, but am obviously present for the two day workshop/discussions and the two days of hacking following up the official workshop.

I want to publicly thank Pablo Neira for organizing this years event. We've now had workshops every year since 2002. They've been very low-profile and small so far. But look at this year's event. It actually has a homepage that's worth mentioning, and the sponsors seem to be literally lining up..

Looking forward to meet lots of fellow hackers, especially those whom I haven't met since last years workshop.

ulogd2 is working

I've managed to bring ulogd2 to a state where it finally does something. The dynamic key resolval/linking of plugin stacks is working, and some basic plugins (NFLOG input, IPV4 packet interpreter (BASE), LOGEMU output) are working, too.

So the remaining work will mostly be in the plugin area. We're currently missing

  • ctnetlink input
  • packet->flow aggregation (basically 'nacctd')
  • IPFIX input and output
  • convert the old mysql/pgsql/sqlite output plugins

If you're interested, patches are always welcome. The code can be downloaded via svn from http://svn.gnumonks.org/branches/ulog/ulogd2/.

ulogd2 about to hit alpha state

Yet another of my projects that never received the amount of attention that was required is ulogd2. If you already know the ulogd-1.x series, then you know it as an efficient packet filter policy violation logging daemon, with backends for files, syslog and various SQL databases.

ulogd2 is much more than that. It's more abstract, and more universal. It's no longer limited to receiving packets from the ULOG target, but is fully modularized, with modules for ULOG, NFLOG (see linux-2.6.14), IPFIX, ctnetlink, ... Now you might wonder why there is something like IPFIX and ctnetlink? That's because ulogd2 can also process (aggregate, export) per-flow information.

The most difficult part of the implementation is the dynamic creation of "plugin stacks", but I think I wrote about this earlier in my blog.

The good news is, that just before I went to bed, ulogd2 compiled for the first time ;) This means I've waded through the tons of errors and warnings created by all the changes introduced since it forked off ulogd-1.x about a year ago.

Now there are some bits of missing functionality here and there, and certainly a large bunch of bugs. But if you are a software developer, you know it's much easier (and rewarding) once the beast actually runs :)

More A780 hacking

Today was a very exciting day of more A780 hacking. You know, from time to time it's quite good to do something else than stupid netfilter development or the like ;)

So what I've been able to do? Well, I analyzed most of the device drivers from userspace side. I now know the key-codes of every keypad or other button/wheel/dial on the device, I know the touch screen and framebuffer. I can control the three different backlights.

Then I've learned a bit more about the architecture of the phone. The Xscale processor (PXA270 Bulverde) actually uses USB to talk to the Neptune chip. Neptune is a DSP with a synthesized ARM7TDMI on-chip. The PXA270 runs in host mode, the Neptune in device mode.

Interestingly, the Motorola developers have debugging callbacks in the stock kernel. So by registering a simple kernel module with the USB rx/tx functions, I now have hexdumps of the USB traffic between those two chips (also called AP and BP).

Then I called the a780, and I immediately received some nice hexdumps in the kernel ring buffer. The first thing I could spot was "IP: "+4930xxxxxxxx",1\r\n". There it was, the incoming phone number :)

Some other nice guy at motorolafans.com has managed to replace the proprietary userspace Bluetooth code with the stock Linux BlueZ codebase. He's working on Bluetooth keyboard support... that would really be nice. Using a Bluetooth keyboard with the Qonsole terminal emulator (or even a framebuffer console) of your phone :)

I'm really confident that the APBP protocol can be worked out fairly quickly. Once this is done, we can start developing our own "phone" programs, and get rid of all the bloated embeddedQT and Java crap that is running on the phone. It has 48MB of physical ram, and the database daemon has a resident size of 2.7MB, the address book 4.5MB, the "phone" program has 6.6MB. This is really ridiculous...

At the end of the road, I'm dreaming of something small and efficient, running uClibc, busybox, DirectFB, ...

The USB device port of the device is called "Extended Mini USB (EMU)", because it apparently can be switched in more than half a dozen of different modes (by assigning various pull-up/pull-down resistors). Apart from a USB device, it can for example run a UART on that port. However, since the USB host port is already used for BulverdeNeptune communication, I don't think it is possible to run the phone in USB host mode. This basically rules out attaching a stock 802.11 wifi USB adapter, which is very sad.

Bringing ftp.gpl-devices.org live

ftp.gpl-devices.org has been up and running for a number of months now. As usual, I never really had the time to take care of it (i.e. feed it with all the vendor-released and 3rd party source code for embedded devices running GPL licensed software).

Luckily, Imre Kaloz was interested in helping me out. He's now in charge of at least putting all the TI AR7 related source tar-balls on the ftp site.

I've already dedicated a 300GB hard disk for the source code, which should be fairly sufficient for some time. At this point, I have no more than 40GB of vendor-supplied source code images at home.. ftp.gpl-devices.org has only some 3GB as of now.

Thanks go to noris.net, the innternet provider where like for almost all of my projects, the server ftp.gpl-devices.org is colocated.

Running netfilter/iptables on your cellphone

Yes, you're reading this right. I've managed to build iptables.o, ipt_*.o, iptable_filter.o, iptable_nat.o, ip_conntrack.o and the like for my Motorola A780 cellphone.

As of now, there's not really all that much need for it... but when I start running dozens of applications on the device, I better make sure to have a decent packet filter to the GPRS/HSCSD world.

But even then, in theory it should now be possible to NAT between the GPRS device one one side, and the usb-lan on the other side. Maybe I should try to bring my whole home network online via the A780 :)

OTOTH this doesn't fix the various security issues on the horizon. The A780 apparently ships zlib-1.1.3. I don't even know how many security vulnerabilities were fixed since then...

More fun with the Motorola A780

I've now successfully built a compatible toolchain for the Motorola A780, thanks to this good site with instructions.

Obviously, one of the first things to do was to build busybox with a config that enables all the missing tools. For some strange reason, the A780 does not ship with the usual uClibc/busybox combination, but with the straight GNU tools (glibc, fileutils, ...). Unfortunately important bits such as less, top, strace, etc. were missing.

I've also managed to build matching ext2,jbd,ext3,sunrpc,nfsd and af_packet kernel modules. The VFAT partition on the TransFlash card was shrunk, and an ext3 partition added. Some hooks into the startup scripts, and now the ext3 is mounted when the phone is switched on. Some PATH and LD_LIBRARY_PATH mangling in .profile, and I have a very workable environment on the phone.

Obviously the most important goal would be to port the EZX arm architecture support into a recent 2.6.x kernel, and then run a full-fledged 2.6.x kernel on the device. With embedded IPsec, packet filtering, etc. That goal is very far, due to stupid proprietary device drivers.

So for now, I'll be looking into the kernel/userspace API's and the userspace/userspace API's in order to develop native userspace applications that can actually use the phone (i.e. make voice/data calls, use the headset/speaker/microphone, ...

Chaosradio on ePassport and Biometrics

Due to the importance of the subject, we will do the second Chaosradio show this year dedicated to electronic passports and biometric identification.

Germany will issue them starting with November this year... so now is about the last possible time to apply for a brand new, shiny, glossy, cheap "old-style" passport that doesn't contain any biometric information.

netfilter developer blogs

I first wrote about this in early 2005: Having developer blogs on people.netfilter.org. Unfortunately I never finished that project so far. I'm not really a web guy at all, so doing stuff related to (X)HTML and CSS always gives me the creeps. Why can't we just have a technically skilled web master volunteer for netfilter.org? *sigh*

For those who're curious, you check out a mirror of this blog, or the early beginning of Gandalf's blog.

Every netfilter developer with an account on people.netfilter.org can easily set up a blog, just by putting blog articles into ~/weblog/.

Work on ulogd2

I've continued work on ulogd2, the next generation netfilter userspace logging daemon. In addition to packet-based logging, it supports flow-based logging.

It turns out my overly-flexible concept of plugin stacks ends up with quite some implementation complexity. The problem can be viewed similar to a linker problem (linking symbols of multiple objects), but in addition resolving dynamically changing dependencies, with some 'symbols' being optional, and with objects that you can ask "if I give you input symbol X, which output symbols can you give me" ?

I really need to do resolve some tax issues before the netfilter workshop, so I'm not sure whether I can finish it before.. especially since I've also started to merge years-old pkttables code into a recent kernel.

Planet FOSS.in has opened

The organizers of FOSS.in have put together a planet site at planet.foss.in, featuring the weblogs of all speakers. Incidentally that includes this blog ;)

If you have trouble resolving the foss.in domain, that's probably due to broken nameserver responses from their current domain hosting provider. At least my bind9 cannot parse their responses... I've now set up a set of 'real' name servers, and Atul is trying to get the whois data updated... sorry for any inconvenience.

released libnfnetlink, libnfnetlink_conntrack and conntrack

This triple-release is in anticipation of a 2.6.14 kernel release. The two libs as well as the conntrack program are userspace counterparts to the "next generation" subsystems inside the kernel netfilter part.

The release involved lots of painful learning-by-doing of autoconf/automake. I'm not a fan of them at all, but I sill think it's less burden than trying to invent everything on your own (like we did with the iptables package) and thus forcing more burden onto the package maintainers of the distributions.

I'll probably release libnfnetlink_log and libnfnetlink_queue tomorrow... but I really don't have any time to work on netfilter at the moment, despite this TODO list :(.

Some bits of ath-driver hacking

This morning I wanted to do something relaxing, so I looked at the ath-driver source code that I'm no hosting for Mateusz at ath-driver.org.

After some hours of digging (and trying to implement channel switching support), I decided that the whole approach of yet-another-driver seems deemed.

If I find some time for Atheros driver hacking, I'll build a Linux driver around the ar5k OpenBSD driver (yes, it will be dual BS/gpl licensed). It's just not worth the pain of re-implementing the HAL functionality for 5210, 5211 and 5212 from scratch...