Back to Berlin

After almost one month of travels (Berlin->Paris->Bangalore->Delhi->Taipei->Delhi->Bangalore->Mumbai->Bangalore->Paris->Berlin), I'm finally back to Berlin. It always feels good to be home, and in fact I'm probably home for more than three consecutive weeks, something that doesn't happen very often.

It has been an exciting time, and I've made quite a bit of progress on both the GSM scanning side as well as on the gnufiish (Linux for E-TEN glofiish) side. Still, lots of work remains, and it's a challenge to see how much time I can spend on it.

During the next couple of weeks I'll be working on VIA related stuff. Most of it is behind-the-scenes work, but I'm also actually going to work on some actual code again. What a relief ;)

Obviously, there is also still a lot to be done on the GSM base station + GSM network front. The interested hacker might already have figured out that I'll be co-presenting with Dieter Spaar on how to run your own GSM network at the 25C3, but I'm mentioning at this blog anyway for the sake of completeness. The code will be released at the 25C3, and we'll hopefully also have some fun at the GSM hackers desk in the hack center.

With some luck, I'll be heading for Taiwan and (for the first time) Korea in January 2009. The other news about 2009 is that I'll likely spend more time than before in India.

Availability of Bollywood DVDs in Bangalore

I've been visiting Bangalore many times throughout the last five years. Every time I've spent a visit to "Planet M" on Brigade Road for buying some Bollywood DVDs.

And for some unknown reason, the size of the racks with Bollywood DVDs is shrinking from year to year. And with it, obviously the choice...

What you can get are sort of the last five major blockbusters, and tons of cheap re-releases of old movies from the seventies through nineties. But what about the last five years? What about anything that was released recently but is no longer part of the top-10? _nothing_!

Oh, and then you can get tons of Bollywood VCDs. But who wants that low quality? It's definitely no pleasure to watch a VCD. Even DVDs have way too little resolution to capture e.g. the details of costumes in a lot-of-people-dancing kind of scene.

It's really sad. Are people no longer buying those DVDs? Do Bollywood fans go to different places? Where should I go in Bangalore for a decent selection? Hints welcome, please send e-mail.

As a side note, yesterday I've also been at Planet M in the Esteem Mall. It's sort of a joke. Never seen a CD/DVD store that small. Obviously no choice at all.

Understanding the GSM Um Layer 1

More or less involuntarily I ended up spending the better part of the last couple of days understanding all the nasty details of the details about the Layer 1 of the GSM Um (MS-BTS) interface, i.e. the RF interface between your mobile phone and the BTS.

You can find a lot of secondary information about GSM both online and offline, but they all seem to address other issues, like layer 2 and layer3 protocols, even the actual logical channel multiplex (at least to some extent), network planning, etc. - But I've not found anything reasonably detailed about the actual channel coding, TDMA and synchronization, ie. what happens after demodulation and before layer 2. So I had to read the specs. Oh well...

I actually thought that all those parts are already there (in gssm and gsm-tvoid), but in reality that was just something like a proof-of-concept implementation. Restricted to only work on Timeslot 0, not demuxing the individual parts of the CCCH (PCH/AGCH/BCCH/..) and not properly dealing with frame-number counting in case of various receiver overflows or the like.

So I've now started on a new codebase specifically focussed on everything that happens between the differentially decoded GSM bursts and layer 2.

Normally, one would not assume a layer1 to be particularly complex. However, in the case of GSM, it really is. There are various different physical channel configurations, each with their own nasty little differences in the channel coding schemes... plus the synchronous nature with its necessity to know a lot of context and correctly count frames in order to correctly interpret and decode the incoming bits.

So where do we start. Most readers of this blog will know that GSM has 8 timeslots on each RF channel. Each timeslot forms one so-called physical channel, which can assume one of many physical channel configurations.

A physical channel configuration determines not only the particular logical channels residing in the physical channel, but also low-level aspects such as parity, convolutional code, training sequence, tail bits, etc.

In every timeslot on a physical channel we transmit something called a burst. Depending on the physical channel configuration, there can be different bursts:

    • Normal: A regular burst transmitting 142 bits of data
      Frequency Correction: A burst that helps us finding the offset between sender and receiver clock rate, compensate for temperature drift, etc.
      Synchronization: A burst for achieving TDMA-frame-number synchronization
      Access: A specially (short) burst used in the uplink (MS->BTS) direction to request a channel allocation by the BTS.
      Dummy: A burst that is transmitted in otherwise unused/empty timeslots, only on the ARFCN (radio frequency channel) that carries the CCCH/BCCH
  • All 8 successive timeslots form what is called a TDMA Frame. Every time the timeslot number changes from 7 to 0, the TDMA Frame Number is incremented by the receiver.

    For all control channels, four such bursts need to be concatenated and go through convolutional decode and parity to form a 23-byte MAC block. On most, but not all control channels, those four bursts are sent in successive TDMA frames on one timeslot. The notable exception is the SACCH (Slow Associated Control Channel) which accommodates every TCH/F (Traffic Channel) and happens to get one burst at the 13th, 101st, 114th, 202nd, ... TDMA frame. Thus, on such TCH/F channels you need to wait for 202 TDMA frames to obtain the four bursts needed to generate one MAC block. A MAC block is what is then passed up to the layer 2.

    Timeslot 0 of the primary radio channel of the BTS is called CCCH (Common Control Channel). It is the only channel about which we can at least make some assumptions about its physical configuration. However, the CCCH carries a number of different channels, such as the BCCH (Broadcast Control Channel), PCH (Paging Channel), RACH (Random Access Channel), SCH (Synchronization Channel), FCCH (Frequency Correction Channel) and possibly a NCH and maybe even a SDCCH/8 multiplexed into it. The "problem" here is that the actual configuration of the logical channels is determined by a layer2 System Information Type 3 message. So you have to work with the minimal guarantees (that there will be a FCCH burst, followed by a SCH burst, followed by four BCCH bursts which give one MAC block, in which you will at some point find the SI Type 3 message telling you how all the other bursts of the CCCH are mapped onto the various other logical channels.

    From the System Information messages you can also learn on which timeslot the CBCH (Cell Broadcast Channel) is located. Typically, it is part of a SDCCH/8+SACCH/8C physical configuration. On all the BTS's I've seen protocol traces of (which are not yet that many), this is on Timeslot 1.

    A SDCCH/8+SACCH/8C configuration is another weird but more logical multiplex. It consists out of 32 consecutive bursts, four for each of the 8 SDCCH instances that it carries. The following 16 bursts are divided to four, and they alternately serve SACCH0..3 or SACCH4..7. So you basically end up with two 51 multiframes , i.e. 102 bursts, out of which 8 bursts go to each of the SDCCH/8 instances, and four go to each SACCH/8C instance. (102-06 = 6, i.e. 3 bursts are dummy in each 51 multiframe.

    For the TCH/F (full-rate traffic channel), the entire viterbi decode and parity is different than the control channels... and which is probably going to be part of another blog post.

    Oh, in case you're interested: you can find the unfinished demultiplex code snippets at this git tree

    Luckily, with a few days more work, we can actually properly demultiplex and decode all incoming frames, learn about the [dynamic] channel assignments and configurations that the BTS does, and reconstruct at least the MAC blocks for all control channels, if not even the actual speech codec data of the TCH's (on those unencrpted BTS that exist in India). However, actually following a conversation is not possible (and not of interest to me anyway) due to the frequency hopping.

    FOSS.out 2008

    FOSS.in 2008 is over. The grand finale was Kalyan Varma's closing keynote on how he thinks fundamental FOSS principles are present in all aspects of his work and life, even in areas completely unrelated to FOSS such as photography and wildlife conversation.

    I could not agree more to what he said. Fundamentally it is about being curious, learning how things work, cooperating with other people with mutual benefit, etc. - as I have to some extent outlined in one of my previous blog posts about reverse engineering.

    One particular spin was also on Security. Having an IT security background like him, with a pretty similar FOSS culture background, I can perfectly understand and support his point of view.

    As for FOSS.in, I think it could also not have been much better. Biggest constraints were probably the conference venue itself. Its lecture halls inevitably create a big divide between a small number of entertainers on the stage and a big auditorium. There also was a constant and severe lack of power outlets at any given time or place - to some extent again a problem with the venue.

    Thanks to the Organizers, Team FOSS.in and the Volunteers. Well Done.

    Update from FOSS.in

    First of all, many people have asked for the slides of my presentations. You can get the keynote slides and the glofiish reverse engineering slides from the FOSS.in website now.

    Giving the latter talk, I was really surprised that nobody in the audience raised their hands when I was asking who had ever done reverse engineering of any sort. I cannot really imagine any of my work, both in the FOSS community as well as professionally without using whatever means to discover how things (devices, drivers, software) work. Isn't it a most natural human trait? You discover something new, and you want to learn how it works. So you take it apart, learn about its components and understand how the individual parts play together.

    I've been doing this with about everything I ever got, even as a kid. Stereo system, reel-to-reel tape recorder, my first 286 based PC, my first motorbike, car, etc. It's simply not acceptable to be in possession of some technical device without understanding how _exactly_ it works.

    So anyway, I hope the talk was at least a bit inspirational and makes some people try. It is not so much important that you actually fully manage to reach the goal (like in my case getting a full-blown Linux implementation of all the drivers done, etc.). The importance is the process, and what you learn while doing it.

    So today I was mostly busy preparing and holding that talk, later at night I was back to working on gsm-tvoid. I'll cover this in a separate post.

    The first two days of FOSS.in over

    I've been having the pleasure of holding the opening keynote at FOSS.in, where I've been (again) using the opportunity to point out the sad situation of Linux in the Embedded space. I think it was good to get this message not only to the CE Linux Conference Europe attendees, but also to the various FOSS interested Indian developers. Many of those work for companies involved in chipsets for Embedded devices, Embedded Systems development or even BSP development.

    Despite that very sad/depressing conference opener, the feedback was overall very positive. Some people mentioned "it was like if you were talking to me personally". So let's see if this kind of grass-roots FOSS lobbying can help to make a bit of a change in those Embedded Linux companies.

    After the keynote I was more or less immediately starting my WorkOut on improving Free Software based GSM protocol analysis. Basically we're looking at GSM-tvoid, gssm, gsmdecode, the wireshark patch of gssm, etc. and coming up with a much improved solution.

    So far we've added the tun-device support from gssm to gsm-tvoid, but that's only a kludge and a temporary solution. Adding fake Ethernet headers to a GSM Um frame and using a non-registered Ethernet protocol type is not really the kind of "implementation quality" that I'd like to see.

    So now I've come up with a 'gsmtap header' similar to what 802.11 solves by the radiotap header. gsm frames including radiotap header can be stored directly as a new linktype in PCAP files, or they can be sent via UDP packets through the regular IP and networking stack, where wireshark can just grab them using the normal network devices.

    We've continued to work on those issues on the second day of FOSS.in, and we'll also continue to work on it today. Tomorrow I'll be presenting on my gnufiish project, i.e. the reverse engineering and Linux port plus driver development for the E-TEN glofiish X800/M800 devices.

    I personally can't really say yet how well the concept of WorkOuts has worked-out in practise. I really need to learn more about the progress that the other workouts have been making. I think at least for the GSM workout, there were not many people who had the skills or knowledge about the protocols and/or the tools involved - which is not a big surprise. But I'd hope that some of the attendees at least got interested in the subject and will contribute to the respective projects. There are many things to be done, including the somewhat tedious exercise of adding dozens and dozens of new dissector code to wireshark. If anyone else (preferably with some generic understanding about network protocols and wireshark dissectors) wants to help with that, please contact me.

    From FreedomHEC Taipei to FOSS.in / Linux and the Taiwanese Hardware Industry

    I'm on my way from Taipei to Bangalore, from FreedomHEC Taipei to FOSS.in. Two very different events in two very different countries with a quite different IT industry.

    I was really happy about FreedomHEC. It is really about time that the Linux world and the Taiwan-based chipset vendors and system integrators start much more interaction. It is a simple economic fact that A lot of hardware development, both in the PC mainboard, Laptop as well as the embedded device space happens in Taiwan. It is also very true, that for whatever reason the gradual Linux revolution in the server and desktop market in the EU, the US and other markets such as Southern America has not really reached Taiwan. At least from all the various contacts that I've made in Taiwan, there are almost no Linux users, and particularly not in a corporate environment.

    My experience in Germany shows that many small and medium sized companies, as well as a noteworthy part of public administration is using Linux, at least on the server side, and to an increasing amount on the desktop side. Many end users have dual-booting machines. Plus, the universities and particularly the computer science departments have a long UNIX-related tradition - and most SunOS and Solaris workstations have since been replaced by Linux based systems, or at least systems with dual-boot configurations.

    If my completely non-representative assessment of the Situation in Taiwan is true, then we just don't see this level of adoption there. And this has a quite big impact:

    • Managers and Engineers underestimate the amount of Linux adoptions in their target markets, since they don't see that much adoption in their domestic market
    • Even if there is a [customer] demand for Linux, the Taiwanese hardware industry has a hard time to properly respond to this demand due to the lack of know-how about Linux and FOSS - both technology-wise, but also regarding the development model
    • There are very few system administrators or software developers with a profound Linux user experience. How are you supposed to administrate or develop for a system that you haven't at least used for a couple of years?

    So as a result of this, I argue that Linux hardware support world wide suffers from the lack of recognition of Linux in Taiwan.

    This needs to change. Recent developments like the Asus eeePC or Linux-based Netbooks in general are not a solution either. They don't mean that Asus suddenly cares about how well e.g. the Linux ACPI implementation interacts with the ACPI BIOS of their non-eeePC Laptops.

    I think any system integrator who understands those facts will likely gain a lot of trust and customer satisfaction. We yet have to see _any_ laptop or mainboard manufacturer who goes public and says "we will test our systems with Linux like we test them with Windows".

    Non-Taiwanese system integrators like Dell or HP have a competitive advantage here. They do understand much better what Linux is, and how to work with it - even though mostly still on the Server side. You will find Linux-based BIOS update tools. You will see ACPI BIOSes that actually work properly and don't just contain random bytes in those parts that Windows doesn't currently use.

    Why not Acer? Why not Asus? Why not MSI? Why not Foxconn? How much of a R&D investment is it really to do even the most minimal testing like booting some Linux Live Distribution from CD and checking if the major features are working? I would assume in the total Laptop R&D cost, it's less than 1%. So if only 1% of the customers will install Linux, it should already be justified.

    Especially right now, nobody has really made the first step. Anyone who starts with the right strategy can be the first one on the market. The opportunity is there. Don't wait until the competition uses it.

    VIA and Openchrome; Graphics Programming Manuals

    Coinciding with FreedomHEC in Taipei, VIA has announced its cooperation with OpenChrome and releases Graphics Programming Manuals.

    This definitely marks a big milestone in VIA's new, much more FOSS friendly Linux support. Not only releasing the source code to VIA's own graphics driver, but actually interoperating with OpenChrome to help to create one future driver base and fight against the fragmentation of the developer and user base.

    After all, there's probably no other family of GPU's where there are so many different Linux/Xorg drivers like VIA's. What a terrible waste of R&D resources to reinvent the wheel over and over again. One reason for doing that (VIA's driver being closed source) has disappeared when it was made open source a couple of months ago.

    So let's hope that this cooperation will be as successful as possible, and we can have one unified driver codebase with the cumulative features of both individual drivers right now. Once that has been done, we can start to think about helping the result to get into mainline X.org and put the entire history to rest.

    I also appreciate and welcome the release of the graphics programming manuals for the two most recent generations of integrated graphics chips. Sure, they are by no means as exhaustive as documentation of major competitors in the GPU market - but then, VIA is a small company and they cannot release documentation which never even existed in the first place. So please accept that VIA is working on releasing the documentation it has, but is unlikely to be able to work on creating additional documentation that doesn't even exist.

    There are still some things to be done, though. We still cannot include MPEG/H.263/H.264 hardware acceleration support in the driver due to unresolved legal issues (working on that, don't worry) and there is still no open source 3D support for the Chrome9 core (VX800 chipset). But then, life would be simple if all of those problems would disappear overnight. In any case, I think VIA can now legitimately claim that it is moving in the right direction, that it is not only trying to become a much better 'Free and Open Source Citizen'.

    There will be more manuals and code up for release at some point, but please excuse that I simply don't want to speak about the tentative schedule of things that haven't happened yet.

    E-TEN glofiish M800 Linux tree now has public git tree and a name

    You can find the git tree here and clone it from git://git.gnufiish.org/gnufiish.git. Thanks to Stefan for setting up the tree and doing the initial push (ssh+git from Taipei is so slow that it cannot finish the initial commit of a kernel Tree before the DSL modem reconnects 12 hours later).

    I've called the project "gnufiish" just for the fun of it. It's quite close to the original name, and therefore a 'language hack'. Though it's obvious that there is no real official connection to the GNU project, since Linux is not part of that.

    Digging into the internals of WinCE / WinMobile

    My E-TEN glofiish Linux porting efforts have made me investigate a lot of internals of the E-TEN ROM file format, WinMobile ROM files in general, XIP, Microsoft flash partition tables, imagefs and other bits and pieces.

    I'm basically able to fully 'decompose' a ROM image into all its individual bits, including the pre-installed CAB's, the pre-linked DLLs in the XIP and the contents of the imagefs. And all of that on Linux, if it wasn't for the weird XPRS / SRPX compression in CE5.x imagefs. Allegedly it's the same Xpress algorithm as used e.g. in hibernation files and certain M$ network protocols, but I was trying that and didn't get anywhere. Luckily, the tools at least ran inside wine.

    It's surprising how little information there is about those internals of the operating system. You can find bits and pieces in the 'ROM cooking' scene, but those are mostly HTC specific and don't always apply to E-TEN. And most of the tools that people tend to create in this community are not FOSS either :(

    Anyway, once I find some time I'll probably pack/publish the stuff that I've done now. Obviously the coolest thing would be to do a GPL'd implementation of e.g. imagefs and get that into Linux. Would be fun (I've never ventured into filesystem land!), but then, it's not like I have any spare time at hands.

    Last night I was trying to make sense of some of the M800 hardware drivers (sergsm.dll, keypad.dll, keybddr.dll, etc.) but it's actually quite a bit harder than I thought.

    I also wrote some perl script that uses haret TRACE capability to reconstruct the I2C command/response stream. so you can basically perform any action on the device, like pushing one of the capacitive touch buttons, and see what kind of I2C communication the CPU initiates as a result. The problem with this, though: The I2C bus runs too fast, so it loses some bytes. I tried to work around it by increasing the I2C clock divider, but it seems the driver actually re-sets the divider with every transfer (rather than just once when bringing the I2C host controller up).

    I'm trying to find other options (I could clock the entire system down, but then this affects things like the LCM refresh and other important clocks), since I believe a clean I2C tracer is the right thing to do.

    I've also spent a bit of time on the Marvell 8686 driver, as there is already some (not entirely polished and thus not mainline yet) GSPI transport code for the libertas driver. However, I didn't finish this since it is not the biggest priority right now. Also, interestingly, the GPIO and other related bits regarding the wifi chip are all present in the winCE registry. Marvell apparently made the driver in a way that E-TEN and others don't need any access to its source code but can fully parameterize it through the registry.

    So as a summary: I was spending basically every awake minute during the last days on this project, but there's no real visible progress yet. I've just learned a lot, and hope to use that information soon to further improve the Linux port.

    Oh, and by the way: It seems like I'll be talking about some of this work (and actually showing how it was done) at FOSS.in 2008 next week. Stay tuned for some good old fun ;)

    As with actual progress on the device itself: I've spent quite a bit today again with reverse engineering some drivers, thereby discovering two GPIO's that seem to be related to GSM modem power management. Maybe that's the reason why my own humble attempt at a Linux GSM driver has so far been unsuccessful. I also seem to find an awful lot of indication that UART0 is actually connected to the GSM modem, too. This might be some strange copy+paste artefact from older glofiish devices' linux driver, or actually they might have two independent communications channel to the modem - wouldn't be the first time to see this.

    Some other bits have hinted at an externally-provided UART clock, but that is apparently just a workaround of a S3C2442 serial controller bug.

    I', still having fun wading through tons of ARM disassembly. It's been a long time since I last had any good reason to use IDA (Interactive Dis-Assembler) that much. It's the only proprietary software that I've been willing to license (and thus pay for) in something like a decade.

    FreedomHEC Taipei 2008

    As I think I've mentioned before elsewhere in this blog, in a few days there is the FreedomHEC Taipei 2008. If you're in Taiwan and are doing some Linux kernel/driver related development, you should come and meet up at this event.

    Looking forward to meeting lots of [new?] Taiwanese Linux developers :)

    TXL-CDG-BLR-DLH-TPE

    This was the route that I was taking to Taipei this time: Berlin, Paris, Bangalore, Delhi, Taipei... with 7 hours in Bangalore and 4 hours in Delhi, resulting in a total travel time of about 38 hours.

    Everything went surprisingly well and I did a lot of work. However, my day/night rhythm is basically completely gone by now. Need to try to synchronize with local time.

    Oh, and if you're asking yourself "why"? Because airline ticket pricing is the most ridiculous thing on this planet, even worse than stock exchange. Any more 'direct' asymmetric Germany->Taiwan->India->Germany flight would have been about three times as expensive as both a Germany->India->Germany and a India->Taiwan->India round trip ticket together.

    Glofiish M800 GSM/UMTS Modem interface reverse engineered

    During my seven hour stopover in Bangalore, I decided not to sleep and rather have a look about what I could do to find out more about the yet-unknown interface between the S3C2442B application processor and the 3.5G Modem in the E-TEN Glofiish M800.

    Some initial poking in the WM6.1 registry led me to the (wrong) conclusion that UART0 might be used. It would have been a lot of data for a UART anyway...

    So as it seems, they're using a SPI based interface. Not a bad choice, considering the various suboptimal alternatives. USB is way too heavyweight and power-consuming, and leads to inevitable problems when you want to resume the application processor from suspend (e.g. on incoming call). You just simply cannot afford the time to enumerate the USB, etc. Some shared memory / dual ported RAM interface like it is found in more integrated chipsets requires quite a bit of software work (synchronization of a shared memory region between two processors that have no common resources!) and requires a quite deep interface into the modem side. Something that E-TEN would unlikely get from Ericsson, I would say.

    So SPI it is. Interestingly, the SPI master is in the modem, the S3C2442 acts as SPI slave. This adds the need for some kind of mechanism how the application processor can tell the modem that it actually wants to transmit an AT command. A simple GPIO line does that trick. The Modem responds by asserting the slave select line.

    Interestingly, they even use DMA accelerated data transfers. So receiving data from the modem is less CPU intensive than reading data from NAND. What a crazy world.

    Some more bits are found in the wiki.

    I've already started to hack up a Linux driver. The SPI side is really simple. What is much harder is the fact that the Linux SPI core has no support for slave mode, and thus neither the in-kernel s3c24xx SPI driver. Furthermore, many of the traditional serial line analogies (baud rate, modem control lines, handshake, break, ...) no longer apply.

    On top of the SPI, they seem to be running pretty standard AT commands. Nothing fancy at all. Thus, I'm optimistic that once the kernel driver is there, FSO or other userland can make use of it quite easily.

    Running Linux on E-TEN glofiish M800

    Ever since my blog post about certain E-TEN glofiish devices in late August, it might have been obvious that I've been up to something.

    In fact, I didn't have much time, as usual. Finally, after something like about two full days of work, I can present some preliminary results:

    root@glofiish-m800:/proc# cat /proc/cpuinfo 
    Processor       : ARM920T rev 0 (v4l)
    BogoMIPS        : 176.53
    
    [...]
    
    Hardware        : Glofiish M800
    Revision        : 0000
    Serial          : 0000000000000000
    root@glofiish-m800:/proc# 
    

    You can also find a preliminary wiki page about the current status of hardware reverse engineering in the OpenEZX wiki. It doesn't really related to EZX or OpenEZX at all, but it somehow is related to the same thing: Bringing kernel+rootfs based 100% on open source to phones without vendor support. It also doesn't really fit into the Openmoko wiki, since as you can assume, this is by no means a project of Openmoko, Inc.

    So far, it was pretty easy. I was taking the 'stable' branch of the Openmoko kernel git tree, adding minimal platform support to it (to get framebuffer, microSD and USB device working), and using haret to boot into a fso-terminal-image located on a microSD card.

    Of course the really hard work now starts, getting all the hardware properly supported, especially the communication with the GSM Modem, as well as the power management related bits. Nonetheless, a foundation is laid, and I expect it to be not too hard to continue from here.

    So maybe, if I can find sufficient time, we will see FSO on a 3G phone at some not-too-distant point in the future.

    Now some of you might be asking: Why am I not working on improving the code for the Openmoko, Inc. handsets GTA01 and GTA02? Isn't it bad to support a non-open hardware manufacturer, plus pay the Windows Mobile license tax on a device, ...? After all, Openmoko, Inc. current business model is centered around the sales of their own hardware to support for the software development!

    I don't think that this is much of a competition to Openmoko. Obviously, everyone wants truly open hardware, such as what Openmoko, Inc. is trying to do. Nonetheless, people (especially geeks/nerds/hackers) want devices with 3.5G or at least 3G, they want devices with real keyboards, higher capacity batteries, better mechanical design, camera, etc. This is just something that Openmoko Inc. has not been able to provide so far. There's probably not many people on this planet who feel as sad about this fact as I do.

    Next generation of OpenPICC in the works

    Yesterday at a brief visit to the Chaos Computer Club Berlin, Henryk introduced me to the prototype to the next-generation OpenPICC. After co-initiating the project with Milosch and Brita some years ago, I unfortunately had to drop out of it due to time constraints. I've heard rumours about plans for a next generation OpenPICC, but now it seems like Milosch actually found some time to get it done.

    The result is really the über-OpenPICC. No shortage of anything. I don't want to reveal any information that's not out in the public already. Nonetheless, I can say: Stay tuned, stay excited. It rocks!

    Open source SIM card emulation, SIM card proxying: BLADOX

    I've been extremely positively surprised as I recently discovered bladox, a company specializing in providing development tools and custom hardware to simulate, proxy and extend SIM cards. And not only that, they are completely based on gcc + binutils as well as their own open source software. There couldn't be any better playground to play with the SIM interface of any mobile phone.

    The general idea is to put an ATMega128 in between your real SIM card and the actual SIM card slot of the GSM mobile phone. On that ATMega128, you can then do whatever you want, e.g. forward only certain commands, such as "perform GSM algorithm" to the actual SIM, and implement the rest in your microcontroller. You can obviously also implement STK applications yourself for demonstration, or you can even block all STK commands and suppress the phone from ever knowing that the SIM actually supports STK.

    This is an excellent playground. And it's even sold at a very affordable price. Now if I only had more time to look into all of its exciting possibilities while not forgetting about my other tasks (and paid work) :)

    Maybe there are some readers of this blog who are just as interested but didn't even know that such products (and all the example code) did actually exist. Now you know :)

    Hearing of German Constitutional Court on voting machines

    Today was a public hearing of the German Constitutional Court (Bundesverfassungsgericht) on the subject of the use of voting machines in elections of the German parliament.

    I've been anticipating this for quite some time. The plaintiff, Dr. Ulrich Wiesner, has been investigating the subject for a long time, just like the CCC has been doing a lot of theoretical analysis as well as practical hands-on hacking of the respective voting machines (actually, rather, Voting computers).

    As most readers of my blog will be well aware, voting using electronic devices, or even more so computers driven by actual software, raises an almost unlimited number of concerns. Both software and hardware manipulations could have tremendous effects on the final result, no regular citizen or even most IT security experts can actually observe the counting of votes and guarantee the correctness of the results.

    The hearing of the constitutional court was for clarification of further questions of the judges to both the plaintiff, the defendant (the German parliament and Ministry of Interior) as well as three independent expert witnesses. While the CCC has earlier been asked by the court to provide an expert study, it was not officially invited to be questioned at this hearing.

    Nonetheless, three senior members of the Berlin CCC (me included) were present in the audience and following the hearing with great anticipation. It was my first 'live' experience at the constitutional court, and I have to say I am no less than impressed. Intellectual discourse on a very high level. The judges were asking very thoughtful and precise questions, were asking for explanations without mercy ;)

    I think the legal representation of the plaintiffs (including a senior legal scholar) was excellent. Good arguments, very eloquent. The various defendants (ranging from representatives of parliament, ministry of interior, the government agency in charge of certifying the voting machines (PTB), as well as the senior election official of the state of Hessen) were making much less impressive performance.

    And at the end of the day, I still cannot get why about every consumer electronics device, from mobile phone to digital TV receiver to game console has about one lightyear more security architecture than the machines that are used to count the votes. No hardware-crypto engine, no encrypted JTAG, no signed bootloader and software (plus automatic mask-rom based signature verification). Plus officials in the public administration who think the trade secrets of the vendor of the machine is more important than the public interest..

    I think the judges very well got that point. You could literally see their disbelief in situations like when it was outlined to them that only the vote-counting machine has to get type approval, but not the PC + software that is used to program the particular election into the vote-counting machine, neither the software used to read out the memory modules and summarize the votes of multiple voting machines. So not even those insufficient small amount of testing and certification that exists does extend to the entire system, rather just to the input unit.

    We'll probably have to wait for some more months (at least weeks) to see the result. I definitely remain very optimistic that the constitutional court will prevent the worst problems of the current situation. I don't think they will completely close the door for voting machines, but at least raise the bar for any such future system very high in order to achieve a level of transparency and trustworthiness similar to that of the traditional paper ballot vote.

    To me, for a long time, the constitutional court is the single remaining still functional and trustworthy entity in the Federal Republic of Germany. It is the last bit of hope against the constant battle of the government administration[s] against civil liberties, post-9-11-security, surveillance/intelligence particular in 'new' technology.

    Some more S3C24xx NAND speed observations

    I've now moved on to other topics, but I still want to mention some of my thoughts on the still quite poor NAND performance on the GTA02 (and generally, the S3C24xx).

    It seems like we are down to a point where the CPU is 100% busy reading from NAND, which is odd. Why would reading from a mass storage device make the CPU so busy? Well, because Samsung "forgot" to add DMA support to all of their integrated NAND controllers, from the old 2410 through the 2440, 2442, 2443 and up to the shiny new 6410, all the NAND controllers don't support DMA. In fact, they don't even have a FIFO or some kind of internal buffer for the received data. This is really weird, considering the facts that

    • every other peripheral (SD/MMC, SPI, UART, ...) can use DMA
    • Samsung as provider of both NAND flash and SoC should be experts in providing good flash performance
    • I cannot see any strong architectural limitation. The data is read into a register. The register should be replaced with a FIFO, and a DMA can regularly read from that register or FIFO and put it somewhere else into memory. It's not any different from e.g. SPI or UART DMA.

    In the current mainline Linux driver for S3C2440 NAND, we busy-wait (poll) for completion of the read request. This is of course sub-optimal. I implemented a version that uses the Read/Busy line IRQ and a 'struct complation' based mechanism and to my big surprise the CPU usage and throughput was identical. It seems like that NAND flash in the GTA02 is fast enough to max out the CPU.

    So probably all that can be done is to optimize the actual code that is executed during the NAND read. It might be worth implementing some small hand-optimized assembly implementation as standalone code (not using the existing driver) to see how far the hardware can actually go.

    Isn't it sad that you use Samsungs SoC and Samsungs NAND (packaged together in one component as multi-chip-package by Samsung) and still get less than 50% of the performance of the NAND chip, according to the data sheets :(

    Some more GSM Abis experiments

    It's been quite some time since I last mentioned Abis in this blog. Recently, I've again found some time to experiment with Abis. The last two days I was working on porting some proof-of-concept code of a fellow hacker from windows to Linux.

    I now also have manufactured all the cables and adapters I need, plus installed a Windows 98 box in order to operate the Wandel+Goltermann MA-10 protocol analyzer that somehow made his way to my lab. The MA-10 can act both a a high-impedance passive sniffer on top of the E1 link (using the Abis Monitor software), or it can actually emulate the BSC by using a terminated E1 Rx+Tx interface and the AbisSim software.

    I don't want to talk too much about things that haven't been implemented yet. But in the next weeks, I'll be diving all the way into mISDN and see what needs to be done to make it talk the specific Q.921 dialect that Abis uses.

    Stay tuned. I'm working towards a first release at the 25C3 congress in late December.

    Android and its perceived 'openness' :(

    As many other people have been blogging and news sites have been reporting: The Android source code has been released. This is definitely good news. However, freedom-loving people already discover in blog posts that there's a remote kill switch by which Google can disable an already installed application and that some features are reserved to vendor-signed applications.

    To me, those things are not a big surprise. As soon as you try to get in bed with the big operators, they will require this level of control. Android is not set out to be a truly open source mobile phone platform, but it's set out to be a sandbox environment for applications.

    And even with all the android code out there, I bet almost (if not all) actual devices shipping with Android and manufactured by the big handset makers will have some kind of DRM scheme for the actual code: A bootloader that verifies that you did not modify the kernel, a kernel that ensures you do not run your own native applications.

    Thus, Android so far was little more to me than yet-another-J2ME. Some sandbox virtual machine environment where people can write UI apps for. In other words: Nothing that gets me excited at all. I want a openness where I can touch and twist the bootloader, kernel, drivers, system-level software - and among other things, UI applications.

    I actually think it's a bit of an insult if people think of Motorola's EZX or MAGX (and now Android) phones as "Linux phones". Because all the freedoms of Linux (writing native applications against native Linux APIs that Linux developers know and love, being able to do Linux [kernel] development) are stripped.

    In the end, to what good is Linux in those devices? Definitely not to any benefit of the user. It's to the benefit of the handset maker, who can skip a pretty expensive Windows Mobile licensing fee. Oh and, yes, they get better memory management than on Symbian ;)

    That's the brave new world. It makes me sick.

    Luckily, it's 50 B.C. and the Romans occupy all of Gaul, except for one small village that has been able to avoid the invaders.

    About the new format / structure of FOSS.in

    There has been quite some discussion on various places on the net about the recently-announced change of the FOSS.in conference format. Instead of lots of talks/presentations, there is an emphasis on workshops and similar more interactive and collaborative types of events.

    I have been speaking to a number of developers who have been to FOSS.in before and who have been putting in proposals for FOSS.in/2008, too. They all think it is a very courageous step: going from a successful, working 'traditional conference' scheme with presentations, sufficient sponsors to cover travel expenses of foreign speakers, etc. to a very different, much more developer-community oriented event.

    I also think it is a courageous experiment. I have not yet heard of any event similar to this before. Sure, there are project days and developer meetings or miniconfs or whatever you might call them. But not to the extent as, at least to my perception, FOSS.in is planning right now.

    In any case, it depends on what your target is. 'typical' Linux conferences are basically focussing on either one (or multiple) of the following:

    • Spread the word about Linux/FOSS, to generate more adoption
    • Provide updates on development progress to various people in the community as well both individual and professional users

    However, if you emphasize on the actual FOSS development, then I think it is quite legitimate to go for a event format that FOSS.in is heading to right now.

    It is exactly FOSS.in who can try such a change, since it is a true community event without any commercial interest and without affiliation to particular companies.

    And after all, who wants to see the same kind of event happening each and every year, with the same kind of people talking? Wouldn't that be boring after some time? Especially if there are a number of other events doing more or less the same?

    In any case, personally I'm planning to do a FOSS.in WorkOut on a USRP+gnuradio based GSM scanner project. India is the perfect place on earth to get this done, since the government mandates A5/0 (no encryption) and thus all the packets can be captured and each and every bit implemented as wireshark plugin.

    Openmoko GTA02 NAND performance improvements

    On Sunday night, after returning from a weekend trip to Hamburg, I sat down and looked at the NAND and S3C2442B data sheet to figure out the actual timing performance. Interestingly, the NAND timings were much more verbose and detailed (and had different names) than the timings described in the NAND controller section of the S3C24xx manual - and both are from Samsung ;)

    Anyway, it seems like the current timing settings for the various stages (reading u-boot by the stepping stone mechanism, reading the kernel by u-boot as well as actual mtd-based access inside the Linux kernels) were extremely suboptimal. They're basically standard timings designed to work with most NAND flashes out there, ignoring the fact that GTA02 uses one specific flash with very good (fast) timings, at least according to the data sheet. There should also be no PCB / routing related issues such as capacitive overload preventing higher speeds, since the NAND flash die is stacked onto the CPU die in one package, and the NAND controller signals are not routed on the PCB anywhere.

    Some initial experiments based on the calculations show that the performance can be easily improved by 41% over the stock GTA02 NAND performance. However, the actual speed (6.59MBytes/sec) is still much lower than the theoretical maximum read performance of 15.64MBytes/sec. It seems there is more room for improvement inside the MTD layer of the Linux kernel.

    It's again quite amazing how much room there is for improvement in GTA02 performance, both power consumption wise (see my recent post about framebuffer blanking), as well as actual data throughput. Those are really low-hanging fruits, and it's very surprising that nobody working for Openmoko or in the Openmoko community has been able to spend some time to look into those...

    FOSS.in 2008 CfP is out

    I have just received news that the FOSS.in 2008 Call for Participation has been released. This is good news, although I think it is quite late, as every year...

    I'm definitely looking forward to FOSS.in, like every year. There's such a huge potential in India, so many software developers. If only some more could be convinced to _effectively_ work on FOSS and contribute their work back to the community, it would be a great gain.

    All details about mifare crypto1 algorithm and protocol disclosed

    Henryk Ploetz has released his thesis on mifare protocol and cryptographic weaknesses, which is to the best of my knowledge the most complete publicly available documentation about the mifare CRYPTO1 system.

    Congratulation to him and his peers (starbug, Karsten Nohl and others) on this great work. I still hope I could have played a more active role in the security research on mifare. But it's good to see that as imperfect as they are, OpenPCD and OpenPICC fulfill their duty as toolkit to enable security research on RFID.