HTC announcement about no more locked-down phones

As it has been covered at various news site, HTC has apparently announced that they will not be shipping Android phones with locked-down bootloaders.

If this is really true, it would mean that more people not only have the theoretical freedom to run modified versions of Linux (granted by GPLv2), but also the practical freedom. If there is no cryptographic restriction on only booting HTC-supplied versions of the Linux kernel (and other software), this is good news!

It comes as a bit of surprise though. "Traditionally", HTC is known for behaving unfriendly towards the community. Not only due to their source code releases being constantly too late, but also due to the fact that their phones were some of the first to use cryptographic signatures to keep people from installing their own versions of Linux (and Android).

The other surprising move has come from Motorola, who probably has the longest tradition of shipping Linux-based phones (in various degrees of GPL compliance), but then using technical means to deprive their customers of the Freedoms the GPL wants to grant to them, i.e. the freedom to run modified versions of the Software (Linux in this case). They did this with the later models of the EZX range, with their MAGX phones, as well as now with their Android phones over the last couple of years. So it was very puzzling to see the same Motorola announce a 180 degree turn in policy at least for their Xoom tablet.

Also, in recent news, Sony Ericsson made a similar announcement that at least some of their Xperia models can be bootloader unlocked.

It's really striking. During the least seven years, I used to be involved in a number of projects that tried to enable the user of mobile smartphones to have the full source code for (at least) the Linux kernel, and to be able to modify, tinker and re-program it any way they want. Now some of the vendors seem to be moving in the right direction.

What's sad is that Samsung is not capitalizing on their potential here. They have always had very timely and complete source code releases for all their Linux based phones at http://opensource.samsung.com/, and they have very rarely tried to lock any of the bootloaders. I don't know if this is intentional or not. But now the other vendors are getting good PR for stopping to do something that (to my knowledge) Samsung has not done, at least not to the extent of the others.

In any case, I still think the Nexus S is the best choice for anyone who wants to have a developer friendly device. It is fully supported in the main AOSP tree, everything in the kernel is GPLv2, and those binary userspace blobs that are required are distributed independently at https://code.google.com/android/nexus/drivers.html so they can be integrated into custom builds. This is by no means perfect, but the best compromise that seems available at this point. I still don't understand why the userspace drivers for the GSM/3G modem, Wifi, Bluetooth and GPS would need to be proprietary. Or even the NFC par, it's sort-of ridiculous to have that proprietary with Free Software RFID stacks like libnfc and librfid around...

Apple not providing LGPL webkit source code for latest iOS 4.3.x

As some people may know, next to a plethora of BSD licensed code, Apple is using some LGPL licensed code in their iPhone products.

So far, it seems they have always provided the respective source code in a timely manner for each and every release they have made on a website www.opensource.apple.com.

However, in recent months it seems they have deviated from that policy for unknown reasons. As my friend and webkit developer zecke has blogged, Apple has stopped to release their webkit source code with iOS release 4.3.0. The corresponding website simply states: "coming soon".

iOS 4.3.0 was released on March 10, 4.3.1 on March 25, 4.3.2 on April 14 and 4.3.3 on May 4. For all of those releases, no source code has been published.

It cannot be a simple oversight, as multiple inquiries have been made to Apple by interested developers. However, the source code yet has to be released.

I think it is time that Apple gets their act together and becomes more straight-forward with LGPL compliance. It is not acceptable to delay the source code release for 8 weeks after shipping a LGPL licensed software. Especially not, if you have already demonstrated in the past that you are well aware of the obligations and have a process and a website to release the corresponding source code under the license conditions.

Jounrees Logiciels Libres / ENSA Tetouan, Morocco

I've been invited to Tetouan, Morocco by the organizers of the second incarnation of the Journees Logiciels Libres. Tomorrow I'll have the pleasure of presenting about Free Software projects related to GSM, including OpenBSC and OsmocomBB.

The organizers have done a great job in caring about the foreign speakers (who include Richard Stallman and myself).

I've been listening to various talks by RMS RMS over the last 16 years or so... but right now I'm listening the first time to him giving a French presentation.

Overall, this trip has done more to improving my understanding French than anything else in a long time. I once had 4 years of French from 1st to 4th grade in school, but never really continued with it. However, what I remember, combined with my knowledge of Portuguese (and even English) is sufficient to e.g. understand all of the French language slides that have been presented at this conference. However, most spoken French is too hard to understand for me.

One striking observation is the apparently much higher percentage of women taking a communications or computer engineering degree here than what I'm used to in Germany or the so-called western world. It reminds me of India where you have the feeling that almost 50% of the IT related students are female. It would still be interesting to see some scientific research why the supposedly open and anti-discriminating, women-rights-embracing 'western world' is seeing less women taking up engineering studies...

PTB kann nicht nur Wahlcomputer nicht prüfen, sondern auch Spielautomaten

To my non-german-speaking blog readers: Deep apologies, this one makes more sense in German. It will remain an exception in this blog.

Eine Gruppe von namhaften öffentlich bestellten und vereidigten Sachverständigen hat ein 25-seitiges Positionspapier herausgegeben, das erläutert, wie die Untätigkeit oder Unfähigkeit der PTB (Physikalisch-Technische Bundesanstalt) dazu führt, daß die gesetzlichen Bestimmungen zur Spielsuchtbekämpfung unterlaufen werden.

Die von der PTB herausgegebenen technischen Richtlinien entsprechen nicht dem Stand der Technik. Nicht nur das, sondern es werden auch noch Sachverständige zugelassen, die sich nicht auf dem Gebiet der Informationsverarbeitung (IT/EDV) auskennen.

Das erinnert mich 1:1 an die extrem peinliche Rolle der PTB im Bereich der Wahlcomputer. Zur Erinnerung: Eine Behörde, deren Vorschriften nicht im Entferntesten geeignet sind, die komplexen informationstechnischen Systeme in adäquater Weise zu prüfen. Eine Behörde, die ihre Prüfvorschriften nicht herausrücken wollte, und deren Prüfberichte nicht veröffentlicht wurden - schließlich wiegt das Geschäftsinteresse der Hersteller schwerer als das Interesse der Bürger an transparenten Wahlen, nicht wahr? Erfreulicherweise hat uns das BVerfG bis auf weiteres von Wahlcomputern befreit, und damit auch die Frage erübrigt, ob die PTB qualifiziert ist, Regeln in diesem Gebiet zu erlassen und/oder Geräte zu prüfen.

Man sollte einfach eine Abteilung für die Prüfung der Spielgeräte beim BSI einrichten. Man kann vom BSI halten, was man will - aber man arbeitet dort einfach auf einem ganz anderem Kompetenzniveau. Man sehe sich die umfang- und detailreichen technischen Richtlinien zur De-Mail an. Da hat jemand über Nachvollziehbarkeit und Sicherheit von IT-Systemen nachgedacht, der wirklich Ahnung hat. (ganz unabhängig davon, ob das System an sich für den Bürger nützlich ist)

Die PTB mag sich ja mit der Eichung von Messgeräten und ähnlichem auskennen - zumindest als es noch um mechanische Waagen oder so geht. Der Name sagt ja schon technisch-physikalisch, nicht etwa Soft-und Hardware von Informationssystemen. Aber genau um letzteres geht es bei den Spielgeräten heutzutage: Moderne Computer, mit komplexer Hardware und Software.

Finally: A parseable MAPv1 ASN.1 specification

After way too much work in extracting the ASN.1 text from word documents, learning about the differences in pre-1990 ASN.1 and pre-1990 TCAP with their respective current-day counterpart, some futile attempts with the incomplete ASN.1 grammar files available for ANTLR3, I have finally dedicated the entire day to manually convert the MAPv1 from ETSI TS 09.02 3.10.0 (1994) to work with present-day TCAP and present-day ASN.1 syntax.

The result not only passes the syntax and semantic checker, but it is accepted by the Erlang asn1ct. It has not been tested/validated against any test data so far. Just in case anyone else needs a 'working' version of MAPv1 some 20 years after it was created, I have published the result here: http://cgit.osmocom.org/cgit/asn1_docextract/tree/output/3.10.0

Any bugfixes/improvements/complaints are obviously welcome.

Now I just need to figure out how to properly work with the ROS/ROSE CONTRACT class and TCAP APPLICATION-CONTEXT class to formally describe the various application contexts, their respective MAP operations and associated application context versions. It's actually quite a bit surprising that the ETSI/3GPP don't specify this formally in 29.002. The information is contained in the human-readable part of the document, but not in the formal ASN.1 notation. I wonder why...

Facebook "Open Compute Project" nothing but hot air

There has been a lot of fuzz about facebook's Open Compute Project, and it seems to originate mostly from journalists without much technical skill.

Did anyone actually bother to check what they released? You can find mechanical designs and simple specification data sheets. More or less every vendor is publishing that kind of information, or at least there are common form factors that are specified (like ATX, eATX, mini-ITX, ...)

It is sad to hear that they are 'openwashing' themselves (like BP is greenwashing itself). Specifically, the following portions are not "open":

  • Any type of electrical schematics (mainboards, power supply, ...)
  • Any type of detailed data sheets or programming manuals on the electronic components used
  • Gerber files of the PCBs

So what this really is all about is: Facebook advertising this is our new mechanical form factor, now we want all of you to adopt it, so we can buy cheaper hardware.

Go home, facebook. Come back if you have something _really_ open.

Following up on my GSM MAP archeology project

In the past weeks, I have been blogging about the work I've been doing in trying to recover the earlier versions of the GSM MAP ASN.1 files in order to support it from my own Osmocom Erlang MAP code.

After having gone through the tiresome exercise of implementing a custom MS Word for DOS file parser to extract the ASN.1 definitions from the 3GPP-published DOC files, I tried to feed them into the Erlang asn1ct code generator.

Unfortunately this didn't work, as the old MAP ASN.1 references very old ITU-T TCAP from 1988. So I went to the ITU website. There again a similar story. From their asn1 website section, you can only get the TCAPMessages from 1997. If you want an older format, like the Q.773 from 1988, you can download a PDF file. Most of that PDF file is actual text. But the ASN.1 sections are included as an image, so there is no way how you can copy+paste them.

So there I went, and manually typed in those few pages of ASN.1 source code. After I was finished, I thought I had finally done enough. Writing a word-for-DOS parser, typing in pages of ASN.1 manually. What more can there be? I should be able to finally generate Erlang code for decoding and encoding the respective messages.

But it would be too simple if it was that easy. The 1988 TCAP as well as the old MAP specifications use the ASN.1 "MACRO" construct to define the individual MAP operations. However, as it seems the MACRO construct had been deprecated in 1994 by the ISO in charge of ASN.1. Successive ISO specs for ASN.1 have completely remove the MACRO construct as similar results can be achieved with newly-introduced information objects. So of course, the Erlang asn1ct (as well as other free software ASN.1 tools) does not support such an antique feature.

So here I am in the year 2011, unable to use information available in the public ITU-T and ETSI/3GPP specifications to build a GSM MAP protocol implementation that can deal with the messages that you encounter on real-world SS7 links. I guess there are now the following different approaches to proceed:

  • write a converter that translates MACRO into OBJECT
  • manually convert the old MAP specs from MACRO to OBJECT
  • add MACRO support to current-day ASN.1 tools
  • find some old ASN.1 code generators that still support MACRO

All in all, this is a really dissatisfying situation, and I would say it is a failure of the standardization bodies to provide useful versions of their specifications. You cannot remove operations and associated message types from a specification, while in real life even 15 years later a whole number of users still use messages according to those (now removed) definitions. Meanwhile, the old specs are getting useless, because you not only deprecate but completely remove a language construct of the formal specification language used.

This renders the idea of having a formal specification useless. If I first need to write my own converters or manually convert that very same formal specification, errors can be introduced in the translation/conversion process, just as much as errors could have been introduced in manually writing encoding and decoding routines based on a non-formal-specified protocol like most of the IETF / Internet protocols.

Deutsche Telekom tried to register a trademark on netfilter

I am currently doing some trademark related research, and just for fun I queried the database of the DPMA (German trademark and patent office) for "netfilter".

To my big surprise, you can find this record, indicating that Deutsche Telekom AG has applied for a trademark on the word "NetFilter" in July 2006.

I find that quite outrageous, as the netfilter project is using the name since about 1999, i.e. 7 years earlier. To our luck, the trademark office refused the application based on the generic nature of the name, i.e. "netfilter" being too generic for anyone obtaining a trademark on it - at least in Germany, under German laws.

Linux Beer, anyone?

During my trademark research, I also discovered: A German beer brewer (St. Georgen Braeu, Buttenheim) has held a registered trademark "LINUX" from 1999 to 2009. This trademark was restricted to "beverages, beer and other alcoholic drinks".

You can find the respective entry in the DPMA trademark database here.

I am not quite certain whether I would have liked the idea of drinking a pint of Linux or not. It would certainly have been a popular gift to bring to international (Linux, Free Software) conferences.

More MS Word for DOS file parsing in the name of GSM

It seems that from TS 09.02 version 4.2.0 on, the editors have actually put markers as "hidden text" inside the Word document, which allow better automatic detection when a given ASN.1 module starts, is interrupted by plain text, continues and ends. The following screen shot (from Section 14 of the above-mentioned document) is human-readable hidden text explaining the syntax:

So now I'm adding this format as a second option to my extraction tool.

Please note: The tool I wrote yesterday (working fine with version 3.x.y of 09.02) is available from asn1_docextract.git. Later today it should also support the >= 4.2.0 annotations outlined in this blog post.

Implementing a custom MS Word for DOS file parser to properly do GSM SS7

Yes, I'm not kidding!

In recent months, I've been writing quite a bit of GSM MAP (Mobile Application Part) code. MAP is the protocol used heavily in the GSM core network and especially on the roaming interfaces between different operators. It is specified in GSM TS 09.02 and later 3GPP TS 29.002.

The protocol specification relies on ASN.1 description of the messages as well as the regular BER encoding rules. ASN.1 is this marvelous technology that allows a protocol to be specified in an abstract and formal notation, in an extensible way, removing all the problems of human-written marshalling code, full of errors and differences due to different developers interpreting a human-readable specification in different ways.

So far so good. You think it should be simple to write a parser and generator for MAP messages: Simply feed them into the ASN.1 compiler of your choice, it will generate code in the target language you require.

As long as both sides of the communication do that using exactly the same revision of the specification (and don't make implementation mistakes), this will work. The reality looks very different, though :( When I test my code against something like one million of real-world messages captured on a production SS7 roaming interface, it produces errors already on packet number six of that trace.

The problem is: The protocol designers have not specified the first versions in a really extensible way, i.e. a given operation originally only returned one atomic data field, and it was later extended to return a sequence of data fields. Thus, there is one additional level of hierarchy in the encoding.

Not only that, but in their infinite wisdom, the designers of MAP have also failed to include versioning information in each and every message header. Instead, it is part of the application context name, which is only part of the first message of every conversation.

Furthermore, different versions of the MAP specifications disagree on whether certain fields are deemed optional or not. This is further complicated by somewhat strange versioning habits. There is the Revision number of the TS 09.02 (like 3.8.10), then there is a different version number encoded in the corresponding ASN.1 files like 'version9(9)' and individual operations then have v1/v2/v3 in their application context name.

Some even more wiser decision must have been to remove the description of older messages from the later versions of the specifications. So even specifications published in the year 2000 no longer include definitions of messages that were still part 5 years earlier. Why does it matter? Because today, in 2011, you still see MAP message on the international SS7 interfaces that are encoded in some of the earliest versions of the MAP protocol!

And if all of this was not enough, the biggest bummer is: For most of the releases of the specification, the ASM.1 text files are not distributed separately, but they are interspersed with human-readable text in the actual specification documents (which can be 600 pages long, nothing you want to cut+paste).

Even worse: If you go to the ETSI homepage and download the PDF version of old 09.02 specs, they will actually provide a PDF with a scanned paper print-out, i.e. no searching and no copy+pasting.

Luckily, the 3GPP has made the history of 3.8.0 and later available on their FTP server. But they are in MS Word for DOS format, like they were written originally. This format can not be opened by OpenOffice, and as far as I know not even by any of the Windows Word versions that MS has released in the last 10 years.

So what did I do? I actually installed MS Word 5.5 for DOS (provided as Freeware from Microsoft) and ran it in DOSEMU, to convert the specs into RTF format. This way I can at least open them and look at them in a modern text processor.

But this still does not solve the copy+paste problem.

I finally found antiword, but it mainly focuses on Word for Windows files and only does rudimentary text extraction from Word for DOS files. But hey, there is an online copy of chapter 16 from the File Formats Handbook, apparently published by Dr.Dobb's (who remembers them!!) at some time in the past.

So what did I do? I wrote some custom parser for those old Word/DOS files, which parses the paragraph format descriptions and tries to identify those sections that contain the ASN.1 code. As they are almost the only part in the specification that is enclosed with a border line on all four pages, this should work pretty fine. Early results are quite promising!

My hope is now that the ETSI stylesheets did not change too much over time, i.e. that this parser will be able to extract the ASN.1 spec for all of the protocol versions that I can find. If that works, I can run them through a validator, then pretty-print them and putt them all in one git tree in chronological order. And maybe at some point in 2011, we will have the marvels of an unified diff between two different MAP versions. The strange part is: Diff was developed in the 1970ies, GSM in the late 1980ies. They should have known about it back then, and used a revision control system like SCCS to record all the changes in the specification they make.

I guess this all is a glimpse how a digital archaeologist of the 22nd century must feel when analyzing ancient artefacts and trying to understand what the heck his ancestors have been doing back then.

UPDATE: The tool can be found at http://cgit.osmocom.org/cgit/asn1_docextract/

Travelling to Belem/Brazil to talk about OpenPCD and OsmocomBB at UFPA

Tomorrow I'll be leaving for a 10-day trip to the signal processing lab of UFPA (Federal University of Para) in Belem, Brazil. I was kindly invited by Prof. Aldebaro Klautau to hold some lectures and lab exercises regarding Free Software (+Hardware) RFID projects like OpenPCD as well as Free Software GSM projects like OsmocomBB.

I would love to use that opportunity to spend some more time in Brazil for holidays, but my schedule really doesn't allow for anything like that at this time. It's always sad to have to miss such a chance. It would be exactly the right time of the year to spend some time at the beaches of Pernambuco or Alagoas... *sigh*

Starting to experiment with Anritsu MD8470A signalling generator

Earlier this week I was able to pick up some new equipment (aka toys) from the customs at Berlin TXL airport. Among other things I learned that despite filing an Internet-Zoll-Anmeldung (IZA) (Internet Customs Declaration), online via the German customs web site, you still have to print two copies of it on actual paper. I only had one copy, and the customs department does not have a copier to produce another copy. Buerocrats :/

In any case, I am now the proud owner of an Anritsu MD8470A signalling generator, which is basically a small 3G (WCDMA) network on steroids, all inside a single box. It can do mobility management, call control, voice calls, sms, packet data service, WAP, etc. It even has a legacy GSM/GPRS radio, so you can do inter-RAT hand-over between 3G and GSM.

But what is even more exciting: It includes (proprietary) APIs that allow you to send sequences hand-crafted messages on RRC and any layer above. This means it is an excellent tool for security and robustness testing of mobile phones.

Struggling with adding Ericsson RBS support to OpenBSC

I've been spending way too much time recently in understanding the low-level aspects of Ericsson RBS 2000 and the associated OM2000 protocol. The goal here is to support this family of BTS from OpenBSC.

The first big obstacle was that the A-bis Layer2 (LAPD) is quite different from what we've seen with Siemens BTS before - and also from what the GSM Specs TS 08.56 says.

In the Ericsson A-bis, there are the following key differences regarding standard A-bis:

  • E1 timeslot is not configured statically. Instead, the BTS scans the entire E1 link and looks for SABM messages to TEI=62/SAPI=62
  • There is no TEI manager
  • LAPD sessions are initiated from BSC to BTS
  • There is not only one OML connection for each BTS, but one additional OML connection for each TRX
  • OML does not follow 08.59/12.21 but is proprietary

All those parts above have now been solved. We can initialize the A-bis link and talk OM2000 to the DXU/IXU of Ericsson RBS 2000, and we can use that to configure and initialize the CF (central function), as well as to configure the IS (Interface Switch) and CON (concentrator).

However, the IS configuration is already quite difficult. In that configuration you connect 1:1 mappings of various ICPs (Interface Connection Points). So you can connect any 64k or even 16k sub-slot of a TRX with any of the E1 interface(s) timeslots. However, the assignment of which TRX(TRU) is represented by which ICP is BTS specific. On a RBS 2401 for example, the first TRX (TRX0) is attached to ICP 512..523 (1x 64kbps signalling + 8x16kbps traffic).

So if we configure the IS to connect those ICPs 512..523 to the ICPs 4..15, then we will get the TRX0 routed to Timeslot 1,2 and 3 of the first E1/T1 interface. You can see some examples in page 89 (pdf page 115) of this document. This works on the RBS 2401, but it seems like the RBS 2308 has different ICP/DCP assignments than the generic RBS 2000 example that they are showing.

If anyone is more familiar with the details of the Interface Switch in RBS2000, and specifically the ICP / DCP mapping inside the RBS 2308, I would definitely want to have a chat with you.

If we cannot figure this out, it is impossible to bring up the per-TRX OML and RSL links, and thus impossible to use those BTS with OpenBSC :(

Public release of Osmocom TETRA code

Today I have publicly released the TETRA receiver code that I've started to work on earlier this year. The gnuradio-based demodulator, PHY and lower MAC code as well as some README file is available from the new http://tetra.osmocom.org/ website.

There's still a lot of work to be done, but fundamentally we are able to receive, demodulate and decode TETRA TMO downlink data. So the task of the software somewhat compares to that of gsm-tvoid or gsm-receiver (part of airprobe).

A corresponding project mailing list has also been made available. Please respect that this is a development-only discussion list right now. If you cannot make the code work and need help with it, chances are high that it doesn't do anything useful for you anyway.

There is some early code that should eventually enable us to run a base-station side transmit implementation, but it's not working yet.

SS7 work: M2UA, MTP3 and ISUP message {de,en}coding

One of my paid contracts required me to start moving directly into the GSM core network, the SS7 domain. While so far I had a fairly good understanding of SCCP, TCAP and MAP (which are required for GSM/3G roaming interfaces), I've now had to look at the actual telephony signalling side of things.

Even in GSM networks, the gateway or inter-working MSC will translate all the GSM TS 04.08 Call Control messages into ISUP, the ISDN User Part, originally designed for call control signalling between ISDN networks.

As apparently always in SS7, there are many, many different standardized and proprietary protocol stackings that can be seen in the wild. I'm now working on a stack that looks like IP->SCTP->M2UA->MTP3->{ISUP, SCCP->TCAP->MAP}.

Luckily, for now there is no need to do any fully-fledged implementation of it, simple message parsing and encoding routines are sufficient for the task at hand.

It's been about time that I'm closing those last gaps in the knowledge of GSM core network protocols. The only part that I'm still missing so far is CAMEL. I know roughly how it works, but I've never played with it or implemented any part of it.

Supporting the HSL 2.75G femtocell from OpenBSC

The last couple of days I've been hacking away on reverse-engineering the proprietary Abis-over-IP variant of the HSL Femtocell. This is required to get this latest newcomer in the GSM femto/picocell market to work with our OpenBSC (and later OsmoSGSN) software. Progress is quite good now, apart from their custom RTP multiplexing format, everything required for signalling, SMS and Voice is working from OpenBSC.

The HSL Femto is a nice and powerful piece of hardware, containing a TI DaVinci ARM+DSP chip, 128MByte DDR2 memory, a Spartan-3A FPGA and 275Ms/s DAC as well as 65 Ms/s ADC. Much too powerful for a single-ARFCN GSM system. This really looks like the vendor wants to do multi-ARFCN software updates later. More details and some initial PCB photographs can be found in the OpenBSC wiki.

The Software side looks a bit like it is still maturing. Most bugs I have found so far are apparently fixed in the SR1.0.1 firmware. The A-bis dialect is quite different (and more simplistic) from what I've seen in any other BTS. More details can once again be found at a page in our wiki.

What's exciting is that there now is a commercially available traditional BTS product at relatively low cost. By traditional I mean it is still only a BTS and not a Um-SIP gateway like OpenBTS.

I hope this will enable more people to use and experiment with OpenBSC, as the cost and availability of the ip.access nanoBTS has always been an issue for many people without a four-digit budget available.

Working on a Free Software TETRA implementation

Those who have been following my twitter feed over the last week will already know: I've been in deep hacking mode implementing the lower levels of TETRA, specifically the PHY and lower MAC but also parts of the upper MAC level.

The primary idea here is to produce something similar to what airprobe is for GSM: An air-interface protocol analyzer that can demodulate/decode/demultiplex information received from a TETRA base station.

I'm not working on this alone, a number of known (and unknown) names from the Osmocom projects have been involved again. Progress has been surprisingly quick. The biggest gain was Dieter discovering that we didn't have to write our own pi4/DQPSK demodulator, but that somebody had already done that as a gnuradio hierarchical block originally intended for the more advanced channel types of APCO25.

So we could concentrate on the PHY and lower MAC layers, i.e. implementing stuff like

    correlator for the various training sequences to achieve frame sync
  • de-scrambler, de-interleaver, convolutional decoder, CRC, Reed-Muller decode
  • Implementing the TDMA multiplex, reading the BSCH (like SCH in GSM)
  • Decoding the MAC-layer PDUs like ACCESS-ASSIGN, SYNC, SYSINFO, RESOURCE
  • Peeking into the higher layers like MM

Before writing the decoder I also wrote encoders/interleaver/scrambler etc. for the transmit side in order to do first testing/validation of the decoder. Using this encoder, we are able to generate continuous downlink SYNC bursts containing BSCH(MAC-SYNC PDU) + BNCH(SYSINFO PDU). As the SYNC bursts can be used for unallocated time-slots, a never-ending sequence of this single burst should provide a valid carrier that TETRA mobiles can lock to.

Working on all this has taught me much. My previous knowledge on convolutional codes, scrambling, interleaving, training sequence corellation, viterbi decoding, etc. has been mostly abstract and theoretical. Now I had the chance of implementing [almost] everything from scratch. Now I can understand what people like Tvoid or Piotr went through when they wrote gsm-tvoid/gsm-receiver (part of airprobe).

For the next couple of weeks I have to turn back to doing some higher priority work again, and in February I want to play with the GSM RF side of the MTK GSM chipsets again, so I don't know when I'll be able to continue with the TETRA related work. However, I hope other developers in the team will pick up where I left and bring the project further forward.

As soon as the code has moved beyond early prototyping, we'll be releasing the demodulator, libosmo-tetra-phy, libosmo-tetra-mac and associated command line tools.

OpenBSC field test at 27c3 over

During the last week I was busy with

  • December 22nd though 24th: Preparing OpenBSC to be ready for the field test at 27c3, i.e.
    • improving the output of the log at "INFO" level to be not too verbose at the expected network load
    • Implement the interface to LCR using a Unix domain socket rather than linking LCR with OpenBSC
    • Configuring all 6 BTS, put them in multi-TRX config, test the setup
    • Manufacturing nanoBTS stacking cable (with their weird RJ-69 plugs that you have to mill a notch off)
    • Install all required software on the machine that will run OpenBSC
  • December 25th and 26th: Setting up the network
    • Physically mounting the nanoBTS units
    • Patching cables throughout the building, installing PoE switches
    • Configuring LCR
    • Interfacing with the Phone Operation Center (POC) via E1 / DSS1
  • December 26th through 30th: Running the network
  • December 30th: De-installing the network

I don't have much time now, still have to unpack lots of boxes full of gear. However, I have finally completed my scripts to graph some of the statistical data of the field test. You can see the graphs in the OpenBSC wiki.

Unfortunately we don't have the same body of statistical data for the previous field tests at 25c3, 26c3 and har2009. However, for all of those three events we have now graphs about the IMSI/Country distribution of all the phones that have ever tried a LOCATION UPDATE with us: 25c3, 26c3, HAR2009, providing some nice statistics on what nationalities are attending the respective events.

Preparations for GSM network at 27C3 conference

Behind the scenes, we've been working on preparing the experimental GSM/GPRS/EDGE network at the 27th Chaos Communication Congress. The regulatory authority was nice enough to grant us 6 ARFCN, which we will split to 3 BTS (2 TRX each), resulting in one BTS with 2 TRX on each of the 3 conference building floors.

I've started a page in the 27C3 conference wiki about the GSM network. Please notice that this information is still preliminary at this point.

The wiki page also contains detailed instructions on how you can participate in the test network. I'm hoping a lot of you will bring a dedicated cellphone that you can put the 27C3 SIM inside and participate in the network.

I'm particularly excited about GPRS/EDGE support. We will be handing out official, world-wide routed, unfiltered IPv4 addresses to each and every phone. This means you are free to run port scans or other attacks (please: No DoS) over an unfiltered IP network directly into your mobile phone.

Learning how GPS _really_ works in order to truly understand RRLP

Back one or two years ago, when I first discovered the RRLP as a mechanism how operators can get very precise GPS positioning of a mobile phone (without any authentication or a way for the user to prevent or at least notice it), I was frankly speaking shocked.

We've done some experiments at HAR2009 and obtained a number of great position fixes, mostly from iPhones. The nice aspect of RRLP is that it is buried down inside Layer3 of the GSM protocol stack in the baseband processor. This is at a much lower level than all the web or App based location based services that are running in an application program in userspace of the application processor.

Now RRLP comes in a number of different flavors. What we have done so far is called ms-based positioning, where the phone works as an autonomous GPS receiver, pretty much like a personal navigation device or any hand-held GPS receiver. So the network simply asks "tell me your GPS coordinates if you know them" and the phone will respond. Some phones ask for assistance data in order to do A-GPS. But that's it.

What has been more of a mystery to me is the ms-assisted GPS RRLP mode, where the phone just performs some measurements and forwards the resulting data to the network. I never really understood the details of how it works, but always wanted to. Last week I finally found some time to do the research required to fully understand it:

The network tells the phone the exact bit timing, Doppler shift and other parameters for each of the satellites that it _knows_ the phone would be receiving given the current cell the phone is registered to. The phone then performs some measurements within very narrow time/frequency/synchronization windows, and passes back the timing of those received signals relative to the current GSM cell signal. Using this information, the actual position estimate will be completely computed inside the network, not inside the phone.

Presumably this ms-assisted mode was implemented to not have to put a full-blown GPS receiver into every phone, requiring sophisticated processing in either hardware or software. Also, this method should be much quicker as the network _knows_ all the current ephemeris data and GPS signal timing, whereas a stand-alone GPS receiver would have to take quite a lot of effort to acquire a signal from cold-start, even if there is some assistance data.

Unfortunately I don't have the time to actually implement the network side for this. It would be a fun project, but I have already way too many of them (and customers who only pay for other features in our Free Software GSM stack).

There's now a RRLP wiki page at security.osmocom.org. As short as it is it still contains more information about RRLP than I could find on any other public source on the network - except the protocol specs.

Wireshark patches enhancing IPA Abis/IP dissector accepted

The wireshark project recently accepted two of my patches related to the Abis/IP dissector. The first of them makes the TCP/UDP port numbers that are interpreted as IPA multiplex a configurable preference. This is really useful, as the actual port numbers used in production setups seem to differ from site to site (with no real standard port numbers and only some that are 'best practise'). Without this patch, in many case you always need to click 'Decode as... GSM IPA' every time you open a pcap file.

The second patch adds support for printing the debug messages that the Hay Systems Ltd. HSL Femtocell includes as stream identifier 0xDD in its Abis/IP variant.

I hope I can find some time to clean up / finish some of the other wireshark patches that we have pending for quite some time. The main problem here is that we imported some definitions from OpenBSC, which use gcc extensions and are thus not permissible for wireshark inclusion.

A US professor who was warning the Indian Government about lack of IT security in Voting machines is being deported from India

According to news reports, J Alex Halderman is refused entry into India and will be deported from the country upon entering. He is one of the authors of the study India's EVMs are vulnerable to fraud which a number of international experts on electronic voting machine security had published in order to warn the Indian government about the flaws in their voting machines.

This is outrageous. Instead of trying to keep those researchers out of the country, the Indian government should invite those experts (who are giving free advice about IT security problems) and have them do a detailed analysis and start an official investigation into why and how the existing machines could ever be used for election purposes.

It seems like the authorities in question have absolutely no clue on how proper incident response is being done. You don't get people to trust your system if you jail activists who outline flaws in voting machines and try to keep foreign experts out of the country. Trust has to be earned. And if there is some serious incident, a public investigation should be started, open to all experts in the field. Trying to cover up by ignoring results of IT security research (academic or otherwise) will not make the system more secure. All this will help is to further undermine trust in the system.

I would like to use this opportunity (and my upcoming trip to FOSS.in/2010) to call upon all my Indian friends: Don't just sit there idle and allow your government to get away with this. The public needs to know how trustworthy the voting machines are. If there are serious objections by academic experts in the field, the system needs to be updated/upgraded or even abolished altogether. Elections are the foundation of a democracy, their results cannot be entrusted to technology that has never received public and independent scrutiny.

UPDATE: It seems that according to indianevm.com, he was only held for 18 hours and later permitted entry into the country. While this is good news in general, it remains unclear why they held him for deportation in the first place, and why the Indian Electoral Commission is so nervous about anyone doing legitimate research on the security of electronic voting in India.