Harald Welte's blog

A real-world practical A5/1 attack using airprobe and Kraken

At Blackhat USA 2010, Karsten Nohl has been presenting on a practical real-world A5/1 cracking attack. For recent years, Karsten, myself and others have been speaking at various opportunities, indicating that a practical attack using readily-available information and tools from the Internet is very possible, and that it is only a matter of time for somebody actually does it.

While Karsten has focused on the actual cryptographic attack, I've been putting in some time in projects like airprobe (a GSM receiver/decoder).

Now finally, a team of friends at the new Security Research Labs (founded by Karsten) in Berlin has put the pieces of the puzzle together.

Airprobe has been extended to fully support decoding of TCH/F (FACCH, SACCH and traffic), as well as SDCCH/SACCH control channels, and to specify the timeslot and physical channel configuration from the command line. Using this, you can

decode the AGCH, wait for an IMMEDIATE ASSIGNMENT of a SDCCH
decode that very SDCCH and wait until encryption is turned on
dump an encrypted burst where you have sufficient known plaintext
use a different program to actually recover the A5/1 ciphering key
feed that key into airprobe and decrypt+decode the ASSIGNMENT COMMAND of the TCH
use airprobe to decrypt+decode that assigned TCH/F

The external program to recover the A5/1 ciphering key is called Kraken and is also available from the SRLabs website.

So what are the limitations? Well, so far this only works on non-hopping cells with a single ARFCN. The limitations are those of the receiver hardware (and SDR software), and not really limitations of the airprobe GSM decoder or the actual software tools.

In the past I would have assumed that non-hopping and/or single-ARFCN cells are rare, but in fact we can find them even inside a big city like Berlin, from at least two of the four German GSM operators. So that's why this attack is very practical, no matter what the GSMA might say.

Implementing the TCAP protocol, heading towards OsmoSGSN SS7 support

The protocol by which traditional GSM core network components interact is called MAP (Mobile Application Part). MAP itself is a user of the TCAP (Transaction Capabilities Application Part) protocol, which in turn runs on a SS7 protocol stack (i.e. SCCP over MTP or M3UA or SUA over SCTP).

For those users of OpenBSC who have a need to interoperate with other GSM networks (roaming), the circuit-switched part of OpenBSC has so far relied on the use of a proprietary MSC (by means of the A interface). This closed MSC then talks MAP/TCAP/SS7 to roaming partners.

However, on the GPRS front, we now have OsmoSGSN. However, as opposed to the BSC on the circuit switched side, the SGSN directly interacts with the core GSM network components (both of the home network and the roaming partners).

So in order to run OsmoSGSN interacting with existing HLRs, we need to add a MAP/TCAP/SS7 interface to it. Once this has been done for the SGSN, we of course can do the same for the MSC-part that is currently integrated with OpenBSC.

As there are existing implementations of SCTP (inside the Linux kernel) and SUA (sualibrary), TCAP is the next step in the protocol stack that needs to be implemented. I've been digging into TCAP for the last week(s), and believe I finally understood every part of its operation.

You can think of TCAP as something that facilitates the transport of request-response type transactions over a datagram oriented transport layer. It intends to have lower overhead than a connection-oriented service (e.g. establishing TCP sessions) and supports features such as aggregating multiple user-messages (called components) in a single actual transport-layer message. The idea is to reduce the overhead of message headers and routing.

TCAP is (unfortunately) specified in ASN.1 and thus requires significant effort to parse and construct. Right now I'm using Lev Walkin's asn1c ASN.1 C code generator to generate the parser and constructor functions. The actual TCAP protocol logic is once again implemented in plain C, using the various concepts and utility functions established in OpenBSC (and now part of libosmocore).

The implementation is making good progress and I hope I can do some early testing in about a week from now, and successively move straight to the MAP protocol, implementing at least those parts that we need for GPRS authentication and attach / routing area updates.

Major update in OpenBSC GPRS/EDGE support

Through the last couple of days, I've been in extreme bug-squashing mode for the GPRS/EDGE code base in OpenBSC (mostly the OsmoSGSN program). I'm now at a point where I can reliably establish PDP contexts and access the Internet from a variety of different phones with different baseband chipsets and GPRS protocol stack implementations. All so-far-known bugs regarding fragmentation/reassembly, sequence numbering and other issues have been fixed. There definitely are plenty more, but we first need to find them.

Since it's working reliably now, it's quite fascinating what the various phones do after connecting to the GPRS network. Like Windows Mobile phones sending Netbios Name Service updates (and requests), which I think is funny considering that they are sent to a network that is typically considered to be the public Internet.

But to be fair and not anti-Windows, my Google/Android G1 also makes some https connections back to Google - and I don't know what they are for [yet].

In any case, with OpenBSC, OsmoSGSN and OpenGGSN anyone interested in doing true security (and privacy) research with mobile phones is now able to do so. Using those programs, you can run your own GPRS+EDGE network and can see first hand what your phones are doing on a cellular network, what kind of data they are sending back home. In this setup, there is no packet filtering, NAT, deep packet inspection and no intrusion detection systems between your PC and the IP stack on your phone.

The reason why you see paging by IMSI in real-world GSM networks

During my work on airprobe and OsmocomBB I've been wondering why you see paging by IMSI in real-world GSM networks.

A quick recap: The IMSI is the world-wide unique serial number of your SIM. Since it is easy to identify and track people, the TMSI was introduced as a temporary identifier that is frequently re-allocated over encrypted channels. The only reason for the TMSI to exist is to prevent tracking of a subscriber by watching where his IMSI appears on the paging channel.

According to the theory, the IMSI is only used when first registering to any GSM network. At that time, a TMSI is allocated to the SIM card in the phone, and this TMSI is used for the next transaction(s). Later, this TMSI is re-allocated and re-allocated, but the IMSI shouldn't show up again in any paging requests.

Even if you switch mobile networks (i.e. in the roaming case), you would once send the IMSI as part of a LOCATION UPDATE REQUEST or IDENTITY RESPONSE, but the network has no need to page the SIM by IMSI.

So far the theory. If you look at the Paging Channel (PCH) of cells in real-world networks, you see a significant (10-20%) amount of paging requests that contain paging by IMSI. This seems strange on first sight, given the theory described above.

I have the following plausible explanation for this:

The VLR keeping the IMSI-TMSI mappings doesn't have non-volatile storage. This means at a VLR restart, all the TMSI allocations will be lost, and the network has to resort to paging by IMSI.
The VLR has a limited amount of RAM, which can store a limited number of IMSI-TMSI mappings. Especially if the operator is interested in saving money, the amount of memory is insufficient for all subscribers in the network. This means, the VLR will expire some old entries in the mapping table to store new entries. Thus, mobile phones whose last transaction with the GSM network was relatively long ago are likely candidates for such VLR expiration. Once a phone for an expired entry needs to be paged again, paging will happen by IMSI.
Last, but not least: GSM networks do not page a phone by the last known cell, but by the last known location area of the phone. A location area might be relatively big. This means that at any cell you will see a lot of paging messages, even for phones that are not even anywhere near this cell. If there is no response within the location area, the MSC might decide to do paging on a larger radius, possibly the entire MSC area. Since such MSC-wide paging is likely to occur for phones that haven't shown activity for a long time (and thus might have moved or disappeared without properly unregistering from the network), those are the exact same phones for which the IMSI-TMSI mappings have expired from the VLR. Thus, the rate of paging-by-IMSI looks disproportionately high.

So the relatively high percentage of paging by IMSI vs. TMSI should not be taken as a measurement with regard to the total number of transactions or even the total number of subscribers. It is simply the mechanics of the network resulting in a distortion of those figures caused by phones that have never properly unregistered from the network.

Back from OpenBTS workshop

I've just returned back from the First OpenBTS workshop held by David Burgess and hosted by Dieter Spaar in south-east Bavaria (Germany). While I'm not involved with OpenBTS so far (except from using it occasionally), I still thought the community surrounding Free Software / Open Source in the GSM field is small enough to make me participate.

On the request of the participants, I also did a short demonstration of both OpenBSC and OsmocomBB. And just like I managed to crash OpenBTS by accidentally sending invalid messages, my OpenBSC demo crashed at some point [due to a not-yet-known bug regarding SMS delivery. I suppose the intrusive changes of the BSC/MSC split are to be blamed for that. But I don't mind, we need that split...

I definitely had a great time meeting the participants of the workshop. There definitely is a very diverse crowd with equally diverse reasons for their interest in using and/or deploying OpenBTS.

Finally, there was a chance to discuss the need for a common 'application interface' in both OpenBSC and OpenBTS. Using that interface, external applications (e.g. implementing USSD or RRLP) could be written in a way to work with both OpenBTS and OpenBSC. I hope we can get started on this soon and remove another bit of fragmentation in what is already a fairly small special interest community...

Given the excellent weather conditions, the motorbike ride to and from the venue went fine - despite being at 650 km distance from my home.

Adding frequency hopping support to OpenBSC

During the last couple of days, I've been adding the bits required to support frequency-hopping BTSs in OpenBSC. Now everything looks great in the protocol traces - but unfortunately it still doesn't work, at least not with the Siemens BS-11 that I have access to.

Will continue to try to make it work. The big advantage of having a hopping BTS under our control is that we can define the hopping sequence - something quite useful once we get to the point where we'd like to add frequency hopping to the telephone-side stack (OsmocomBB).

The good news is that I had to fix lots of bugs in the A-bis OML dissector for wireshark that I wrote some time ago. It's now much more complete and definitely a big step further towards eventually getting it included in wireshark mainline.

A fairy tale about ICCIDs, IMSIs and iPads

One of the big news of the last week is AT&T's leak of 114,000 iPad customer records including the e-mail address and ICCID

While that leak is certainly a big issue in itself, there are some people, most notably Chris Paget, who claim that this is much more serious than generally assumed. The main claim here seems to be that ...in order to translate an ICCID into an IMSI, you need to query the HLR.

I have been reading GSM protocol specifications on every level for the past years, and never have I seen the ICCID being mentioned anywhere. The GSM specifications do not require this information to be stored in the HLR, and the MAP protocol (used on the C interface between MSC and HLR, see 3GPP TS 29.002) does not even know how to encode/specify it.

Also, there is no technical need for it. The ICCID is never used nor needed in any part of the GSM protocol. Also, the GSM network typically doesn't store any information that is not absolutely necessary for its operation. The only identifier of a SIM card that the network protocols care about is the IMSI.

So unless the US operators in question have either some kind of proprietary extensions to both their HLR and the MAP protocol, there is to the best of my knowledge no way how you can relate the ICCID to the IMSI.

And thus, as a result, the IMSI-catcher attack described will not work since you don't know the IMSI of the SIM card (associated with the customer record) that you want to catch.

If anyone can show me hard technical facts about ICCIDs being used in the HLRs of the operators in question, I am happy to post here I was wrong. Otherwise, I would hope everyone else could also come down to the hard technical facts, i.e. which particular MAP message is used for this alleged ICCID-to-IMSI query.

UPDATE: As some people have discovered, the three US operators themselves have decided that they use the same number to generate both the ICCID and the IMSI. So if you have one, you can compute the other. No need for HLR access, no need for the MAP protocol. So the information leak is in fact unrelated to the GSM protocol but simply a matter of how unfortunate those particular three operators assign their unique identifiers.

Wanted: Packet traces of the MAP+TCAP based C/Gc interface

I'm looking for any example pcap files (packet traces) of the so-called "C" and "Gc" Interfaces, i.e. the interfaces of the HLR (Home Location Register). If anyone has such pcap files or can generate some, I would very much appreciate it.

The protocol levels I'm interested in is SCCP, TCAP and MAP. The lower layers (MTP) are not important now.

Specifically, I'm looking for traces of any of the following MAP operations:

updateLocation
cancelLocation
restoreData
sendParameters
updateGprsLocation
sendAuthenticationInfo
purgeMS
sendRoutingInfo
sendRoutingInfoForSM
reportSM-DeliveryStatus
readyForSM
noteSubscriberPresent
sendRoutingInfo
anytimeInterrogation
statusReport
{register,erase,activate,deactivate,interrogate}SS
sendIMSI
sendRoutingInfoForGprs
insertSubscriberData
deleteSubscriberData
checkIMEI

If anyone is able to produce the respective traces, I would appreciate it a lot. I'd need them as examples to make sure I fully understand the TS 09.02 in combination with Q.77x before actually starting to implement it...

First functional HTTP transfer in my own GPRS/EDGE network

Today marks the day where finally, after months of (non-full-time) work, I have made the first successful HTTP connection through my own GPRS/EDGE network.

Ever since we started to seriously get into OpenBSC to run GSM networks, I've been looking forward to running GPRS networks, too. What most people don't know: GPRS is radically different from GSM. It basically only shares the frequencies and timeslot architecture of the physical layer, while having it's own layer1, layer2 and various other protocol layers. Also, its signalling and data completely bypass the usual BSC and MSC components of a GSM core network.

So what I've been working on is now called OsmoSGSN. Using OpenBSC, you can provision an ip.access nanoBTS (or any other BTS with a Gb Interface) to broadcast the GPRS/EDGE capabilities to the handsets. The BTS then establishes the Gb interface (consisting of NS and BSSGP) to the SGSN.

The SGSN takes care of GPRS Mobility Management (GMM) and Session Management(SM) in the signalling plane, as well as the LLC + SNDCP protocol layers. On the other end, it uses the GTP protocol to connect to a GGSN. In our case, this is the OpenGGSN project which I recently adopted.

OpenGGSN then creates a virtual network device (tun0), through which the actual IP packets are entering/leaving the GPRS network. From there you can route and/or NAT them just like any other IP packets.

The current code is still incomplete in many places and known to be unstable. But it's really rewarding that after a long time of development, layer after layer of the stack, finally actual TCP/IP can be provisioned to phones.

The code is in the current master of the openbsc git repository, but I don't think there's much point in trying it just yet. I suppose in a week from now things should be much more stable.

OpenGGSN Version 0.90 released

Only three weeks ago I blogged about OpenGGSN, a seemingly-abandoned Free Software implementation of the GGSN node of the GPRS core network.

Things have developed quite a bit ever since. As the original author didn't respond to any of my mails and sourceforge.net was not able to reach him either, they have approved my the 'abandoned project takeover' (APT) request.

I've now switched the project from CVS to git, removed links to the non-existing openggsn.org homepage and released version 0.90, containing nothing less than a fix for remote DoS vulnerability that was pending for more than 5 years.

So far I'm only exercising the PDP context activation/deactivation parts of OpenGGSN from OsmoSGSN (the SGSN I write as sister-project to OpenBSC), but I hope we can use OpenGGSN in a production GPRS network soon...

Doing even more encapsulations than the GPRS Gb interface already has

Back in October 2009, I blogged about the incredibly deep protocol stack on the GPRS Gb interface between a BSS and the SGSN.

Today I had the pleasure of implementing an even more odd variant of the Gb interface, where NS does not get encapsulated in UDP/IP/Ethernet, but in FrameRelay/GRE/IP/Ethernet. The total protocol stack thus then looks like: HTTP/TCP/IP/SNDCP/LLC/BSSGP/NS/FR/GRE/IP/Ethernet, with an optional PPP between IP and SNDCP. If anyone does the math to calculate the total protocol overhead, please let me know[...

The reason for that oddity is apparently that there are Cisco and other routers that can encapsulate Frame Relay in GRE. So using a old circuit-switched SGSN with E1 interfaces and such a router, you can convert from Frame Relay on E1 to Frame Relay on GRE/IP/Ethernet.

Both the Gb Proxy as well as the upcoming OsmoSGSN use the same NS implementation, i.e. they can now both talk NS/FR/GRE and the NS/UDP variants - even at the same time, as the encapsulation is a property of each individual NS-VC.

Back from a week of GSM/GPRS protocol coding/testing in Iceland

With only 16 hours delay (which isn't all that much considering the volcanic ash situation) I arrived back in Berlin from one week of OpenBSC software hacking, particularly on the GPRS side of things.

It was really nice to see to what extent OpenBSC software is already used at On-Waves, providing GSM and now also GPRS services to thousands of users.

My work was mostly focused on the Gb-Proxy, a multiplexer/proxy for GPRS Gb links running the NSIP (NS-over-IP) protocol. It combines elements of the idea of a network address translator with that of a proxy, combined with a little bit of packet-based routing. This really makes me feel like I'm back to packet-switched networking, which is great. Especially the fact that we use the VTY code from quagga and its interactive command line sometimes lets you forget that you're not working with classic TCP/IP routing daemons or the like ;)

Aside from that, I continued my work on the upcoming OsmoSGSN, using which we will be able to run an autonomous GPRS network with no dependency on external proprietary components. In this setup, the PCU in the BTS connects over Gb/IP to OsmoSGSN, which then talks over GTPv1 to the OpenGGSN.

Also, work was spent on an abstract rate_counter implementation (now part of libosmocore). The idea is to have a counter that will count certain events (like number of packets/bytes, number of link failures, etc), but also keep a small history about how many of those events happened in the last second, last minute, last hour and last day. There is also common code to store those counters in the database, as well as to print them on the VTY. The new counters are so far only used in the GB-Proxy, but they will soon likely be added to OpenBSC (bsc_hack) and other programs of our Free Software GSM network portfolio.

Heading for 4 days of Iceland to work on OpenBSC GPRS

Having just returned from Croatia the day before yesterday, I'm about to head on a four-day trip to Iceland, where I'll be doing some testing and bug fixing of the current OpenBSC GPRS support at On-Waves.

Zecke is also going to be there, working on other aspects of OpenBSC. This will make the trip even more fun!

CECT C3100: Not a phone, but a flashlight with integrated phone

I've seen many [mobile] phones in my life, but nothing like the CECT C3100 so far. It's made of the cheapest hard plastic, like cheap kids toys. In addition to the phone keyboard, it has a mechanical switch on its side. If you slide that switch, five powerful bright white LEDs at the top of the phone will turn the entire device into a flashlight.

However, it is one of the most basic phones with one of the older/simpler MTK baseband chips inside (MT6223). Also, as we have determined by a PCB delamination analysis, the test pads next to the MT6223 really are its ARM JTAG pins.

JTAG is something not commonly found in MTK phone designs, but it is definitely a big win for bootstrapping any system-level software such as drivers on the unit.

Right now I don't have the time to work on MT6223, we still have many issues to fix in the current Ti Calypso code. But I can't wait to find time to see if we can extend our hardware support to MTK GSM chipsets...

GPRS progress in OpenBSC

In recent weeks, I have been able to pick up my work at GPRS support for OpenBSC again. What has been done is:

Add OML support to configure nanoBTS for EDGE
Add RR (System Information) support to indicate EDGE support
Make a OpenBSC + nanoBTS setup inter-operate with an existing SGSN
Develop a proxy that can aggregate the Gb-interfaces of multiple BTS into one Gb link to a real SGSN. This way the SGSN has only one Gb link for all the cells under the control of a BSC, as opposed to one Gb link for each and every BTS

What I'm working on now is the actual SGSN implementation. The SGSN is mainly responsible for GPRS mobility management (GMM) and for terminating the Layer2 (LLC) protocol from the MS. This is very different from circuit-switched GSM, where Layer2 (LAPDm) already terminates at the BTS.

The layering stack of GPRS is a real nightmare, I am sure I have indicated this in this blog before. The Current OsmoSGSN code (available from the regular openbsc.git repository) implements the NS, BSSGP and LLC layers, as well as the basic GSM 04.08 GPRS Mobility Management messages like GPRS ATTACH/DETACH and ROUTEING AREA UPDATE. The LLC code is still somewhat limited, but for the time being it is sufficient.

What drove me crazy for a couple of days is the number of parameters that are exchanged at PDP CONTEXT ATTACH time. There are no less than 26 different quality of service (QoS) parameters negotiated (see struct gsm48_qos at the bottom of this link), each of them from a wide range of values. It's almost impossible to imagine more than 1% of all the possible combinations have ever been used in production networks. The QoS parameter negotiation works by the phone sending a list of requested parameters, to which the SGSN responds with its selected parameters. My first thought was: Lets be smart and simply echo back the QoS parameters - the phone must accept what it has requested. That didn't work either: While the QoS structure is the same in both ways, the actual values in uplink and downlink directions are encoded differently. Who on earth defines such an encoding?

Next item was the XID exchange which is at the boundary between LLC and the SNDCP (Sub-Network Dependent Convergence Protocol). It works like this: The phone proposes an endless list of parameters, which the SGSN can evaluate, and then depending on the parameter type either negotiate up or down. According to the spec, sending an empty XID response should mean "I agree with all your parameters". However, at least those phones that I tested were not happy with that. So I decided to simply send back the entire XID block to the phone. And believe it or not, as opposed to the QoS parameters, this time it even worked

So now I'm facing the implementation of the actual SNDCP-to-GTP interworking, which is nothing less but the guts of the SGSN. GTP is the protocol used on the GGSN side. At least this time, GTP is sent directly over TCP or UDP, i.e. the stacking inside the SGSN is only one layer deep, while on the Gb-interface it is four (NS,BSSGP,LLC,SNDCP).

SNDCP interacts with the GPRS Mobility Management, GPRS Session Management (both GSM 04.08 over LLC), the GTP interface to the GGSN, as well as other parts. I expect many pitfalls on the way to getting it working, and given the complexity involved I have already decided to stick much closer to the specification than I usually did with the OpenBSC work. This means properly implementing all the state machines with all their transitions, exceptions and timers. I'm sure it's going to be "fun". The good part of it is: Most of the SGSN will be re-used once we finally get around adding support for 3G/UMTS/WCDMA cells.

OpenGGSN - An open source GGSN implementation

As a friend pointed out to me at exactly the right point in time: there already is an Free implementation of a GGSN. In case you don't know what a GGSN is: It is one of the two core components of a GPRS network. So, in order to extend a OpenBSC GSM network with GPRS support, there are two components required: The SGSN (on which I'm working currently, project name OsmoSGSN), and the GGSN. Due to the good news about OpenGGSN, I'm quite confident that I will not need to implement the latter part.

OpenGGSN is not only a Free Software implementation of the GGSN, but it is also licensed under GPLv2, making it compatible with the OpenBSC codebase (which is GPLv2 or later). This means I will be able to link the OpenGGSN-provided libgtp library (implementing both sides of the GTP protocol between SGSN and GGSN) from OsmoSGSN, further reducing the amount of work required to get this working.

However, despite seeming like a fairly advanced/complete implementation of the GGSN specification: OpenGGSN seems like a project that was abandoned many years ago. The latest CVS commit is from 2005, and all of the bug fixes that people have submitted to the bug tracker have not been merged. The homepage is defunct, and the openggsn.org domain name seems to have been expired (and grabbed).

I've tried to contact the author by e-mail about his intentions for the project, let's see if there is any response. Meanwhile, I have generated a git repository from the OpenGGSN CVS repository at sourceforge and applied all the pending fixes to a local branch. See my related mailing list post for more information.

Working on GPRS support for OpenBSC again

This has been on my TODO list for at least the last six months or so: Growing the experimental GPRS branch of OpenBSC into something more useful.

Right now, you can use OpenBSC with a GPRS-capable BTS - but only if you have an existing SGSN to serve the Gb interface of the BTS. This somehow defeats the point. We want to offer a 'GSM network in a box' solution, where no other non-free Software is required to run a fully functional small network.

So right now I'm cleaning up the 08.16 (Network Services) Implementation, and will move my way up through the existing 08.18 (BSSGP) and LLC code that I wrote some time ago.

With some luck, in a couple of weeks we should be able to run a self-sufficient combined GSM + GPRS (+ EDGE) network out of OpenBSC.

Paper: Anatomy of contemporary GSM cellphones

During the last days, I was working on an introductory paper on how a GSM cellphone actually works. It is titled Anatomy of contemporary GSM cellphone hardware and should provide a good technical text for anyone who generally is into technology and understands a bit about both software, computer architecture as well as radio, but who still feels like he has no clue what is actually happening inside the phone, particularly the hardware side.

The text does not cover the GSM protocols itself, as there is much more information available on this already.

Feel free to let me know what you think, I'm always happy to extend or clarify it based on your feedback. I hope some people find the text useful.

OpenBTS workshop in June (Pfarrkirchen/Germany)

If you've always been interested in Open Source / Free Software GSM using OpenBTS, you might be interested in the upcoming three-day OpenBTS workshop in Pfarrkirchen/Germany.

The workshop is held by OpenBTS project leader David Burgess, and will cover subjects ranging from OpenBTS installation, configuration and deployment down to actually extending OpenBTS with some custom code.

p.s.: Please don't misunderstand, the workshop really is about OpenBTS not OpenBSC - though there might be OpenBSC folks hanging around ;)

OsmocomBB now performing location updating procedure against GSM cell

I haven't had much time for blogging recently, too much exciting work going on at OsmocomBB:

we now have simplistic support for Uplink (transmit) on SDCCH/4
we have a minimal Layer2 (LAPDm) implementation
we can send LOCATION UPDATING REQUEST to the network, and receive the respective response
there's wireshark integration, i.e. all packets on the L1-L2 interface can be sent into wireshark for protocol analysis

There are still many limitations, but this is a major milestone in the project: We have working bi-directional communication from the phone to the network!

The limitations include:

The cell has to use a combined CCCH (SDCCH/4 on timeslot 0)
The cell has to use no encryption/authentication
The layer2 is not finished, especially re-transmissions will not work yet
There's no power control loop yet
There's no timing advance correction

However, most of those are more or less simple we know what needs to be done, its just a matter of getting it done kind of tasks. There are no big unknowns involved, and particularly no further reverse-engineering of the hardware is required.

Also, the existence of a stable bi-directional communications channel between the network and the phone means that anyone interested in working on the higher layers can now actually do so. Completing and testing layer2 as well as RR/MM/CC on layer3 is a major task in itself, and it definitely requires the lower layers to be there.

The other good part is that development of layer2 and layer3 can happen entirely on the host PC, where debugging is much easier and there's no need for cross-compilation and we can use all the usual debugging options (gdb, valgrind, ...)

I'm now almost heading off for holidays (starting March 10), so don't expect any major progress from me anytime soon. I hope other interested developers will be able to take it from here and fill in some missing gaps until I'll get back.

Looking for documentation on sunplus SPMA100B

In the Motorola/Compal C155 phone supported by OsmocomBB, we have found a ringtone melody chip called SPMA100B from sunplus.

As strange as it might seem, this is the only part used in the phone for which we have not been able to find any kind of programming information. So if you know anything about how to program this part from software (register map, programming manual, ...) please let me know!

And no, we don't need electrical/mechanical data sheets, thanks :)

Restructuring OpenBSC and OsmocomBB code

I've spent the better part of the day with , renaming files/functions/include paths, Makefiles, autotools and the like.

The result of this is a new sub-project called libosmocore that gathers all the shared code between the network-side GSM implementation OpenBSC and the phone-side implementation OsmocomBB. The library is portable enough that it can run on a proper OS (like GNU/Linux) but also be cross-compiled to work on the actual phone without any OS.

On the other hand we now have a master Makefile in OsmocomBB to build libosmocore for host PC and target (phone), as well as the osmocon and layer2 host programs and the phone firmware itself.

Let's hope I can now return to writing actual code...

Announcing OsmocomBB: Free Software / Open Source GSM Baseband firmware

Last, but not least, I am proud to announce the OsmocomBB project publicly. During the last 7 weeks, a small group of skilled developers has been working on this

It has now reached a point where we can

scan the spectrum for the strongest signal GSM channels
lock onto them and performing AFC (automatic frequency control)
decode the SCH burst to obtain BSIC and GSM frame time
decode the BCCH of the cell, pass it over to the host PC and feed it into wireshark

Since this in itself is a valuable and useful milestone of the project, it was the ideal opportunity to take this project public.

There's still a lot of work to be done in many areas. Most of them are not even related to the GSM air interface. So if you're familiar with C development on an ARM7TDMI based microcontroller, know your way around I2C and SPI, are familiar with the GNU toolchain for ARM and want to help us out: Please join the baseband-devel mailing list right away!

In six weeks from bare hardware to receiving BCCHs

After six weeks of full-time hacking, with the help of a few friends, we have made it to receiving actual BCCH data from a GSM cell.

So what does this mean? As I have indicated publicly at the 26C3 conference: Now, that we have managed to create a working GSM network-side implementation (OpenBSC) during the last year, we will proceed to do the same with the phone side.

Initially we spent quite a bit of thinking on building our own custom hardware. But while planning for the first prototype, we realized that it would simply distract us too much from what we actually wanted to do. We don't want to take care of component sourcing, prototype generations, quality assurance in production, production testing, etc. -- All we want is to write a Free Software GSM protocol implementation for a phone.

Unfortunately (as usually in the industry), the silicon and device makers do not publish sufficient documentation about their devices to enable third-party developers to go ahead and write their own software: The never ending problem of Free Software in many areas beyond more-or-less standardized hardware like in the PC industry.

So, if you want to write Free Software for such a device, you have two options:

Reverse engineering the existing hardware and writing your code based on that information
Building your own hardware and then writing the software you wanted to write.

I've been involved in both approaches multiple times while looking only at the application processor (the PDA side) of mobile phones: OpenEZX and gnufiish are two more or less abandoned projects aimed at reverse engineering. Openmoko was the project that had to build its own hardware as a dependency to be fulfilled before writing software.

If you're not a company and don't want to sell anything, the reverse engineering approach looks more promising. You can piggy-back on existing hardware, don't need to take care of sourcing/production/certification/shipping and other tedious bits.

If you are a company and want to generate revenue, then of course you want to build the hardware and ship it, as it is what you derive your profits from.

So, just to be clear on this: Neither OpenEZX, nor gnufiish nor Openmoko were ever about writing Free Software for the GSM baseband processor, i.e. the beast that exchanges messages with the actual GSM operator network. But this is what we're working on right now.

It's about time, don't you agree? after 19 years of only proprietary software on the baseband chips in billions of phones, it is more than time for bringing the shining light of Freedom into this area of computing.

To me personally, it is the holy grail of Free Software: Driving it beyond the PC, beyond operating systems and application programs. Driving it into the billions of embedded devices where everyone is stuck with proprietary software without an alternative. Everybody takes it for granted to run megabytes of proprietary object code, without any memory protection, attached to an insecure public network (GSM). Who would do that with his PC on the Internet, without a packet filter, application level gateways and a constant flow of security updates of the software? Yet billions of people do that with their phones all the time.

I hope with our work there will be a time where the people who paid for their phones will be able to actually own and control what it does. If I have paid for it, I determine what software it runs and when it send which message or doesn't.

Oh, getting back to what our work: It will be published as soon as it is sufficiently stable and fit for public consumption. You won't be able to make phone calls yet, but we'll get there at some later point this year.

Symbian is Open Soruce - Really?

In recent news, the Symbian Foundation announced that "All 108 packages containing the source code of the Symbian platform can now be downloaded from Symbian's developer web site". This is great news!

This morning I tried to look at the parts most interesting to me: phonesrv (implementing call engine, cell broadcast and SIM toolkit APIs) and poc (implementing push-to-talk). Their pages don't have the usual "source code" tab at the bottom right which links to mercurial and tarball download pages!

Either I'm too stupid, or I am unable to find any source code for those two components. I'm quite sure something essential like the API's for making phone calls are considered part of the Symbian platform. So how does that match with the statement that all packages containing the Symbian platform can now be downloaded?

Sorry for new blog updates

I've been busy day and night, hacking away on my latest pet project in the GSM field. In fact, it's been a long time since I've been able to dedicate so much time and energy into one particular project, without many distractions at all. The project is now finally looking quite promising and making nice progress throughout the last three weeks.

If progress continues, I hope in another week I'll be able to reveal what this is all about. I haven't felt this level of excitement since the early days of Openmoko :)

Illusions about MagicJack at CES

Many people have pointed out the MagicJack Femtocell product that has been announced at CES. I cannot really understand the big hype and news about it. Why? read further...

On the technical side, there is hardly anything new. Using projects like OpenBTS or OpenBSC, you can run your own GSM network and connect it to VoIP. Sure, the retail price of the MagicJack is much lower, but that's the economics of scale. As soon as OpenBSC support for one of the recent femtocells is done, we also have a much lower cost solution to the same problem.

On the legal and business side, I can see many problems for MagicJack:

To operate equipment in the GSM or 3G spectrum, you need a license. Since a nationwide GSM operator license is very expensive in about any country of the world, it is economically not an option. Selling the MagicJack devices without a license and leaving the spectrum license to the user will not work, or at least not long, since regulatory authorities and commercial operators are not going to let anyone deploy systems that interfere with their networks.
If you keep the Operator's SIM in the phone, and use that SIM on your own network you might at least violate the terms of services of the operator. The SIM card normally belongs to the operator, and it is part of the users business relationship with that operator. As such, you can not really use it with other networks. Sure, if you do this at home with your OpenBTS/OpenBSC installation, nobody will care. But if somebody is doing this commercially, and in a way that affects the sale of talk time of the regular operators, again it will not take long until the commercial operator will sue you.
Security. If you run your phone with a "foreign" SIM, you do not know the Ki on the SIM and thus cannot do cryptographic authentication and/or encryption. This is a big security issue. It is once again not an issue in your personal testbed setup, but it is going to be one if you do this at large scale as a mass market product.

So, as you can see: It's neither technically something exceptionally new, nor is it something that has a very promising business or legal outlook. The only way how a product like this would work is if it is authorized by the respective operator. But why would the operator authorize something that will take talk time away from his network and thus his revenue stream?

Planning for a GSM development board

I've finally found enough spare time to work on detailed plans for a GSM development board. The idea here is to have a 100% open hardware design with 100% free software that provides an inexpensive platform for GSM related R&D.

Initially the focus is on having a board that can behave like a GSM cellphone, next steps would be to have a multi-channel frequency-hopping receiver, and finally the option of using it as a BTS.

The idea is fairly simple: Take a commercial off-the-shelf analog baseband and RF circuit for GSM and attach it to a general purpose DSP, add some glue logic and go ahead. But the devil is in the details:

You want something where you can still find the parts on the market, but which still has sufficient leaked documentation that you can write an open source driver for it.
The requirements for a MS and a BTS are quite different. A phone never needs to continuously receive and transmit in all timeslots, e.g.
The requirement for a multi-channel frequency-hopping receiver is mainly a high number of receiver circuits, so the solution needs to scale to a larger number of receivers.
The analog baseband circuits often have quite obscure control interfaces which need to be driven. Custom peripherals in programmable logic (CPLD/FPGA) are required.
The TDMA nature has strict timing requirements. Normal processors and software-based mechanisms are not sufficient to trigger the required events and their sequencing at a high enough precision.

Anyway, there is sort of a first plan now, and the next weeks and months will be spent with refining the plans and building a first proof-of-concept prototype. Once that has proven to work, we want to go ahead with finishing the design for a real board, to be manufactured in sufficient quantities for interested parties.

Unless you have worked in GSM phone or base station hardware design or have a similar level of EE and DSP skills, please refrain from contacting me right now about how to join the project. We want to focus on getting things going first, then make a public release at a point where there is something that works sufficiently well that we can support a larger community of developers.

2010-01-02 / 2pm CET: Radio Interview at Deutschlandradio

The German radio station Deutschlandradio Kultur has invited Constanze Kurz (46halbe) and myself for interviews during today's Breitband radio show. We'll be talking about the 26C3, the Chaos Computer Club and - of course - GSM [in]security.

OpenBSC now has handover support

So far, OpenBSC already implemented mobility management, i.e. keeping track of which location area a mobile phone is locate in. However, this only works during idle mode, i.e. while there is no actual phone call in progress.

Throughout the last week, I've been working on getting real handover support into OpenBSC. This is now actually working very well! You can move from one cell into another cell while your phone call continues just like it is supposed to do.

The signalling part is actually not all that hard to implement. However, it has some dependencies on things like measurement reports, which in turn require us to send proper neighbor lists, which in turn requires proper generation of system information messages, etc.

The actual order of events in a successful handover case is as follows:

OpenBSC send the neighbor cell information in the BCCH and SACCH
OpenBSC regularly receives measurement reports from the phone, where it tells us how well neighbor cells are being received. OpenBSC then averages those measurements and decides
allocates a channel on the new BTS
waits until the BTS acknowledges this
sends a HANDOVER COMMAND to the phone through the old BTS
waits for HANDOVER ACCESS bursts and a HANDOVER COMPLETE on the new BTS
closes the old channel on the old BTS
takes care of re-routing the voice channels

As indicated, the signalling part was relatively easy, and once the measurement processing and neighbor lists were in place, this worked really quick. What turned out to be a much bigger PITA was the handling of the actual voice streams.

Let's assume you have a call from A to B, where B is changing cells and now becomes B*. In this case, after switching cells, the speech frames from A need to be re-routed to B* instead of B. That's simple and works very easy. In the other direction, switching off B is easy. However, a completely new channel B* suddenly sends speech frames to A. In case of classic TRAU frames on E1 that is simple as they don't have any context.

In the case of ip.access nanoBTS, the speech frames are transported using RTP. Changing the source of your stream will change its CSRC (synchronization source identifier), timestamps and sequence numbers. The receiver in BTS A is not happy at all about this. So with handover, it is no longer possible to send RTP streams directly between BTS's, but OpenBSC's RTP proxy needs to process the RTP packets and hide the details of the changed source.

This is further complicated by the fact that during handover you are losing speech frames, somewhere between 10 and 40 in the cases that I've seen. This means that when sending the new RTP frames from B*, the sequence number and timestamp needs to account for those lost frames, i.e. incremented by the respective loss count. Otherwise the RTP receiver in BTS A will think it is only receiving old frames and will discard all of them.

Luckily, all of this seems to have been sorted out now and I'm confident we will have actual full handover at the GSM network at 26th annual Chaos Communication Congress in a few days from now. We'll be running 3 BTS's with a total number of 5 TRX's inside the conference building.

German National Education and Research Network reports on OpenBTS and OpenBSC

Issue 77 of the regular publication "DFN Mitteilungen" of the German National Research Network (DFN) reports on Open Source software for GSM networks, specifically OpenBTS and OpenBSC.

I'm happy to see that at least some parts in academia are now discovering this software and use it for research purpose. That's great news!

GSM and UMTS: The Creation of Global Mobile Communication

There is yet another really exciting book that I've been reading lately: GSM and UMTS: The Creation of Global Mobile Communication. It's a book on the history of GSM. From the early days at CEPT, through the creation of ETSI and the GSM MoU Organization, the 3GPP, ...

It puts the development into historical context, indicates who were the key participants at that time, political aspects of the European PTTs that initiated the project, the role of the European Commission, etc.

I've always been looking for this kind of information online anywhere, but there really is nothing that provides any level of detail. Wikipedia e.g. has only two paragraphs (which I believe to be even partially incorrect) on GSM's history. Contrast that with the many writings on the history of the Internet.

The book is accompanied by a CD-ROM with many old meeting notes and other documents from the various stages of the GSM development process.

Re-discovering the marvels of Nokia DCT-3: Blacksphere, MADos & Co.

About 4-5 years ago I first discovered Project Blacksphere, a group of hackers who were working on reverse engineering debug interfaces, as well as the actual phone firmware and hardware of Nokia DCT-3 phones like the 3310. Unfortunately, those projects have meanwhile been discontinued and seem to have died off.

When I last looked at that project, the status was still very limited with regard to the actual GSM side of things. You could run MADos on your phone and run some games inside it. Sure, being able to use the battery charger, keypad, LCM, etc. from your own software on an undocumented device is already great achievement, but if I want a small device without GSM then I just simply use any random PDA or build something myself.

The point about reverse engineering an actual phone is exactly to get what you cannot get from any other piece of hardware: Get access to the lower layers of the GSM protocol stack. Since MADos still looked quite far away from that, I didn't find it particularly interesting at that time.

Today I found a mirror of the project blacksphere, and discovered that apparently they had come much further with reverse engineering the interface between the DSP and the CPU, which is the interface between layer 1 and layer 2 in the GSM stack. If you fully understand that interface, you can write your own GSM stack on the phone and have a true open source phone. The code and information available is not quite at that stage at yet, but very close!

So since I have some 3310 phones (I constantly use them for OpenBSC testing) and the FBUS and MBUS cables, I am definitely going to play a bit with MADos and nlib in its latest known state. It might be the easiest way to write a MS-side layer2 + layer 3 GSM protocol implementation and put it onto an existing Layer 1.

OpenBSC: System Information + Rest Octet generation

During the flight to Bangalore I kept working on the system_information branch of OpenBSC. This branch has been lingering in git for quite some time, but I haven't yet felt confident enough to merge it into the official master.

In OpenBSC so far, the SYSTEM INFORMATION messages (type 1 through 6) are not really generated by actual code. Rather, we use some templates that are patched here and there with actual operational parameters such as the ARFCN of the current cell. This has been easy for the very early start of the project, but it has started to become more of a problem lately.

One example are neighbor cell lists. If you operate a network with multiple cells, then of course you want to provide proper neighbor cell lists. At HAR2009, we solved the problem by manually hard-coding the respective bitmasks. That's of course not a proper solution.

Another problem is the notoriously complex encoding of the rest octets, which culminates in the SI13 REST OCTETS describing the GPRS parameters of a cell.

After a couple of hours in-flight hacking at the code in the sytem_information branch, I am now confident that it provides superior quality SI messages and rest octets than the hard-coded templates we used to have before.

Neighbor Cell lists and even SI13 rest octets are looking great when checking them in the wireshark dissectors. I think it's ready for prime time now, and the code should get merged into the master branch ASAP.

Now I am only left with one question: Should I consider this the first part of the FOSS.in GSM workout? ;)

Performance Enhancements in a Frequency Hopping GSM Network

Dieter Spaar had pointed out this book some months ago when I first raised some questions regarding frequency hopping and the orthogonal nature of hopping sequences with the same HSN but different MAIO.

Last week while David Burgess was with me, he also indicated that this book was great and he unfortunately didn't think of bringing it along with him.

Meanwhile, I have immediately ordered the book and am already at something like 30% completion. It is a most interesting book to read, approaching GSM from an advanced network planning angle, with a specific focus on the effects of frequency hopping, uplink/downlink power control and DTX on the overall system performance of a GSM network.

The theoretical foundations are always put in a GSM network simulator with detailed channel model, but also actually implemented in a real-world GSM network in Denmark.

Next to all the GSM specifications with their plethora of options and operator dependent settings, this book gives a detailed (but still very technical) background on how and why an Operator would configure his network to maximize the service quality offered to his subscribers.

From the results, you can for example very clearly see that

frequency hopping over a cyclic sequence gives higher gain improvement than random hopping, especially if the number of channels in the mobile allocation is low
frequency hopping gain is very dependent on the speed at which the MS moves. At 3kph, the gain when hopping over 8 channels can be 7dB, while at 50kph the same hopping will only provide 1.5dB
MAIO management (using different MAIO but same HSN) for all sectors in a cell gives significant FER improvements
handover algorithms differ quite a bit between non-frequency-hopping and frequency-hopping networks

In the end, it seems, network planning is never about allocating your channels in a way they don't overlap. That would limit the network capacity way too much. Network planning seems to only be about averaging out the interference that cells inevitably have with each other and ensure that the quality of the system only degrades with increasing load.

Reverse engineering 16-in-1 Super SIM cards

In order to support some real cryptographic authentication in OpenBSC, we have to use SIM cards with a known Ki (secret key). For cards that are issued by a real GSM operator, the Ki is only stored in the SIM and in the Authentication center of the network. Since we cannot obtain it from either of those two sources, we have to program our own SIM cards.

Unfortunately, SIM cards with privileges and/or documentation how to set Ki, IMSI and other data are not readily available on the open market. There are a couple of other solutions, though:

Use one of the old/cheap 6-in-1 / 16-in-1 SIM cards from the SIM cloning scene
Implement the GSM 11.11 SIM card spec on a programmable card such as a PIC microcontroller card or Java Card
Use something like the Bladox products to implement a SIM card or a SIM card proxy

The cheapest option with little R&D overhead is to use the so-called 16-in-1 SIM cards. They allow the user to set some of the interesting bits (Ki, PMLNsel, ICCID, IMSI, SMSP): Sufficient for authentication, but not sufficient for doing arbitrary tricks with the SIM.

Today I spent the better part of the day reverse engineering how both the SIM card as well as the included SIM card reader work. The result can be found in the OpenBSC wiki.

As I've already implemented+tested general authentication and encryption support in OpenBSC, all that is left to be done is some integration, configuration and testing. With some luck we can soon operate OpenBSC with full authentication and encryption support. This in turn will of course help with cryptanalysis and other experiments in a controlled environment :)

David Burgess (OpenBTS) visiting me for a couple of days in Berlin

On Friday, David Burgess of the OpenBTS project has come to visit me in Berlin. We're working on the final preparation of the two-day Deepsec 2009 GSM Security Workshop which will happen in Vienna next week.

David has more than 10 years experience in implementing GSM Layer 1 as well as the higher-layer protocols, so it's always great to talk with him and tap into his experience. Unfortunately the preparations for the workshop kept us too busy to work on some actual code.

The more than 200 slides for the workshop will be published after the workshop is over.

India setting up service stations to program IMEI into phones

This is not really current news, as it was released much earlier this year. However, I'm not following Indian news that closely so it has slipped my attention:

India's COAI is setting up hundreds of service centers where end users can have an IMEI programmed into their phone. This apparently relates to the fact that there are plenty of phones of Chinese origin with an all-zero IMEI in India.

Since there is a government law that requires every phone to have an unique IMEI number, operators have been ordered to refuse phones with an all-zero IMEI onto their network.

I personally find all of this very funny:

Firstly, what law enforcement typically cares about is the subscriber identity (the SIM). Persons are identified by their phone number. The phone number in turn is associated with the SIM. The SIM is issued by the operator and has a world-wide unique number called IMSI. So why would you care about the phone serial number? People can switch their phones much more easily than they can switch their SIM card, since the latter would always mean using a different number.
Secondly, for most common phones (and particularly the cheaper Chinese phones), tools to program/reset the IMEI are readily available on the Internet. So what's the point for a government initiative to even create more such software, distributed widely across the country. Chances are high that software leaks.
Finally, if I get a COAI-issued IMEI programmed into my phone, I can at any time erase it again and go back to the COAI to have a new IMEI programmed into it.

So from a real IT security point of view, this entire exercise is nothing but an annoyance to keep people busy and create employment for the staff operating those IMEI programmers.

Tho those involved: Work smarter, not harder ;)

Enabling jabber in WebOS on the Palm Pre using a binary patch

One of my main complaints about the palm Pre is that there is no support for the major IM protocol's such as jabber, icq, aim, msn, ...

As I discovered, they're actually using a library (libpurple) that supports all those protocols. It's just the UI and the intermediate LibpurpleAdapter program which artificially restrict the features that this library offers.

So it sounds to me like palm is getting money or other favors from Google to artificially restrict the capabilities of the Webos messenger.

As I have described in this mail to the webos-internals mailing list, you can actually use a very simple one-byte binary patch to LibpurpleAdapter to enable jabber support.

After that binary patch, you can add jabber contacts with the regular user@jabber-server.doma.in address and use the regular messenger application for keeping in touch with your jabber contacts. Just like how it is supposed to be.

Legal notice: Making this binary patch is legal, since LibpurpleAdapter is actually released under LGPL. If you have a working build environment for the Pre with all the libpurple headers, you can of course modify the source code and recompile it (as explained in the mail).

Side note: The libpurple-adapter source code that Palm has published on opensource.palm.com does not correspond to the actual binary code. This is a LGPL violation. However, since palm is the copyright holder, nobody can really do anything about it. But it once again shows that the software build/release process does not automatically generate the source code packages and that there is an erroneous manual process involved :(