LaForge's home page (Posts about networking)https://laforge.gnumonks.org/blog/tags/networking.atom2022-09-24T16:07:23ZHarald WelteNikolaudtrace - Unix domain socket tracinghttps://laforge.gnumonks.org/blog/20180330-udtrace/2018-03-30T00:00:00+02:002018-03-30T00:00:00+02:00Harald Welte<p>When developing applications that exchange data over sockets, every so
often you'd like to analyze exactly what kind of data is exchanged over
the socket.</p>
<p>For TCP/UDP/SCTP/DCCP or other IP-based sockets, this is rather easy by
means of libpcap and tools like tcpdump, tshark or wireshark. However,
for unix domain socket, unfortunately no such general capture/tracing
infrastructure exists in the Linux kernel.</p>
<p>Interestingly, even after searching for quite a bit I couldn't find
any existing tools for this. This is surprising, as unix domain sockets
are used by a variety of programs, from sql servers to bind8 <cite>ndc</cite> all
the way to the <cite>systemctl</cite> tool to manage systemd.</p>
<p>In absence of any kernel support, the two technologies I can think of to
implement this is either <a class="reference external" href="https://sourceware.org/systemtap/">systemtap</a> or
a <a class="reference external" href="https://blog.cryptomilk.org/2014/07/21/what-is-preloading/">LD_PRELOAD wrapper</a>.</p>
<p>However, I couldn't find an example for using either of those two to get
traces of unix domain soocket communications.</p>
<p>Ok, so I get to write my own. My first idea hence was to implement
something based on top of systemtap, the Linux kernel tracing framework.
Unfortunately, systemtap was broken in Debian unstable (which I use for
decades) at the time, so I went back to the good old LD_PRELOAD shim
library / wrapper approach.</p>
<p>The result is called udtrace and can be found at</p>
<blockquote>
<p>git clone <a class="reference external" href="https://gitea.osmocom.org/laforge/udtrace">https://gitea.osmocom.org/laforge/udtrace</a></p>
</blockquote>
<p>or alternatively via its <a class="reference external" href="https://github.com/laf0rge/udtrace">github mirror</a>.</p>
<p>Below is a copy+paste of its README file. Let's hope this tool is
useful to other developers, too:</p>
<section id="udtrace-unix-domain-socket-tracing">
<h2>udtrace - Unix Domain socket tracing</h2>
<p>This is a LD_PRELOAD wrapper library which can be used to trace the data
sent and/or received via unix domain sockets.</p>
<p>Unlike IP based communication that can be captured/traced with pcap
programs like tcpdump or wireshark, there is no similar mechanism
available for unix domain sockets.</p>
<p>This LD_PRELOAD library intercepts the C library function calls of
dynamically linked programs. It will detect all file descriptors
representing unix domain sockets and will then print traces of all
data sent/received via the socket.</p>
<section id="usage">
<h3>Usage</h3>
<p>Simply build <strong>libudtrace.so</strong> using the <strong>make</strong> command, and then
start your to-be-traced program with</p>
<blockquote>
<p>LD_PRELOAD=libudtrace.os</p>
</blockquote>
<p>e.g.</p>
<blockquote>
<p>LD_PRELOAD=libudtrace.so systemctl status</p>
</blockquote>
<p>which will produce output like this:</p>
<blockquote>
<pre class="code python doctest">>>> UDTRACE: Unix Domain Socket Trace initialized (TITAN support DISABLED)
>>> UDTRACE: Adding FD 4
>>> UDTRACE: connect(4, "/run/dbus/system_bus_socket")
4 sendmsg W 00415554482045585445524e414c20
4 sendmsg W 3331333033303330
4 sendmsg W 0d0a4e45474f54494154455f554e49585f46440d0a424547494e0d0a
[...]
</pre>
</blockquote>
</section>
<section id="output-format">
<h3>Output Format</h3>
<p>Currently, <strong>udtrace</strong> will produc the following output:</p>
<p>At time a FD for a unix domain socket is created:</p>
<blockquote>
<pre class="code python doctest">>>> UDTRACE: Adding FD 8
</pre>
</blockquote>
<p>At time a FD for a unix domain socket is closed:</p>
<blockquote>
<pre class="code python doctest">>>> UDTRACE: Removing FD 8
</pre>
</blockquote>
<p>At time a FD for a unix domain socket is bound or connected:</p>
<blockquote>
<pre class="code python doctest">>>> UDTRACE: connect(9, "/tmp/mncc")
</pre>
</blockquote>
<p>When data is read from the socket:</p>
<blockquote>
<p>9 read R 00040000050000004403000008000000680000001c0300002c03000000000000</p>
</blockquote>
<p>When data is written to the socket:</p>
<blockquote>
<p>9 write W 00040000050000004403000008000000680000001c0300002c03000000000000</p>
</blockquote>
<dl class="simple">
<dt>Where</dt>
<dd><ul class="simple">
<li><p><em>9</em> is the file dsecriptor on which the event happened</p></li>
<li><p><em>read/write</em> is the name of the syscall, could e.g. also be sendmsg / readv / etc.</p></li>
<li><p><em>R|W</em> is Read / Write (from the process point of view)</p></li>
<li><p>followed by a hex-dump of the raw data. Only data successfully
written (or read) will be printed, not the entire buffer passed to
the syscall. The rationale is to only print data that was actually
sent to or received from the socket.</p></li>
</ul>
</dd>
</dl>
</section>
<section id="titan-decoder-support">
<h3>TITAN decoder support</h3>
<p>Getting hex-dumps is nice and fine, but normally one wants to have a
more detailed decode of the data that is being passed on the socket.</p>
<p>For TCP based protocols, there is wireshark. But most protocols on unix
domain sockets don't follow inter-operable / public standards, so even
if one was to pass the traces into wireshark somehow, there would be no
decoder.</p>
<p>In the <a class="reference external" href="https://osmocom.org/">Osmocom project</a>, we already had some type
definitions and decoders for our protocols written in the TTCN-3
programming language, using <a class="reference external" href="https://projects.eclipse.org/projects/tools.titan">Eclipse TITAN</a>.
In order to build those decoders fro MNCC and PCUIF, please use</p>
<blockquote>
<p>make ENABLE_TITAN=1</p>
</blockquote>
<p>when building the code.</p>
<p>Please note that this introduces a run-time dependency to
libttcn3-dynamic.so, which is (at least on Debian GNU/Linux) not
installed in a default library search path, so you will have to use
something like:</p>
<blockquote>
<p>LD_LIBRARY_PATH=/usr/lib/titan LD_PRELOAD=libudtrace.so systemctl status</p>
</blockquote>
</section>
</section>