NavigaTor: Finding Faster Paths to Anonymity

NavigaTor facilitates large-scale measurements on the live Tor network, without stressing it. To this end, NavigaTor includes a custom Tor path generator and measures Round-Trip-Time (RTT), Time-To-First-Byte (TTFB), and throughput of Tor circuits.

NavigaTor was used to produce results for the paper “NavigaTor: Finding Faster Paths to Anonymity” by Robert Annessi and Martin Schmiedecker. The paper appeared in the proceedings of the 1st IEEE European Symposium on Security and Privacy (Euro SP 2016). You can find a pre-print here and the final version here (copyright IEEE).

Source code

The source code is published under a free license (GPLv2) and can be obtained with:

git clone https://github.com/naviga-tor/navigator.git

NavigaTor.py

NavigaTor.py measures Round-Trip-Time (RTT), Time-To-First-Byte (TTFB), and throughput of Tor circuits. Measurement results are written as serialized, LZO-compressed objects to files for further processing. Besides a Python interpreter, the following Python libraries are required: pkg_resources, lzo, stem, and pycurl.

Before running NavigaTor, a (patched) Tor client must be started with the provided configuration:

./tor -f torrc

The Tor client needs to be of the 0.2.3.x series and must be patched with the dumpguards, log_cbt, descriptors and findpath patches. A patched Tor client binary for x86 is provided with statically linked OpenSSL, libevent, and Zlib libraries. If you have troubles running any of the scripts below, you can use our Linux chroot environment that contains all necessary dependencies.

A run of NavigaTor may look as simple as this:

./NavigaTor.py

By default, NavigaTor connects to a local Tor client on TCP port 9051 and builds one circuit using a single thread. RTT, TTFB, and throughput for this circuit are each measured one time. Measurement data are written to files prefixed “probe_”.

Various command line options can be used to control NavigaTor.py:

Option	Description
--circuits	Number of circuits to build and measure. (Default: 1)
--rttprobes	Number of RTT measurements on each circuit. (Default: 1)
--ttfbprobes	Number of TTFB measurements on each circuit. (Default: 1)
--bwprobes	Number of throughput measurements on each circuit. (Default: 1)
--probesleep	Waiting interval between probes in seconds. (Default: 0.0)
--threads	Number of parallel measurement threads. (Default: 1)
--output	Name prefix for output files. (Default: “probe_”)
--network-protection	Prevent hammering the Tor network. (Default: Enabled)
--port	Control port of the Tor client. (Default: 9051)

testdata.py

After NavigaTor.py ran successfully, its output can be verified using testdata.py. A typical run may look like this:

cat probe_001 | ./testdata.py

truncatedata.py

truncatedata.py truncates data from NavigaTor measurements. A typical run may look like this:

cat probe_001 | ./truncatedata.py 1> probe_001.e

merge.py

merge.py both merges multiple truncated measurement data files into one and sorts measurements by date. A typical run may look like this:

./merge.py probe_001.e probe_002.e probe_003.e 1> probe

cat.py

cat.py calculates the congestion delay for circuit. A typical run may look like this:

./cat.py --input probe --output probe.cat

r9.py

r9.py adds statistical information to circuit probes. A typical run may look like this:

./r9.py --input probe.cat --output probe.stat

json_export.py

json_export.py converts NavigaTor measurements to JSON format. A typical run may look like this:

./json_export.py --input probe.stat --output probe.json

Measurement data

Gathered measurement data (608MB Lzip compressed) are provided under a free license (Creative Commons Attribution 4.0 International). The data format, JSON, is human readable and an open standard; libraries for parsing exist for virtually every programming language. Raw (untruncated, serialized) measurement data are available on request (~30GB compressed).

Directory structure

“planetlab_latency” contains 19 files. Each file corresponds to a single measurement run conducted on one Planetlab host. See paper Sections 7.2 (Statistical Analysis), 7.3 (Latency Evaluation), and 7.4 (Anonymity Evaluation) for details on the measurements.
“planetlab_throughput” contains 7 files. Each file corresponds to a single measurement run conducted on one Planetlab host. See paper Section 7.5 (Throughput Evaluation) for details on the measurements.
“single_host” contains 2 files: “single-host_latency.json” corresponds to latency measurements and “single-host_throughput.json” to throughput measurements. See paper Sections 7.6 (Congestion-Aware Scheme) and 7.7 (Latency over Time) for details on the measurement.

Data format

Every line represents the measurement of one circuit in JSON format, containing the following items:

date of measurement
entry node's fingerprint
middle node's fingerprint
exit node's fingerprint
CBT in milliseconds
(calculated) CBT percentile
(whether backup data have been used for calculating CBT percentile)
list of RTTs in milliseconds
(calculated) First-RTT percentile
(whether backup data have been used for calculating First-RTT percentile)
list of TTFBs in milliseconds
list of throughput measurements in Mbit/s
(calculated) congestion delay
(calculated) congestion delay percentile
(whether backup data have been used for calculating congestion delay percentile)

readdata.py

A simple script showing how to read the provided measurement data.

Tor path generator

NavigaTor includes a custom Tor path generator that is identical to Tor’s default path selection algorithm. To this end, only small, non-intrusive changes were made to Tor’s source code. This ensures that, given the same consensus data, the path generator produces the same paths as Tor would and also keeps to changes portable to other versions of the Tor software.

Although the path generator is included in NavigaTor.py, we provide a simple example, which shows that it also can be used independently:

./pathgen.py

Like Navigator.py, the path generator also depends on a patched Tor client; however, it only depends on the findpath patch strictly.