Ooniprobe

-------------------------

Who said friday 17th is only bad luck?

* Add two new report entry keys test_start_time and test_runtime

* Fix bug that lead to ooniresources not working properly

-------------------------

* Introduce a new tool for generating ooniprobe test decks called oonideckgen.

* Introduce a new tool for updating resources used for geoip lookup and deck
generation.

* Add support for policy aware bouncing in the client.
https://trac.torproject.org/projects/tor/ticket/12579

* Various improvements to the bridge_reachability test (enable better tor
logging and also log obfsproxy)

* Fix backward compatibility with twisted 13.1 and add regression tests for
this.
https://trac.torproject.org/projects/tor/ticket/13139

-------------------------

* Update MANIFEST.in to include the manpages for ooniprobe and oonireport.

* Raise a more specific exception when multiple test cases are in a single
nettest file and the usageOptions are incoherent.

-------------------------

In this new release of ooniprobe we have added a new command line tool for
listing the reports that have not been published to a collector and that allows
the probe operator to choose which ones they would like to upload.

We have also made some privacy improvements to the reports (we will sanitize
all things that may look like file paths) and added metadata associated with
the maxmind database being used by the probe operator.

Here is a more detailed list of what has been done:

* Annotate on disk which reports we have submitted and which ones we have not:
https://trac.torproject.org/projects/tor/ticket/11860

* Add tool called oonireport for publishing unpublished ooniprobe reports to a
collector: https://trac.torproject.org/projects/tor/ticket/11862

* Probe Report does not leak filepaths anymore:
https://trac.torproject.org/projects/tor/ticket/12706

* Reports now include version information about the maxmind database being
used: https://trac.torproject.org/projects/tor/ticket/12771

* We now do integrity checks on the ooniprobe.conf file so that we don't start
the tool if the config file is missing some settings or is not consistent:
https://trac.torproject.org/projects/tor/ticket/11983
(thanks to Alejandro López (kudrom))

* Improvements have been made to the sniffer subsystem (thanks to Alejandro
López (kudrom))

* Fix the multi protocol traceroute test.
https://trac.torproject.org/projects/tor/ticket/12883

Minor bug fixes:

* Fix dns_spoof test (by kudrom)
https://trac.torproject.org/projects/tor/ticket/12486

* ooni might not look at requiresTor:
https://trac.torproject.org/projects/tor/ticket/11858

* ooni spits out gobs of tracebacks if Tor is not running and the OONI config
says it will be:
https://trac.torproject.org/projects/tor/ticket/11859

* The README for ooni-probe should mention the bugtracker and repository
https://trac.torproject.org/projects/tor/ticket/11980

------------------------

* Add ooniprobe manpage.

* Fix various security issues raised by the least authority audit.

* Add a test that checks for Tor bridge reachability.

* Record the IP address of the exit node being used in torified requests.

* Captive portal test now uses the ooni-probe test templates.

* Have better test naming consistency.

-------------------------

* Fix bugs in the traceroute test that lead to not all packets being collected.

* All values inside of http_requests test are now initialized inside of setUp.

* Fix a bug that lead to the input value of the report not being set in some
circumstances.

* Add bridge_reachability test