Cloudfeaster

Added

* bin/install-dev-env-scripts.sh can now be used by spider repos to add dev env host scripts to a spider repo's host env

Changed

* Nothing

Removed

* Nothing

Added

* added run-all-spiders.sh and run-spider.sh
* by default Chrome now started with --no-sandbox which should mean that
Chrome can run as root which simplifies a whole host of complexity

Changed

* Nothing

Removed

* Nothing

Added

* .travis.yml now runs run_repo_security_scanner.sh
* added xe_exchange_rates.py sample spider
* added sha1 hash of spiders args to spider output

Changed

* [ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver) 2.38 -> 2.46
* Selenium 3.12.0 -> 3.141.0
* twine 1.11.0 -> 1.12.1
* dateutil 2.7.3 -> 2.7.5
* material simplifcation of way to use run_sample.sh

Removed

* removed bank_of_canada_daily_exchange_rates.py sample spider
* removed spiderhost.py, spiderhost.sh, spiders.py and spiders.sh

Added

* Nothing

Changed

* Selenium 3.11.0 -> 3.12.0
* python-dateutil 2.7.2 -> 2.7.3
* spider metadata changed to camel case instead of snake case to get closer to [these](https://google.github.io/styleguide/jsoncstyleguide.xml) JSON style guidelines
* crawl results metadata now grouped in the _metadata property and use camel case instead of snake case
* crawl results are now validated against [this](cloudfeaster/jsonschemas/spider_output.json) jsonschema
* added [spiders.sh](bin/spiders.sh) and [spiderhost.sh](bin/spiderhost.sh) to enable the API for a docker image container spiders to be expressed in a manner that's independant from Python and Webdriver

Removed

* Nothing

Added

* support [pip 10.x](https://pip.pypa.io/en/stable/news/)

Changed

* simonsdave/cloudfeaster docker image is now based on Ubuntu 16.04
* [ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver) 2.37 -> 2.38

Removed

* Nothing

Added

* added [cloudfeaster/samples/pypi.py](cloudfeaster/samples/pypi.py) sample spider

Changed

* spiders meta data - url string property is now validated using jsonschema
uri format instead of pattern
* selenium 3.9.0 -> 3.11.0
* python-dateutil 2.6.1 -> 2.7.2
* [ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver) 2.35 -> 2.37
* twine 1.10.0 -> 1.11.0
* identifying_factors and authenticating_factors properties will now always appear
in spiders.py output

Removed

* Nothing