Cr8

================

Breaking
--------

- Changed the default output format to ``text``. In addition, the values of
``--output-format`` were renamed from ``full`` and ``short`` to ``json`` and
``text``.

Miscellaneous improvements
--------------------------

- Added a ``--keep-data`` option to ``run-crate``. If this is set the data
folder isn't removed if the process is stopped.

- The ``version`` argument of ``run-crate`` can now also be a fs path to a
CrateDB tarball.

- Various error handling and ``Ctrl+c`` improvements.

- Added ``--logfile-info`` and ``--logfile-result`` options to ``run-spec`` and
``run-track``.

================

- Fixed a regression that caused ``run-spec`` to save results into ``hosts``
instead of ``result-hosts``.

================

insert-fake-data improvements
-----------------------------

- Multiple cores are now utilized better for fake data generation.

- Adopted internal queries to be compatible with Crate versions > ``0.57``.

- ``insert-fake-data`` will now insert the accurate number of rows specified
instead of rounding to the nearest bulk size.

Miscellaneous
-------------

- ``run-crate latest-stable`` now correctly launches the latest released stable
version of Crate.
It incorrectly retrieved the version of the latest Java client release.

- ``run-crate`` now outputs the postgres port if found in the logs.

- Added a ``--action`` argument to ``run-spec`` which can be used to only run a
subset of a spec file.

- Extended the track-file format to allow re-using a setup across multiple spec
files.

- Added a ``--version`` option.
Best feature ever.

- Changed the ``--help`` output formatting so it's easier to read.

================

Breaking (but only a little)
----------------------------

- The ``run-track`` subcommand now does not fail any more if a single
spec file of the track fails. To achieve the same behaviour as before, you
can use the ``--failfast`` command line option.

Improvements
------------

- Added a ``--output-fmt`` option to most commands.
This option can be used to get a succinct output.

- Added a new ``auto_inc`` fake data provider for ``insert-fake-data``.
This provider may be a bit slow. This is due to the fact that the fake data
generation utilizes multiple processes and this provider requires
synchronization. But it's still awesome.

- Spec files now support a ``min_version`` setting.
This can be used to skip certain queries if the server doesn't meet the
``min_version`` requirement.

- Improved the error handling a bit.

- Statements and arguments in spec files can now be defines as callables.

- Added ``meta`` object column to results table.
It's now possible to add a name to the spec so the benchmark results can
easily be identified by this spec label.

- Added Crate build date column to version_info in benchmark result table.

- ``timeit`` now shows a progress bar.

- The ``--setting`` and ``-env`` options of ``run-crate`` are now repeatable.


Fixes
-----

- Fixed an issue with the ``num-records`` option of ``insert-fake-data``.
It didn't work correctly if the number of records specified was smaller than
the bulk size.

- Fixed some issues with the way Crate is launched using ``run-track``.
If Crate produced a lot of logging output it could get stuck.

================

Breaking
--------

- ``hosts`` and ``table`` is now always a named argument.
This affects ``timeit``, ``insert-json``, ``insert-blob`` and
``insert-fake-data``


Features 🍒
-----------

run-track
~~~~~~~~~

Added a new ``run-track`` command.
This command can be used to execute ``track`` files. A ``track`` file is a file
in ``TOML`` format containing a matrix definition of Crate versions, Crate
configurations and spec files.

The command will run each listed Crate version with each configuration and run
all listed spec files against it.


Other improvements
~~~~~~~~~~~~~~~~~~

- Added a new ``run-crate`` command.

- Added a fake-data provider for ``geo_point`` columns.

- Improved the ``--help`` output of most commands.

- Run-spec output is now proper JSON

- Spec files can be written in python

- ``args`` and ``bulk_args`` can now be specified in ``toml`` spec files.


Fixes 💩
--------

- ``runtime_stats['n']`` is no longer capped to 1000

- ``insert-json`` now ignores empty lines instead of causing an error.

================

Breaking 💔
-----------

Pretty much everything:

- Renamed ``blob upload`` to ``insert-blob``

- Renamed ``json2insert`` to ``insert-json``

- Renamed ``fill-table`` to ``insert-fake-data``

- Removed ``find-perf-regressions``

New & shiny features ✨
-----------------------

run-spec
~~~~~~~~

Added a new command which can be used to "run" spec files. Spec files are
either ``.json`` or ``.toml`` files which contain setup, queries and tear-down
directives. A minimal example::

[setup]
statement_files = ["sql/create_countries.sql"]

[[setup.data_files]]
target = "countries"
source = "data/countries.json" paths are relative to the spec file

[[queries]]
statement = "select count(*) from countries"
iterations = 1000

[teardown]
statements = ["drop table countries"]


``run-spec`` will execute the given specification and output runtime statistics.
The result can also directly be inserted into a Crate cluster.

insert-fake-data & insert-json
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Column names are now quoted in the insert statement

insert-fake-data
~~~~~~~~~~~~~~~~

- No longer tries to generate data for generated columns

- Speed improvements

- Added default provider mappings for columns of type ``float``, ``double`` and
``ip``

insert-json
~~~~~~~~~~~

- Prints runtime stats after the inserts are finished

timeit
~~~~~~

- Added a histogram and percentiles to the runtime statistics that are printed

- Added a concurrency option