Pyperf

------------------------

* ``stats`` command: count the number of outliers
* Rewrite the calibration code to support PyPy:

- On PyPy, calibrate also the number of warmups
- On PyPy, recalibrate the number of loops and warmups
- Loop calibration now uses the number of warmups and values instead
of 1 to compute warmup values
- A worker process cannot calibrate the number of loops and compute values.
These two operations now require two worker processes.

* Command line interface (CLI): the ``--benchmark``, ``--include-benchmark``
and ``--exclude-benchmark`` options can now be specified multiple times.
* Rewrite ``dump`` command:

- Writes one value per line
- Now display also metadata of calibration runs
- Enhance formatting of calibration runs
- Display number of warmup, value and loop

* Add new run metadata:

- ``calibrate_loops``, ``recalibrate_loops``: number of loops
of loop calibration/recalibration runs
- ``calibrate_warmups``, ``recalibrate_warmups``: number of warmups
of warmup calibration/recalibration runs

------------------------

* Add a new "perf command" command to measure the timing of a program
* ``Runner.bench_command()`` now measures also the maximum RSS memory if
available.
* Fix Windows 32bit issue on Python 2.7, fix by yattom.
* ``Runner.bench_func()`` now uses ``functools.partial()`` if the function
has argument. Calling ``partial()`` is now 1.07x faster (-6%) than calling
``func(*args)``.
* Store memory values as integers, not float, when tracking memory usage
(``--track-memory`` and ``--tracemalloc`` options)

------------------------

Enhancements:

* ``stats`` command now displays percentiles
* ``hist`` command now also checks the benchmark stability by default
* dump command now displays raw value of calibration runs.
* Add ``Benchmark.percentile()`` method

Backward incompatible changes:

* Remove the ``compare`` command to only keep the ``compare_to`` command
which is better defined
* Run warmup values must now be normalized per loop iteration.
* Remove ``format()`` and ``__str__()`` methods from Benchmark. These methods
were too opinionated.
* Rename ``--name=NAME`` option to ``--benchmark=NAME``
* Remove ``perf.monotonic_clock()`` since it wasn't monotonic on Python 2.7.
* Remove ``is_significant()`` from the public API

Other changes:

* check command now only complains if min/max is 50% smaller/larger than
the mean, instead of 25%.

--------------------------

Major change:

* Display ``Mean +- std dev`` instead of ``Median +- std dev``

Enhancements:

* Add a new ``Runner.bench_command()`` method to measure the execution time of
a command.
* Add ``mean()``, ``median_abs_dev()`` and ``stdev()`` methods to ``Benchmark``
* ``check`` command: test also minimum and maximum compared to the mean

Major API change, rename "sample" to "value":

* Rename attributes and methods:

- ``Benchmark.bench_sample_func()`` => ``Benchmark.bench_time_func()``.
- ``Run.samples`` => ``Run.values``
- ``Benchmark.get_samples()`` => ``Benchmark.get_values()``
- ``get_nsample()`` => ``get_nvalue()``
- ``Benchmark.format_sample()`` => ``Benchmark.format_value()``
- ``Benchmark.format_samples()`` => ``Benchmark.format_values()``

* Rename Runner command line options:

- ``--samples`` => ``--values``
- ``--debug-single-sample`` => ``--debug-single-value``

Changes:

* ``convert``: Remove ``--remove-outliers`` option
* ``check`` command now tests stdev/mean, instead of testing stdev/median
* setup.py: statistics dependency is now installed using ``extras_require`` to
support setuptools 18 and newer
* Add setup.cfg to enable universal builds: same wheel package for Python 2
and Python 3
* Add ``perf.VERSION`` constant: tuple of int
* JSON version 6: write metadata common to all benchmarks (common to all runs
of all benchmarks) at the root; rename 'samples' to 'values' in runs.

--------------------------

* Add ``--python-names`` option to the :ref:`Runner CLI <runner_cli>`
* ``system show`` command now checks if the system is ready for benchmarking
* Fix ``--compare-to`` option: the benchmark was run twice with the reference
Python, instead of being run first with reference Python and then changed
Python.
* Runner now raises an exception if a benchmark name is not unique.
* ``compare_to`` command now keeps the original order of benchmarks, only
sort if ``--by-speed`` option is used.
* Fix ``system`` command on macOS on non-existent ``/proc`` and ``/sys``
pseudo-files.
* Fix ``system`` bugs on systems with more than 32 processors.

--------------------------

New features:

* Add ``--compare-to`` option to the :ref:`Runner CLI <runner_cli>`
* :ref:`compare_to <compare_to_cmd>` command: Add ``--table`` option to render a table

Bugfixes:

* Fix the ``abs_executable()`` function used to find the absolute path to the
Python program. Don't follow symbolic links to support correctly virtual
environments.