Lab

-----------------

Lab
^^^
* Parser: don't try to parse missing files. Print message to stdout instead.
* Add soft memory limit of "memory_per_cpu * 0.98" for Slurm runs to safeguard against cgroup failures.
* Abort if report contains duplicate attribute names.
* Make reports even if fetcher detects unexplained errors.
* Use ``flags=''`` for :meth:`lab.parser.Parser.add_pattern` by default again.
* Include node names in standard reports and warn if report mixes runs from different partitions.
* Add new example experiment using a simple vertex cover solver.
* ``BaselSlurmEnvironment``: don't load Python 2.7.11 since it might conflict with an already loaded module.
* Raise default ``nice`` value to 5000.

Downward Lab
^^^^^^^^^^^^
* Support new Fast Downward exitcodes (Silvan).
* Parse "planner_wall_clock_time" attribute in planner parser.
* Include "planner_wall_clock_time" and "raw_memory" attributes in unexplained errors table.
* Make PlanningReport more generic by letting derived classes override the new
``PREDEFINED_ATTRIBUTES``, ``INFO_ATTRIBUTES`` and ``ERROR_ATTRIBUTES`` class members (Augusto).
* Don't compute the "quality" attribute automatically. The docs and ``showcase-options.py`` show
how to add the two filters that together add the IPC quality score to each run.

-----------------

Lab
^^^
* Add :meth:`exp.add_parser() <lab.experiment.Experiment.add_parser>` method. See also :ref:`parsing` (Silvan).
* Add :meth:`exp.add_parse_again_step() <lab.experiment.Experiment.add_parse_again_step>` method for running parsers again (Silvan).
* Require that the ``build``, ``start_runs`` and ``fetch`` steps are added explicitly (see :class:`~lab.experiment.Experiment`).
* Remove *required* argument from ``add_resource()``. All resources are now required.
* Use stricter naming rules for commands and resources. See respective ``add_*`` methods for details.
* Use ``required=False`` and ``flags='M'`` by default for :meth:`lab.parser.Parser.add_pattern`.
* Only support custom command line arguments for locally executed steps.
* Log errors to stderr.
* Log exit codes and wall-clock times of commands to driver.log.
* Add unexplained error if driver.log is empty.
* Let fetcher fetch ``properties`` and ``static-properties`` files.
* Remove deprecated possibility of passing Step objects to ``add_step()``.
* Remove deprecated ``exp.__call__()`` method.

Downward Lab
^^^^^^^^^^^^
* Add "planner_timer" and "planner_memory" attributes.
* Reorganize parsers and don't add any parser implicitly. See :ref:`downward-parsers`.
* Add anytime-search parser that parses only "cost", "cost:all" and "coverage".
* Revise and simplify single-search parser.
* Parse new Fast Downward exit codes (http://issues.fast-downward.org/issue739).
* Don't exclude (obsolete) "benchmarks" directory when caching revisions.
* Only copy "raw_memory" value to "memory" when "total_time" is present.
* Rename "fast-downward" command to "planner".
* Make "error" attribute optional for reports.

-----------------

Lab
^^^
* BaselSlurmEnvironment: Use ``infai_1`` and ``normal`` as default Slurm partition and QOS.
* Remove ``OracleGridEngineEnvironment``.

Downward Lab
^^^^^^^^^^^^
* Use ``--overall-time-limit=30m`` and ``--overall-memory-limit=3584M`` for all Fast Downward runs by default.
* Don't add ``-j`` option to build options (``build.py`` now uses all CPUs automatically).

-----------------

Lab
^^^
* Print run and task IDs during local experiments.
* Make warnings and error messages more informative.
* Abort after fetch step if fetcher finds unexplained errors.
* Improve examples and docs.

Downward Lab
^^^^^^^^^^^^
* Don't parse preprocessor logs anymore.
* Make regular expressions stricter in parsers.
* Don't complain if SAS file is missing.

-----------------

Lab
^^^
* Add BaselSlurmEnvironment (Florian).
* Support running experiments in virtualenv (Shuwa).
* Redirect output to ``driver.log`` and ``driver.err`` as soon as possible.
* Store all observed unexplained errors instead of a single one (Silvan).
* Report unexplained error if ``run.err`` or ``driver.err`` contain output.
* Report unexplained error if "error" attribute is missing.
* Add configurable soft and hard limits for output to ``run.log`` and ``run.err``.
* Record grid node for each run and add it to warnings table.
* Omit \toprule and \bottomrule in LaTeX tables.
* Add ``lab.reports.Table.set_row_order()`` method.
* Only escape text in table cells if it doesn't contain LaTeX or HTML markup.
* Allow run filters to change a run's ID (needed for renaming algorithms).
* Add ``merge`` kwarg to ``add_fetcher()`` (Silvan).
* Exit with returncode 1 if fetcher finds unexplained errors.
* Let fetcher show warning if ``slurm.err`` is not empty.
* Include content of ``slurm.err`` in reports if it contains text.
* Add continuous integration testing.
* Add ``--skip-experiments`` option for ``tests/run-tests`` script.
* Clean up code.
* Polish documentation.

Downward Lab
^^^^^^^^^^^^
* For each error outcome show number of runs with that outcome in summary table and dedicated tables.
* Add standalone exit code parser. Allow removing translate and search parsers (Silvan).
* Allow passing ``Problem`` instances to ``FastDownwardExperiment.add_suite()`` (Florian).
* Don't filter duplicate coordinates in scatter plots.
* Don't round scatter plot coordinates.
* Remove output.sas instead of compressing it.
* Fix scatter plots for multiple categories **and** the default ``None`` category (Silvan).

-----------------

Lab
^^^
* Show warning and ask for action when evaluation dir already exists.
* Add ``scale`` parameter to Attribute. It is used by the plot reports.
* Add ``digits`` parameter to Attribute for specifying the number of digits after the decimal point.
* Pass name, function, args and kwargs to ``exp.add_step()``. Deprecate passing Step objects.
* After calling ``add_resource("mynick", ...)``, use resource in commands with "{mynick}".
* Call: make ``name`` parameter mandatory, rename ``mem_limit`` kwarg to ``memory_limit``.
* Store grid job files in ``<exp-dir>-grid-steps``.
* Use common ``run-dispatcher`` script for local and remote experiments.
* LocalEnvironment: support randomizing task order (enabled by default).
* Make ``path`` parameter optional for all experiments.
* Warn if steps are listed explicitly and ``--all`` is used.
* Change main experiment step name from "start" to "run".
* Deprecate ``exp()``. Use ``exp.run_steps()`` instead.
* Don't filter ``None`` values in ``lab.reports`` helper functions.
* Make logging clearer.
* Add example FF experiment.
* Remove deprecated code (e.g. predefined Step objects, ``tools.sendmail()``).
* Remove ``Run.require_resource()``. All resources have always been available for all runs.
* Fetcher: remove ``write_combined_props`` parameter.
* Remove ``Sequence`` class.
* Parser: remove ``key_value_patterns`` parameter. A better solution is in the works.
* Remove ``tools.overwrite_dir()`` and ``tools.get_command_output()``.
* Remove ``lab.reports.minimum()``, ``lab.reports.maximum()``, ``lab.reports.stddev()``.
* Move ``lab.reports.prod()`` to ``lab.tools.product()``.
* Rename ``lab.reports.gm()`` to ``lab.reports.geometric_mean()`` and
``lab.reports.avg()`` to ``lab.reports.arithmetic_mean()``.
* Many speed improvements and better error messages.
* Rewrite docs.

Downward Lab
^^^^^^^^^^^^
* Always validate plans. Previous Lab versions don't add ``--validate``
since older Fast Downward versions don't support it.
* HTML reports: hide tables by default, add buttons for toggling visibility.
* Unify "score_*", "quality" and "coverage" attributes: assign values in range [0, 1]
and compute only sum and no average.
* Don't print tables on commandline.
* Remove DownwardExperiment and other deprecated code.
* Move ``FastDownwardExperiment`` into ``downward/experiment.py``.
* Rename ``config`` attribute to ``algorithm``. Remove ``config_nick`` attribute.
* Change call name from "search" to "fast-downward".
* Remove "memory_capped", and "id_string" attributes.
* Report raw memory in "unexplained errors" table.
* Parser: remove ``group`` argument from ``add_pattern()``, and always use group 1.
* Remove ``cache_dir`` parameter. Add ``revision_cache`` parameter to ``FastDownwardExperiment``.
* Fetcher: remove ``copy_all`` option.
* Remove predefined benchmark suites.
* Remove IpcReport, ProblemPlotReport, RelativeReport, SuiteReport and TimeoutReport.
* Rename CompareConfigsReport to ComparativeReport.
* Remove possibility to add ``_relative`` to an attribute to obtain relative results.
* Apply filters sequentially instead of interleaved.
* PlanningReport: remove ``derived_properties`` parameter. Use two filters
instead: one for caching results, the other for adding new properties
(see ``QualityFilters`` in ``downward/reports/__init__.py``).
* PlotReport: use fixed legend location, remove ``category_styles`` option.
* AbsoluteReport: remove ``colored`` parameter and always color HTML reports.
* Don't use domain links in Latex reports.
* AbsoluteReport: Remove ``resolution`` parameter and always use ``combined`` resolution.
* Rewrite docs.