Scriptworker

---------------------

.. _added-43:

Added
~~~~~

- added ``get_loggable_url`` to avoid logging secrets
- added integration test for private artifacts

.. _changed-38:

Changed
~~~~~~~

- ``create_artifact`` now has a default expiration of the task
expiration date.
- ``get_artifact_url`` now supports signed URLs for private artifacts
- ``get_artifact_url`` no longer returns unquoted urls (breaks signed
urls)
- ``validate_artifact_url`` unquotes paths before returning them

.. _fixed-35:

Fixed
~~~~~

- fix integration tests for osx py36
`135 <https://github.com/mozilla-releng/scriptworker/issues/135>`__

.. _removed-16:

Removed
~~~~~~~

- removed the config for ``artifact_expiration_hours``.
- removed support for taskcluster 0.3.x

.. _section-79:

--------------------

.. _added-44:

Added
~~~~~

- added support for bouncer scriptworker

.. _changed-39:

Changed
~~~~~~~

- renamed ``run_loop`` to ``run_tasks``
- ``run_tasks`` now shuts down gracefully after receiving a SIGTERM: it
finishes the current task(s), and exits.

.. _fixed-36:

Fixed
~~~~~

- ``run_tasks`` now sleeps 5 if there were no tasks claimed.

.. _section-80:

--------------------

.. _fixed-37:

Fixed
~~~~~

- Freeze aiohttp to 2.x.y

.. _section-81:

--------------------

.. _added-45:

Added
~~~~~

- ``valid_vcs_rules``, ``source_env_prefix``,
``extra_run_task_arguments`` depend on ``cot_product``
- ``cot_product`` is defined in example configuration
- Support for ship-it tasks

.. _section-82:

--------------------

.. _added-46:

Added
~~~~~

- Added ``scriptworker.cot.verify.get_jsone_template``, because action
tasks use actions.json instead of .taskcluster.yml

.. _changed-40:

Changed
~~~~~~~

- Added a ``tasks_for`` argument to ``populate_jsone_context``.
- Used ``format_json`` instead of ``pprint.pformat`` in most
``scriptworker.cot.verify`` functions.

.. _removed-17:

Removed
~~~~~~~

- Removed ``scriptworker.utils.render_jsone``, since it reduced to a
``jsone.render`` call.
- Removed the now-unused
``scriptworker.constants.max_jsone_iterations``

.. _section-83:

--------------------

.. _added-47:

Added
~~~~~

- Added ``scriptworker.cot.verify.verify_parent_task_definition``. This
is the core change in this release, aka CoT version 2. We now use
json-e to rebuild the decision/action task definitions from the tree.
- Added ``json-e`` and ``dictdiffer`` dependencies.
- ``arrow``, ``certifi``, ``multidict``, ``taskcluster``, and ``yarl``
have updated their major version numbers.
- Added ``Context.projects`` and ``Context.populate_projects``.
- Added ``load_json_or_yaml_from_url``.
- Added ``DEFAULT_CONFIG['cot_version']`` and
``DEFAULT_CONFIG['min_cot_version']``; this is cotv2. If
``min_cot_version`` is 1, we allow for falling back to the old cot v1
logic.
- Added ``DEFAULT_CONFIG['project_configuration_url']`` and
``DEFAULT_CONFIG['pushlog_url']``.
- Added ``scriptworker.task.KNOWN_TASKS_FOR``,
``scriptworker.task.get_action_name``,
``scriptworker.task.get_commit_message``,
``scriptworker.task.get_and_check_project``,
``scriptworker.task.get_and_check_tasks_for``
- Added ``scriptworker.utils.remove_empty_keys`` since the taskgraph
drops key/value pairs where the value is empty. See
https://github.com/taskcluster/json-e/issues/223
- Added ``scriptworker.utils.render_jsone`` to generically render
json-e.
- Added ``max_jsone_iterations`` pref; sometimes the values to replace
template values are several layers deep.
- Added ``scriptworker.cot.verify.get_pushlog_info``,
``scriptworker.cot.verify.get_scm_level``,
``scriptworker.cot.verify.populate_jsone_context``, and
``scriptworker.cot.verify.compare_jsone_task_definition``.
- Added test files to ``scriptworker/test/data/cotv2/``.

.. _changed-41:

Changed
~~~~~~~

- Renamed ``load_json`` to ``load_json_or_yaml``. This now takes a
``file_type`` kwarg that defaults to ``json``.
- Moved ``get_repo``, ``get_revision``, ``is_try``, and ``is_action``
from ``scriptworker.cot.verify`` to ``scriptworker.task``
- Moved the sub-function path callback from ``scriptworker.cot.verify``
to ``scriptworker.utils.match_url_path_callback``
- ``scriptworker.cot.verify.guess_task_type`` takes a 2nd arg,
``task_defn``, to differentiate action tasks from decision/cron
tasks.
- ``scriptworker.cot.verify.get_all_artifacts_per_task_id`` adds
``public/actions.json`` and ``public/parameters.yml`` to decision
task artifacts to download, for action task verification.
- Removed the ``firefox`` from ``scriptworker.cot.verify`` function
names.
- Tweaked the task ID logging in ``verify_cot``.

.. _fixed-38:

Fixed
~~~~~

- Updated ``path_regexes`` to identify most (all?) valid hg.m.o repo
paths, instead of returning ``None``.

.. _removed-18:

Removed
~~~~~~~

- Removed ``scriptworker.cot.verify.verify_decision_task`` and
``scriptworker.cot.verify.verify_action_task`` in favor of
``scriptworker.cot.verify.verify_parent_task``.

.. _section-84: