Scriptworker

----------------------

.. _fixed-57:

Fixed
~~~~~

- balrog tasks are now verifiable in chain of trust.

----------------------

.. _added-70:

Added
~~~~~

- ``verify_signed_tag``, which verifies the tag’s signature and makes
sure we’re updated to it.

.. _changed-65:

Changed
~~~~~~~

- ``rebuild_gpg_homedirs`` now uses git tags instead of checking for
signed commits.
- ``get_git_revision`` now takes a ``ref`` kwarg; it finds the revision
for that ref (e.g., tag, branch).
- ``update_signed_git_repo`` ``revision`` kwarg is now named ``ref``.
It also verifies and updates to the signed git tag instead of
``ref``.
- ``update_signed_git_repo`` now returns a tuple (revision, tag)
- ``build_gpg_homedirs_from_repo`` now uses ``verify_signed_tag``
instead of ``verify_signed_git_commit``, and takes a new ``tag`` arg.

.. _fixed-58:

Fixed
~~~~~

- the curl command in ``Dockerfile.gnupg`` now retries on failure.

.. _removed-28:

Removed
~~~~~~~

- ``verify_signed_git_commit_output``
- ``verify_signed_git_commit``

----------------------

.. _added-71:

Added
~~~~~

- beetmover and balrog scriptworker support in chain of trust
verification
- ``cot_restricted_trees`` config, which maps branch-nick to branches

.. _changed-66:

Changed
~~~~~~~

- Changed ``cot_restricted_scopes`` to be a scope to branch-nick dict,
indexed by ``cot_product``

.. _fixed-59:

Fixed
~~~~~

- nuke then move the tmp gpg homedir, rather than trying to [wrongly]
use ``overwrite_gpg_home`` on a parent dir

----------------------

.. _added-72:

Added
~~~~~

- Dockerfiles: one for general testing and one for gpg homedir testing,
with readme updates
- ``flake8_docstrings`` in tox.ini
- log chain of trust verification more verbosely, since we no longer
have real artifacts uploaded alongside

.. _changed-67:

Changed
~~~~~~~

- download cot artifacts into ``work_dir/cot`` instead of
``artifact_dir/public/cot``, to avoid massive storage dups
- ``download_artifacts`` now returns a list of full paths instead of
relative paths. Since ``upstreamArtifacts`` contains the relative
paths, this should be more helpful.
- ``contextual_log_handler`` now takes a ``logging.Formatter`` kwarg
rather than a log format string.

.. _changed-68:

Changed
~~~~~~~

- check for a new gpg homedir before ``run_loop``, because puppet will
now use ``rebuild_gpg_homedirs``

.. _fixed-60:

Fixed
~~~~~

- updated all docstrings to pass ``flake8_docstrings``
- switched to a three-phase lockfile for gpg homedir creation to avoid
race conditions (locked, ready, unlocked)
- catch ``aiohttp.errors.DisconnectedError`` and
``aiohttp.errors.ClientError`` in ``run_loop`` during
``upload_artifacts``
- compare the built docker-image tarball hash against
``imageArtifactHash``

.. _removed-29:

Removed
~~~~~~~

- the ``create_initial_gpg_homedirs`` entry point has been removed in
favor of ``rebuild_gpg_homedirs``.

----------------------

.. _changed-69:

Changed
~~~~~~~

- ``scriptworker.cot.verify.raise_on_errors`` now takes a kwarg of
``level``, which defaults to ``logging.CRITICAL``. This is to support
fuzzy task matching, where not matching a task is non-critical.
- ``scriptworker.cot.verify.verify_link_in_task_graph`` now supports
fuzzy task matching. If the Link’s ``task_id`` isn’t in the task
graph, try to match the task definition against the task graph
definitions, and throw ``CoTError`` on failure. This is to support
Taskcluster retriggers.
- ``verify_cot`` is now an entry point, rather than a helper script in
``scriptworker/test/data/``.

.. _fixed-61:

Fixed
~~~~~

- allowed for ``USE_SCCACHE`` as a build env var

----------------------

.. _added-73:

Added
~~~~~

- ``scriptworker.cot.verify`` now verifies the chain of trust for the
graph.
- ``scriptworker.exceptions.CoTError`` now marks chain of trust
validation errors.
- ``scriptworker.task.get_task_id``, ``scriptworker.task.get_run_id``,
``scriptworker.task.get_decision_task_id``,
``scriptworker.task.get_worker_type``
- ``scriptworker.log.contextual_log_handler`` for short-term logs
- added framework for new docs

.. _changed-70:

Changed
~~~~~~~

- config files are now yaml, to enable comments.
``config_example.json`` and ``cot_config_example.json`` have been
consolidated into ``scriptworker.yaml.tmpl``. ``context.cot_config``
items now live in ``context.config``.
- ``validate_artifact_url`` now takes a list of dictionaries as rules,
leading to more configurable url checking.
- ``scriptworker.cot`` is now ``scriptworker.cot.generate``. The
``get_environment`` function has been renamed to
``get_cot_environment``.
- ``scriptworker.gpg.get_body`` now takes a ``verify_sig`` kwarg.
- ``download_artifacts`` now takes ``valid_artifact_task_ids`` as a
kwarg.
- ``max_connections`` is now ``aiohttp_max_connections``
- scriptworker task definitions now expect an ``upstreamArtifacts``
list of dictionaries

.. _fixed-62:

Fixed
~~~~~

- docstring single backticks are now double backticks
- catch aiohttp exceptions on upload

.. _removed-30:

Removed
~~~~~~~

- removed all references to ``cot_config``
- removed the credential update, since puppet restarts scriptworker on
config change.

.. _section-120: