Scriptworker

--------------------

.. _added-74:

Added
~~~~~

- ``gpg_lockfile`` and ``last_good_git_revision_file`` in config
- ``get_last_good_git_revision`` and ``write_last_good_git_revision``
now return the last good git revision, and write it to
``last_good_git_revision_file``, respectively.
- ``get_tmp_base_gpg_home_dir`` is a helper function to avoid
duplication in logic.
- ``rebuild_gpg_homedirs`` is a new entry point script that allows us
to recreate the gpg homedirs in a tmpdir, in a separate process
- ``is_lockfile_present``, ``create_lockfile``, and ``rm_lockfile`` as
helper functions for the two gpg homedir entry points.

.. _changed-71:

Changed
~~~~~~~

- ``sign_key``, ``rebuild_gpg_home_flat``, ``rebuild_gpg_home_signed``,
``build_gpg_homedirs_from_repo`` are no longer async.
- ``overwrite_gpg_home`` only keeps one backup.
- ``update_signed_git_repo`` now returns the latest git revision,
instead of a boolean marking whether the revision is new or not. This
will help avoid the scenario where we update, fail to generate the
gpg homedirs, and then stay on an old revision until the next push.
- ``update_logging_config`` now takes a ``file_name`` kwarg, which
allows us to create new log files for the ``rebuild_gpg_homedirs``
and ``create_initial_gpg_homedirs`` entry points.

.. _fixed-63:

Fixed
~~~~~

- ``build_gpg_homedirs_from_repo`` now waits to verify the contents of
the updated git repo before nuking the previous base gpg homedir.
- ``create_initial_gpg_homedirs`` now creates a logfile

.. _removed-31:

Removed
~~~~~~~

- ``rebuild_gpg_homedirs_loop`` is no longer needed, and is removed.

.. _section-121:

--------------------

.. _changed-72:

Changed
~~~~~~~

- logged the stacktrace if the ``main`` loop hits an exception. No
longer catch and ignore ``RuntimeError``, since it wasn’t clear why
that was put in.
- updated ``check_config`` to make sure taskcluster-related configs
match taskcluster requirements

.. _fixed-64:

Fixed
~~~~~

- changed the way the polling loop works: ``async_main`` is now a
single pass, which ``main`` calls in a ``while True`` loop. This
should fix the situation where polling was dying silently while the
git update loop continued running every 5 minutes.

.. _section-122:

--------------------

.. _fixed-65:

Fixed
~~~~~

- explicitly pass ``taskId`` and ``runId`` to ``claim_task``. There’s a
new ``hintId`` property that appears in ``message_info['task_info']``
that broke things.

.. _section-123:

--------------------

.. _added-75:

Added
~~~~~

- added ``git_key_repo_dir``, ``base_gpg_home_dir``, ``my_email``, and
``gpg_path`` to ``config_example.json``
- added ``cot_config_example.json``, ``cot_config_schema.json``, and
``scriptworker.config.get_cot_config`` for ChainOfTrust config
- added ``update_signed_git_repo``, ``verify_signed_git_commit``,
``build_gpg_homedirs_from_repo``, ``rebuild_gpg_homedirs_loop``, and
``create_initial_gpg_homedirs`` for gpg homedir creation and updates
in the background.
- added a background call to update the gpg homedirs in
``scriptworker.worker.async_main``
- added another entry point, ``create_initial_gpg_homedirs``, for
puppet to create the first gpg homedirs

.. _changed-73:

Changed
~~~~~~~

- default config filename is now ``scriptworker.json`` instead of
``config.json``
- moved ``scriptworker.config.get_context_from_cmdln`` out of
``scriptworker.worker.main``; now using argparse
- changed default ``sign_chain_of_trust`` to True
- ``scriptworker.gpg.sign_key``,
``scriptworker.gpg.rebuild_gpg_home_flat``, and
``scriptworker.gpg.rebuild_gpg_home_signed`` are now async, so they
can happen in parallel in the background
- renamed ``scriptworker.gpg.latest_signed_git_commit`` to
``scriptworker.gpg.verify_signed_git_commit_output``
- combined ``scriptworker.log.log_errors`` and
``scriptworker.log.read_stdout`` into
``scriptworker.log.pipe_to_log``
- added ``taskGroupId`` to the list of default valid ``taskId``\ s to
download from. This logic will need to change in version 0.9.0 due to
the new `chain of trust dependency traversal
logic <https://gist.github.com/escapewindow/a6a6944f51d4219d08284ededc65aa30>`__

.. _fixed-66:

Fixed
~~~~~

- added missing docstrings to the ``download_artifacts`` and
``download_file`` functions
- fixed coverage version in ``tox.ini py35-coveralls``
- ``sign_key`` now supports signing keys with multiple subkeys

.. _section-124:

--------------------

.. _added-76:

Added
~~~~~

- added ``DownloadError`` exception for ``download_file``
- added ``scriptworker.task.download_artifacts``
- added ``scriptworker.util.download_file``

.. _changed-74:

Changed
~~~~~~~

- ``DEFAULT_CONFIG``, ``STATUSES``, and ``REVERSED_STATUSES`` have
moved to ``scriptworker.constants``.
- ``list_to_tuple`` has been renamed ``freeze_values``, and also
converts dict values to frozendicts.

.. _section-125:

--------------------

.. _added-77:

Added
~~~~~

- significant gpg support
- ability to create new gpg homedirs
- scriptworker now requires ``pexpect`` for gpg key signing
- docstrings!
- helper scripts to generate 1000 pubkeys and time importing them.
- added ``scriptworker.utils.rm`` as an ``rm -rf`` function

.. _changed-75:

Changed
~~~~~~~

- ``utils.makedirs`` now throws ``ScriptWorkerException`` if the path
exists and is not a directory or a softlink pointing to a directory.
- gpg functions now take a ``gpg_home`` kwarg to specify a different
homedir
- moved ``scriptworker.client.integration_create_task_payload`` into
``scriptworker.test``
- renamed ``scriptworker.util.get-_hash`` kwarg ``hash_type`` to
``hash_alg``
- renamed ``firefox_cot_schema.json`` to ``cot_v1_schema.json``; also,
the schema has changed.
- the chain of trust schema has changed to version 1.

.. _fixed-67:

Fixed
~~~~~

- pass a ``task`` to ``scriptworker.task.reclaimTask`` and exit the
loop if it doesn’t match ``context.task``
- we now verify that ``context.task`` is the same task we scheduled
``reclaim_task`` for.

.. _removed-32:

Removed
~~~~~~~

- Removed ``get_temp_creds_from_file``, since we’re not writing
``temp_creds`` to disk anymore
- Removed ``scriptworker.task.get_temp_queue``, since we already have
``context.temp_queue``
- Removed ``pytest-asyncio`` dependency. It doesn’t play well with
``pytest-xdist``.
- Removed ``scriptworker.task.get_temp_queue``; we can use
``context.temp_queue``
- Removed ``pytest-asyncio`` usage to try to use ``pytest-xdist``, then
turned that back off when it conflicted with the event loop

.. _section-126: