Taskcluster

GENERAL

▶ [patch]
Upgrades go to 1.21.0 and node to 18.17.1

USERS

▶ [patch] [6440](https://github.com/taskcluster/taskcluster/issues/6440)
Generic Worker now allocates a pseudo tty when running Docker Worker tasks, to
emulate Docker Worker behavior. Previously it did not allocate a tty, which
could result in e.g. output not being colored.

OTHER

▶ Additional changes not described here: [6400](https://github.com/taskcluster/taskcluster/issues/6400), [#6414](https://github.com/taskcluster/taskcluster/issues/6414), [#6442](https://github.com/taskcluster/taskcluster/issues/6442).

Automated Package Updates

<details>
<summary>11 Dependabot/Renovate updates</summary>

* build(deps-dev): bump eslint in /clients/client-web (7b132ce69)
* build(deps-dev): bump zen-observable from 0.8.15 to 0.10.0 (b6098fade)
* build(deps): bump github.com/iancoleman/strcase from 0.2.0 to 0.3.0 (deea49585)
* build(deps-dev): bump babel/core in /clients/client-web (c5e7d8ab6)
* build(deps-dev): bump karma-coverage in /clients/client-web (8c0c79322)
* build(deps): bump pyyaml from 6.0 to 6.0.1 in /taskcluster (f818ba817)
* build(deps): bump taskcluster from 54.1.2 to 54.4.1 in /taskcluster (06d02af96)
* build(deps-dev): bump nock from 13.3.1 to 13.3.2 in /clients/client (58f394be6)
* build(deps): bump aws-sdk from 2.1376.0 to 2.1426.0 (bfb701686)
* build(deps): bump golang.org/x/tools from 0.11.0 to 0.11.1 (7420158d5)
* build(deps): bump certifi from 2023.5.7 to 2023.7.22 in /taskcluster (5f364270a)

</details>

GENERAL

▶ [patch]
Upgrade Node.js to 18.17.0.

ADMINS

▶ [patch] [6405](https://github.com/taskcluster/taskcluster/issues/6405)
Expire artifacts job no longer logs errors for each missing artifact. Instead it reports the number of missing artifacts at the end of the job.

USERS

▶ [patch]
Adds a task log letting the user know their Docker Worker payload is being converted to a Generic Worker payload using d2g.

Automated Package Updates

<details>
<summary>1 Dependabot/Renovate updates</summary>

* build(deps): bump aiohttp from 3.8.4 to 3.8.5 in /taskcluster (56a9903ae)

</details>

USERS

▶ [minor] [5961](https://github.com/taskcluster/taskcluster/issues/5961)
Generic Worker now supports the `osGroups` feature on macOS, Linux and
FreeBSD. Support was already added to Windows in Generic Worker 6.0.0.

Example Linux/macOS task (requires `docker` to be installed on worker):


created: <timestamp>
deadline: <timestamp>
workerType: my-worker-type
provisionerId: mv-provisioner-id
scopes:
- generic-worker:os-group:my-provisioner-id/my-worker-type/docker
payload:
osGroups:
- docker
command:
- - docker
- run
- --rm
- ubuntu:latest
- /usr/bin/echo
- hello
maxRunTime: 60
metadata:
name: Ubuntu - docker test
owner: pmooremozilla.com
source: https://github.com/taskcluster/taskcluster/pull/6397
description: Test calling docker from a Generic Worker task

GENERAL

▶ [patch] [6420](https://github.com/taskcluster/taskcluster/issues/6420)
Fixes generic worker issue where artifacts were no longer being uploaded.

ADMINS

▶ [patch] [6405](https://github.com/taskcluster/taskcluster/issues/6405)
Expire artifacts handles the case where the artifact is not found during deletion. GCS behaves differently to S3 here, as it will throw an error if the artifact is not found, where S3 would always return 204.

USERS

▶ [minor] [5967](https://github.com/taskcluster/taskcluster/issues/5967)
This change integrates the `d2g` tool into Generic Worker so that it can accept a valid, Docker Worker payload.

▶ [patch] [6417](https://github.com/taskcluster/taskcluster/issues/6417)
Generic Worker: Interactive sessions suffered from a race condition that was introduced in Generic Worker 54.2.0. This has been fixed.

USERS

▶ [minor] [6405](https://github.com/taskcluster/taskcluster/issues/6405)
Expire artifacts supports both bulk deletion and single deletion. This can be configured for the deployment using `AWS_USE_BULK_DELETE` environment variable (`false` by default). This is needed because not all S3 compatible storages support bulk delete, specifically [GCS](https://cloud.google.com/storage/docs/migrating#methods-comparison).
`EXPIRE_ARTIFACTS_BATCH_SIZE` can be used to control how many records to process at once, i.e. how many parallel delete requests would be sent to storage service (`100` by default).

DEVELOPERS

▶ [patch] [6395](https://github.com/taskcluster/taskcluster/issues/6395)
Fixed local development environment where artifacts could not be loaded in the UI. This was caused by not using pinned `minio/*` images.

▶ [patch] [6395](https://github.com/taskcluster/taskcluster/issues/6395)
Local development environment now supports live log.