Dagster-cloud

New

- A handful of changes have been made to URLs in Dagit:
- The `/instance` URL path prefix has been removed. E.g. `/instance/runs` can now be found at `/runs`.
- The `/workspace` URL path prefix has been changed to `/locations`. E.g. the URL for job `my_job` in repository `foobar` can now be found at `/locations/foobar/jobs/my_job`.
- In Dagit, the “Workspace” navigation item in the top nav has been moved to be a tab under the “Deployment” section of the app, and is renamed to “Definitions”.

Dependency Changes

- We’ve upgraded our `dagster/dagster-cloud-agent` Docker image’s base from `python:3.8.12-slim` to `python:3.8.15-slim` to patch [some vulnerabilities in the old base image](https://snyk.io/test/docker/python%3A3.8.12-slim).

New

- A new “Environment Variables” section in the Deployment tab allows you to set environment variables that will be included whenever your Dagster code runs. See [the docs](https://docs.dagster.io/dagster-cloud/developing-testing/environment-variables-and-secrets) for more information. In order for environment variables to be included, your code must be using dagster version 1.0.17 or higher, and if you’re using a Hybrid agent, it must also be at dagster version 1.0.17 or higher.

Bugfixes

- Fixed an issue where stack traces from errors that occurred within user code were sometimes not displayed in Dagit.
- Fixed an issue where failure during automatic retries before run creation would result in alerts not being sent. Now, if a failure occurs during the automatic retry of a run before the next run has been created, a specific alert will be sent.

Documentation

- Updated [Environment variables and secrets documentation](https://docs.dagster.io/dagster-cloud/developing-testing/environment-variables-and-secrets) to include info about using the new “Environment Variables” section in the Deployment tab
- Added a dedicated guide for [setting environment variables using a Hybrid agent’s configuration](https://docs.dagster.io/dagster-cloud/developing-testing/setting-environment-variables-dagster-cloud-agents)

Documentation

- Added documentation for [using environment variables](https://docs.dagster.io/dagster-cloud/developing-testing/environment-variables-and-secrets) in Dagster Cloud.
- Added a [configuration reference for the Docker agent](https://docs.dagster.io/dagster-cloud/deployment/agents/docker/configuration-reference).
- Added example configuration for environment variables and secrets to the [Kubernetes](https://docs.dagster.io/dagster-cloud/deployment/agents/kubernetes/configuration-reference#environment-variables-and-secrets) and [Amazon ECS](https://docs.dagster.io/dagster-cloud/deployment/agents/amazon-ecs/configuration-reference#environment-variables-and-secrets) agent configuration references.

New

- [dagit] The new Overview and Workspace pages have been enabled for all users, after being gated with a feature flag for the last several releases. These changes include design updates, virtualized tables, and more performant querying.
- When running a Dagster Cloud agent, the Python logging configuration of the agent process can now be configured by passing in a `--agent-logging-config-path` option to the `dagster-cloud agent run` command. This config can also be set in the Dagster Cloud Helm chart using the `loggingConfig` key.
- Added a `dagster-cloud settings saml remove-identity-provider-metadata` command that can be used to reset the SAML configuration for an organization.
- You can now re-execute a run from failure in a Serverless deployment in Dagster Cloud without needing to define an IO manager

Bugfixes

- Fixed the `createdBefore` filter on the run GraphQL endpoint.

New

- The default Docker image for Serverless no longer runs `pip install --upgrade pip`.

Experimental

- Non-isolated runs: Previously all runs launched in an isolated run environment. Opting in to this new feature adds a default option to the Launchpad to execute without isolation. The run will start faster, but with fewer compute resources. This is great for testing, but not recommended for compute or memory intensive production jobs. This will only apply to runs launched via the UI. Scheduled runs won't be affected. To enable:
- Upgrade your agent to at least 1.0.15
- In the UI, click on profile icon → User Settings → Enable non-isolated runs

New

- The Amazon ECS agent will now re-use existing task definitions for the services that it spins up for each code location when the configuration for the service’s task definition has not changed. Previously, it would register a new task definition every time the service was re-deployed.

Bugfixes

- Fixed an issue where Serverless images would error on the absence of a script `dagster_cloud_pre_install.sh` inside of the source folder.