Dagster

Bugfixes

- Fixed an issue where `dagster dev` or the Dagster UI would display an error when loading jobs created with op or asset selections.

New

- Latency improvements to the scheduler when running many simultaneous schedules.

Bugfixes

- The performance of loading the Definitions snapshot from a code server when large `multi_asset` s are in use has been drastically improved.
- The snowflake quickstart example project now renames the “by” column to avoid reserved snowflake names. Thanks [jcampbell](https://github.com/jcampbell)!
- The existing group name (if any) for an asset is now retained if `the_asset.with_attributes` is called without providing a group name. Previously, the existing group name was erroneously dropped. Thanks [ion-elgreco](https://github.com/ion-elgreco)!
- [dagster-dbt] Fixed an issue where Dagster events could not be streamed from `dbt source freshness`.
- [dagster university] Removed redundant use of `MetadataValue` in Essentials course. Thanks [stianthaulow](https://github.com/stianthaulow)!
- [ui] Increased the max number of plots on the asset plots page to 100.

Breaking Changes

- The `tag_keys` argument on `DagsterInstance.get_run_tags`is no longer optional. This has been done to remove an easy way of accidentally executing an extremely expensive database operation.

Dagster Cloud

- The maximum number of concurrent runs across all branch deployments is now configurable. This setting can now be set via GraphQL or the CLI.
- [ui] In Insights, fixed display of table rows with zero change in value from the previous time period.
- [ui] Added deployment-level Insights.
- [ui] Fixed an issue causing void invoices to show up as “overdue” on the billing page.
- [experimental] Branch deployments can now indicate the new and modified assets in the branch deployment as compared to the main deployment. To enable this feature, turn on the “Enable experimental branch deployment asset graph diffing” user setting.

New

- [ui] When viewing logs for a run, the date for a single log row is now shown in the tooltip on the timestamp. This helps when viewing a run that takes place over more than one date.
- Added suggestions to the error message when selecting asset keys that do not exist as an upstream asset or in an `AssetSelection.`
- Improved error messages when trying to materialize a subset of a multi-asset which cannot be subset.
- [dagster-snowflake] `dagster-snowflake` now requires `snowflake-connector-python>=3.4.0`
- [embedded-elt] `sling_assets` accepts an optional name parameter for the underlying op
- [dagster-openai] `dagster-openai` library is now available.
- [dagster-dbt] Added a new setting on `DagsterDbtTranslatorSettings` called `enable_duplicate_source_asset_keys` that allows users to set duplicate asset keys for their dbt sources. Thanks hello-world-bfree!
- Log messages in the Dagster daemon for unloadable sensors and schedules have been removed.
- [ui] Search now uses a cache that persists across pageloads which should greatly improve search performance for very large orgs.
- [ui] groups/code locations in the asset graph’s sidebar are now sorted alphabetically.

Bugfixes

- Fixed issue where the input/output schemas of configurable IOManagers could be ignored when providing explicit input / output run config.
- Fixed an issue where enum values could not properly have a default value set in a `ConfigurableResource`.
- Fixed an issue where graph-backed assets would sometimes lose user-provided descriptions due to a bug in internal copying.
- [auto-materialize] Fixed an issue introduced in 1.6.7 where updates to ExternalAssets would be ignored when using AutoMaterializePolicies which depended on parent updates.
- [asset checks] Fixed a bug with asset checks in step launchers.
- [embedded-elt] Fix a bug when creating a `SlingConnectionResource` where a blank keyword argument would be emitted as an environment variable
- [dagster-dbt] Fixed a bug where emitting events from `dbt source freshness` would cause an error.
- [ui] Fixed a bug where using the “Terminate all runs” button with filters selected would not apply the filters to the action.
- [ui] Fixed an issue where typing a search query into the search box before the search data was fetched would yield “No results” even after the data was fetched.

Community Contributions

- [docs] fixed typo in embedded-elt.mdx (thanks [cameronmartin](https://github.com/cameronmartin))!
- [dagster-databricks] log the url for the run of a databricks job (thanks [smats0n](https://github.com/smats0n))!
- Fix missing partition property (thanks [christeefy](https://github.com/christeefy))!
- Add op_tags to observable_source_asset decorator (thanks [maxfirman](https://github.com/maxfirman))!
- [docs] typo in MultiPartitionMapping docs (thanks [dschafer](https://github.com/dschafer))
- Allow github actions to checkout branch from forked repo for docs changes (ci fix) (thanks [hainenber](https://github.com/hainenber))!

Experimental

- [asset checks] UI performance of asset checks related pages has been improved.
- [dagster-dbt] The class `DbtArtifacts` has been added for managing the behavior of rebuilding the manifest during development but expecting a pre-built one in production.

Documentation

- Added example of writing compute logs to AWS S3 when customizing agent configuration.
- "Hello, Dagster" is now "Dagster Quickstart" with the option to use a Github Codespace to explore Dagster.
- Improved guides and reference to better running multiple isolated agents with separate queues on ECS.

Dagster Cloud

- Microsoft Teams is now supported for alerts. [Documentation](https://docs.dagster.io/dagster-cloud/managing-deployments/setting-up-alerts)
- A `send sample alert` button now exists on both the alert policies page and in the alert policies editor to make it easier to debug and configure alerts without having to wait for an event to kick them off.

Bugfixes

- [dagster-embedded-elt] Fixed a bug in the `SlingConnectionResource` that raised an error when connecting to a database.

Experimental

- [asset checks] `graph_multi_assets` with `check_specs` now support subsetting.

New

- Added a new `run_retries.retry_on_op_or_asset_failures` setting that can be set to false to make run retries only occur when there is an unexpected failure that crashes the run, allowing run-level retries to co-exist more naturally with op or asset retries. See [the docs](https://docs.dagster.io/deployment/run-retries#combining-op-and-run-retries) for more information.
- `dagster dev` now sets the environment variable `DAGSTER_IS_DEV_CLI` allowing subprocesses to know that they were launched in a development context.
- [ui] The Asset Checks page has been updated to show more information on the page itself rather than in a dialog.

Bugfixes

- [ui] Fixed an issue where the UI disallowed creating a dynamic partition if its name contained the “|” pipe character.
- AssetSpec previously dropped the metadata and code_version fields, resulting in them not being attached to the corresponding asset. This has been fixed.

Experimental

- The new `multi_observable_source_asset` decorator enables defining a set of assets that can be observed together with the same function.
- [dagster-embedded-elt] New Asset Decorator `sling_assets` and Resource `SlingConnectionResource` have been added for the `[dagster-embedded-elt.sling](http://dagster-embedded-elt.sling)` package. Deprecated `build_sling_asset`, `SlingSourceConnection` and `SlingTargetConnection`.
- Added support for op-concurrency aware run dequeuing for the `QueuedRunCoordinator`.

Documentation

- Fixed reference documentation for isolated agents in ECS.
- Corrected an example in the Airbyte Cloud documentation.
- Added API links to OSS Helm deployment guide.
- Fixed in-line pragmas showing up in the documentation.

Dagster Cloud

- Alerts now support Microsoft Teams.
- [ECS] Fixed an issue where code locations could be left undeleted.
- [ECS] ECS agents now support setting multiple replicas per code server.
- [Insights] You can now toggle the visibility of a row in the chart by clicking on the dot for the row in the table.
- [Users] Added a new column “Licensed role” that shows the user's most permissive role.

New

- Dagster officially supports Python 3.12.
- `dagster-polars` has been added as an integration. Thanks danielgafni!
- [dagster-dbt] `dbt_assets` now supports loading projects with semantic models.
- [dagster-dbt] `dbt_assets` now supports loading projects with model versions.
- [dagster-dbt] `get_asset_key_for_model` now supports retrieving asset keys for seeds and snapshots. Thanks aksestok!
- [dagster-duckdb] The Dagster DuckDB integration supports DuckDB version 0.10.0.
- [UPath I/O manager] If a non-partitioned asset is updated to have partitions, the file containing the non-partitioned asset data will be deleted when the partitioned asset is materialized, rather than raising an error.

Bugfixes

- Fixed an issue where creating a backfill of assets with dynamic partitions and a backfill policy would sometimes fail with an exception.
- Fixed an issue with the type annotations on the `asset` decorator causing a false positive in Pyright strict mode. Thanks tylershunt!
- [ui] On the asset graph, nodes are slightly wider allowing more text to be displayed, and group names are no longer truncated.
- [ui] Fixed an issue where the groups in the asset graph would not update after an asset was switched between groups.
- [dagster-k8s] Fixed an issue where setting the `security_context` field on the `k8s_job_executor` didn't correctly set the security context on the launched step pods. Thanks krgn!

Experimental

- Observable source assets can now yield `ObserveResult`s with no `data_version`.
- You can now include `FreshnessPolicy`s on observable source assets. These assets will be considered “Overdue” when the latest value for the “dagster/data_time” metadata value is older than what’s allowed by the freshness policy.
- [ui] In Dagster Cloud, a new feature flag allows you to enable an overhauled asset overview page with a high-level stakeholder view of the asset’s health, properties, and column schema.

Documentation

- Updated docs to reflect newly-added support for Python 3.12.

Dagster Cloud

- [kubernetes] Fixed an issue where the Kubernetes agent would sometimes leave dangling kubernetes services if the agent was interrupted during the middle of being terminated.