Dagster

New

- ops or assets with multiple outputs that are all required and return type `None`/ `Nothing` will interpret an explicitly or implicitly returned value `None` to indicate that all outputs were successful.
- The `skip_reason` argument to the constructor of `SensorResult` now accepts a string in addition to a `SkipReason`.
- [dagster-k8s] Added a `step_k8s_config` field to `k8s_job_executor` that allows you to customize the raw Kubernetes config for each step in a job. See [the docs](https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment#kubernetes-configuration-on-every-step-in-a-run) for more information.
- [dagster-k8s] Launched run pods now have an additional code location label.
- [dagster-ui] The runs table now lets you toggle which tags are always visible.
- [dagster-dbt] `dagster-dbt project scaffold` now creates the scaffold in multiple files:
- `constants.py` contains a reference to your manifest and dbt project directory
- `assets.py` contains your initial dbt assets definitions
- `definitions.py` contains the code to load your asset definitions into the Dagster UI
- `schedules.py` contains an optional schedule to add for your dbt assets
- [dagster-dbt] Added new methods `get_auto_materialize_policy` and `get_freshness_policy` to `DagsterDbtTranslator`.
- [dagster-fivertran] Sync options can now be passed to `load_assets_from_fivetran_instance`.
- [dagster-wandb] W&B IO Manager now handles partitions natively. (Thanks [chrishiste](https://github.com/chrishiste)!)

Bugfixes

- Previously, canceling large asset backfills would cause the daemon to time out and display a “not running” error. This has been fixed.
- [dagster-ssh] Previously the `SSHResource` would warn when `allow_host_key_change` was set. Now known hosts are always loaded from the system hosts file, and the `allow_host_key_change` parameter is ignored.
- Previously, when using AutoMaterializePolicies, partitioned assets downstream of partitioned observable source assets could be materialized before their parent partitions were observed. This has been fixed.

Documentation

- `graph_multi_asset` now has an API docs entry.
- The `GCSComputeLogManager` example in the [Dagster Instance reference](https://docs.dagster.io/deployment/dagster-instance#gcscomputelogmanager) is now correct.
- Several outdated K8s documentation links have been removed from the [Customizing your Kubernetes deployment guide](https://docs.dagster.io/deployment/guides/kubernetes/customizing-your-deployment).
- Added callouts to the [GitHub](https://docs.dagster.io/dagster-cloud/managing-deployments/branch-deployments/using-branch-deployments-with-github) and [GitLab](https://docs.dagster.io/dagster-cloud/managing-deployments/branch-deployments/using-branch-deployments-with-gitlab) Branch Deployment guides specifying that some steps are optional for Serverless users.
- The “Graphs” page under the “Concepts” section has been renamed to “Op Graphs” and moved inside under the “Ops” heading.
- [dagster-dbt] Added API examples for `dbt_assets` for the following use-cases:
- Running dbt commands with flags
- Running dbt commands with `--vars`
- Running multiple dbt commands
- Retrieving dbt artifacts after running a dbt command
- Invoking other Dagster resouces alongside dbt
- Defining and accessing Dagster config alongside dbt

Dagster Cloud

- The viewer role now has permission to edit their own user tokens.

New

- `graph_asset` now takes a `config` parameter equivalent to the parameter on `graph`.
- Added an optional `dynamic_partitions_store` argument to `DynamicPartitionsDefinition` for multi-partition run properly with dynamic partitions (Thanks elzzz!).
- [dagster-grpahql] Added ` partitionsByAssets` to `backfillParams` for ranged partition backfill (Thanks ruizh22!).
- [dagster-dbt] Support for `dbt-core==1.6` has been added.
- [dagster-dbt] `DbtCliResource` now supports configuring `profiles_dir`.
- [dagster-k8s] Allow specifying `restart_policy` on `k8s_job_op` (Thanks Taadas!).
- [dagster-snowflake] Added `authenticator` to `SnowflakePandasIOManager`, which allows specifying the authentication mechanism to use (Thanks pengw0048!).
- [ui] The Asset > Events page now allows you to filter by event type, making it easy to hide observations.
- [ui] The Asset > Partitions page checks to see if the selected partition is stale and displays stale causes in the details pane.
- [ui] Hovering over "Fresh" tags now provides detailed information about why the last materialization meets the asset's freshness policy.
- [ui] The global asset graph can now be filtered to display a subset of the available asset groups.

Bugfixes

- In some situations, multiple materializations of the same asset could be kicked off when using a lazy `AutoMaterializePolicy` with assets that had at least one source asset parent and at least one non-source asset parent. This has been fixed.
- After applying an eager `AutoMaterializePolicy` to a time-partitioned asset downstream of an unpartitioned asset, the latest partition would only ever be materialized a single time, rather than updating in response to any parent updates. This has been fixed.
- Fixed an issue that would cause the creation of a `StaticPartitionsDefinition` containing many thousands of partitions could take a significant amount of time.
- The run coordinator daemon now uses a fresh request context on each iteration, fixing an issue where stale grpc server references could be used in certain high volume conditions.
- Automatically generated data versions for partitioned assets now correctly reflect the data versions of upstream partitions. Previously, they were computed using the data versions from the most recent materializations of upstream assets regardless of partition.
- [dagster-airbyte] Previously, attempting to load assets from an Airbyte instance in which some of the tables had hyphens in their name would result in an error. This has been fixed.
- [dagster-dbt] Previously, attempting to load assets from a dbt project in which some of the models had hyphens in their name would result in an error. This has been fixed.
- [dagstermill] Fixed a bug where known state for executing dagstermill ops was not correctly passed in (Thanks motuzov!).
- [ui] Pressing the up or down arrow key without a selection in the asset graph no longer causes a page error.

Documentation

- Added the starter project’s template for Dagster University.
- Fixed an incorrect method name in DagsterDbtTranslator Docs (Thanks akan72!).

Dagster Cloud

- When importing a dbt project on the Dagster Cloud setup page, an `Unexpected exception` error would be raised when scaffolding a pull request on a repository with no `profiles.yml`. This behavior has been updated to raise a more descriptive error message on the repo selection page.
- The running multiple agents guide has been revamped to discuss running agent replicas and zero-downtime deployment of the agent.
- The `agentReplicas` config setting on the helm chart has been renamed to `isolatedAgents`. In order to use this config setting, your user code dagster version needs to be `1.4.3` or greater.

New

- [ui] When viewing a run for auto-materialized assets, show a tag with information about the assets that were materialized.
- [ui] In the Auto-materialize History view, when one or more of an asset’s parents have been updated, the set of updated parents will be viewable.
- [ui] Link to the auto-materialized history for an asset from the asset DAG view.
- [ui] For runs that were the result of auto-observation, show a tag for this in the Runs list view.
- Added warnings for storage incompatibility with the experimental global op concurrency.

Bugfixes

- [dagster-dbt] Fixed an issue where `dagster-dbt project scaffold` didn’t create a project directory with all the scaffolded files.
- Fixed an issue which could cause errors when using the `SpecificPartitionsPartitionMapping` with auto-materialization.

Breaking Change

- Previously, it was possible to set `max_materializations_per_minute` on an `AutoMaterializePolicy` to a non-positive number. This will now result in an error.

Community Contributions

- Fix for loading multipartitions paths in `upath_io_manager` from harrylojames; thank you!
- Docs typo fix from C0DK; thank you!

Documentation

- Revamped the dagster-dbt tutorial to take advantage of `dagster project scaffold` and the new dagster-dbt APIs.

New

- [dagster-dbt] When invoking`dagster-dbt project scaffold` on a dbt project directory, if a `profiles.yml` exists in the root of the directory, its contents are used to add dbt adapter packages to the scaffolded `setup.py`.
- The default sentinel value for the multiprocessing executor’s `max_concurrent` field has been changed from `0` to `None` to more clearly signal its intent. A value of `0` is still interpreted as the sentinel value which dynamically allocates `max_concurrent` based on detected CPU count.

Bugfixes

- IO managers defined on jobs will now be properly merged with resources defined in `Definitions`, so that jobs are able to override the IO manager used.
- [dagster-fivetran] Fixed an issue where `EnvVars` in a `FivetranResource` would not be evaluated when loading assets from the Fivetran instance.
- [dagster-airbyte] Fixed an issue where `EnvVars` in an `AirbyteResource` would not be evaluated when loading assets from the Airbyte resource.

Documentation

- [dagster-dbt] Added API docs for `DbtCliResource`, `DbtCliInvocation`, `dbt_assets`, `DagsterDbtTranslator`, `dagster-dbt project scaffold`
- [dagster-dbt] Expanded references for new APIs:
- Added documentation to customize asset definition attributes for dbt assets
- Added documentation to define upstream and downstream dependencies to dbt assets
- Added documentation to define schedules for dbt assets

Dagster Cloud

- The experimental agent config setting `agent_replicas`has been deprecated in favor of a new name `isolated_agents` (`agentReplicas` --> `isolatedAgents` in the helm chart). Upgrading to the new name requires all code locations to be on `1.4.3` or greater.

Bugfixes

- Fixes a bug in `dagster-dbt` that was preventing it from correctly materializing subselections of dbt asset.

Bugfixes

- Fixes a bug in `dagster-dbt` that was preventing it efficiently loading dbt projects from a manifest.