Dagster

**Breaking Changes**

- The `dagit` key is no longer part of the instance configuration schema and must be removed from `dagster.yaml` files before they can be used.
- `-d` can no longer be used as a command-line argument to specify a mode. Use `--mode` instead.
- Use `--preset` instead of `--preset-name` to specify a preset to the `pipeline launch` command.
- We have removed the `config` argument to the `ConfigMapping`, `composite_solid`, `solid`, `SolidDefinition`, `executor`, `ExecutorDefinition`, `logger`, `LoggerDefinition`, `resource`, and `ResourceDefinition` APIs, which we deprecated in 0.8.0. Use `config_schema` instead.

**New**

- Python 3.8 is now fully supported.
- `-d` or `--working-directory` can be used to specify a working directory in any command that
takes in a `-f` or `--python_file` argument.
- Removed the deprecation of `create_dagster_pandas_dataframe_type`. This is the currently
supported API for custom pandas data frame type creation.
- Removed gevent dependency from dagster
- New `configured` API for predefining configuration for various definitions: https://legacy-docs.dagster.io/overview/configuration/#configured
- Added hooks to enable success and failure handling policies on pipelines. This enables users to set up policies on all solids within a pipeline or on a per solid basis. Example usage can be found [here](https://legacy-docs.dagster.io/examples/hooks)
- New instance level view of Scheduler and running schedules
- dagster-graphql is now only required in dagit images.

**Breaking Changes**

- `AssetMaterializations` no longer accepts a `dagster_type` argument. This reverts the change
billed as "`AssetMaterializations` can now have type information attached as metadata." in the
previous release.

**New**

- Added new GCS and Azure file manager resources
- `AssetMaterializations` can now have type information attached as metadata. See the materializations tutorial for more
- Added verification for resource arguments (previously only validated at runtime)

**Bugfixes**

- Fixed bug with order-dependent python module resolution seen with some packages (e.g. numpy)
- Fixed bug where Airflow's `context['ts']` was not passed properly
- Fixed a bug in celery-k8s when using `task_acks_late: true` that resulted in a `409 Conflict error` from Kubernetes. The creation of a Kubernetes Job will now be aborted if another Job with the same name exists
- Fixed a bug with composite solid output results when solids are skipped
- Hide the re-execution button in Dagit when the pipeline is not re-executable in the currently loaded repository

**Docs**

- Fixed code example in the advanced scheduling doc (Thanks wingyplus!)
- Various other improvements

**New**

- `CeleryK8sRunLauncher` supports termination of pipeline runs. This can be accessed via the
“Terminate” button in Dagit’s Pipeline Run view or via “Cancel” in Dagit’s All Runs page. This
will terminate the run master K8s Job along with all running step job K8s Jobs; steps that are
still in the Celery queue will not create K8s Jobs. The pipeline and all impacted steps will
be marked as failed. We recommend implementing resources as context managers and we will execute
the finally block upon termination.
- `K8sRunLauncher` supports termination of pipeline runs.
- `AssetMaterialization` events display the asset key in the Runs view.
- Added a new "Actions" button in Dagit to allow to cancel or delete mulitple runs.

**Bugfixes**

- Fixed an issue where `DagsterInstance` was leaving database connections open due to not being
garbage collected.
- Fixed an issue with fan-in inputs skipping when upstream solids have skipped.
- Fixed an issue with getting results from composites with skippable outputs in python API.
- Fixed an issue where using `Enum` in resource config schemas resulted in an error.

**New**

- The new `configured` API makes it easy to create configured versions of resources.
- Deprecated the `Materialization` event type in favor of the new `AssetMaterialization` event type,
which requires the `asset_key` parameter. Solids yielding `Materialization` events will continue
to work as before, though the `Materialization` event will be removed in a future release.
- We are starting to deprecate "system storages" - instead of pipelines having a system storage
definition which creates an intermediate storage, pipelines now directly have an intermediate
storage definition.
- We have added an `intermediate_storage_defs` argument to `ModeDefinition`, which accepts a
list of `IntermediateStorageDefinition`s, e.g. `s3_plus_default_intermediate_storage_defs`.
As before, the default includes an in-memory intermediate and a local filesystem intermediate
storage.
- We have deprecated `system_storage_defs` argument to `ModeDefinition` in favor of
`intermediate_storage_defs`. `system_storage_defs` will be removed in 0.10.0 at the earliest.
- We have added an `intermediate_storage` decorator, which makes it easy to define intermediate
storages.
- We have added `s3_file_manager` and `local_file_manager` resources to replace the file managers
that previously lived inside system storages. The airline demo has been updated to include
an example of how to do this:
https://github.com/dagster-io/dagster/blob/0.8.8/examples/airline_demo/airline_demo/solids.py#L171.
- The help panel in the dagit config editor can now be resized and toggled open or closed, to
enable easier editing on smaller screens.

**Bugfixes**

- Opening new Dagit browser windows maintains your current repository selection. 2722
- Pipelines with the same name in different repositories no longer incorrectly share playground state. 2720
- Setting `default_value` config on a field now works as expected. 2725
- Fixed rendering bug in the dagit run reviewer where yet-to-be executed execution steps were
rendered on left-hand side instead of the right.

**Breaking Changes**

- Loading python modules reliant on the working directory being on the PYTHONPATH is no longer
supported. The `dagster` and `dagit` CLI commands no longer add the working directory to the
PYTHONPATH when resolving modules, which may break some imports. Explicitly installed python
packages can be specified in workspaces using the `python_package` workspace yaml config option.
The `python_module` config option is deprecated and will be removed in a future release.

**New**

- Dagit can be hosted on a sub-path by passing `--path-prefix` to the dagit CLI. 2073
- The `date_partition_range` util function now accepts an optional `inclusive` boolean argument. By default, the function does not return include the partition for which the end time of the date range is greater than the current time. If `inclusive=True`, then the list of partitions returned will include the extra partition.
- `MultiDependency` or fan-in inputs will now only cause the solid step to skip if all of the
fanned-in inputs upstream outputs were skipped

**Bugfixes**

- Fixed accidental breaking change with `input_hydration_config` arguments
- Fixed an issue with yaml merging (thanks shasha79!)
- Invoking `alias` on a solid output will produce a useful error message (thanks iKintosh!)
- Restored missing run pagination controls
- Fixed error resolving partition-based schedules created via dagster schedule decorators (e.g. `daily_schedule`) for certain workspace.yaml formats