Astronomer-cosmos

------------------

New Features

* Introduction of experimental support to run dbt BQ models using Airflow deferrable operators by pankajkoti pankajastro tatiana in 1224 1230.
This is a first step in this journey and we would really appreciate feedback from the community.

For more information, check the documentation: https://astronomer.github.io/astronomer-cosmos/getting_started/execution-modes.html#airflow-async-experimental

This work has been inspired by the talk "Airflow at Monzo: Evolving our data platform as the bank scales" by
jonathanrainer ed-sparkes given at Airflow Summit 2023: https://airflowsummit.org/sessions/2023/airflow-at-monzo-evolving-our-data-platform-as-the-bank-scales/.

* Support using ``DatasetAlias`` and fix orphaning unreferenced dataset by tatiana in 1217 1240

Documentation: https://astronomer.github.io/astronomer-cosmos/configuration/scheduling.html#data-aware-scheduling

* Add GCP_CLOUD_RUN_JOB execution mode by ags-de 1153

Learn more about it: https://astronomer.github.io/astronomer-cosmos/getting_started/gcp-cloud-run-job.html

Enhancements

* Create single virtualenv when ``DbtVirtualenvBaseOperator`` has ``virtualenv_dir=None`` and ``is_virtualenv_dir_temporary=True`` by kesompochy in 1200
* Consistently handle build and imports in ``cosmos/__init__.py`` by tatiana in 1215
* Add enum constants to init for direct import by fabiomx in 1184

Bug fixes

* URL encode dataset names to support multibyte characters by t0momi219 in 1198
* Fix invalid argument (``full_refresh``) passed to DbtTestAwsEksOperator (and others) by johnhoran in 1175
* Fix ``printer_width`` arg type in ``DbtProfileConfigVars`` by jessicaschueler in 1191
* Fix task owner fallback by jmaicher in 1195

Docs

* Add scarf to readme and docs for website analytics by cmarteepants in 1221
* Add ``virtualenv_dir`` param to ``ExecutionConfig`` docs by pankajkoti in 1173
* Give credits to LennartKloppenburg in CHANGELOG.rst by tatiana 1174
* Refactor docs for async mode execution by pankajkoti in 1241

Others

* Remove PR branch added for testing a change in CI in 1224 by pankajkoti in 1233
* Fix CI wrt broken coverage upload artifact pankajkoti in 1210
* Fix CI issues - Upgrade actions/upload-artifact & actions/download-artifact to v4 and set min version for packaging by pankajkoti in 1208
* Resolve CI failures for Apache Airflow 2.7 jobs by pankajkoti in 1182
* CI: Update GCP manifest file path based on new secret update by pankajkoti in 1237
* Pre-commit hook updates in 1176 1186, 1186, 1201, 1219, 1231

--------------------

New Features

* Add support for loading manifest from cloud stores using Airflow Object Storage by pankajkoti in 1109
* Cache ``package-lock.yml`` file by pankajastro in 1086
* Support persisting the ``LoadMode.VIRTUALENV`` directory LennartKloppenburg and tatiana in 1079 and 611
* Add support to store and fetch ``dbt ls`` cache in remote stores by pankajkoti in 1147
* Add default source nodes rendering by arojasb3 in 1107
* Add Teradata ``ProfileMapping`` by sc250072 in 1077

Enhancements

* Add ``DatabricksOauthProfileMapping`` profile by CorsettiS in 1091
* Use ``dbt ls`` as the default parser when ``profile_config`` is provided by pankajastro in 1101
* Add task owner to dbt operators by wornjs in 1082
* Extend Cosmos custom selector to support + when using paths and tags by mvictoria in 1150
* Simplify logging by dwreeves in 1108

Bug fixes

* Fix Teradata ``ProfileMapping`` target invalid issue by sc250072 in 1088
* Fix empty tag in case of custom parser by pankajastro in 1100
* Fix ``dbt deps`` of ``LoadMode.DBT_LS`` should use ``ProjectConfig.dbt_vars`` by tatiana in 1114
* Fix import handling by lazy loading hooks introduced in PR 1109 by dwreeves in 1132
* Fix Airflow 2.10 regression and add Airflow 2.10 in test matrix by pankajastro in 1162

Docs

* Fix typo in azure-container-instance docs by pankajastro in 1106
* Use Airflow trademark as it has been registered by pankajastro in 1105

Others

* Run some example DAGs in Kubernetes execution mode in CI by pankajastro in 1127
* Install requirements.txt by default during dev env spin up by CorsettiS in 1099
* Remove ``DbtGraph.current_version`` dead code by tatiana in 1111
* Disable test for Airflow-2.5 and Python-3.11 combination in CI by pankajastro in 1124
* Pre-commit hook updates in 1074, 1113, 1125, 1144, 1154, 1167

------------------

Bug fixes

* Fix getting temporary AWS credentials with assume_role by piotrkubicki in 1081
* Fix issue 'No such file or directory' by tatiana in 1097

Others

* Change Cosmos dev status from alpha to prod by tatiana in 1098
* Pre-commit hook updates in 1083, 1092

------------------

New Features

* Speed up ``LoadMode.DBT_LS`` by caching dbt ls output in Airflow Variable by tatiana in 1014
* Support to cache profiles created via ``ProfileMapping`` by pankajastro in 1046
* Support for running dbt tasks in AWS EKS in 944 by VolkerSchiewe
* Add Clickhouse profile mapping by roadan and pankajastro in 353 and 1016
* Add node config to TaskInstance Context by linchun3 in 1044

Bug fixes

* Support partial parsing when cache is disabled by tatiana in 1070
* Fix disk permission error in restricted env by pankajastro in 1051
* Add CSP header to iframe contents by dwreeves in 1055
* Stop attaching log adaptors to root logger to reduce logging costs by glebkrapivin in 1047

Enhancements

* Support ``static_index.html`` docs by dwreeves in 999
* Support deep linking dbt docs via Airflow UI by dwreeves in 1038
* Add ability to specify host/port for Snowflake connection by whummer in 1063

Docs

* Fix rendering for env ``enable_cache_dbt_ls`` by pankajastro in 1069

Others

* Update documentation for DbtDocs generator by arjunanan6 in 1043
* Use uv in CI by dwreeves in 1013
* Cache hatch folder in the CI by tatiana in 1056
* Change example DAGs to use ``example_conn`` as opposed to ``airflow_db`` by tatiana in 1054
* Mark plugin integration tests as integration by tatiana in 1057
* Ensure compliance with linting rule D300 by using triple quotes for docstrings by pankajastro in 1049
* Pre-commit hook updates in 1039, 1050, 1064
* Remove duplicates in changelog by jedcunningham in 1068

------------------

Bug fixes

* Bring back ``dataset`` as a required field for BigQuery profile by pankajkoti in 1033

Enhancements

* Only run ``dbt deps`` when there are dependencies by tatiana and AlgirdasDubickas in 1030

Docs

* Fix docs so it does not reference non-existing ``get_dbt_dataset`` by tatiana in 1034

------------------

Bug fixes

* Fix the invocation mode for ``ExecutionMode.VIRTUALENV`` by marco9663 in 1023
* Fix Cosmos ``enable_cache`` setting by tatiana in 1025
* Make ``GoogleCloudServiceAccountDictProfileMapping`` dataset profile arg optional by oliverrmaa and pankajastro in 839 and 1017
* Athena profile mapping set ``aws_session_token`` in profile only if it exists by pankajastro in 1022

Others

* Update dbt and Airflow conflicts matrix by tatiana in 1026
* Enable Python 3.12 unittest by pankajastro in 1018
* Improve error logging in ``DbtLocalBaseOperator`` by davidsteinar in 1004
* Add GitHub issue templates for bug reports and feature request by pankajkoti in 1009
* Add more fields in bug template to reduce turnaround in issue triaging by pankajkoti in 1027
* Fix ``dev/Dockerfile`` + Add ``uv pip install`` for faster build time by dwreeves in 997
* Drop support for Airflow 2.3 by pankajkoti in 994
* Update Astro Runtime image by RNHTTR in 988 and 989
* Enable ruff F linting by pankajastro in 985
* Move Cosmos Airflow configuration to settings.py by pankajastro in 975
* Fix CI Issues by tatiana in 1005
* Pre-commit hook updates in 1000, 1019