Astronomer-cosmos

--------------------

Bug Fixes

* Fix import error in dbt bigquery adapter mock for ``dbt-bigquery<1.8`` for ``ExecutionMode.AIRFLOW_ASYNC`` by pankajkoti in 1548
* Fix ``operator_args`` override configuration by ghjklw in 1558
* Fix missing ``install_dbt_deps`` in ``ProjectConfig`` ``__init__`` method by ghjklw in 1556
* Fix dbt project parsing ``dbt_vars`` behavior passed via ``operator_args`` by AlexandrKhabarov in 1543
* Avoid reading the connection during DAG parsing of the async BigQuery operator by joppevos in 1582
* Fix: Workaround to incorrectly raised ``gcsfs.retry.HttpError`` (Invalid Credentials, 401) by tatiana in 1598
* Fix the async execution mode read sql files for dbt packages by pankajastro in 1588
* Improve BQ async error handling by tatiana in 1597
* Fix path selector when ``manifest.json`` is created using MS Windows by tatiana in 1601
* Fix log that prints 'Total filtered nodes' by tatiana in 1603
* Fix select behaviour using ``LoadMode.MANIFEST`` and a path with star by tatiana in 1602
* Support ``on_warning_callback`` with ``TestBehavior.BUILD`` and ``ExecutionMode.LOCAL`` by corsettigyg in 1571
* Fix ``DbtRunLocalOperator.partial()`` support by tatiana ashb in 1609
* fix: ``container_name`` is null for ecs integration by nicor88 in 1592

Docs

* Improve MWAA getting-started docs by removing unused imports by jx2lee in 1562

Others

* Disable ``example_cosmos_dbt_build.py`` DAG in CI by pankajastro in 1567
* Upgrade GitHub Actions Ubuntu version by tatiana in 1561
* Update GitHub bug issue template by pankajastro in 1586
* Enable DAG ``example_cosmos_dbt_build.py`` in CI by pankajastro in 1573
* Run async DAG in DAG without setup/teardown task by pankajastro in 1599
* Add test case that fully covers recent select issue by tatiana in 1604
* Add CI job to test multiple dbt versions for the async DAG by pankajkoti in 1535
* Improve unit tests speed from 89s to 14s by tatiana in 1600
* Pre-commit updates: 1560, 1583, 1596

--------------------

Breaking changes

* When using ``LoadMode.DBT_LS``, Cosmos will now attempt to use the ``dbtRunner`` as opposed to subprocess to run ``dbt ls``.
While this represents significant performance improvements (half the vCPU usage and some memory consumption improvement), this may not work in
scenarios where users had multiple Python virtual environments to manage different versions of dbt and its adaptors. In those cases,
please, set ``RenderConfig(invocation_mode=InvocationMode.SUBPROCESS)`` to have the same behaviour Cosmos had in previous versions.
Additional information `here <https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls>`_ and `here <https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#how-to-run-dbt-ls-invocation-mode>`_.

Features

* Use ``dbtRunner`` in the DAG Processor when using ``LoadMode.DBT_LS`` if ``dbt-core`` is available by tatiana in 1484. Additional information `here <https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls>`_.
* Allow users to opt-out of ``dbtRunner`` during DAG parsing with ``InvocationMode.SUBPROCESS`` by tatiana in 1495. Check out the `documentation <https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#how-to-run-dbt-ls-invocation-mode>`_.
* Add structure to support multiple db for async operator execution by pankajastro in 1483
* Support overriding the ``profile_config`` per dbt node or folder using config by tatiana in 1492. More information `here <https://astronomer.github.io/astronomer-cosmos/profiles/#profile-customise-per-node>`_.
* Create and run accurate SQL statements when using ``ExecutionMode.AIRFLOW_ASYNC`` by pankajkoti, tatiana and pankajastro in 1474
* Add AWS ECS task run execution mode by CarlosGitto and aoelvp94 in 1507
* Add support for running ``DbtSourceOperator`` individually by victormacaubas in 1510
* Add setup task for async executions by pankajastro in 1518
* Add teardown task for async executions by pankajastro in 1529
* Add ``ProjectConfig.install_dbt_deps`` & change operator ``install_deps=True`` as default by tatiana in 1521
* Extend Virtualenv operator and mock dbt adapters for setup & teardown tasks in ``ExecutionMode.AIRFLOW_ASYNC`` by pankajkoti, tatiana and pankajastro in 1544

Bug Fixes

* Fix select complex intersection of three tag-based graph selectors by tatiana in 1466
* Fix custom selector behaviour when the model name contains periods by yakovlevvs and 60098727 in 1499
* Filter dbt and non-dbt kwargs correctly for async operator by pankajastro in 1526

Enhancement

* Fix OpenLineage deprecation warning by CorsettiS in 1449
* Move ``DbtRunner`` related functions into ``dbt/runner.py`` module by tatiana in 1480
* Add ``on_warning_callback`` to ``DbtSourceKubernetesOperator`` and refactor previous operators by LuigiCerone in 1501
* Gracefully error when users set incompatible ``RenderConfig.dbt_deps`` and ``operator_args`` ``install_deps`` by tatiana in 1505
* Store compiled SQL as template field for ``ExecutionMode.AIRFLOW_ASYNC`` by pankajkoti in 1534

Docs

* Improve ``RenderConfig`` arguments documentation by tatiana in 1514
* Improve callback documentation by tatiana in 1516
* Improve partial parsing docs by tatiana in 1520
* Fix typo in selecting & excluding docs by pankajastro in 1523
* Document ``async_py_requirements`` added in ``ExecutionConfig`` for ``ExecutionMode.AIRFLOW_ASYNC`` by pankajkoti in 1545

Others

* Ignore dbt package tests when running Cosmos tests by tatiana in 1502
* Refactor to consolidate async dbt adapter code by pankajkoti in 1509
* Log elapsed time for sql file(s) upload/download by pankajastro in 1536
* Remove the fallback operator for async task by pankajastro in 1538
* GitHub Actions Dependabot: 1487
* Pre-commit updates: 1473, 1493, 1503, 1531

--------------------

Bug Fixes

* Fix ``httpx.get`` exception handling while emitting telemetry by tatiana in 1439
* Fix (not) rendering detached tests in ``TestBehavior.NONE`` and ``AFTER_ALL`` by tatiana in 1463
* Fix detached test tasks names so they do not exceed 250 chars by tatiana in 1464

Enhancement

* Allow users to opt-in or out (default) of detached test nodes by tatiana in 1470. Learn more about this `here <https://astronomer.github.io/astronomer-cosmos/configuration/testing-behavior.html>`_.

Docs

* Docs: Fix broken links and rendering by pankajastro in 1437
* Update ``operator args`` docs to include ``install_deps`` by tatiana in 1456
* Improve Cosmos ``select`` docs to include latest graph operator support by tatiana in 1467

Others

* Upgrade GitHub action artifacts upload-artifact & download-artifact to v4 by pankajkoti in 1445
* Enable Depandabot to scan outdated Github Actions dependencies by tatiana in 1446
* Pre-commit hook updates in 1459, 1441
* Dependabot Github action updates in 1451, 1452, 1453, 1454, 1455

--------------------

Bug Fixes

* Fix rendering dbt tests with multiple parents by tatiana in 1433
* Add ``kwargs`` param in DocsOperator method ``upload_to_cloud_storage`` by pankajastro in 1422

Docs

* Improve OpenLineage documentation by tatiana in 1431

Others

* Enable Docs DAG in CI leveraging existing CI connections by pankajkoti in 1428
* Install providers with airflow by pankajkoti in 1432
* Remove unused docs dependency by pankajastro in 1414
* Pre-commit hook updates in 1424

--------------------

New Features

* Support customizing Airflow operator arguments per dbt node by wornjs in 1339. `More information <https://astronomer.github.io/astronomer-cosmos/getting_started/custom-airflow-properties.html>`_.
* Support uploading dbt artifacts to remote cloud storages via callback by pankajkoti in 1389. `Read more <https://astronomer.github.io/astronomer-cosmos/configuration/callbacks.html>`_.
* Add support to ``TestBehavior.BUILD`` by tatiana in 1377. `Documentation <https://astronomer.github.io/astronomer-cosmos/configuration/testing-behavior.html>`_.
* Add support for the "at" operator when using ``LoadMode.DBT_MANIFEST`` or ``CUSTOM`` by benjy44 in 1372
* Add dbt clone operator by pankajastro in 1326, as documented in `here <https://astronomer.github.io/astronomer-cosmos/getting_started/operators.html>`_.
* Support rendering tasks with non-ASCII characters by t0momi219 in 1278 `Read more <https://astronomer.github.io/astronomer-cosmos/configuration/task-display-name.html>`_.
* Add warning callback on source freshness by pankajastro in 1400 `Read more <https://astronomer.github.io/astronomer-cosmos/configuration/source-nodes-rendering.html#on-warning-callback-callback>`_.
* Add Oracle Profile mapping by slords and pankajkoti in 1190 and 1404
* Emit telemetry to Scarf during DAG run by tatiana in 1397
* Save tasks map as ``DbtToAirflowConverter`` property by internetcoffeephone and hheemskerk in 1362

Bug Fixes

* Fix the mock value of port in ``TrinoBaseProfileMapping`` to be an integer by dwolfeu 1322
* Fix access to the ``dbt docs`` menu item outside of Astro cloud by tatiana in 1312
* Add missing ``DbtSourceGcpCloudRunJobOperator`` in module ``cosmos.operators.gcp_cloud_run_job`` by anai-s in 1290
* Support building ``DbtDag`` without setting paths in ``ProjectConfig`` by tatiana in 1307
* Fix parsing dbt ls outputs that contain JSONs that are not dbt nodes by tatiana in 1296
* Fix Snowflake Profile mapping when using AWS default region by tatiana in 1406
* Fix dag rendering for taskflow + DbtTaskGroup combo by pankajastro in 1360

Enhancements

* Improve dbt command execution logs to troubleshoot ``None`` values by tatiana in 1392
* Add logging of stdout to dbt graph run_command by KarolGongola in 1390
* Add ``profile_config`` for Docker by andrewhlui in 1347
* Support rendering build operator task-id with non-ASCII characters by pankajastro in 1415

Docs

* Remove extra ` char from docs by pankajastro in 1345
* Add limitation about copying target dir files to remote by pankajkoti in 1305
* Generalise example from README by ReadytoRocc in 1311
* Add security policy by tatiana, chaosmaw and lzdanski in 1385
* Mention in documentation that the callback functionality is supported in ``ExecutionMode.VIRTUALENV`` by pankajkoti in 1401

Others

* Restore Jaffle Shop so that ``basic_cosmos_dag`` works as documented by tatiana in 1374
* Remove Pytest durations from tests scripts by tatiana in 1383
* Remove typing-extensions as dependency by pankajastro in 1381
* Pin dbt-databricks version to < 1.9 by pankajastro in 1376
* Refactor ``dbt-sqlite`` tests to use ``dbt-postgres`` by pankajastro in 1366
* Remove 'dbt-core<1.8.9' pin by tatiana in 1371
* Remove dependency ``eval_type_backport`` by tatiana in 1370
* Enable kubernetes tests for dbt>=1.8 by pankajastro 1364
* CI Workaround: Pin dbt-core, Disable SQLite Tests, and Correctly Ignore Clone Test to Pass CI by pankajastro in 1337
* Enable Azure task in the remote store manifest example DAG by pankajkoti in 1333
* Enable GCP remote manifest task by pankajastro in 1332
* Add exempt label option in GH action stale job by pankajastro in 1328
* Add integration test for source node rendering by pankajastro in 1327
* Fix vulnerability issue on docs dependency by tatiana in 1313
* Add postgres pod status check for k8s tests in CI by pankajkoti in 1320
* [CI] Reduce the amount taking to run tests in the CI from 5h to 11min by tatiana in 1297
* Enable secret detection precommit check by pankajastro in 1302
* Fix security vulnerability, by not pinning Airflow 2.10.0 by tatiana in 1298
* Fix Netlify build timeouts by tatiana in 1294
* Add stalebot to label/close stale PRs and issues by tatiana in 1288
* Unpin dbt-databricks version by pankajastro in 1409
* Fix source resource type tests by pankajastro in 1405
* Increase performance tests models by tatiana in 1403
* Drop running 1000 models in the CI by pankajkoti in 1411
* Fix releasing package to PyPI by tatiana in 1396
* Address review comments on PR 1347 regarding profile_config for ExecutionMode.Docker by pankajkoti in 1413
* Pre-commit hook updates in 1394, 1373, 1358, 1340, 1331, 1314, 1301

------------------

Bug fixes

* Fix ``DbtVirtualenvBaseOperator`` to use correct virtualenv Python path by kesompochy in 1252
* Fix displaying dbt docs as menu item in Astro by tatiana in 1280
* Fix: Replace login by user for clickhouse profile by petershenri in 1255

Enhancements

* Improve dbt Docs Hosting Debugging -- Update dbt_docs_not_set_up.html by johnmcochran in 1250
* Minor refactor on VirtualenvOperators & add test for PR 1253 by tatiana in 1286

Docs

* Add Welcome Section and "What Is Cosmos" Blurb to Home Page by cmarteepants and yanmastin-astro in 1251
* Update the URL for sample dbt docs hosted in Astronomer S3 bucket by pankajkoti in 1283
* Add dedicated scarf tracking pixel to readme by cmarteepants in 1256


Others

* Update ``CODEOWNERS`` to track all files by pankajkoti in 1284
* Fix release after the ``raw`` rst directive was disabled in PyPI by tatiana in 1282
* Update issue template ``bug.yml`` - cosmos version update in the dropdown by pankajkoti in 1275
* Pre-commit hook updates in 1285, 1274, 1254, 1244