- First alpha release of dbt-spark-cde adapter that supports connection to Cloudera Data Engineering backend using CDE APIs
dbt-spark 1.2.0 (April 28, 2022)
- This file provides a full account of all changes to `dbt-spark`.
- Changes are listed under the (pre)release in which they first appear. Subsequent releases include changes from previous releases.
- "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-spark/blob/main/CONTRIBUTING.md#adding-changelog-entry)
dbt-spark 1.3.0 - October 12, 2022
Features
- Support python model through notebook, currently supported materializations are table and incremental ([417](https://github.com/dbt-labs/dbt-spark/issues/417), [#377](https://github.com/dbt-labs/dbt-spark/pull/377))
- Add changie to dbt-spark ([416](https://github.com/dbt-labs/dbt-spark/issues/416), [#418](https://github.com/dbt-labs/dbt-spark/pull/418))
- merge exclude columns for spark models ([5260](https://github.com/dbt-labs/dbt-spark/issues/5260), [#390](https://github.com/dbt-labs/dbt-spark/pull/390))
- Array macros ([453](https://github.com/dbt-labs/dbt-spark/issues/453), [#454](https://github.com/dbt-labs/dbt-spark/pull/454))
- implement testing for type_boolean in spark ([470](https://github.com/dbt-labs/dbt-spark/issues/470), [#471](https://github.com/dbt-labs/dbt-spark/pull/471))
- Support job cluster in notebook submission method, remove requirement for user for python model submission ([444](https://github.com/dbt-labs/dbt-spark/issues/444), [#467](https://github.com/dbt-labs/dbt-spark/pull/467))
- Migrate dbt-utils current_timestamp macros into core + adapters ([483](https://github.com/dbt-labs/dbt-spark/issues/483), [#480](https://github.com/dbt-labs/dbt-spark/pull/480))
Fixes
- Pin pyodbc to version 4.0.32 to prevent overwriting libodbc.so and libltdl.so on Linux ([397](https://github.com/dbt-labs/dbt-spark/issues/397), [#398](https://github.com/dbt-labs/dbt-spark/pull/398))
- python incremental model tmp table using correct schema ([441](https://github.com/dbt-labs/dbt-spark/issues/441), [#445](https://github.com/dbt-labs/dbt-spark/pull/445))
- change to get_columns_in_relation to fix cache inconsistencies to fix cache issues in incremental models causing failure on on_schema_change ([447](https://github.com/dbt-labs/dbt-spark/issues/447), [#451](https://github.com/dbt-labs/dbt-spark/pull/451))
Under the Hood
- Support core incremental refactor ([4402](https://github.com/dbt-labs/dbt-spark/issues/4402), [#394](https://github.com/dbt-labs/dbt-spark/pull/394))
- specify supported_languages for materialization that support python models ([437](https://github.com/dbt-labs/dbt-spark/issues/437), [#440](https://github.com/dbt-labs/dbt-spark/pull/440))
- Submit python model with Command API by default. Adjusted run name ([424](https://github.com/dbt-labs/dbt-spark/issues/424), [#442](https://github.com/dbt-labs/dbt-spark/pull/442))
- Better interface for python submission ([452](https://github.com/dbt-labs/dbt-spark/issues/452), [#452](https://github.com/dbt-labs/dbt-spark/pull/452))
- ignore mypy typing issues ([461](https://github.com/dbt-labs/dbt-spark/issues/461), [#462](https://github.com/dbt-labs/dbt-spark/pull/462))
- Enable Pandas and Pandas-on-Spark DataFrames for dbt python models ([468](https://github.com/dbt-labs/dbt-spark/issues/468), [#469](https://github.com/dbt-labs/dbt-spark/pull/469))
- Convert df to pyspark DataFrame if it is koalas before writing ([473](https://github.com/dbt-labs/dbt-spark/issues/473), [#474](https://github.com/dbt-labs/dbt-spark/pull/474))
Dependency
- Bump pyodbc from 4.0.32 to 4.0.34 ([417](https://github.com/dbt-labs/dbt-spark/issues/417), [#459](https://github.com/dbt-labs/dbt-spark/pull/459))
- Bump black from 22.3.0 to 22.8.0 ([417](https://github.com/dbt-labs/dbt-spark/issues/417), [#458](https://github.com/dbt-labs/dbt-spark/pull/458))
- Update click requirement from ~=8.0.4 to ~=8.1.3 ([417](https://github.com/dbt-labs/dbt-spark/issues/417), [#457](https://github.com/dbt-labs/dbt-spark/pull/457))
- Bump mypy from 0.950 to 0.971 ([417](https://github.com/dbt-labs/dbt-spark/issues/417), [#456](https://github.com/dbt-labs/dbt-spark/pull/456))
- Bump thrift-sasl from 0.4.1 to 0.4.3 ([417](https://github.com/dbt-labs/dbt-spark/issues/417), [#455](https://github.com/dbt-labs/dbt-spark/pull/455))
Contributors
- [barberscot](https://github.com/barberscot) ([#398](https://github.com/dbt-labs/dbt-spark/pull/398))
- [chamini2](https://github.com/chamini2) ([#469](https://github.com/dbt-labs/dbt-spark/pull/469))
- [colin-rogers-dbt](https://github.com/colin-rogers-dbt) ([#480](https://github.com/dbt-labs/dbt-spark/pull/480), [#462](https://github.com/dbt-labs/dbt-spark/pull/462))
- [dave-connors-3](https://github.com/dave-connors-3) ([#390](https://github.com/dbt-labs/dbt-spark/pull/390))
- [dbeatty10](https://github.com/dbeatty10) ([#454](https://github.com/dbt-labs/dbt-spark/pull/454), [#469](https://github.com/dbt-labs/dbt-spark/pull/469), [#474](https://github.com/dbt-labs/dbt-spark/pull/474))
- [graciegoheen](https://github.com/graciegoheen) ([#454](https://github.com/dbt-labs/dbt-spark/pull/454))
- [jpmmcneill](https://github.com/jpmmcneill) ([#471](https://github.com/dbt-labs/dbt-spark/pull/471))
- [ueshin](https://github.com/ueshin) ([#474](https://github.com/dbt-labs/dbt-spark/pull/474))
Previous Releases
For information on prior major and minor releases, see their changelogs:
- [1.2](https://github.com/dbt-labs/dbt-spark/blob/1.2.latest/CHANGELOG.md)
- [1.1](https://github.com/dbt-labs/dbt-spark/blob/1.1.latest/CHANGELOG.md)
- [1.0](https://github.com/dbt-labs/dbt-spark/blob/1.0.latest/CHANGELOG.md)
- [0.21](https://github.com/dbt-labs/dbt-spark/blob/0.21.latest/CHANGELOG.md)
- [0.20](https://github.com/dbt-labs/dbt-spark/blob/0.20.latest/CHANGELOG.md)
- [0.19 and earlier](https://github.com/dbt-labs/dbt-spark/blob/0.19.latest/CHANGELOG.md)