Dbt-databricks

Features

- Support Python 3.10 ([158](https://github.com/databricks/dbt-databricks/pull/158))

Features

- Add grants to materializations ([dbt-labs/dbt-spark366](https://github.com/dbt-labs/dbt-spark/issues/366), [dbt-labs/dbt-spark#381](https://github.com/dbt-labs/dbt-spark/pull/381))
- Add `connection_parameters` for databricks-sql-connector connection parameters ([135](https://github.com/databricks/dbt-databricks/pull/135))
- This can be used to customize the connection by setting additional parameters.
- The full parameters are listed at [Databricks SQL Connector for Python](https://docs.databricks.com/dev-tools/python-sql-connector.html#connect-method).
- Currently, the following parameters are reserved for `dbt-databricks`. Please use the normal credential settings instead.
- server_hostname
- http_path
- access_token
- session_configuration
- catalog
- schema

Fixes

- Incremental materialization updated to not drop table first if full refresh for delta lake format, as it already runs _create or replace table_ ([dbt-labs/dbt-spark286](https://github.com/dbt-labs/dbt-spark/issues/286), [dbt-labs/dbt-spark#287](https://github.com/dbt-labs/dbt-spark/pull/287))

Under the hood

- Update `SparkColumn.numeric_type` to return `decimal` instead of `numeric`, since SparkSQL exclusively supports the former ([dbt-labs/dbt-spark380](https://github.com/dbt-labs/dbt-spark/pull/380))
- Make minimal changes to support dbt Core incremental materialization refactor ([dbt-labs/dbt-spark402](https://github.com/dbt-labs/dbt-spark/issue/402), [dbt-labs/dbt-spark#394](httpe://github.com/dbt-labs/dbt-spark/pull/394), [136](https://github.com/databricks/dbt-databricks/pull/136))
- Add new basic tests `TestDocsGenerateDatabricks` and `TestDocsGenReferencesDatabricks` ([134](https://github.com/databricks/dbt-databricks/pull/134))
- Set upper bound for `databricks-sql-connector` when Python 3.10 ([154](https://github.com/databricks/dbt-databricks/pull/154))
- Note that `databricks-sql-connector` does not officially support Python 3.10 yet.

Contributors

- [grindheim](https://github.com/grindheim) ([dbt-labs/dbt-spark#287](https://github.com/dbt-labs/dbt-spark/pull/287/))

Features

- Support for Databricks CATALOG as a DATABASE in DBT compilations ([95](https://github.com/databricks/dbt-databricks/issues/95), [#89](https://github.com/databricks/dbt-databricks/pull/89), [#94](https://github.com/databricks/dbt-databricks/pull/94), [#105](https://github.com/databricks/dbt-databricks/pull/105))
- Setting an initial catalog with `session_properties` is deprecated and will not work in the future release. Please use `catalog` or `database` to set the initial catalog.
- When using catalog, `spark_build_snapshot_staging_table` macro will not be used. If trying to override the macro, `databricks_build_snapshot_staging_table` should be overridden instead.

Fixes

- Block taking jinja2.runtime.Undefined into DatabricksAdapter ([98](https://github.com/databricks/dbt-databricks/pull/98))
- Avoid using Cursor.schema API when database is None ([100](https://github.com/databricks/dbt-databricks/pull/100))

Under the hood

- Drop databricks-sql-connector 1.0 ([108](https://github.com/databricks/dbt-databricks/pull/108))

Features

- Add support for [Delta constraints](https://docs.databricks.com/delta/delta-constraints.html) ([#71](https://github.com/databricks/dbt-databricks/pull/71))

Under the hood

- Port testing framework changes from [dbt-labs/dbt-spark299](https://github.com/dbt-labs/dbt-spark/pull/299) and [dbt-labs/dbt-spark#314](https://github.com/dbt-labs/dbt-spark/pull/314) ([#70](https://github.com/databricks/dbt-databricks/pull/70))

Fixes

- Make internal macros use macro dispatch pattern ([72](https://github.com/databricks/dbt-databricks/pull/72))

Features

- Support for setting table properties as part of a model configuration ([33](https://github.com/databricks/dbt-databricks/issues/33), [#49](https://github.com/databricks/dbt-databricks/pull/49))
- Get the session_properties map to work ([57](https://github.com/databricks/dbt-databricks/pull/57))
- Bump up databricks-sql-connector to 1.0.1 and use the Cursor APIs ([50](https://github.com/databricks/dbt-databricks/pull/50))