Dbt-dremio

Changes

- **Breaking:** With the previous implementation of the twin strategy, the connector **always** created a view matching the schema and name of a new table when the `clone` strategy was set (default). This differs from the behavior described in our [documentation](https://github.com/dremio/dbt-dremio/wiki/Using-Materializations-with-Dremio#user-content-optional-twin-strategy-configuration) since the twin strategy is only supposed to overwrite **conflicting** views.
- With this change, we introduce a check that will only overwrite existing views, not create new ones on table creation when the `clone` strategy is enabled. When a view is created with a conflicting table, the view will always select the existing table.
- To avoid failing queries, double-check that database and schema for freshly created table point to the physical space (datalake). This is especially relevant when pointing a source to a physical tables, since dbt defaults to the database (views), but the table will only lie in the datalake with this fix.
- Adds check for catalog/space existence during model creation. It would unblock using non admin users in DC to run dbt-dremio.
- Adds support for dbt model constraints
- Adds support for dbt model contracts
- Let the adapter pass down the defined schema to Dremio during both table and view creation

Features

- [278](https://github.com/dremio/dbt-dremio/pull/278) Fix twin strategy implementation
- [281](https://github.com/dremio/dbt-dremio/pull/281) Add support for dbt model constraints / contracts
- [271](https://github.com/dremio/dbt-dremio/pull/271) Pass down schema on table and view creation

Changes

- When naming reflections, if a `name` config is not set, the `alias` config parameter will be used instead. If also undefined, it will refer to the model name instead of using `Unnamed Reflection`
- Grants can now be set for both users and for roles. A prefix was added to handle this, with `user:` and `role:` being the valid prefixes. For example, `user:dbt_test_user_1` and `role:dbt_test_role_1`. If no prefix is provided, defaults to user for backwards compatibility.
- Moves the `raw_on_schema_change` variable back into scope for the config validator
- Adds `BaseIncrementalOnSchemaChange` test to test_incremental.py
- Changed logic for partitioning when materializing tables. Double quoting issue has been removed, now letting the user decide the quoting
- New example: `partition_by=['month("datetime_utc")']`
- Fixed a bug with grabbing the column schemas from queries (names and data types)
- Invisible to the users but it was the cause for some other bugs users were facing that have now been fixed

Features

- [259](https://github.com/dremio/dbt-dremio/pull/259) Added support for roles in grants
- [273](https://github.com/dremio/dbt-dremio/pull/273) Fix issue with on_schema_change config
- [282](https://github.com/dremio/dbt-dremio/pull/282) Fix issue with get_column_schema_from_query

Changes

- Added [DremioRestClient](dbt/adapters/dremio/api/rest/client.py) to isolate all Dremio API calls inside one class
- [256](https://github.com/dremio/dbt-dremio/pull/256) Reflections are now handled through the Rest API
- Non-admin users are now able to use reflections
- It is now possible to set a custom name for reflections
- If a reflection already exists in the dataset with the same name defined in the model, it will be updated instead of creating a new one
- New `date_dimensions` parameter was added to the reflection materialization, to set fields that have a `DATE` granularity
- Added Distribution Fields under `distribute_by`
- Added partition transformations under `partition_transform`
- Defaults to Original/Identity if not defined
- `year/month/day/hour/bucket(n)/truncate(n)`
- Computations default to `SUM, COUNT` if mapped measure is numeric, `COUNT` if not
- `reflections_enabled` adapter option has been renamed to `reflections_metadata_enabled` (requires user privileges to run in dremio)
- Removing duplicated macros array_append, array_concat as Dremio already has SQL functions analogues.
- [250](https://github.com/dremio/dbt-dremio/pull/250) Possibility to integrate wikis and tags by enabling `relation` option from `persist_docs` configuration
- New macro `dremio__persist_docs` created
- Views also perform `persist_docs` macro
- Integration via REST API

Features

- [250](https://github.com/dremio/dbt-dremio/pull/250) Implementation of wikis and tags feature
- [250](https://github.com/dremio/dbt-dremio/pull/256) Reflections are now handled through the Rest API

Dependency

- [222](https://github.com/dremio/dbt-dremio/issues/222) Upgrade dbt-core to 1.8.8 and dbt-tests-adapter to 1.8.0

Features

- [223](https://github.com/dremio/dbt-dremio/issues/224) Implement merge strategy for incremental materializations
- [229](https://github.com/dremio/dbt-dremio/issues/229) Add max operator to get_relation_last_modified macro

Changes

- [195](https://github.com/dremio/dbt-dremio/issues/195) Ensure the adapter does not try and create folders in object storage source
- [220](https://github.com/dremio/dbt-dremio/pull/220) Optimize networking performance with Dremio server

Changes

- [199](https://github.com/dremio/dbt-dremio/issues/199) Populate PyPI's `long_description` with contents of `README.md`
- [167](https://github.com/dremio/dbt-dremio/issues/167) Remove parentheses surrounding views in the create_view_as macro. In more complex queries, the parentheses cause performance issues.
- [211](https://github.com/dremio/dbt-dremio/issues/211) Make fetching model data false by default. This improves performance where job results do not need to be populated.
- [203](https://github.com/dremio/dbt-dremio/issues/203) Allow for dots in schema name, by surrounding in single and double quotes.
- [193](https://github.com/dremio/dbt-dremio/issues/193) Fixes Reflection bug: The name argument to ref() must be a string, got <class 'jinja2.runtime.Undefined'>
- [Versioning](https://github.com/dremio/dbt-dremio/pull/210) Added optional parameter v to the ref macro