Dbt-maxcompute

Bug Fixes
In `incremental` materialization, avoid redundant temp relation creation when `on_schema_change` is enabled with `merge` strategy, improving stability and efficiency.

New Features

Auto-partitioned tables now support custom configuration for the name of the generated partition column. The `partition_by` parameter introduces a new option `generate_column_name` to define the column name.
Example:
sql
{{ config(
materialized='table',
partition_by={"fields": "some_date", "data_types": "timestamp", "generate_column_name": "pt1", "granularity": "month"}
) }}
SELECT * FROM {{ source('raw', 'seed') }}

Bug Fixes
For the `incremental` materialization strategy, temporary tables are now deleted before materialization to prevent the "table already defined" error. Previously, cleanup only occurred after materialization.

New Features
- Added support for `dbt source freshness` functionality.

Bug Fixes
- Fixed an issue where using the `incremental` materialization with the `insert_overwrite` strategy would cause errors for auto-partitioned tables.
- Fixed an issue where temporary tables created using the `incremental` materialization were not automatically deleted.

New Features
- Adaptation to the latest MaxCompute SQL syntax:
- `DATE_TRUNC` now supports `datepart` values: `'quarter'`, `'isoweek'`, `'week'`, and `'weekday'`
- `DATEADD` now supports `datepart` values: `'millisecond'` and `'microsecond'`
- `DATEDIFF` now supports `datepart` values: `'microsecond'`, `'isoweek'`, `'week'`, and `'weekday'`
- `persist_doc` now supports `view` materialization
- New `calculate_freshness_from_metadata` function for calculating data freshness
- `lifecycle` parameter support for `table` and `incremental` materialization to specify table lifecycle

Bug Fixes
- Resolved an issue where specifying a partition column as the first column during materialization would add an extra comma.

Bug Fixes
- Fixed an issue where the drop_relation function failed to properly delete views.

Important Changes

In dbt, there are two styles of incremental materialization logic for `insert_overwrite`:
- **Original Logic**: Maintained by dbt-adapters, it requires specifying a `unique_key` to deduplicate data based on the unique key.
- **Enhanced Logic**: Used by dbt-bigquery, it requires specifying a `partition_by` field and overwrites only the new partition each time.

In the **v1.9.0-a0** release, `dbt-maxcompute` supported both logics and named them `insert_overwrite` (original logic) and `bq_insert_overwrite` (enhanced logic). However, considering the following reasons:
- The majority of MaxCompute tables are partitioned tables, and users utilize the enhanced logic (`bq_insert_overwrite`) far more frequently than the original logic.
- Maintaining two sets of logic incurs higher costs.

We have decided to adopt the enhanced logic as the default implementation and made the following adjustments:
- Renamed `bq_insert_overwrite` to `insert_overwrite`.
- Renamed `bq_microbatch` to `microbatch`.
- Removed the original `insert_overwrite` and `microbatch` logic.

> **Note**: This change may impact existing project configurations. Please update your code according to the new naming conventions.


New Features

View Creation Optimization
- Added the `OR REPLACE` keyword when creating views to ensure successful updates if the view already exists, avoiding conflicts caused by duplicate views.

Enhanced Authentication Methods
- Introduced the `alibabacloud_credential` module, supporting multiple authentication methods, including:
- STSToken
- RAM Role
- credential provider chain
- etc.
- For more information, refer to the [documentation](https://github.com/aliyun/dbt-maxcompute/blob/master/docs/authentication.md).

Bug Fixes

- Resolved compatibility issues with `pyodps 0.12.2`, ensuring smooth operation with the latest dependency libraries.
- Fixed an issue where the default schema specified in `raw` materialization mode was inconsistent with the actual runtime schema, improving data consistency.