Ahjo

Update dependencies
Updated requirements for azure-identity and pyodbc:
- `azure-identity >= 1.16.1, < 1.18`
- `pyodbc >= 5.0, < 5.2`

Test results to database table
Added ability to save test action results to a database table. See [Test](https://github.com/ALMPartners/ahjo?tab=readme-ov-file#test) for more information.

New master actions
Added new master actions `create_test_table` and `create_test_view` for creating a table and a view for test action results.

Retry logic for database connection
Added support for retrying database connection if the connection fails. The number of retries and the interval between retries can be defined with `connect_retry_count` and `connect_retry_interval` parameters in the config file.

Database logging
Added support for database logging. The feature can be enabled by setting `enable_database_logging` to `true` in the config file. The log entries are written into a log table (name and schema can be defined in the config file).

Alembic API
Alembic commands can be executed through ahjo with `alembic-api` command. The command could be used in a situation where Ahjo is installed with MSI package and one needs to run alembic commands without python environment.

Metadata update speedup
Metadata deployment is now 95% faster. The speedup is achieved by reducing the number of queries to database.

Enable optimized ORM bulk insert by default
[SQLAlchemy's optimized ORM bulk insert](https://docs.sqlalchemy.org/en/20/changelog/whatsnew_20.html#optimized-orm-bulk-insert-now-implemented-for-all-backends-other-than-mysql) is now enabled by default. Previously, the feature was disabled because it was incompatible with ahjo's pyodbc bulk insert implementation. The incompatibility has been resolved.

SQLAlchemy logging
Added support for [SQLAlchemy logging](https://docs.sqlalchemy.org/en/20/core/engines.html#configuring-logging). The feature can be enabled by setting `enable_sqlalchemy_logging` to `true` in the config file.

Pre-commit hook installation parameterized
Git pre-commit hook can now be installed with user-defined paths to scan rules and ignore rules files.

Regression fix: Git version query fails if Timestamp column is not found in git version table
Fixed a regression where the git version query fails if the Timestamp column is not found in the git version table.

Updates to ahjo scan
- New search rules: `sql_object_modification`, `alembic_table_modification`, `sql_insert` and `email`
- Added ability to define custom search rules with regex patterns
- Scan results can be ignored based on rules or matches
- Added command for scan config initialization
- The structure of the ignore file has changed
- An example of the new structure can be found in the `README.md` file
- Scan rules are now defined in a YAML file

Custom command-line arguments for actions
Added ability to pass custom command-line arguments to actions. The arguments are accessible from the context object in the action.

New arguments for ahjo command
Added new arguments `--skip-metadata-update`, `--skip-alembic-update` and `--skip-git-update` to ahjo command. These arguments can be used to skip updating documented extended properties to database, running alembic migrations and updating current git version to git version table.

Timestamp column to git table
Added a new column `Timestamp` to git table. The column holds the timestamp of the commit. The timestamp is updated when the git version is updated to the git table.

Updates to pipelines
- Added a new pipeline for running database tests
- Run ahjo build & sign pipeline only when a tag is created
- MSI installer for both user and system targets
- Added automatic file transfer from ADO to Teams
- Fixed Safety check command in pipeline

Ability to connect with SQLAlchemy URL
Added support for connecting to database with [SQLAlchemy URL](https://docs.sqlalchemy.org/en/20/core/engines.html#database-urls). The URL can be used to define more complex connection settings, e.g. [pyodbc connection attributes](https://learn.microsoft.com/en-us/sql/connect/odbc/dsn-connection-string-attribute?view=sql-server-ver16). Use `sqlalchemy.url` setting in config file to define the URL. If the URL is defined, it overrides the values of `dialect`, `sql_port`, `sql_driver`, `target_server_hostname` and `target_database_name`.

Support for SQLAlchemy engine configuration
Previously ahjo was hard-coded to use SQLAlchemy's default engine configuration. Now it is possible to customize the engine with SQLAlchemy's [create_engine](https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine) function parameters. The parameters can be defined in config file under `sqlalchemy.*` key. For example `sqlalchemy.pool_size: 10` is passed as pool_size=10 to `create_engine` function.

Support for SQLAlchemy URL query configuration
SQLAlchemy's class method `sqlalchemy.engine.URL.create` accepts a [query](https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.engine.URL.create.params.query) dictionary to be passed to the dialect and/or the DBAPI upon connect. The dictionary can be defined in ahjo config file under `sqla_url_query_map` key. The dictionary can be used for example to define [pyodbc connection attributes](https://learn.microsoft.com/en-us/sql/connect/odbc/dsn-connection-string-attribute?view=sql-server-ver16).

ODBC Driver 18 for SQL Server support
In previous versions, support for ODBC Driver 18 for SQL Server was limited due to hard-coded connection attributes. Now this has been fixed and the driver is supported.

The ODBC Driver 18.0 allows users to send long data types as max data types with the `LongAsMax` connection attribute. In ahjo, this is set to `Yes` by default.

**Notice that the connection encryption defaults have changed in ODBC Driver 18 for SQL Server.** The default value for `Encrypt` is `yes` and the default value for `TrustServerCertificate` is `no`. This means that the driver will encrypt the connection by default and it will not trust the server certificate by default. If you want to use the old defaults, e.g. in development environment, you need to define `Encrypt=no` and/or `TrustServerCertificate=yes` in `sqla_url_query_map` or `sqlalchemy.url` settings in config file.

Deprecated config parameters
`odbc_trust_server_certificate` and `odbc_encrypt` settings are deprecated and will be removed in the future. Use `sqla_url_query_map` or `sqlalchemy.url` settings in config file instead.

Build & Sign pipeline for ahjo MSI installation package
Added a build & sign pipeline for ahjo MSI installation package. The pipeline builds the package and signs it with a certificate. The signed package is published to Azure Artifacts.

Azure-identity to ahjo MSI installation package
In previous version of ahjo MSI installation package, azure-identity was not included. This caused an error when trying to use azure-identity authentication if ahjo was installed with MSI package. This has now been fixed.

Regression fix: UnboundLocalError in drop_sqlfile_objects
Fixed a regression bug where `UnboundLocalError` was raised when `drop_sqlfile_objects` was called.

Regression fix: git version table is not updated when using git version info file
In previous version, git version table was not updated when using git version info file. This has now been fixed.

Disable pyodbc pooling
Disabled pyodbc pooling by default since SQLAlchemy has its own pooling behavior. See [Pyodbc Pooling / connection close behavior](https://docs.sqlalchemy.org/en/20/dialects/mssql.html#pyodbc-pooling-connection-close-behavior) for more information.

Close SQLAlchemy connection if an error occurs
If an error occurs when running an action, the SQLAlchemy connection is now closed so that the connection is not left open.