Tiledb

Deprecation announcements

* All deprecated APIs will be removed in version 2.26. TileDB can be built with `--remove-deprecations` (`bootstrap`) or `-DTILEDB_REMOVE_DEPRECATIONS` (CMake) to validate that projects are not using any deprecated APIs.

Improvements

* Improve diagnostics when an Azure endpoint is not configured. [4845](https://github.com/TileDB-Inc/TileDB/pull/4845)
* Do not attempt Azure shared key authentication if no account name is specified. [4856](https://github.com/TileDB-Inc/TileDB/pull/4856)
* Clarify the documentation for the non empty domain CAPI. [4885](https://github.com/TileDB-Inc/TileDB/pull/4885)
* Make closing a group that is not open a no-op. [4917](https://github.com/TileDB-Inc/TileDB/pull/4917)

Defects removed

* Fix wrong fallback cell order for Hilbert. [4924](https://github.com/TileDB-Inc/TileDB/pull/4924)
* Config serialization should take into account environment variables. [4865](https://github.com/TileDB-Inc/TileDB/pull/4865)
* Vac files should only be removed if paths removal was fully successful. [4889](https://github.com/TileDB-Inc/TileDB/pull/4889)
* Fix C query condition examples. [4912](https://github.com/TileDB-Inc/TileDB/pull/4912)

C++ API

* `Query::submit_async` is deprecated. Call `Query::submit()` on another thread instead. [4879](https://github.com/TileDB-Inc/TileDB/pull/4879)
* Overloads of methods and constructors in `Array` and `ArraySchema` that accept encryption keys are deprecated. Specify the encryption key with the `sm.encryption_type` and `sm.encryption_key` config options instead. [4879](https://github.com/TileDB-Inc/TileDB/pull/4879)
* Add C++ API for tiledb_array_consolidate_fragments. [4884](https://github.com/TileDB-Inc/TileDB/pull/4884)

Build System Changes

* Vcpkg is always enabled; turning the `TILEDB_VCPKG` option off is no longer supported and fails. [4570](https://github.com/TileDB-Inc/TileDB/pull/4570)
* Update the `TILEDB_REMOVE_DEPRECATIONS` option to exclude all deprecated C and C++ APIs from the library binary and the headers. [4887](https://github.com/TileDB-Inc/TileDB/pull/4887)

Internal Improvements

* Implement iterator facade for external sort. [4914](https://github.com/TileDB-Inc/TileDB/pull/4914)
* Implement var_length_view and unit tests for external sort. [4918](https://github.com/TileDB-Inc/TileDB/pull/4918)
* Implement permutation view for external sort. [4920](https://github.com/TileDB-Inc/TileDB/pull/4920)
* Implement proxy sort for external sort. [4922](https://github.com/TileDB-Inc/TileDB/pull/4922)
* Implement alt var length view for external sort. [4925](https://github.com/TileDB-Inc/TileDB/pull/4925)

Deprecation announcements

* Support for downloading dependencies with CMake `ExternalProject`s by specifying `-DTILEDB_VCPKG=OFF` will be removed in 2.23. Vcpkg will be downloaded and used to manage dependencies by default. See https://github.com/TileDB-Inc/TileDB/blob/dev/doc/dev/BUILD.md for help on how to disable automatically downloading vcpkg and acquire dependencies from the system.

Configuration changes

* Add `vfs.gcs.service_account_key` config option that specifies a Google Cloud service account credential JSON string. [4855](https://github.com/TileDB-Inc/TileDB/pull/4855)
* Add `vfs.gcs.workload_identity_configuration` config option that specifies a Google Cloud Workload Identity Federation credential JSON string. [4855](https://github.com/TileDB-Inc/TileDB/pull/4855)

New features

* Support Microsoft Entra ID authentication to Azure. [4126](https://github.com/TileDB-Inc/TileDB/pull/4126)

Improvements

* Allow to set buffers for geometry types with CPP API more easily. [4826](https://github.com/TileDB-Inc/TileDB/pull/4826)

Defects removed

* Throw error when an upload fails due to bad state. [4815](https://github.com/TileDB-Inc/TileDB/pull/4815)
* Single-process sub-millisecond temporal disambiguation of random labels. [4800](https://github.com/TileDB-Inc/TileDB/pull/4800)
* Expose VFSExperimental and relocate CallbackWrapperCPP. [4820](https://github.com/TileDB-Inc/TileDB/pull/4820)
* Do not load group metadata when getting it from REST. [4821](https://github.com/TileDB-Inc/TileDB/pull/4821)
* Fix crash getting file size on non existent blob on Azure. [4836](https://github.com/TileDB-Inc/TileDB/pull/4836)

Defects removed

* Fix crash getting file size on non existent blob on Azure. [4836](https://github.com/TileDB-Inc/TileDB/pull/4836)

Defects removed

* Do not load group metadata when getting it from REST. [4821](https://github.com/TileDB-Inc/TileDB/pull/4821)

Deprecation announcements

* Building with Visual Studio 2019 (MSVC toolchains v14.29 and earlier) will become unsuported in version 2.28, to be released in Q3 2024.
* HDFS support will be removed in version 2.28, to be released in Q3 2024.
* Support for GLIBC 2.17 (manylinux2014, based on CentOS 7) will be removed in TileDB version 2.31, to be released in Q4 2024. It will be replaced by support for GLIBC 2.28 (manylinux_2_28; RHEL8; Rocky Linux 8).

New features

* Add datatypes GEOM_WKB and GEOM_WKT. [4640](https://github.com/TileDB-Inc/TileDB/pull/4640)

TileDB backend API support:

* Query Plan REST support. [4519](https://github.com/TileDB-Inc/TileDB/pull/4519)
* Consolidation Plan REST support. [4537](https://github.com/TileDB-Inc/TileDB/pull/4537)

Improvements

* Optimize sparse reads when full non empty domain is requested. [4710](https://github.com/TileDB-Inc/TileDB/pull/4710)
* Lazily initialize the Azure SDK client. [4703](https://github.com/TileDB-Inc/TileDB/pull/4703)
* Remove QueryCondition requirement for Enumeration values to be valid. [4707](https://github.com/TileDB-Inc/TileDB/pull/4707)
* Convert dimension function pointer members to subclass methods. [4774](https://github.com/TileDB-Inc/TileDB/pull/4774)
* Implement dense dimension aggregates. [4801](https://github.com/TileDB-Inc/TileDB/pull/4801)

Defects removed

* Fix documentation of `Subarray::set_config` to match `tiledb_subarray_set_config`. [4672](https://github.com/TileDB-Inc/TileDB/pull/4672)
* Fix stats bug in StorageManager. [4708](https://github.com/TileDB-Inc/TileDB/pull/4708)
* Add explicit casts to fix compile errors. [4712](https://github.com/TileDB-Inc/TileDB/pull/4712)
* Fix bug with new minio behavior. [4725](https://github.com/TileDB-Inc/TileDB/pull/4725)
* Support reading V1 group details with explicit version in the name. [4744](https://github.com/TileDB-Inc/TileDB/pull/4744)
* Update crop_range to clamp to domain range. [4781](https://github.com/TileDB-Inc/TileDB/pull/4781)
* Fix segfault in ArraySchema::check. [4787](https://github.com/TileDB-Inc/TileDB/pull/4787)
* Fix crash in aggregates on dimensions. [4794](https://github.com/TileDB-Inc/TileDB/pull/4794)
* Fix defects with query channels. [4786](https://github.com/TileDB-Inc/TileDB/pull/4786)
* Remove trailing slash from `rest.server_address` configuration option. [4790](https://github.com/TileDB-Inc/TileDB/pull/4790)
* Provide a clear exception message when performing remote incomplete queries with aggregates. [4798](https://github.com/TileDB-Inc/TileDB/pull/4798)

API changes

C API

* Support VFS `ls_recursive` API for posix filesystem. [4778](https://github.com/TileDB-Inc/TileDB/pull/4778)

Test only changes

* Add unit_vfs to tests for windows/osx/linux. [4748](https://github.com/TileDB-Inc/TileDB/pull/4748)

Build System Changes

* Update vcpkg version baseline to https://github.com/microsoft/vcpkg/commit/72010900b7cee36cea77aebb97695095c9358eaf. [#4553](https://github.com/TileDB-Inc/TileDB/pull/4553)
* Add vcpkg triplets for RelWithDebInfo. [4669](https://github.com/TileDB-Inc/TileDB/pull/4669)
* Reintroduce the `TILEDB_STATIC` option under a deprecation warning. [4732](https://github.com/TileDB-Inc/TileDB/pull/4732)
* Fix linker errors when building with MSVC. [4759](https://github.com/TileDB-Inc/TileDB/pull/4759)
* CPack in Release Workflow. [4694](https://github.com/TileDB-Inc/TileDB/pull/4694)
* Single release CMake file that contains all required links/hashes. [4631](https://github.com/TileDB-Inc/TileDB/pull/4631)
* Improve build performance when GCS is enabled. [4777](https://github.com/TileDB-Inc/TileDB/pull/4777)
* Fix compiling unit tests when HDFS is enabled. [4795](https://github.com/TileDB-Inc/TileDB/pull/4795)
* Add CPack support to CMakeLists.txt. [4645](https://github.com/TileDB-Inc/TileDB/pull/4645)

Defects removed

* Support reading V1 group details with explicit version in the name. [4744](https://github.com/TileDB-Inc/TileDB/pull/4744)

Build System Changes

* Reintroduce the `TILEDB_STATIC` option under a deprecation warning. [4732](https://github.com/TileDB-Inc/TileDB/pull/4732)