Tiledb

Improvements
* Consolidate `_nonempty_domain_var` into `nonempty_domain` [632](https://github.com/TileDB-Inc/TileDB-Py/pull/632)
* Support more valid Python syntax for `QueryCondition` statements [636](https://github.com/TileDB-Inc/TileDB-Py/pull/636)
* Addition of `ascii` dtype to `Attr` allows `QueryCondition` to support var-length strings [637](https://github.com/TileDB-Inc/TileDB-Py/pull/637)

TileDB-Py 0.9.4 Release Notes

Improvements
* Support pickling for arrays in write-mode [626](https://github.com/TileDB-Inc/TileDB-Py/pull/626)

Bug Fixes
* Fixed multi-range indexer to default to explicitly pass in the non-empty domain if dimensions are unspecified [630](https://github.com/TileDB-Inc/TileDB-Py/pull/630)

TileDB-Py 0.9.3 Release Notes

Packaging Notes
* Due to a packaging issue released with 0.9.3 (NumPy ABI compatibility with NumPy < 1.20 for Python 3.8), this section is intentionally left blank.

TileDB-Py 0.9.2 Release Notes

Packaging Notes
* Fixed release builder ordering issue which led to CRLF line endings in 0.9.1 source distribution.

API Changes
* Deprecate `Array.timestamp` and replace with `Array.timestamp_range` [616](https://github.com/TileDB-Inc/TileDB-Py/pull/616)

Improvements
* Set `ArraySchema.tile_order=None` for Hilbert-ordered arrays [609](https://github.com/TileDB-Inc/TileDB-Py/pull/609)
* Use CIBW to build release wheels on Linux [613](https://github.com/TileDB-Inc/TileDB-Py/pull/613)
* Addition of Pickling functionality for `SparseArray` [618](https://github.com/TileDB-Inc/TileDB-Py/pull/618)

TileDB-Py 0.9.1 Release Notes

TileDB Embedded updates:

Improvements
* Support more env selectable options in both azure-windows.yml and azure-windows-release.yml [2384](https://github.com/TileDB-Inc/TileDB/pull/2384)
* Enable Azure/Serialization for windows CI artifacts [2400](https://github.com/TileDB-Inc/TileDB/pull/2400)

Bug fixes
* Correct check for last offset position so that undefined memory is not accessed. [2390](https://github.com/TileDB-Inc/TileDB/pull/2390)
* Fix ch8416, failure to read array written with tiledb 2.2 via REST [2404](https://github.com/TileDB-Inc/TileDB/pull/2404)
* Fix ch7582: use the correct buffer for validity deserialization [2407](https://github.com/TileDB-Inc/TileDB/pull/2407)

Improvements
* Support passing a timestamp range for consolidation and vacuuming [603](https://github.com/TileDB-Inc/TileDB-Py/pull/603)

Bug Fixes
* FragmentInfo API's to_vacuum_uri() function corrected to iterate through `to_vacuum_num` rather than `fragment_num`[603](https://github.com/TileDB-Inc/TileDB-Py/pull/603)
* Return "NA" For ArraySchema.tile_order if "Hilbert" [605](https://github.com/TileDB-Inc/TileDB-Py/pull/605)

TileDB-Py 0.9.0 Release Notes

TileDB Embedded updates:
* TileDB-Py 0.9.0 includes TileDB Embedded [TileDB 2.3](https://github.com/TileDB-Inc/TileDB/releases/tag/2.3) with a significant
number of new features and improvements.

Packaging Notes
* Windows wheels are now built with TileDB Cloud REST support enabled [541](https://github.com/TileDB-Inc/TileDB-Py/pull/541)

Improvements
* Addition of `QueryCondition` API to filter query on attributes [576](https://github.com/TileDB-Inc/TileDB-Py/pull/576)

Bug Fixes
* Fixed `from_pandas` append error for sparse arrayse: no need to specify 'row_start_idx' [593](https://github.com/TileDB-Inc/TileDB-Py/pull/593)
* Fixed 'index_dims' kwarg handling for `from_pandas` [590](https://github.com/TileDB-Inc/TileDB-Py/pull/590)

API Changes
* `from_dataframe` function has been removed; deprecated in TileDB-Py 0.6 and replaced by `from_pandas`.

---

TileDB-Py 0.8.11 Release Notes

Bug fixes
* Fixed incorrect NumPy ABI target in Linux wheels [590](https://github.com/TileDB-Inc/TileDB-Py/pull/590)
* QueryCondition API will cast condition values to the datatype of the corresponding attribute [589](https://github.com/TileDB-Inc/TileDB-Py/pull/589)
* QueryCondition API errors out when there are mismatched attributes to `query`'s `attr_cond` and `attrs` arguments [589](https://github.com/TileDB-Inc/TileDB-Py/pull/589)
* QueryCondition API can now parse negative numbers [589](https://github.com/TileDB-Inc/TileDB-Py/pull/589)


TileDB-Py 0.8.10 Release Notes

Improvements
* Disabled libtiledb Werror compilation argument for from-source builds via setup.py [574](https://github.com/TileDB-Inc/TileDB-Py/pull/574)
* Relaxed NumPy version requirements for from-source builds via setup.py [575](https://github.com/TileDB-Inc/TileDB-Py/pull/575)

Bug fixes
* Fixed FragmentInfoList where context was not being passed to ArraySchema [573](https://github.com/TileDB-Inc/TileDB-Py/pull/573)
* Fixed FragmentInfoList where context was not being passed to ArraySchema [578](https://github.com/TileDB-Inc/TileDB-Py/pull/578)
* Fixed read bug due to large estimated result size [579](https://github.com/TileDB-Inc/TileDB-Py/pull/579)
* Fixed bug reading nullable attributes due to missing buffer resize [581](https://github.com/TileDB-Inc/TileDB-Py/pull/581)
* Fixed Python output for `tiledb.stats_dump` [586](https://github.com/TileDB-Inc/TileDB-Py/pull/586)

TileDB-Py 0.8.9 Release Notes

TileDB Embedded updates:

Disk Format
* Format version incremented to 9. [2108](https://github.com/TileDB-Inc/TileDB/pull/2108)

Breaking behavior
* The setting of \`sm.read_range_oob\` now defaults to \`warn\`, allowing queries to run with bounded ranges that errored before. [2176](https://github.com/TileDB-Inc/TileDB/pull/2176)
* Removes TBB as an optional dependency [2181](https://github.com/TileDB-Inc/TileDB/pull/2181)

New features
* Support TILEDB_DATETIME_{SEC,MS,US,NS} in arrow_io_impl.h [2228](https://github.com/TileDB-Inc/TileDB/pull/2228)
* Adds support for filtering query results on attribute values [2141](https://github.com/TileDB-Inc/TileDB/pull/2141)
* Adding support for time datatype dimension and attribute [2140](https://github.com/TileDB-Inc/TileDB/pull/2140)
* Add support for serialization of config objects [2164](https://github.com/TileDB-Inc/TileDB/pull/2164)
* Add C and C++ examples to the `examples/` directory for the `tiledb_fragment_info_t` APIs. [2160](https://github.com/TileDB-Inc/TileDB/pull/2160)
* supporting serialization (using capnproto) build on windows [2100](https://github.com/TileDB-Inc/TileDB/pull/2100)
* Config option "vfs.s3.sse" for S3 server-side encryption support [2130](https://github.com/TileDB-Inc/TileDB/pull/2130)
* Name attribute/dimension files by index. This is fragment-specific and updates the format version to version 9. [2107](https://github.com/TileDB-Inc/TileDB/pull/2107)
* Smoke Test, remove nullable structs from global namespace. [2078](https://github.com/TileDB-Inc/TileDB/pull/2078)

Improvements
* replace ReadFromOffset with ReadRange in GCS::read() to avoid excess gcs egress traffic [2307](https://github.com/TileDB-Inc/TileDB/pull/2307)
* Hilbert partitioning fixes [2269](https://github.com/TileDB-Inc/TileDB/pull/2269)
* Stats refactor [2267](https://github.com/TileDB-Inc/TileDB/pull/2267)
* Improve Cap'n Proto cmake setup for system installations [2263](https://github.com/TileDB-Inc/TileDB/pull/2263)
* Runtime check for minimum validity buffer size [2261](https://github.com/TileDB-Inc/TileDB/pull/2261)
* Enable partial vacuuming when vacuuming with timestamps [2251](https://github.com/TileDB-Inc/TileDB/pull/2251)
* Consolidation: de-dupe FragmentInfo [2250](https://github.com/TileDB-Inc/TileDB/pull/2250)
* Consolidation: consider non empty domain before start timestamp [2248](https://github.com/TileDB-Inc/TileDB/pull/2248)
* Add size details to s3 read error [2249](https://github.com/TileDB-Inc/TileDB/pull/2249)
* Consolidation: do not re-open array for each fragment [2243](https://github.com/TileDB-Inc/TileDB/pull/2243)
* Support back compat writes [2230](https://github.com/TileDB-Inc/TileDB/pull/2230)
* Serialization support for query conditions [2240](https://github.com/TileDB-Inc/TileDB/pull/2240)
* Make SubarrayPartitioner's member functions to return Status after calling Subarray::get_range_num. [2235](https://github.com/TileDB-Inc/TileDB/pull/2235)
* Update bzip2 super build version to 1.0.8 to address [CVE-2019-12900](https://nvd.nist.gov/vuln/detail/CVE-2019-12900) in libbzip2 [#2233](https://github.com/TileDB-Inc/TileDB/pull/2233)
* Timestamp start and end for vacuuming and consolidation [2227](https://github.com/TileDB-Inc/TileDB/pull/2227)
* Fix memory leaks reported on ASAN when running with leak-detection. [2223](https://github.com/TileDB-Inc/TileDB/pull/2223)
* Use relative paths in consolidated fragment metadata [2215](https://github.com/TileDB-Inc/TileDB/pull/2215)
* Optimize `Subarray::compute_relevant_fragments` [2216](https://github.com/TileDB-Inc/TileDB/pull/2216)
* AWS S3: improve is_dir [2209](https://github.com/TileDB-Inc/TileDB/pull/2209)
* Add nullable string to nullable attribute example [2212](https://github.com/TileDB-Inc/TileDB/pull/2212)
* AWS S3: adding option to skip Aws::InitAPI [2204](https://github.com/TileDB-Inc/TileDB/pull/2204)
* Added additional stats for subarrays and subarray partitioners [2200](https://github.com/TileDB-Inc/TileDB/pull/2200)
* Introduces config parameter "sm.skip_est_size_partitioning" [2203](https://github.com/TileDB-Inc/TileDB/pull/2203)
* Add config to query serialization. [2177](https://github.com/TileDB-Inc/TileDB/pull/2177)
* Consolidation support for nullable attributes [2196](https://github.com/TileDB-Inc/TileDB/pull/2196)
* Adjust unit tests to reduce memory leaks inside the tests. [2179](https://github.com/TileDB-Inc/TileDB/pull/2179)
* Reduces memory usage in multi-range range reads [2165](https://github.com/TileDB-Inc/TileDB/pull/2165)
* Add config option \`sm.read_range_oob\` to toggle bounding read ranges to domain or erroring [2162](https://github.com/TileDB-Inc/TileDB/pull/2162)
* Windows msys2 build artifacts are no longer uploaded [2159](https://github.com/TileDB-Inc/TileDB/pull/2159)
* Add internal log functions to log at different log levels [2161](https://github.com/TileDB-Inc/TileDB/pull/2161)
* Parallelize Writer::filter_tiles [2156](https://github.com/TileDB-Inc/TileDB/pull/2156)
* Added config option "vfs.gcs.request_timeout_ms" [2148](https://github.com/TileDB-Inc/TileDB/pull/2148)
* Improve fragment info loading by parallelizing fragment_size requests [2143](https://github.com/TileDB-Inc/TileDB/pull/2143)
* Allow open array stats to be printed without read query [2131](https://github.com/TileDB-Inc/TileDB/pull/2131)
* Cleanup the GHA CI scripts - put common code into external shell scripts. [2124](https://github.com/TileDB-Inc/TileDB/pull/2124)
* Reduced memory consumption in the read path for multi-range reads. [2118](https://github.com/TileDB-Inc/TileDB/pull/2118)
* The latest version of `dev` was leaving behind a `test/empty_string3/`. This ensures that the directory is removed when `make check` is run. [2113](https://github.com/TileDB-Inc/TileDB/pull/2113)
* Migrating AZP CI to GA [2111](https://github.com/TileDB-Inc/TileDB/pull/2111)
* Cache non_empty_domain for REST arrays like all other arrays [2105](https://github.com/TileDB-Inc/TileDB/pull/2105)
* Add additional stats printing to breakdown read state initialization timings [2095](https://github.com/TileDB-Inc/TileDB/pull/2095)
* Places the in-memory filesystem under unit test [1961](https://github.com/TileDB-Inc/TileDB/pull/1961)
* Adds a Github Action to automate the HISTORY.md [2075](https://github.com/TileDB-Inc/TileDB/pull/2075)
* Change printfs in C++ examples to cout, edit C print statements to fix format warnings [2226](https://github.com/TileDB-Inc/TileDB/pull/2226)

Deprecations
* The following APIs have been deprecated: `tiledb_array_open_at`, `tiledb_array_open_at_with_key`, `tiledb_array_reopen_at`. [2142](https://github.com/TileDB-Inc/TileDB/pull/2142)

Bug fixes
* Fix a segfault on `VFS::ls` for the in-memory filesystem [2255](https://github.com/TileDB-Inc/TileDB/pull/2255)
* Fix rare read corruption in S3 [2253](https://github.com/TileDB-Inc/TileDB/pull/2253)
* Update some union initializers to use strict syntax [2242](https://github.com/TileDB-Inc/TileDB/pull/2242)
* Fix race within `S3::init_client` [2247](https://github.com/TileDB-Inc/TileDB/pull/2247)
* Expand accepted windows URIs. [2237](https://github.com/TileDB-Inc/TileDB/pull/2237)
* Write fix for unordered writes on nullable, fixed attributes. [2241](https://github.com/TileDB-Inc/TileDB/pull/2241)
* Fix tile extent to be reported as domain extent for sparse arrays with Hilbert ordering [2231](https://github.com/TileDB-Inc/TileDB/pull/2231)
* Do not consider option sm.read_range_oob for set_subarray() on Write queries [2211](https://github.com/TileDB-Inc/TileDB/pull/2211)
* Change avoiding generation of multiple, concatenated, subarray flattened data. [2190](https://github.com/TileDB-Inc/TileDB/pull/2190)
* Change mutex from basic to recursive [2180](https://github.com/TileDB-Inc/TileDB/pull/2180)
* Fixes a memory leak in the S3 read path [2189](https://github.com/TileDB-Inc/TileDB/pull/2189)
* Fixes a potential memory leak in the filter pipeline [2185](https://github.com/TileDB-Inc/TileDB/pull/2185)
* Fixes misc memory leaks in the unit tests [2183](https://github.com/TileDB-Inc/TileDB/pull/2183)
* Fix memory leak of \`tiledb_config_t\` in error path of \`tiledb_config_alloc\`. [2178](https://github.com/TileDB-Inc/TileDB/pull/2178)
* Fix check for null pointer in query deserialization [2163](https://github.com/TileDB-Inc/TileDB/pull/2163)
* Fixes a potential crash when retrying incomplete reads [2137](https://github.com/TileDB-Inc/TileDB/pull/2137)
* Fixes a potential crash when opening an array with consolidated fragment metadata [2135](https://github.com/TileDB-Inc/TileDB/pull/2135)
* Corrected a bug where sparse cells may be incorrectly returned using string dimensions. [2125](https://github.com/TileDB-Inc/TileDB/pull/2125)
* Fix segfault in serialized queries when partition is unsplittable [2120](https://github.com/TileDB-Inc/TileDB/pull/2120)
* Always use original buffer size in serialized read queries serverside. [2115](https://github.com/TileDB-Inc/TileDB/pull/2115)
* Fix an edge-case where a read query may hang on array with string dimensions [2089](https://github.com/TileDB-Inc/TileDB/pull/2089)

API additions

C API
* Added tiledb_array_set_open_timestamp_start and tiledb_array_get_open_timestamp_start [2285](https://github.com/TileDB-Inc/TileDB/pull/2285)
* Added tiledb_array_set_open_timestamp_end and tiledb_array_get_open_timestamp_end [2285](https://github.com/TileDB-Inc/TileDB/pull/2285)
* Addition of `tiledb_array_set_config` to directly assign a config to an array. [2142](https://github.com/TileDB-Inc/TileDB/pull/2142)
* tiledb_query_get_array now returns a deep-copy [2184](https://github.com/TileDB-Inc/TileDB/pull/2184)
* Added \`tiledb_serialize_config\` and \`tiledb_deserialize_config\` [2164](https://github.com/TileDB-Inc/TileDB/pull/2164)
* Add new api, `tiledb_query_get_config` to get a query's config. [2167](https://github.com/TileDB-Inc/TileDB/pull/2167)
* Removes non-default parameter in "tiledb_config_unset". [2099](https://github.com/TileDB-Inc/TileDB/pull/2099)

C++ API
* Added Array::set_open_timestamp_start and Array::open_timestamp_start [2285](https://github.com/TileDB-Inc/TileDB/pull/2285)
* Added Array::set_open_timestamp_end and Array::open_timestamp_end [2285](https://github.com/TileDB-Inc/TileDB/pull/2285)
* add Query::result_buffer_elements_nullable support for dims [2238](https://github.com/TileDB-Inc/TileDB/pull/2238)
* Addition of `tiledb_array_set_config` to directly assign a config to an array. [2142](https://github.com/TileDB-Inc/TileDB/pull/2142)
* Add new api, `Query.config()` to get a query's config. [2167](https://github.com/TileDB-Inc/TileDB/pull/2167)
* Removes non-default parameter in "Config::unset". [2099](https://github.com/TileDB-Inc/TileDB/pull/2099)
* Add support for a string-typed, variable-sized, nullable attribute in the C++ API. [2090](https://github.com/TileDB-Inc/TileDB/pull/2090)

Improvements
* Support for iterating over incomplete query results [548](https://github.com/TileDB-Inc/TileDB-Py/pull/548)
- This feature provides the capability to consume partial query results with a fixed maximum buffer size
rather than the the default behavior of resizing buffers and resubmitting to completion.
Usage example: `examples/incomplete_iteration.py`
(along with test in: `test_libtiledb.py:test_incomplete_return`)
* Rename FragmentsInfo to FragmentInfoList [551](https://github.com/TileDB-Inc/TileDB-Py/pull/551)
* Dataframe creation uses Zstd default compression level (-1) [552](https://github.com/TileDB-Inc/TileDB-Py/pull/552)
* Rename Fragment Info API's `non_empty_domain` attribute to `nonempty_domain` [553](https://github.com/TileDB-Inc/TileDB-Py/pull/553)
* Added configuration option `py.alloc_max_bytes` to control maximum initial buffer allocation [557](https://github.com/TileDB-Inc/TileDB-Py/pull/557)

Bug fixes
* Fixed incorrected error raised in .df[] indexer when pyarrow not installed [554](https://github.com/TileDB-Inc/TileDB-Py/pull/554)
* Fixed `from_pandas(attr_filters=None, dim_filters=None)` (previously used internal defaults) [564](https://github.com/TileDB-Inc/TileDB-Py/pull/554)
* Fixed `from_pandas` write bug due to incorrect classification of str/bytes columns [562](https://github.com/TileDB-Inc/TileDB-Py/pull/562)
* Fix segfault due to mismatched validity num and data buffer sizes [567](https://github.com/TileDB-Inc/TileDB-Py/pull/567)

TileDB-Py 0.8.8 Release Notes

TileDB Embedded updates:

TileDB-Py 0.8.7 Release Notes

Improvements
* ArraySchema support for `cell_order="hilbert"` [535](https://github.com/TileDB-Inc/TileDB-Py/pull/535)

Bug fixes
* Fixed regression in `from_pandas` with string-valued index dimensions [526](https://github.com/TileDB-Inc/TileDB-Py/pull/526)
* Fixed GC lifetime bug in string buffer conversion [525](https://github.com/TileDB-Inc/TileDB-Py/pull/526)
* Fixed `FilterList`'s `repr()` method [528](https://github.com/TileDB-Inc/TileDB-Py/pull/528)

TileDB-Py 0.8.6 Release Notes

TileDB Embedded updates: