Tiledbsoma

New features added

* [python] Blockwise SciPy sparse and Arrow Table iterator (1792) (1858)
* [python] Outgest `uns` (1770) (1801)
* [python] Handle string columns with all-NaN values (1834) (1859)
* [python] Support default-coord reads of corner-written dense ND arrays (1810) (1812)
* [python] Allow uns-key restriction for ingest and outgest (1815) (1826)
* [python] Allow `X_layer_name=None` for outgest of X-free experiments (1830) (1832)
* [r] Write string attrs as UTF-8 (Python compatibility) (1843) (1850)

Minor changes

* [r] Set the package version to 1.5.0, and update NEWS.md (1856) (1860)
* [r] Unit-test queryability of enumerated columns (1884) (1885)
* [r] Depend on TileDB-R 0.21.3 (1882) (1883)
* [r] Install required tiledb package version from r-universe (1863) (1881)
* [r] Suppress deprecation warnings in `write_soma.DataFrame()` (1876) (1878)
* [r] Force exporting v3 assays with SeuratObject v5 installed (1809) (1841)
* [ci/r] No more "Last 13 lines of output:" in CI (1871) (1873)
* [ci/python] Make Python blockwise tests less resource-intensive (1877) (1879)
* [ci/python] Excess `pytest` duration (1866) (1868)
* [c++/python] Depend on core 2.17.3 and TileDB-Py 0.23.2 (1838) (1842)
* [c++] Remove `spdlog` requirement (1852) (1857)
* [c++/python] Expose `uri`, `result_order`, and `column_names` for `SOMAArray` (1814) (1827)
* [c++/python/r] Depend on core 2.17.4, tiledb-py 0.23.4, and tiledb-r 0.21.2 (1854) (1855)

Bug fixes

* [python] Enumerations-API refinement (1755) (1844)
* [python] Non-default measurement propagation on `from_anndata` (1820) (1821)
* [python/c++] Support enumerations with Pandas 2.0+
* [python/c++] Expanded enumeration support in `ArrowAdapter::to_arrow` (1848) (1861)
* [ci/python] Enable more verbose python unit test logging (1870) (1872)
* [r] Update `SOMADataFrame` creation and writes (1835) (1847)
* [c++] Correct `results_complete` (1816) (1831)
* [ci/c++] Avoid intermittent CI fail building `catch2` (1828) (1833)

New features added

* [Backport release-1.5] [python] Non-default measurement propagation on `from_anndata` by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1821
* [Backport release-1.5] [python] Allow uns-key restriction for ingest and outgest by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1826
* [Backport release-1.5] [python] Allow `X_layer_name=None` for outgest of X-free experiments by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1832
* [Backport release-1.5] [python] Support new set-membership query condition by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1788
* [Backport release-1.5] [python] Create separate `set_condition` by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1796
* [Backport release-1.5] [python] Outgest `uns` by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1801
* [Backport release-1.5] [python] Support default-coord reads of corner-written dense ND arrays by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1812

Minor changes

* [Backport release-1.5] [python] Use AnnData 0.10.1 by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1786
* [Backport release-1.5] [r] Articulate R version 1.4.4.1 by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1805
* [Backport release-1.5] [python] Add `py.typed` to package manifest by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1813
* [Backport release-1.5] [python/c++] Expose `uri`, `result_order`, and `column_names` for `SOMAArray` by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1827
* [Backport release-1.5] [c++] Correct `results_complete` by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1831
* [Backport release-1.5] [c++] Avoid intermittent CI fail building `catch2` by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1833

Bug fixes

* [Backport release-1.5] [python] Update chunk-sizing estimator for chunked matrix write by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1800

**Full Changelog**: https://github.com/single-cell-data/TileDB-SOMA/compare/1.5.0rc2...1.5.0rc3

New features added

* [Backport release-1.5] [python] Support `obs_id_name` and `var_id_name` in `tiledbsoma.io.from_anndata` by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1750
* [Backport release-1.5] [python] Allow `adata.X` to be `None` by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1772
* [Backport release-1.5] [python] Ingest 2D uns string arrays (e.g. color labels) by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1773

Minor changes

* [Backport release-1.5] [ci/r] Use TileDB-R 0.21.1 binaries by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1751
* [Backport release-1.5] [ci] Update tools used in pre-commit testing by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1754
* [Backport release-1.5] [r] Remove duplicated step in workflow by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1760
* [Backport release-1.5] [r] Update installed packages in workflow by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1761
* [Backport release-1.5] [python] Temporarily pin to AnnData 0.9.* by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1771
* Manually backport 1757 by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1781
* Manually backport 1779 by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1782

Bug fixes

* [Backport release-1.5] [python] Check for downgraded categorical write by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1752
* [Backport release-1.5] [c++] Rework `SOMAArray` reader iterator sequence by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1753
* [Backport release-1.5] [python] Fix and auto-test notebook paths by github-actions in https://github.com/single-cell-data/TileDB-SOMA/pull/1780

**Full Changelog**: https://github.com/single-cell-data/TileDB-SOMA/compare/1.5.0rc1...1.5.0rc2

New features added

* [Backport release-1.5] [python] Support enumerations in `update_obs`/`update_var` by github-actions in 1736
* [Backport release-1.5] [r] Support enumerations in `update_obs`/`update_var` by github-actions in 1734
* [Backport release-1.5] [r] Support `factor` and `ordered` in `SOMADataFrameCreate` by github-actions in 1709

Minor changes

* [Backport release-1.5] [python] Consolidation and vacuuming are now platform configuration options by github-actions in 1696
* [Backport release-1.5] [r] Replace three local utility functions with use of RcppInt64 by github-actions in 1724
* [Backport release-1.5] [ci] MacOS CI updates by github-actions in 1697
* [Backport release-1.5] [python] update to numba 0.58 by github-actions in 1726
* [python][release-1.5] Revert to numba 0.57 until it's available in Conda by johnkerl in 1735
* [Backport release-1.5] Minor follow-up to 1695 by github-actions in 1705

Bug fixes

* [Backport release-1.5] Depend on core 2.17.1, TileDB-Py 0.23.1, TileDB-R 0.21.1 by github-actions in 1732
* [Backport release-1.5] [python] Flatten categorical `soma_joinid` if presented at `write` by github-actions in 1699
* [Backport release-1.5] [python] `update_obs`/`update_var` with original or readback source by github-actions in 1725
* [Backport release-1.5] [c++] Fix for empty-indexed multi-dimensional array by github-actions in 1708
* [Backport release 1.5] [r] Update `SOMADataFrame$create()` to automatically mark `ordered()` by johnkerl in 1727

**Full Changelog**: https://github.com/single-cell-data/TileDB-SOMA/compare/1.5.0rc0...1.5.0rc1

New features added

Primary: enumerated types AKA categoricals AKA factors:

* [python] Support `Enumeration` in Python API by nguyenv in https://github.com/single-cell-data/TileDB-SOMA/pull/1511
* [r] Add enumeration support by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1559
* [r] Update `write_soma.data.frame()` to support categoricals by mojaveazure in https://github.com/single-cell-data/TileDB-SOMA/pull/1686
* [c++] Support `Enumeration` in C++ codebase by nguyenv in https://github.com/single-cell-data/TileDB-SOMA/pull/1519
* [c++] Depend on core 2.17.0 by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1677

As well:

* [python] Append-mode pre-check logic by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1554
* [python] Add parameters to `ExperimentAxis` query methods by ebezzi in https://github.com/single-cell-data/TileDB-SOMA/pull/1553
* [python] Improve guidance on non-ingestible NaN data in categorical-int obs/var columns by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1568
* [python] Append-mode label/ID-mapping logic by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1556
* [python] `update_obs`/`update_var` by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1586
* [python] Support H5AD ingest directly from S3 in backed mode by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1629
* [python] Append-mode I/O step by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1587
* [python] Check that `update_obs`/`update_var` do not update row counts by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1639
* [python] Port bounding-box logic from R to Python by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1648
* [python] `append_obs`/`append_var` by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1634
* [python] Support X-free AnnData files by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1672
* [r] Add SOMA Experiment Query -> SingleCellExperiment outgestor by mojaveazure in https://github.com/single-cell-data/TileDB-SOMA/pull/1279
* [r] Add SingleCellExperiment -> SOMA Ingestor by mojaveazure in https://github.com/single-cell-data/TileDB-SOMA/pull/1547
* [r] Add bounding box to arrays by mojaveazure in https://github.com/single-cell-data/TileDB-SOMA/pull/1513
* [r] Adjust bounding box to be spec compliant by mojaveazure in https://github.com/single-cell-data/TileDB-SOMA/pull/1644
* [r] Support updating obs and var by aaronwolen in https://github.com/single-cell-data/TileDB-SOMA/pull/1643

Minor changes

* [python] Add a missing log-line for uns ndarray ingest by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1551
* [python] Usability improvements to profiler by atolopko-czi in https://github.com/single-cell-data/TileDB-SOMA/pull/1516
* [python] Replace dataclasses with attrs by thetorpedodog in https://github.com/single-cell-data/TileDB-SOMA/pull/1557
* [python] Make `tiledbsoma.io._create_or_open_coll` compatible with 1.2.7 by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1592
* [python] Decouple 1587 and 1592 by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1594
* [python] Update to `_create_or_open_coll` for 1.2.7 compatibility by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1606
* [python] Don't accept lists for Python metadata entries, but on purpose by thetorpedodog in https://github.com/single-cell-data/TileDB-SOMA/pull/1645
* [python] Export `update_obs` and `update_var` from `tiledbsoma.io` by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1660
* [r] Docs and NAMESPACE cleanup by aaronwolen in https://github.com/single-cell-data/TileDB-SOMA/pull/1566
* [r] Temporarily install tiledb-r from r-universe by aaronwolen in https://github.com/single-cell-data/TileDB-SOMA/pull/1570
* [r] Experiments for r-universe dependencies on Mac by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1571
* [r] Include system prerequisites in install notes by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1580
* [r] Trivial whitespace mod on README.md by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1583
* [r] Remove workaround for tiledb-r 0.20.3 while CRAN was on break by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1581
* [r] Add ability to install BioC packages to r-ci.sh [ci skip] by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1589
* [r] Bump version in `apis/r/DESCRIPTION` file by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1596
* [r] Accomodate RedHat install path in package-internal path by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1633
* [r] Additional polish for R CMD check --as-cran by r-devel by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1641
* [r] Add a finalizer for the SOMAArray instance and assure valgrind tests pass by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1636
* [r] Add three valgrind options by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1658
* [r] Refactor iterated reader lifetime by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1657
* [r] More tweaks for tests by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1659
* [r] Use the new tiledb-r release in CI by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1663
* [c++] Addition of `SOMA{Dense,Sparse}NDArray` by nguyenv in https://github.com/single-cell-data/TileDB-SOMA/pull/1529
* [c++] Add new constructor, factor validation helper by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1560
* [c++] Addition of `SOMACollection`, `SOMAMeasurement`, `SOMAExperiment` by nguyenv in https://github.com/single-cell-data/TileDB-SOMA/pull/1541
* [c++] `SOMACollection` Takes Start and End Timestamp by nguyenv in https://github.com/single-cell-data/TileDB-SOMA/pull/1576
* [c++] Support reading member-to-uri mapping in read and write mode by nguyenv in https://github.com/single-cell-data/TileDB-SOMA/pull/1627
* [c++] `SOMAObject` Metadata Methods Should Be Virtual by nguyenv in https://github.com/single-cell-data/TileDB-SOMA/pull/1631
* Extract method for `obsm`/`varm` outgest by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1646
* [c++] Don't use Python to generate C++ test inputs by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1668
* [c++] Removed unused-variable warning by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1680
* [ci] Interop-CI update by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1679
* [ci] Trivial typofixes by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1681
* [ci] Have clang-format show version by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1682
* Pd/add output on failure by davisp in https://github.com/single-cell-data/TileDB-SOMA/pull/1688

Bug fixes

* [python] Generalize `add_X_layer` URI handling by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1548
* [python] Adding missing context in ingest uns ndarray by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1552
* [python] Fix For 1456: Invalid Result With PyArrow Array Coords by nguyenv in https://github.com/single-cell-data/TileDB-SOMA/pull/1573
* [python] Avoid unnecessary slowdown on empty sparse arrays by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1649
* [python] Typo fix in `devtools/outgestor` by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1652
* [python] Lazily initialize a default global TileDB context. by thetorpedodog in https://github.com/single-cell-data/TileDB-SOMA/pull/1661
* [python] Leverage bounding-box feature for obsm/varm outgest robustness by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1650
* [python] Restore efficiency on zero-cell `X` by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1671
* [python] Initialize TileDB Contexts extremely lazily. by thetorpedodog in https://github.com/single-cell-data/TileDB-SOMA/pull/1664
* [python] Pass user configuration/timestamp to query condition init by Shelnutt2 in https://github.com/single-cell-data/TileDB-SOMA/pull/1687
* [python] Categorical-write corner-case by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1691
* [python] Fix pandas categorical test by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1693
* [r] Ensure numeric axis query coordinates are converted by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1540
* [r] Update Rd/ directory to quieten checks by eddelbuettel in https://github.com/single-cell-data/TileDB-SOMA/pull/1647

New Contributors

* davisp made their first contribution in https://github.com/single-cell-data/TileDB-SOMA/pull/1688

**Full Changelog**: https://github.com/single-cell-data/TileDB-SOMA/compare/1.3.0...1.5.0rc0

This release is a single-bug-fix release affecting TileDB-SOMA Python. There are no changes for TileDB-SOMA R; however, it is our policy to stamp all implementation languages with the same release tag regardless of how many mods affect each given language. Thus, this is a no-op release for TileDB-SOMA R.

* Backport 1690 to `release-1.4` by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1794
* [r] Bump to 1.4.4 in `apis/r/DESCRIPTON` on `release-1.4` by johnkerl in https://github.com/single-cell-data/TileDB-SOMA/pull/1802

**Full Changelog**: https://github.com/single-cell-data/TileDB-SOMA/compare/1.4.3...1.4.4