Narwhals

Changes

- docs: fix api completness generation in CI (1863)

✨ Enhancements

- feat: pyspark and duckdb selectors (1853)
- feat: completely refactor alias tracking and support `nw.all`, `nw.nth`, and selectors across the API (1866)
- feat: raise if expression changes length in `.over()` (1867)
- feat: add DuckDB `join_asof` (1860)
- feat: allow any ddof value for duckdb `var` and `std` (1858)
- feat: add DuckDB `Struct` dtype (1851)
- feat: add DuckDB `list.len` method (1850)
- feat: enforce that group-by aggregations actually aggregate (1844)
- feat: add `mean_horizontal` for DuckDB (1846)

🐞 Bug fixes

- fix: support various reductions in pyspark (1870)
- fix: address `&` / `|` operator errors for PySpark / chore: use F.lit in maybe_evaluate for pyspark, like we do for duckdb (1872)
- fix: allow any name in expression parsing (1871)
- fix: `nw.selectors.by_dtype(nw.Datetime)` was excluding tz-aware for Polars (1869)
- patch: fix duckdb `narwhals_to_native_dtype` for `Int32` and `UInt32` (1857)
- fix: raise on selectors addition (1854)
- fix: rename ewm and rolling `min_periods` to `min_samples` for polars 1.21 compatibility (1864)
- fix: address `&` / `|` operator errors for PySpark / chore: use F.lit in maybe_evaluate for pyspark, like we do for duckdb (1872)

🛠️ Other improvements

- test: skip Ibis tests if deps aren't installed, set defaults in `PandasLikeSeries.__array__` to fix cuDF is_in (1862)
- chore: parse strings as columns names at narwhals-level (1856)
- chore: some typing and default removal (1852)
- chore: validate expressions shape at narwhals/expr.py level (1845)
- chore: simplify internal renaming in _pandas_like/series.py (1847)

Thank you to all our contributors for making this release possible!
FBruzzesi, MarcoGorelli and raisadz

Changes

- ci: downstream test for formulaic (1817)
- docs: Removed class, added polars to the table, str.head and str.tail are a… (1801)

✨ Enhancements

- feat: deprecate `maintain_order` in `Expr.unique` and `LazyFrame.tail` (but keep around in `stable.v1`) (1839)
- feat: track whether expressions change length but don't aggregate, and only allow length-changing expressions if they're followed by aggregations in the lazy API (1828)
- feat: add DuckDB: nw.nth, nw.sum_horizontal, nw.concat_str, group_by with drop_null_keys (1832)
- feat: remove `eager_or_interchange` from `from_native` in main namespace, switch Ibis' support from interchange to lazy in main namespace (but preserve status-quo in stable.v1) (1829)
- feat: add `is_nan` and `is_finite` for duckdb, `is_nan` for pyspark (1825)
- chore: rename time zone tests (1830)
- feat: Deprecate Expr.arg_true (but keep Series.arg_true, and keep both available in stable.v1) (1827)
- feat: add `Expr.dt` methods to `PySpark` (1835)
- feat: add pyspark str namespace `to_datetime` (1826)
- feat: add total duration methods for DuckDB (1831)
- feat: add `.over` method for `SparkLikeExpr` (1808)
- feat: pyspark group by `n_unique` and no aggregation (1819)
- feat: Disallow order-dependent expressions from being passed to nw.LazyFrame (1806)
- feat: add `SparkExpr.cast` for basic types (1812)
- feat: pyspark and duckdb `Expr.name` namespace (1809)
- feat: add `AnonymousExprError` (1816)
- feat: deprecate Expr.head, Expr.tail, Expr.sort, Expr.gather_every, Expr.sample (but keep them in stable.v1) (1791)
- feat: add duckdb dataframe `drop_nulls` (1811)
- feat: add `DataFrame` and `Series` `to_polars` (1803)
- feat: add support for `SparkLikeNamespace.when` (1805)

🐞 Bug fixes

- fix: when/then/otherwise output name was not consistent across backends (1833)
- fix: raise on shape mismatch in filter (1814)
- patch: fix when-then double lit case (1810)

🛠️ Other improvements

- chore: simplify imports (1838)
- test: fixup test suite for cudf.pandas (1837)
- test: decouple pyspark constructor from pandas (1834)
- chore: refactor pandas-like `narwhals_to_native_dtype` (1824)
- chore: refactor `isinstance_or_issubclass` to allow tuples (1820)

Thank you to all our contributors for making this release possible!
Dhanunjaya-Elluri, EdAbati, FBruzzesi, MarcoGorelli, luke396, marvinl803 and raisadz

Changes

✨ Enhancements

- feat: improve error message on check_columns_exist (1799)
- feat: DuckDB `date` (1798)
- feat: DuckDB `dt.to_string` (1794)
- feat: DuckDB `dt.ordinal_day` (1796)
- feat: DuckDB `dt.weekday` (1795)
- feat: add extra overload for concat for when it is not known statically whether inputs are eager or lazy (1783)
- feat: duckdb replace_all with regex (1784)
- feat: implement anti-join, str.len_chars, and null_count for DuckDB (1777)
- feat: `SparkLikeNamespace` methods (1779)
- feat: add `SparkLikeStrNamespace` methods (1781)
- feat: add `all`, `any` and `null_count` Spark Expressions (1724)
- feat: add missing dunder methods in `SparkLikeExpr` and `SparkLikeNamespace.lit` (1708)
- feat: implement cross-join for duckdb (1773)
- feat: semi-join for duckdb (1767)
- feat: implement `n_unique` for DuckDB (1762)
- feat: implement when/then/otherwise for DuckDB (1759)
- feat: add few missing `SparkLikeExpr` methods (1721)

🐞 Bug fixes

- fix: to_py_scalar was raising for decimals (1800)
- fix: fix skew for duckdb with fewer than 3 elements (1785)
- fix: duckdb join was failing if column names contained spaces (1775)
- fix: parse_version was not parsing duckdb pre-preleases correctly (1763)
- fix: update Spark min version in `utils.py` (1760)

📖 Documentation

- docs: remove docstring examples from narwhals/stable/v1 (1797)
- docs: Restore API complentess page (1788)
- docs: Increased width for content (1769)
- docs: fix `is_between` type hint in signature (1766)

🛠️ Other improvements

- chore: factor out check_columns_exist (1792)
- chore: split namespaces out from expr and series (1782)
- chore: move pyspark tests into main test suite (1761)
- chore: dask nightly (1768)
- chore: Remove some unnecessary trailing commas (1757)
- chore: validate predicates in `nw.when` one level higher (1756)
- chore: remove some expr._kwargs defaults (1747)
- chore: Filter left join warning (1753)

Thank you to all our contributors for making this release possible!
DeaMariaLeon, Dhanunjaya-Elluri, EdAbati, FBruzzesi, MarcoGorelli, camriddell, lucas-nelson-uiuc, marvinl803 and raisadz

Changes

✨ Enhancements

- feat: add `Series|Expr.rank` (1342)

🐞 Bug fixes

- fix: fix license classifier (1751)

📖 Documentation

- docs: update api-completeness with `duckdb` (1740)

🛠️ Other improvements

- test: remove `cudf` from `tests/expr_and_series/replace_time_zone_test.py::test_replace_time_zone_none[cudf]` (1748)
- chore: filter old pyarrow/pandas warnings we cant do anything about (1746)
- [pre-commit.ci] pre-commit autoupdate (1741)

Thank you to all our contributors for making this release possible!
AlessandroMiola, Dhanunjaya-Elluri, EdAbati, FBruzzesi, MarcoGorelli, anopsy

Changes

- fix: always pass Column to pandas.spark.functions (1712)
- tests: add 'core' to nox dependencies (1713)
- feat: add `weekday` to Pandas, Dask and Arrow dt namespaces (1680)
- Link to Avoiding the UserWarning error while using Pandas group_by page when warning is thrown (1683)

✨ Enhancements

- feat: Implement partial "lazy" support for DuckDB (even with this PR, DuckDB support is work-in-progress!) (1725)
- feat: validate library minimum version in compliant objects (1727)
- feat: enable to cast to `Struct` dtype (1717)
- feat: Expressify `clip` arguments (1709)
- feat: add is_nan expression & series method (1625)
- feat: show native object (if possible) in repr (1702)
- feat: add `sum` and `sum_horizontal` for `SparkLike` (1693)
- feat: add some `SparkLikeLazyFrame` methods (1633)

🐞 Bug fixes

- fix: fix broken link from warning (1732)
- fix: nw.lit(date, dtype=nw.Date), loosen Dask minimum back to 2024.8 (1730)
- chore: simplify definitions of rhs expressions, bump dask minimum to 2024.10 (1720)
- fix: dask group by with kwargs (1676)
- fix: `DaskLazyFrame` and `ArrowDataFrame` `.with_row_index` column order (1706)
- fix: scalar reductions on empty inputs (1715)
- fix: remove `maintain_order` from LazyFrame.unique (1687)
- fix: modin dtype interoperability (1692)
- fix: Dask was raising for scalar vs Series binary operations (1684)
- fix: casting to List for cudf (1686)

📖 Documentation

- docs: `Expr` method' docstrings (1733)
- docs: let `nw.LazyFrame` docstrings examples run on Polars and Dask only (1700)
- docs: update null handling page with `is_nan` reference (1716)
- docs: add spark-like in backend completeness (1710)
- docs: `Series` method' docstrings (1699)
- docs: `DataFrame` method' docstrings (1688)

🛠️ Other improvements

- test: catch some cudf failures (1735)
- chore: refactor root_names and output_names tracking (1731)
- tests: remove unnecessary pytest filterwarnings (1691)
- chore: remove some lazy tests (1719)
- fix: Catch cuDF warning for concat with empty elements, and ComputeError for non-float is_nan (1718)
- chore: use `None` instead of `float('nan')` to check for null values in tests (1697)
- test: allow to run tests for Polars[gpu] (1698)
- test: Make constructors configurable when running tests (1694)

Thank you to all our contributors for making this release possible!
AlessandroMiola, DeaMariaLeon, EdAbati, FBruzzesi, MarcoGorelli, camriddell, dependabot, dependabot[bot], lucas-nelson-uiuc and lucianosrp

Changes

🐞 Bug fixes

- fix: `is_duplicated` was returning wrong-length result for PyArrow and Dask (1679)

Thank you to all our contributors for making this release possible!
DeaMariaLeon and MarcoGorelli