Daft

Changes

✨ New Features

- [FEAT] (ACTORS-2) Add optimization pass to split Project into ActorPoolProject jaychia (2627)
- [FEAT] Stream results from native executor into python colin-ho (2667)
- [FEAT]: huggingface integration universalmind303 (2701)

🚀 Performance Improvements

- [PERF] Fix excessive parquet metadata reading Vince7778 (2694)

👾 Bug Fixes

- [BUG] Use python logging level colin-ho (2705)
- [BUG] Add a with\_execution/planning\_config context manager and fix tests for splitting of parquet jaychia (2713)
- [BUG] Fix Resource Request Serialization and factor our Serialize Object as bincode samster25 (2707)

📖 Documentation

- [DOCS] Partitioning user guide and small doc fixes jaychia (2717)
- [FEAT] (ACTORS-2) Add optimization pass to split Project into ActorPoolProject jaychia (2627)
- [BUG] Add a with\_execution/planning\_config context manager and fix tests for splitting of parquet jaychia (2713)
- Update PreCommit Hooks samster25 (2715)
- [FEAT]: huggingface integration universalmind303 (2701)

🧰 Maintenance

- [CHORE] switch CI to use UV samster25 (2716)
- [CHORE] Add no benchmark flag for integration io tests colin-ho (2711)
- [CHORE] Speed up parquet integration test colin-ho (2706)
- [CHORE] Enable codspeed benchmarking colin-ho (2696)

Changes

✨ New Features

- [FEAT]: sql case/when universalmind303 (2591)
- [FEAT] Add comparison of timestamps with same timezone Vince7778 (2604)
- [FEAT] Add support for pyiceberg v0.7 kevinzwang (2594)
- [FEAT] Make the `end` argument for `.list.slice()` optional desmondcheongzx (2593)

🚀 Performance Improvements

- [PERF] Add physical plan optimizer and optimization Vince7778 (2557)

👾 Bug Fixes

- [BUG]: remove simsimd dependency universalmind303 (2605)
- [BUG] Fix parquet reads when a top-level column's final row spans more than one data page desmondcheongzx (2586)
- [BUG]: accept "iterable[pa.Table]" for from\_arrow universalmind303 (2583)

📖 Documentation

- [CHORE] Fix imports on jupyter notebook examples kevinzwang (2600)

🧰 Maintenance

- [CHORE] Fix imports on jupyter notebook examples kevinzwang (2600)
- [CHORE]: ignore ".zed" directory universalmind303 (2595)

Changes

✨ New Features

- [FEAT] Fix resource accounting in PyRunner jaychia (2567)
- [FEAT] Add `.str.count_matches()` Vince7778 (2580)
- [FEAT]: streaming json universalmind303 (2582)
- [FEAT]: `embedding.cosine_distance` function universalmind303 (2526)
- [FEAT] Enable buffered iteration on plans jaychia (2566)
- [FEAT] Streaming CSV Reads colin-ho (2565)
- [FEAT] Support Reading Iceberg Merge-on-Read Position Deletes kevinzwang (2563)
- [FEAT]: daft sql universalmind303 (2558)
- [FEAT] Initializes daft-sql and defines the daft.sql(..) function. RCHowell (2559)
- [FEAT] Add image mode casting Vince7778 (2562)
- [FEAT] Add tracing to local execution engine samster25 (2556)
- [FEAT] Create file when writing dataframe with no rows kevinzwang (2540)
- [FEAT] Add string tokenize expression Vince7778 (2503)
- [FEAT]: support optional rowgroups to `read_parquet` universalmind303 (2534)
- [FEAT] Add hashjoin, sort, and hashagg ops to new executor colin-ho (2530)

👾 Bug Fixes

- [BUG] Fix `.str.length()` on Unicode strings Vince7778 (2579)
- [BUG] Fix table filters with scalar mask colin-ho (2542)
- [BUG] Enable sorting bool columns colin-ho (2529)

📖 Documentation

- [FEAT] Add `.str.count_matches()` Vince7778 (2580)
- [DOCS]: small update to "scaling-up" universalmind303 (2577)
- [DOCS] quickstart-revision avriiil (2124)
- [DOCS] Update Ray Dataset link kevinzwang (2564)
- [FEAT] Add image mode casting Vince7778 (2562)
- [DOCS] Fix notebook CI jaychia (2544)
- [FEAT] Add string tokenize expression Vince7778 (2503)
- [DOCS] Do not generate PDFs jaychia (2539)
- [DOCS] Build PDF and htmlzip formats of docs for offline consumption jaychia (2538)

🧰 Maintenance

- [CHORE] Remove daft-execution colin-ho (2553)
- [CHORE] Enable all features for rust-analyzer Vince7778 (2560)
- [CHORE] Add TPC-H questions 11-22 to benchmarks (currently skipped) kevinzwang (2299)
- [CHORE]: move minhash to daft-functions universalmind303 (2518)

⬆️ Dependencies

- Bump async-compression from 0.4.10 to 0.4.12 dependabot (2548)

Changes

👾 Bug Fixes

- [BUG] Fix bug with map\_groups UDFs that return more than 1 output row for empty partitions jaychia (2532)
- [BUG] Use shared thread pool for multiple running instances of df on pyrunner jaychia (2502)
- [BUG] Fix bug with multi-partition `any_value` Vince7778 (2531)
- [BUG] Allow for Parquet reading from files with differing schemas jaychia (2514)
- [BUG] With\_new\_children not implemented for sample colin-ho (2528)

🧰 Maintenance

- [CHORE] add pytest benchmarking for local testing of execution engine samster25 (2523)

Changes

✨ New Features

- [FEAT] Decouple pipeline building and running from new executor colin-ho (2522)
- [FEAT] Add concat to new execution model + buffered intermediate ops colin-ho (2519)
- [FEAT] Math Ops for FixedSizeList / FixedShapeTensor / Embedding Type samster25 (2507)
- [FEAT] adding transform functionality otacilio-psf (2498)
- [FEAT] List chunk expression desmondcheongzx (2491)
- [FEAT] Refactors and agg improvements for new local execution model colin-ho (2497)
- [FEAT]: dyn function registry universalmind303 (2466)
- [FEAT] New Local Execution Model colin-ho (2437)
- [FEAT] Implement hashing and groupby on lists Vince7778 (2464)
- [FEAT] Add upload functionality to binary columns jaychia (2461)
- [FEAT] List slice expression desmondcheongzx (2479)
- [FEAT] add bit shift functions murex971 (2453)
- [FEAT] Implements trigonometry expressions: arctanh arccosh arcsinh fedemagnani (2476)
- [FEAT] Microsoft Fabric support in AzureConfig kevinzwang (2465)

🚀 Performance Improvements

- [PERF]: dont read parquet metadata multiple times universalmind303 (2358)
- [PERF] Local Execution Plan samster25 (2489)
- [PERF] Optimize string normalization Vince7778 (2474)

👾 Bug Fixes

- [BUG] merge conflict for python catalog scan task samster25 (2517)
- [BUG] Add retries for write timeout errors mauriceweber (2508)
- [BUG] Use Daft s3 credentials chain for deltalake reads jaychia (2486)
- [BUG] Support delta-rs version >0.17.4 in deltalake writes jaychia (2488)
- [BUG] Fix anti-join on different column names Vince7778 (2477)

📖 Documentation

- [FEAT] adding transform functionality otacilio-psf (2498)
- [DOCS] Tutorial: FOTW data access avriiil (2384)
- [FEAT] List chunk expression desmondcheongzx (2491)
- [DOCS] Enhance Dataframe / Expressions examples sunaysanghani (2360)
- [FEAT] add bit shift functions murex971 (2453)
- [FEAT] Implements trigonometry expressions: arctanh arccosh arcsinh fedemagnani (2476)
- [DOCS] Adds RunLLM widget vsreekanti (2462)
- [DOCS] Fix broken example URL kevinzwang (2467)
- [FEAT] Microsoft Fabric support in AzureConfig kevinzwang (2465)
- [DOCS] Remove unsupported map get doc example kevinzwang (2452)

🧰 Maintenance

- [CHORE] remove tokio-stream as dep samster25 (2521)
- [CHORE] drop unused deps and add machete CI check samster25 (2520)
- [CHORE] fix incremental builds with vscode rust analyzer samster25 (2515)
- [CHORE] Disable Python as default feature and have maturin enable it by default samster25 (2516)
- [CHORE]: remove daft-core from daft-io universalmind303 (2513)
- [CHORE]: move uri functions to new "daft-functions" crate universalmind303 (2501)
- [CHORE] Add better typing for class UDFs jaychia (2388)
- [CHORE] Add a helpful message in build-artifact-s3 workflow jaychia (2480)