Xorbits

What's new in 0.4.0 (2023-06-21)
These are the changes in xorbits v0.4.0.
New features
* FEAT: Support xgboost wrapper by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/502
* FEAT: Add lightGBM wrapper by yifeis7 in https://github.com/xprobe-inc/xorbits/pull/507
Enhancements
* ENH: Fix the incompatibility with the latest version of SQLAlchemy. by yifeis7 in https://github.com/xprobe-inc/xorbits/pull/520
Bug fixes
* BUG: fix `sort_index` bug of multi-level indexed Dataframe with multi-chunk by pangyoki in https://github.com/xprobe-inc/xorbits/pull/506
* BUG: fix dataframe sample by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/532
* Bug: Fix numpy CI by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/537
* BUG: fix lightgbm by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/541
* BUG: Suppress scipy warnings by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/543
Documentation
* DOC: Init Xgboost Doc by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/525
* DOC: Updated doc for python support version by shark-21 in https://github.com/xprobe-inc/xorbits/pull/518
* DOC: Install xgboost for doc building by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/539
* DOC: Add lightgbm doc by yifeis7 in https://github.com/xprobe-inc/xorbits/pull/540


**Full Changelog**: https://github.com/xprobe-inc/xorbits/compare/v0.3.2...v0.4.0

What's new in 0.3.2 (2023-06-13)
These are the changes in xorbits v0.3.2.
New features
* FEAT: Support Applymap Operand by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/445
* FEAT: Add JuiceFs as an alternative storage by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/453
* FEAT: groupby rolling by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/448
* FEAT: support pivot op by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/460
* FEAT: rolling cov by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/450
* FEAT: implement jax-based runtime graph fusion optimizer by RandomY-2 in https://github.com/xprobe-inc/xorbits/pull/440
* FEAT: support pivot_table op by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/474
* FEAT: create gpu feature on dashboard by RayJi01 in https://github.com/xprobe-inc/xorbits/pull/477
Enhancements
* ENH: Support numpy.ndarray fallback to numpy by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/434
* ENH: update pivot_table op by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/485
* ENH: update tokenizing numpy by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/486
* ENH: update pivot op by Hank0626 in https://github.com/xprobe-inc/xorbits/pull/483
* ENH: update warning msg for NPM absence by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/509
* ENH: Make memory allocation more aggressive on local by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/514
* BLD: Restrict numpy version in docker for python 3.8 and 3.9 by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/449
* BLD: Change conda numpy and scipy channel in docker build for python 3.8 by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/452
* BLD: add doc deps to setup.cfg by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/472
* BLD: fix doc build by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/481
Bug fixes
* BUG: Value Error when using dataframes' results to initialize a new dataframe by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/436
* BUG: series.tolist()[0] get Attribute Error by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/430
* BUG: pd.pivot by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/480
* BUG: fix read pickle bug and add UT by aa452948257 in https://github.com/xprobe-inc/xorbits/pull/484
* BUG: fix sort_index bug when level attribute is None by pangyoki in https://github.com/xprobe-inc/xorbits/pull/496
* BUG: Try to fix UCX communication backend argument parser by luweizheng in https://github.com/xprobe-inc/xorbits/pull/490
* BUG: Fix the GPU display issue in worker and dashboard page by RayJi01 in https://github.com/xprobe-inc/xorbits/pull/512
* BUG: Fix errors for WSL machines with GPU. by yifeis7 in https://github.com/xprobe-inc/xorbits/pull/498
Documentation
* DOC: fix the docs issue link by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/451
* DOC: correct dev doc by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/455
* DOC: git usage zh_cn by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/457
* DOC: update pot files by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/459
* DOC: Check doc building in CI by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/463
* DOC: translate documentation development by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/458
* DOC: remove `fuzzy` statements by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/468
* DOC: correct & translate creating dev env by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/470
* DOC: translation for the contributing doc by RayJi01 in https://github.com/xprobe-inc/xorbits/pull/469
* DOC: Fix docker development doc by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/473
* DOC: remove the readthedocs related content from dev docs by shark-21 in https://github.com/xprobe-inc/xorbits/pull/476
* DOC: Update Xorbits build and installation process by yifeis7 in https://github.com/xprobe-inc/xorbits/pull/495
* DOC: Adds guide to use `-f` flag to modify default config. by shark-21 in https://github.com/xprobe-inc/xorbits/pull/508
Others
* CHORE: Modify mergify by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/467

New Contributors
* Hank0626 made their first contribution in https://github.com/xprobe-inc/xorbits/pull/445
* RayJi01 made their first contribution in https://github.com/xprobe-inc/xorbits/pull/469
* shark-21 made their first contribution in https://github.com/xprobe-inc/xorbits/pull/476
* pangyoki made their first contribution in https://github.com/xprobe-inc/xorbits/pull/496
* yifeis7 made their first contribution in https://github.com/xprobe-inc/xorbits/pull/495

**Full Changelog**: https://github.com/xprobe-inc/xorbits/compare/v0.3.1...v0.3.2

What's new in 0.3.1 (2023-05-16)
These are the changes in xorbits v0.3.1.

Highlights
Xorbits is now able to pass all TPC-H queries for v0.3.1 on a single NVIDIA GPU card.
New features
* FEAT: implement sklearn.model_selection.ParameterGrid by RandomY-2 in https://github.com/xprobe-inc/xorbits/pull/421
Enhancements
* CLN: Remove k8s deploy in _mars by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/431
Bug fixes
* BUG: AttributeError when calling df.col by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/410
* BUG: Wrong dataframe id value by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/422
* BUG: Fix agg('size') execute map result, which should be a series rather than a df by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/427
* BUG: groupby same agg funcs leads to wrong results by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/433
* BUG: Fix groupby apply on GPU by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/425
* BUG: generate proper code for custom aggregation func by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/428
Others
* CHORE: Fix pyarrow version by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/423
* CHORE: Compatible with Arrow 12.0.0 by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/426


**Full Changelog**: https://github.com/xprobe-inc/xorbits/compare/v0.3.0...v0.3.1

What's new in 0.3.0 (2023-04-28)
These are the changes in xorbits v0.3.0.

Highlights
For v0.3.0, Xorbits is generally available on a single NVIDIA GPU card. For TPC-H benchmark, Xorbits is able to pass almost all queries except for q8 and q12. Multiple GPUs support will be added in the coming v0.3.x versions.
New features
* FEAT: `np.tri` by aa452948257 in https://github.com/xprobe-inc/xorbits/pull/338
* FEAT: add alluxio storage lib by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/392
Enhancements
* ENH: Support Scale Up on K8S by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/311
* ENH: GPU benchmark script by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/342
* ENH: lazy init for pandas fallback by jiayini1119 in https://github.com/xprobe-inc/xorbits/pull/350
* ENH: implement synchronous SGD for Logistic Regression by RandomY-2 in https://github.com/xprobe-inc/xorbits/pull/323
* ENH: Support DF and Series agg on GPU by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/346
* ENH: Compatible with pandas 2.0 by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/328
* ENH: remove global vars in adapters by jiayini1119 in https://github.com/xprobe-inc/xorbits/pull/372
* ENH: Support arrow list type for `read_parquet` by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/379
* ENH: Move the logic for handling this `n_io_process` to xorbits by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/401
* CLN: remove mutable service by qianduoduo0904 in https://github.com/xprobe-inc/xorbits/pull/374
Bug fixes
* BUG: Fix local init for multiple GPU devices by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/325
* BUG: Int Auto Conversion by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/336
* BUG: unexpected error from `DataFrame.__mars_tensor__` by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/340
* BUG: No permission to chmod when multi users on the same machine by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/326
* BUG: equality assertion on lazy loaded cupy by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/247
* BUG: Xorbits.numpy raises TypeError by Zhou1213CN in https://github.com/xprobe-inc/xorbits/pull/343
* BUG: fetch vram data from cluster by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/355
* BUG: fetch vram data from local cluster by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/357
* BUG: Fix pd.__dir__() by jiayini1119 in https://github.com/xprobe-inc/xorbits/pull/370
* BUG: Fix df.groupby.size on GPU by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/354
* BUG: fetching from GPU by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/381
* BUG: destory session in xorbits.shutdown by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/388
* BUG: DataFrameBinOp tile output dtype is wrong by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/367
* BUG: GPU `DataFrame` init from tensors or pandas dataframes by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/385
* BUG: Fix batch put and fetch data on GPU by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/383
* BUG: Fix `estimate_size` of `DataFrameReadParquet` by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/398
* BUG: Fix `DataFrame.set_axis` on GPU by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/368
* BUG: add attribute '__array__' for mars dataframe by Bojun-Feng in https://github.com/xprobe-inc/xorbits/pull/282
* BUG: series initializer on GPU by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/414
* BUG: GPU index initializer by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/409
* BUG: Fix merge and join on GPU by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/411
* BUG: gpu dataframe iloc by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/419
Tests
* TST: Fix pandas version by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/322
* TST: refine ut and add 'convert_to_abspath' ut by aa452948257 in https://github.com/xprobe-inc/xorbits/pull/320
* TST: fix GPU tests by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/373
* TST: reuse existing tests for GPU testing by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/386
Documentation
* DOC: add deployment doc for slurm by luweizheng in https://github.com/xprobe-inc/xorbits/pull/317
* DOC: added SLURM doc links by qianduoduo0904 in https://github.com/xprobe-inc/xorbits/pull/332

New Contributors
* luweizheng made their first contribution in https://github.com/xprobe-inc/xorbits/pull/317
* Zhou1213CN made their first contribution in https://github.com/xprobe-inc/xorbits/pull/336
* Bojun-Feng made their first contribution in https://github.com/xprobe-inc/xorbits/pull/282

**Full Changelog**: https://github.com/xprobe-inc/xorbits/compare/v0.2.2...v0.3.0

What's new in 0.2.2 (2023-04-03)
These are the changes in xorbits v0.2.2.
New features
* FEAT: impl rolling.corr by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/313
Enhancements
* ENH: globally enable task info collection by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/228
* ENH: Support xorbits.numpy.random fallback to numpy by RandomY-2 in https://github.com/xprobe-inc/xorbits/pull/244
* ENH: dump runtime stats to cloud storage systems by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/243
* ENH: Session id is prefixed with namespace in the k8s environment by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/259
* ENH: Use `get_index_and_uuid` to get device uuid by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/242
* ENH: logging improvements by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/257
* ENH: do not use global variable for profiling by qianduoduo0904 in https://github.com/xprobe-inc/xoscar/pull/11
* ENH: remove tblib by qianduoduo0904 in https://github.com/xprobe-inc/xorbits/pull/293
* ENH: use a single tempdir by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/296
* ENH: init extension entry point when importing by qianduoduo0904 in https://github.com/xprobe-inc/xorbits/pull/299
* ENH: support numpy fallback by RandomY-2 in https://github.com/xprobe-inc/xorbits/pull/284
* BLD: Fixes cython compilation failures in Python 3.11. by sighingnow in https://github.com/xprobe-inc/xorbits/pull/252
* BLD: Fix requirements for building wheel by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/263
* BLD: Add python *cp311* wheel support & drop *cp37* wheel build by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/272
* BLD: Drop Python 3.7 and support Python 3.11 for docker and CI by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/274
* BLD: Xoscar in Xorbits base image by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/281
* BLD: Trigger docker cd manually by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/286
* CLN: Remove `_is_exiting_future` by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/230
* CLN: Remove code related to caching function by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/239
* CLN: Remove `read_raydataset` by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/307
* REF: use Xoscar as dependency for actor layer by qianduoduo0904 in https://github.com/xprobe-inc/xorbits/pull/279
Bug fixes
* BUG: add sleep when waiting for endpoint ready in K8s by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/241
* BUG: delete node_modules dir in docker by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/265
* BUG: Fix np.einsum error when computing trace by RandomY-2 in https://github.com/xprobe-inc/xorbits/pull/262
* BUG: handle interactive exec exception by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/270
* BUG: fix error when inputting 1d array for np.tril by jiayini1119 in https://github.com/xprobe-inc/xorbits/pull/283
* BUG: default logging not applied in interactive mode by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/287
* BUG: Stop uploading node info after interpreter shutdown by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/291
* BUG: runtime error caused by read_csv and read_parquet with relative path by aa452948257 in https://github.com/xprobe-inc/xorbits/pull/292
Tests
* TST: GPU CI by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/240
* TST: Add "_mars" directory to coverage by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/305
Documentation
* DOC: logging by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/306
* DOC: K8s drops support for 3.7 and adds support for 3.11 by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/314
Others
* Chore(deps-dev): Bump webpack from 5.75.0 to 5.76.0 in /python/xorbits/web/ui by dependabot in https://github.com/xprobe-inc/xorbits/pull/275
* CHORE: Move gpu workflow into python.yaml by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/256
* CHORE: Fix conditions of mergify by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/260
* CHORE: Fix url in setup.cfg by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/261
* CHORE: Update codespell by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/264
* CHORE: Fix openssl version in k8s by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/266
* CHORE: Add permissions for mergify by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/294
* CHORE: Revert "Add permissions for mergify (294)" by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/302

New Contributors
* jiayini1119 made their first contribution in https://github.com/xprobe-inc/xorbits/pull/283
* aa452948257 made their first contribution in https://github.com/xprobe-inc/xorbits/pull/292

**Full Changelog**: https://github.com/xprobe-inc/xorbits/compare/v0.2.1...v0.2.2

What's new in 0.2.1 (2023-03-17)
These are the changes in xorbits v0.2.1.
Enhancements
* ENH: globally enable task info collection by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/228
* ENH: Support xorbits.numpy.random fallback to numpy by RandomY-2 in https://github.com/xprobe-inc/xorbits/pull/244
* ENH: dump runtime stats to cloud storage systems by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/243
* ENH: Session id is prefixed with namespace in the k8s environment by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/259
* ENH: Use `get_index_and_uuid` to get device uuid by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/242
* ENH: logging improvements by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/257
* BLD: Fixes cython compilation failures in Python 3.11. by sighingnow in https://github.com/xprobe-inc/xorbits/pull/252
* BLD: Fix requirements for building wheel by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/263
* BLD: Add python *cp311* wheel support & drop *cp37* wheel build by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/272
* BLD: Drop Python 3.7 and support Python 3.11 for docker and CI by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/274
* CLN: Remove `_is_exiting_future` by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/230
* CLN: Remove code related to caching function by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/239
Bug fixes
* BUG: add sleep when waiting for endpoint ready in K8s by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/241
* BUG: delete node_modules dir in docker by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/265
* BUG: Fix np.einsum error when computing trace by RandomY-2 in https://github.com/xprobe-inc/xorbits/pull/262
* BUG: handle interactive exec exception by UranusSeven in https://github.com/xprobe-inc/xorbits/pull/270
Tests
* TST: GPU CI by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/240
Others
* Chore(deps-dev): Bump webpack from 5.75.0 to 5.76.0 in /python/xorbits/web/ui by dependabot in https://github.com/xprobe-inc/xorbits/pull/275
* CHORE: Move gpu workflow into python.yaml by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/256
* CHORE: Fix conditions of mergify by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/260
* CHORE: Fix url in setup.cfg by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/261
* CHORE: Update codespell by aresnow1 in https://github.com/xprobe-inc/xorbits/pull/264
* CHORE: Fix openssl version in k8s by ChengjieLi28 in https://github.com/xprobe-inc/xorbits/pull/266

New Contributors
* sighingnow made their first contribution in https://github.com/xprobe-inc/xorbits/pull/252

**Full Changelog**: https://github.com/xprobe-inc/xorbits/compare/v0.2.0...v0.2.1