Koalas

NumPy's universal function (ufunc) compatibility

We added the compatibility of NumPy ufunc (1127). Virtually all ufunc compatibilities in Koalas DataFrame were implemented. See the example below:

python
>>> import databricks.koalas as ks
>>> import numpy as np
>>> kdf = ks.range(10)
>>> np.log(kdf)
id
0 NaN

NumPy's universal function (ufunc) compatibility

We added the compatibility of NumPy ufunc (1096, 1106). Virtually all ufunc compatibilities in Koalas Series were implemented. See the example below:

python
>>> import databricks.koalas as ks
>>> import numpy as np
>>> kdf = ks.range(10)
>>> kser = np.sqrt(kdf.id)
>>> type(kser)
<class 'databricks.koalas.series.Series'>
>>> kser

Multi-index columns support

We continue improving multi-index columns support. We made the following APIs support multi-index columns:

- `nunique` (980)
- `to_csv` (983)

Documentation

Now, we have installation guide, design principles and FAQ in our public documentation (914, 944, 963, 964)

Other new features and improvements

We added the following new features:

koalas

- `merge` (969)

koalas.DataFrame:

- `keys` (937)
- `ndim` (947)

koalas.Series:

- `keys` (935)
- `mode` (899)
- `truncate` (928)
- `xs` (921)
- `where` (922)
- `first_valid_index` (936)

koalas.Index:

- `copy` (939)
- `unique` (912)
- `ndim` (947)
- `has_duplicates` (946)
- `nlevels` (945)

koalas.MultiIndex:

- `copy` (939)
- `ndim` (947)
- `has_duplicates` (946)
- `nlevels` (945)

koalas.Expanding

- `count` (978)

Along with the following improvements:

- Fix passing options as keyword arguments (968)
- Make is_monotonic~ work properly for index (930)
- Fix Series.\_\_getitem\_\_ to work properly (934)
- Fix reindex when all the given columns are included the existing columns (975)
- Add datetime as the equivalent python type to TimestampType (957)
- Fix is_unique to respect the current Spark column (981)
- Fix bug when assign None to name as Index (974)
- Use name_like_string instead of str directly. (942, 950)

Koalas Logo

Now that we have an official logo!


We can see the cute logo in our documents as well.

Documentation

Also we improved the documentation: https://koalas.readthedocs.io/en/latest/

- Added the logo (831)
- Added a Jupyter notebook for 10 min tutorial (843)
- Added the tutorial to the documentation (853)
- Add some examples for plot implementations in their docstrings (847)
- Move contribution guide to the official documentation site (841)

Binder integration for the 10 min tutorial

You can run a live Jupyter notebook for 10 min tutorial from [](https://mybinder.org/v2/gh/databricks/koalas/master?filepath=docs%2Fsource%2Fgetting_started%2F10min.ipynb).

Multi-index columns support

We continue improving multi-index columns support. We made the following APIs support multi-index columns:

- `transform` (800)
- `round` (802)
- `unique` (809)
- `duplicated` (803)
- `assign` (811)
- `merge` (825)
- `plot` (830)
- `groupby` and its functions (833)
- `update` (848)
- `join` (848)
- `drop_duplicate` (856)
- `dtype` (858)
- `filter` (859)
- `dropna` (857)
- `replace` (860)

Plots

We also continue adding plot APIs as follows:

For DataFrame:

- `plot.kde()` (784)

Other new features and improvements

We added the following new features:

koalas.DataFrame:

- `pop` (791)
- `__iter__` (836)
- `rename` (806)
- `expanding` (840)
- `rolling` (840)

koalas.Series:

- `aggregate` (816)
- `agg` (816)
- `expanding` (840)
- `rolling` (840)
- `drop` (829)
- `copy` (869)

koalas.DataFrameGroupBy:

- `expanding` (840)
- `rolling` (840)

koalas.SeriesGroupBy:

- `expanding` (840)
- `rolling` (840)

Along with the following improvements:

- Add squeeze argument to read_csv (812)
- Raise a more helpful error for duplicated columns in Join (820)
- Issue with ks.merge to Series (818)
- Fix `MultiIndex.to_pandas()` and `__repr__()`. (832)
- Add unit and origin options for to_datetime (839)
- Fix on wrong error raise in DataFrame.fillna (844)
- Allow str and list in aggfunc in DataFrameGroupby.agg (828)
- Add `index_col` argument to `to_koalas()`. (863)