What's Changed
> [!WARNING]
> The LH5 I/O routines have been refactored! Some function names have changed and new methods for loading and viewing data have been added. Read the migration guide below for more details. This release is fully backward compatible, but deprecation warnings will show up when using the old methods. Upgrade to the new recommended syntax to suppress them.
**NEW**: the package now offers support for viewing LGDO data (Tables, in particular) as [Awkward arrays](https://awkward-array.org/) through the `LGDO.view_as()` interface. Awkward Array is a library for nested, variable-sized data, including arbitrary-length lists, records, mixed types, and missing data, using NumPy-like idioms.
Please consult the API documentation on https://legend-pydataobj.readthedocs.io to learn about the new methods.
Migration Guide
Imports
LH5 I/O related routines have been moved to a dedicated subpackage: `lgdo.lh5`
Old syntax:
py
from lgdo.lh5_store import LH5Store, ls
store = LH5Store()
ls("file.lh5")
New recommended syntax:
py
from lgdo import lh5
store = lh5.LH5Store()
lh5.ls("file.lh5")
Read/write LGDOs to disk
Old syntax:
py
store = LH5Store()
obj, _ = store.read_object("obj", "file.lh5")
store.write_object(obj, "obj", "file.lh5")
New syntax:
py
store = lh5.LH5Store()
obj, _ = store.read("obj", "file.lh5")
store.write(obj, "obj", "file.lh5")
Convert LGDO to another format
`LGDO.view_as()` is the new recommended way to view (i.e. without performing a copy) LGDOs in alternative formats (Pandas, Numpy, Awkward...)
Old syntax:
py
table = Table(...)
table.get_dataframe()
New syntax:
py
table.view_as("pd")
Old syntax:
py
from lgdo.lh5_store import load_nda, load_dfs
load_nda("file.lh5", ["obj"])
load_dfs("file.lh5", ["tbl"])
New syntax:
py
from lgdo import lh5
lh5.read_as("obj", "file.lh5", library="np")
lh5.read_as("obj", "file.lh5", library="pd")
New syntax (longer alternative):
py
from lgdo import lh5
store = lh5.LH5Store()
obj, _ = store.read("obj", "file.lh5")
obj.view_as("np")
tbl, _ = store.read("tbl", "file.lh5")
tbl.view_as("pd")
Full list of changes
* Fixed bug in LH5Iterator when number of entries for file is zero by iguinn in https://github.com/legend-exp/legend-pydataobj/pull/39
* Refactor of LH5 I/O routines, deprecation of existing methods by MoritzNeuberger in https://github.com/legend-exp/legend-pydataobj/pull/24
* Support (environment) variables for tweaking Numba at runtime by gipert in https://github.com/legend-exp/legend-pydataobj/pull/44
* Add vectorized operations to VectorOfVectors by iguinn in https://github.com/legend-exp/legend-pydataobj/pull/42
* Add LGDO format conversion utilities by MoritzNeuberger in https://github.com/legend-exp/legend-pydataobj/pull/30
* Added depth option to show and lh5ls by iguinn in https://github.com/legend-exp/legend-pydataobj/pull/52
* Reimplement `Table.eval()`, now handling `VectorOfVectors` by gipert in https://github.com/legend-exp/legend-pydataobj/pull/53
* Deprecate `load_nda()` and `load_dfs()` in favour of `.view_as()` by gipert in https://github.com/legend-exp/legend-pydataobj/pull/56
* Support setting a fill value when "exploding" `VectorOfVectors` into NumPy arrays in `.view_as("np")` by gipert in https://github.com/legend-exp/legend-pydataobj/pull/57
* Migrate to pyproject.toml, upgrade pre-commit config by gipert in https://github.com/legend-exp/legend-pydataobj/pull/59
* Fix for reading just first row of VectorOfVectors by ggmarshall in https://github.com/legend-exp/legend-pydataobj/pull/63
* Feature: `lh5.read_as()` to read LH5 data straight into third party data views by gipert in https://github.com/legend-exp/legend-pydataobj/pull/62
* Added warning when adding a column to a table with different length by MoritzNeuberger in https://github.com/legend-exp/legend-pydataobj/pull/58
* Add first version of CITATION.cff by gipert in https://github.com/legend-exp/legend-pydataobj/pull/64
* Bug fix in `LH5Store.read()`: check for `n_rows` longer than `idx`s before dropping by ggmarshall in https://github.com/legend-exp/legend-pydataobj/pull/65
* Bugfix for varlen error msgs and specify nda in view_as "ak" so dtype correctly inferred by ggmarshall in https://github.com/legend-exp/legend-pydataobj/pull/67
* Add Patrick to CITATION.cff by gipert in https://github.com/legend-exp/legend-pydataobj/pull/68
* `Table.view_as()` performance fixes by gipert in https://github.com/legend-exp/legend-pydataobj/pull/70
New Contributors
* MoritzNeuberger made their first contribution in https://github.com/legend-exp/legend-pydataobj/pull/24
**Full Changelog**: https://github.com/legend-exp/legend-pydataobj/compare/v1.4.2...v1.5.0