Erlab

Breaking Changes

- Deprecated module `erlab.io.utilities` is removed. Use `erlab.io.utils` instead. ([e189722](https://github.com/kmnhan/erlabpy/commit/e189722f129d55cab0d2ec279e5303929cb09979))
- Deprecated module `erlab.interactive.utilities` is removed. Use `erlab.interactive.utils` instead. ([af2c81c](https://github.com/kmnhan/erlabpy/commit/af2c81c676455ddfa19ae9bbbbdbdd68d257f26c))
- Deprecated module `erlab.characterization` is removed. Use `erlab.io.characterization` instead. ([8d770bf](https://github.com/kmnhan/erlabpy/commit/8d770bfe298253c020aeda6d61a9eab625facf6c))
- Deprecated module `erlab.analysis.utils` is removed. Use `erlab.analysis.transform.shift` and `erlab.analysis.gold.correct_with_edge`. ([0b2ca44](https://github.com/kmnhan/erlabpy/commit/0b2ca44844cc5802d32d9ed949e831b534525183))
- Deprecated alias `slice_along_path` in `erlab.analysis` is removed. Call from `erlab.analysis.interpolate` instead. ([305832b](https://github.com/kmnhan/erlabpy/commit/305832b7bb18aa3d1fda21f4cd0c0992b174d639))
- Deprecated aliases `correct_with_edge` and `quick_resolution` in `erlab.analysis` are removed. Call from `erlab.analysis.gold` instead. ([075eaf8](https://github.com/kmnhan/erlabpy/commit/075eaf8cd222044aa5cc0c3459698ab33568958c))
- Removed deprecated aliases `load_igor_ibw` and `load_igor_pxp`. Use `xarray.open_dataarray` and `xarray.open_dataset` instead. ([7f07ad2](https://github.com/kmnhan/erlabpy/commit/7f07ad2c46f80d48c255d408f3f200ae01930060))
- The default attribute name for the sample temperature is changed to `sample_temp` from `temp_sample`. This will unfortunately break a lot of code that relies on the key `temp_sample`, but will be easy to refactor with find and replace. ([32e1cd5](https://github.com/kmnhan/erlabpy/commit/32e1cd5fb45bce12cfa83c520e8c61af96a8cb39))
- All dataloaders must now add a new keyword argument to `load_single`, but implementing it is not mandatory.

Also, dataloaders that implements summary generation by overriding `generate_summary` must now switch to the new method.

See the summary generation section in the updated user guide.

Furthermore, the `isummarize` method is no longer public; code that uses this method should use `summarize` instead.

The `usecache` argument to the `summarize` method is no longer available, and the cache will be updated whenever it is outdated. ([0f5dab4](https://github.com/kmnhan/erlabpy/commit/0f5dab46e3d3a75fc77908b4072f08aa89059acd))

Features

- **io.igor:** enable loading experiment files to DataTree ([1835be0](https://github.com/kmnhan/erlabpy/commit/1835be0d08ed899b2edbb06fb442cd9addb40929))

Added methods to the backend to allow using `xarray.open_datatree` and `xarray.open_groups` with Igor packed experiment files. Closes [29](https://github.com/kmnhan/erlabpy/issues/29)
- add `qinfo` accessor ([eb3a742](https://github.com/kmnhan/erlabpy/commit/eb3a74297211aae8f13e6974563e6da819bfbedb))

Adds a `qinfo` accessor that prints a table summarizing the data in a human readable format. Closes [27](https://github.com/kmnhan/erlabpy/issues/27)
- **interactive.kspace:** pass lattice parameters and colormap info to `ktool` ([6830af3](https://github.com/kmnhan/erlabpy/commit/6830af343326e0367a6dfb016728a6cf1325cf64))

Added the ability to pass lattice vectors and colormaps to `ktool`.
- **interactive.kspace:** add circle ROI to ktool ([304e1a5](https://github.com/kmnhan/erlabpy/commit/304e1a53f189ebed9a890680c3499a756c586498))

Added a button to the visualization tab which creates a circle ROI. The position and radius can be edited by right-clicking on the roi.
- **interactive.colors:** add zero center button to right-click colorbar ([c037de1](https://github.com/kmnhan/erlabpy/commit/c037de1f4387c0daf7cc7aa252124f01269bc633))
- **interactive.imagetool:** add `.ibw` and `.pxt` files to load menu ([73c3afe](https://github.com/kmnhan/erlabpy/commit/73c3afef306109be858d23dbf8511617c5d203dd))
- **io.dataloader:** allow passing rcParams to interactive summary plot ([a348366](https://github.com/kmnhan/erlabpy/commit/a34836673315fdc9acc0ed52d8e56edc90c18456))
- **io.dataloader:** implement automatic summary generation ([0f5dab4](https://github.com/kmnhan/erlabpy/commit/0f5dab46e3d3a75fc77908b4072f08aa89059acd))

It is now much easier to implement a summary generation mechanism. This commit also adds a new keyword argument to `load_single` that can greatly speed up summary generation.
- **io.dataloader:** support callable objects in `additional_attrs` ([e209499](https://github.com/kmnhan/erlabpy/commit/e209499c8044f0085fda74b7dc491517a695099c))

Bug Fixes

- **interactive.imagetool:** fix copy cursor value for numpy 2 ([dc19c82](https://github.com/kmnhan/erlabpy/commit/dc19c827c4082989e47b0f8e2d7adda45ad62aaa))
- **io.dataloader:** retain selected dimension in interactive summary ([9d54f8b](https://github.com/kmnhan/erlabpy/commit/9d54f8b3402767cf15e6cf5ab00ee5a1b766d172))
- **accessors.general:** keep associated coords in `qsel` when averaging ([03a7b4a](https://github.com/kmnhan/erlabpy/commit/03a7b4a30b4c6a635f904fcab377298b06b86f66))
- **io.dataloader:** ignore old summary files ([bda95fc](https://github.com/kmnhan/erlabpy/commit/bda95fc1f0aaec73c179fd47258f6fde8056aaf9))
- **io.plugins.kriss:** fix KRISS ibw file match pattern ([7ced571](https://github.com/kmnhan/erlabpy/commit/7ced57152edb802bd14f831c77494a6f805f5097))
- **analysis.gold:** retain attributes in `quick_resolution` ([504acdc](https://github.com/kmnhan/erlabpy/commit/504acdc1d7d9b8dcd4613ca97551d78c366f0337))
- do not require qt libs on initial import ([118ead6](https://github.com/kmnhan/erlabpy/commit/118ead603b89867e56b29932f59bd02b476ab43b))

Code Refactor

- **io.igor:** change wave dimension name handling ([3e0586a](https://github.com/kmnhan/erlabpy/commit/3e0586ae34893698317136bfccc5fd839b91332e))

Waves with both dim and unit labels provided were given a dim label formatted like `dim(unit)`. This update changes this so that the dim label is just `dim`, and the unit is inserted to coordinate attrs.
- **io:** remove deprecated module ([e189722](https://github.com/kmnhan/erlabpy/commit/e189722f129d55cab0d2ec279e5303929cb09979))
- **interactive:** remove deprecated module ([af2c81c](https://github.com/kmnhan/erlabpy/commit/af2c81c676455ddfa19ae9bbbbdbdd68d257f26c))
- remove deprecated module `erlab.characterization` ([8d770bf](https://github.com/kmnhan/erlabpy/commit/8d770bfe298253c020aeda6d61a9eab625facf6c))
- **analysis:** remove deprecated module ([0b2ca44](https://github.com/kmnhan/erlabpy/commit/0b2ca44844cc5802d32d9ed949e831b534525183))
- **analysis:** remove deprecated alias ([305832b](https://github.com/kmnhan/erlabpy/commit/305832b7bb18aa3d1fda21f4cd0c0992b174d639))
- **analysis:** remove deprecated aliases ([075eaf8](https://github.com/kmnhan/erlabpy/commit/075eaf8cd222044aa5cc0c3459698ab33568958c))
- **interactive.imagetool.manager:** add prefix to temporary directories for better identification ([e56163b](https://github.com/kmnhan/erlabpy/commit/e56163ba7fe7d92f3a01ec78098c2d0194ea0302))
- **io.plugins:** implement DA30 file identification patterns in superclass ([f6dfc44](https://github.com/kmnhan/erlabpy/commit/f6dfc4412b56fc1d83efceb4a65070eb9ef1c2b1))
- **io:** remove deprecated aliases ([7f07ad2](https://github.com/kmnhan/erlabpy/commit/7f07ad2c46f80d48c255d408f3f200ae01930060))
- change temperature attribute name ([32e1cd5](https://github.com/kmnhan/erlabpy/commit/32e1cd5fb45bce12cfa83c520e8c61af96a8cb39))

Changes `temp_sample` to `sample_temp` for all data loaders and analysis code.
- **utils.formatting:** change formatting for numpy arrays ([95d9f0b](https://github.com/kmnhan/erlabpy/commit/95d9f0b602551141232eb5a2fa10c421d11d2233))

For arrays with 2 or more dimensions upon squeezing, only the minimum and maximum values are shown. Also, arrays with only two entries are displayed as a list.
- **io.dataloader:** disable parallel loading by default ([fed2428](https://github.com/kmnhan/erlabpy/commit/fed2428229e3ef70fc95a35670fc75ace44024bd))

Parallel loading is now disabled by default since the overhead is larger than the performance gain in most cases.
- change some warnings to emit from user level ([e81f2b1](https://github.com/kmnhan/erlabpy/commit/e81f2b121d2931b327d30b146db1e77e7a3b3ec2))
- **io.dataloader:** cache summary only if directory is writable ([85bcb80](https://github.com/kmnhan/erlabpy/commit/85bcb80bdf27ea12edb9314247a978f71c8be6dc))
- **io.plugins:** improve warning message when a plugin fails to load ([9ee0b90](https://github.com/kmnhan/erlabpy/commit/9ee0b901b1b904dabb38d29f4c166dca07c9a7e9))
- **io:** update datatree to use public api ([6c27e07](https://github.com/kmnhan/erlabpy/commit/6c27e074c5aceb16eb9808cca38b8ba73748f07e))

Also bumps the minimum supported xarray version to 2024.10.0.
- **io.dataloader:** make `RegistryBase` private ([df7079e](https://github.com/kmnhan/erlabpy/commit/df7079e4fc96b195d34436bcc93684e10ddecdad))
- **io.dataloader:** rename loader registry attribute `default_data_dir` to `current_data_dir` ([d87eba7](https://github.com/kmnhan/erlabpy/commit/d87eba7db6cea051f76b61ea7b0834e439460810))

The attribute `default_data_dir` has been renamed to `current_data_dir` so that it is consistent with `current_loader`. Accessing the old name is now deprecated.

Also, the `current_loader` and `current_data_dir` can now be assigned directly with a syntax like `erlab.io.loaders.current_loader = "merlin"`.

Performance

- **io.plugins.da30:** faster summary generation for DA30 zip files ([22b77bf](https://github.com/kmnhan/erlabpy/commit/22b77bf0ee787fe1236fb85262702b79265e3b8d))
- **io.igor:** suppress `igor2` logging ([5cd3a8c](https://github.com/kmnhan/erlabpy/commit/5cd3a8c273b143d1a83f3286678638fede1ddd01))
- **analysis.interpolate:** extend acceleration ([84daa88](https://github.com/kmnhan/erlabpy/commit/84daa8866ec4223555568f441b6010bb4936a413))

The fast linear interpolator now allows more general interpolation points like interpolating 3D data on a 2D grid. This means that passing `method='linearfast'` to `DataArray.interp` is faster in many cases.

Features

- **interactive.imagetool:** add normalization option to View menu ([53e2cf2](https://github.com/kmnhan/erlabpy/commit/53e2cf2b6e49de70a6857b782acc8ec0815a93b5))
- **io.dataloader:** allow passing additional arguments to `load_single` ([1652c20](https://github.com/kmnhan/erlabpy/commit/1652c20de102f7bb5fa4a26360652341f3249c2e))
- **io.plugins:** add support for two new beamlines, closes [61](https://github.com/kmnhan/erlabpy/issues/61) ([368263e](https://github.com/kmnhan/erlabpy/commit/368263ef282ef3ec39cc6c8bbe23f26ddcf18b8f))

Added plugins with preliminary support for Diamond I05 and ALBA BL20 LOREA.
- **io:** add `nexusutils` module for working with NeXus files ([2532941](https://github.com/kmnhan/erlabpy/commit/25329416d837cd318bab51fe7ff747e5f83cbc19))

This commit adds a new submodule `io.nexusutils` that contains utilities for converting NeXus data to xarray data structures.

Bug Fixes

- **erlab.io.plugins.merlin:** resolve typo in file dialog methods ([39caa99](https://github.com/kmnhan/erlabpy/commit/39caa998cc6857cd197dc1a0a7efc961ad3afe46))
- **erlab.accessors.general:** make `qsel` accessor work along dimensions with no coordinates ([7f0d259](https://github.com/kmnhan/erlabpy/commit/7f0d259f75626316d28497a3e25e1009d7c05851))
- **interactive.imagetool:** avoid errors on termination ([9fd044b](https://github.com/kmnhan/erlabpy/commit/9fd044bae170e025f5e27eb392fc600c19ec30b8))

Code Refactor

- **io:** return path-like objects instead of strings in `get_files` ([2eb9166](https://github.com/kmnhan/erlabpy/commit/2eb9166e4183222e9cb282fc0bf4984d76bf3124))

Bug Fixes

- **io.dataloader:** fix `coordinate_attrs` not being propagated ([278675b](https://github.com/kmnhan/erlabpy/commit/278675b54d2e12471ce8629fbd6d249aa7184c0e))

Code Refactor

- add app icon for imagetool manager ([e1cbcd2](https://github.com/kmnhan/erlabpy/commit/e1cbcd29b045cb2d586baa3c6272fd60cfd05979))

Features

- **io.dataloader:** add new argument that can control combining ([bdec5ff](https://github.com/kmnhan/erlabpy/commit/bdec5ff24e02e82597cef10d225997599efaa257))

Adds a new parameter `combine` to `io.load`. If `False`, returns a list of post-processed files without attempting to concatenate or merge the data into a single object. If `True`, retains the current default behavior.

Bug Fixes

- **imagetool:** allow coords of any dtype coercible to float64 ([4342ebc](https://github.com/kmnhan/erlabpy/commit/4342ebc1bc4be01fcc9c7883ecfbaef0f5857e5d))
- **io.dataloader:** properly handle combining multi-axis scans ([2cd22c7](https://github.com/kmnhan/erlabpy/commit/2cd22c7998cd22d399e59a86131e4c5712127b23))

Code Refactor

- **io.plugins:** update type hints ([54d0c5d](https://github.com/kmnhan/erlabpy/commit/54d0c5d7e55cc7e6af6b4d83feeb9d6c863e52f6))
- remove unused imports ([f1e35de](https://github.com/kmnhan/erlabpy/commit/f1e35ded993f1fb2be04c549aa241809c7d68a4d))
- **interactive:** add informative error message for missing pyqt ([1347a02](https://github.com/kmnhan/erlabpy/commit/1347a0231698f8104be71116823f797caeccc9a6))
- **io.plugins:** add warning when plugin load fails ([ed5b184](https://github.com/kmnhan/erlabpy/commit/ed5b184538cde60c1fa2ba2421ff30c27acb1eed))

Features

- **io.plugins:** add loader for beamline ID21 ESM at NSLS-II ([c07e490](https://github.com/kmnhan/erlabpy/commit/c07e490c37b52706aa9407d6adb5aa7787e2c1b0))

This commit adds a new data loader for beamline ID21 ESM at NSLS-II, Brookhaven National Laboratory.
- **io.dataloader:** add formatters ([2ee9a4a](https://github.com/kmnhan/erlabpy/commit/2ee9a4a2361b91727c4e964adc00a26398863f2f))

A new attribute named `formatters` and a new method `get_formatted_attr_or_coord` has been added to loaders. This allows custom per-attribute pretty-printing behavior.
- **io:** add parallel argument to `load` ([88cd924](https://github.com/kmnhan/erlabpy/commit/88cd924efd9375ebe89417b504c80e55f7071404))
- **io:** add xarray backend for igor files ([1fe5ca5](https://github.com/kmnhan/erlabpy/commit/1fe5ca514777f4356f1b67251dc1a6f21b320d48))

`.pxt`, `.pxp`, and `.ibw` files can now be opened with xarray methods such as `xr.open_dataset` and `xr.open_dataarray`. See the updated user guide for more information.

Bug Fixes

- **io.dataloader:** properly reorder coordinates ([3ebfb0f](https://github.com/kmnhan/erlabpy/commit/3ebfb0fdbc21e690e51c185da92ed9c88921d3b1))

Coordinate order was broken for loaders which assign coordinates in inherited `post_process`. This is now fixed, and returned data will be consistently ordered with respect to the mapping, with the dimension coordinates coming first.
- **erlab.io.plugins.maestro:** temporary fix for xarray issue ([c2d04a3](https://github.com/kmnhan/erlabpy/commit/c2d04a31ee98cbbcc403dafce6cf8dd6e37a09e5))
- **io.plugins.da30:** properly handle output types ([6297aba](https://github.com/kmnhan/erlabpy/commit/6297aba2474ce4a28e865f48320d37e47c4ba1f7))

The DA30 loader now tries to return a dataset from `.zip` files only when there are no coordinate conflicts. In the case of conflicts, the loader will return a `DataTree`.

Also, single region DA30 `.pxt` files will now return a `DataArray` consistent with the equivalent `.ibw` file.
- **interactive.fermiedge:** allow transposed input to fermi edge fitting tool ([dcae75e](https://github.com/kmnhan/erlabpy/commit/dcae75e718dd6b0e2cc773e43407e3a6a2cd97f0))
- **interactive.imagetool:** retain attrs when exporting slice ([1bed572](https://github.com/kmnhan/erlabpy/commit/1bed572dc7309d723152237902b021d27d0dbb40))

When accessing the data of a single slice from the right-click menu of `ImageTool`, the attributes of the original data are now kept. This allows saved slices or data opened in other tools to retain their attributes.
- resolve gui script not working on windows with conda ([62253d0](https://github.com/kmnhan/erlabpy/commit/62253d0d64d9cf458ee3e58e20fc890914c13c18))
- **io.dataloader:** clear plot before loading in interactive summaries ([21d6dea](https://github.com/kmnhan/erlabpy/commit/21d6dea9876066c64b9a9d2dc62dd9d41ba3ec9b))
- **plotting.general:** fix `plot_slices` compatibility with slice object as argument ([5948a7b](https://github.com/kmnhan/erlabpy/commit/5948a7b64fb415cfe86668bb17f3c656df1466a6))
- **io:** disable memmapping when loading data ([c39da1b](https://github.com/kmnhan/erlabpy/commit/c39da1b64ad28447260b9131f38206131bd6c0cb))

Memmapping seemed to interfere loading multiple files when called through ipywidgets.

Code Refactor

- **io.utils:** use pathlib in `get_files` ([b7a0f5b](https://github.com/kmnhan/erlabpy/commit/b7a0f5b33e9a1f7a6840413f431fd60a10bb754e))
- **io.plugins.merlin:** combine ImageTool file menu into single entry ([6e28ac2](https://github.com/kmnhan/erlabpy/commit/6e28ac21d7b534f5243f25a40710910eefb3a2b8))
- **io:** add postprocessing and validation for `DataTree` objects ([3fb3ff5](https://github.com/kmnhan/erlabpy/commit/3fb3ff5ec3755a40d6f2a66e9f242e6be9529816))
- **io:** add warning when file is ambiguous ([8daabb8](https://github.com/kmnhan/erlabpy/commit/8daabb85eb4b4313fc05199fa0eb8ed4de6c80f6))
- **io:** remove renaming steps from `load_single` in multi-file loaders ([542f4f2](https://github.com/kmnhan/erlabpy/commit/542f4f2874f5255e71a53ecc58c0b87c68c84aaa))

Combining before renaming coords should be more straightforward
- **io:** allow missing alpha coord ([c9deed4](https://github.com/kmnhan/erlabpy/commit/c9deed43a0ecce20a70145d9fe6a21b2a1ea1693))

Validation checks will not warn about missing detector angle, allowing XPS measurements.
- **io.dataloader:** only allow real file or folder names as input ([b9a59cc](https://github.com/kmnhan/erlabpy/commit/b9a59cc4c4cd16c44c93237b24b519ab270aff79))

The previous behavior allowed passing `f_001.pxt` to load `f_001_S001.pxt`, `f_001_S002.pxt`... but this was confusing since there is no file named `f_001.pxt`. This commit disallows such input.
- cleanup erplot namespace ([007eedb](https://github.com/kmnhan/erlabpy/commit/007eedb8862fcdcd8a54bb170c5ec0272db3f194))
- **io:** implement metaclass ([e787b1c](https://github.com/kmnhan/erlabpy/commit/e787b1c63b1cf786797ddcd2c360438619574f80))

Whenever the `identify()` method failed to find any files, subclasses had to explicitly raise `FileNotFoundError`. This resulted in a lot of boilerplate code and ambiguous error messages. Now, all subclasses can just return `None` in `identify()` when no files are found. The appropriate error is automatically raised.
- **io.igor:** raise OSError on load fail ([6c7a4c4](https://github.com/kmnhan/erlabpy/commit/6c7a4c471874c8fb92c267f6c551e332e35d0bc6))
- move dataloader cell formatting implementation to utils module ([0f2cb1c](https://github.com/kmnhan/erlabpy/commit/0f2cb1c70b2c7dbadd5ab936770290486b50200a))
- **io:** deprecate calling igor functions from top level namespace ([bb8af7c](https://github.com/kmnhan/erlabpy/commit/bb8af7c76b1bf31f481de2a43f104f8c8a638225))

Calling `erlab.io.load_wave` and `erlab.io.load_experiment` is deprecated. When writing new code, use `xarray.load_dataarray` and `xarray.load_dataset` instead.

Performance

- **interactive.imagetool:** improve manager speed ([891c4ee](https://github.com/kmnhan/erlabpy/commit/891c4eed0921fdb5b4ebd2cb17e6c932afa79ccc))