Echopype

Overview

This release includes a number of small fixes and enhancements. An important component is to set the max supported python version to <3.12, before we fully migrate to using the new [`xarray.DataTree`](https://github.com/pydata/xarray/blob/main/DATATREE_MIGRATION_GUIDE.md) in the next release with breaking changes.

Enhancements
* Remove min max attributes for `compute_MVBS` by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1380
* Update setup-services.py by spacetimeengineer in https://github.com/OSOceanAcoustics/echopype/pull/1398
* Add support for AZFP 130kHz by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1412
* Update calibrate_base.py to include EA640 in backscatter size check. by torsteinnh in https://github.com/OSOceanAcoustics/echopype/pull/1407

Bug and warning fixes
* Handle `NaN` in `apply_mask` input mask by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1376
* EK60 Conversion: Drop channels that don't have power data by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1383
* Skip Parsing Invalid EK80 Environment Datagram(s) by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1387
* Fix 'Division by Zero' error when converting EK80 files without Sound Velocity Profile Depth value(s) by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1381
* Add error warning for all NaN lat/lon in `get_distance_from_latlon` by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1403
* Updated paths in test_align_to_ping_time_glider_azfp to use pytest fixture by oftfrfbf in https://github.com/OSOceanAcoustics/echopype/pull/1414

Documentation
* Update installation doc page by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1389

Others
* Set max python version <3.12 [all tests ci] by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1422

New Contributors
* spacetimeengineer made their first contribution in https://github.com/OSOceanAcoustics/echopype/pull/1398
* oftfrfbf made their first contribution in https://github.com/OSOceanAcoustics/echopype/pull/1414
* torsteinnh made their first contribution in https://github.com/OSOceanAcoustics/echopype/pull/1407

**Full Changelog**: https://github.com/OSOceanAcoustics/echopype/compare/v0.9.0...v0.9.1

Overview

This release includes a number of new features and performance enhancements, as well as fixes for bugs and warnings. We have fully switched to using the [trunk-based](https://www.atlassian.com/continuous-delivery/continuous-integration/trunk-based-development) development workflow in this release.


New features
* Add noise and attenuation functions from Ryan et. al. 2015 and echopy by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1316
* Add `.bot` and `.idx` files to EK60/EK80 parsing by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1318
* Add index aggregation option for cleaning mask functions by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1326
* Added alpha-version code for new AZFP6 format by dash-uvic in https://github.com/OSOceanAcoustics/echopype/pull/1323
* Use latest MRU parser version and enable `ep.consolidate.add_location` can be based off datagram type by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1359

Enhancements
* Support lazy-loaded EK80 broadband-complex data by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1311
* Allow `MVBS` into `ep.mask.apply_mask`, and add alignment check for target variable and mask by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1345
* Enhance `ep.calibrate.compute_Sv` scalability by enabling EchoData chunking and removing eager computation by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1331
* Use Echodata `Platform` and `Beam_group1` variables to compute depth for EK60/EK80 by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1319
* Allow waveform mode "FM" in calibrate by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1363
* Allow external arrays in `add_depth` and add an universal ping time alignment function by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1369

Bug and warning fixes
* Fix duck array `RuntimeWarning` for EK60 `ep.open_raw` by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1337
* Coding: `channel_mode` dtype from `float16` to `float32` by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1350 and https://github.com/OSOceanAcoustics/echopype/pull/1351
* Fix problems in applying `compute_Sv` on combined `EchoData` object by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1328
* Fix data type check for padding short complex pings by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1353
* Update `xr.drop` to `xr.drop_vars` by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1364

Documentation
* Update docs for visualization by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1347
* Merge data processing level changes from `stable` by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1355
* Add the Ping acoustic library to other resources by gavinmacaulay in https://github.com/OSOceanAcoustics/echopype/pull/1362
* Update contributing guide and README/index by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1356 and https://github.com/OSOceanAcoustics/echopype/pull/1373

Others
* Remove the deprecated `visualize` subpackage by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1336
* Temporarily skip datagrams generated by EC150 by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1357
* Modify dask image in requirement.txt by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1361

New Contributors
* dash-uvic made their first contribution in https://github.com/OSOceanAcoustics/echopype/pull/1323

**Full Changelog**: https://github.com/OSOceanAcoustics/echopype/compare/v0.8.4...v0.9.0

Overview

This release includes a few performance enhancements, bug fixes, and improving warnings/error messages. We are also preparing to switch to using the [trunk-based](https://www.atlassian.com/continuous-delivery/continuous-integration/trunk-based-development) development workflow in the next release to streamline new contributions.

Enhancements
* Upgrade `compress_pulse` to use parallelized convolution by anantmittal in https://github.com/OSOceanAcoustics/echopype/pull/1208
* Optimize `harmonize_env_param_time` by anantmittal in https://github.com/OSOceanAcoustics/echopype/pull/1235
* Optimize frequency differencing with dask by anantmittal in https://github.com/OSOceanAcoustics/echopype/pull/1198
* Optimize `get_vend_cal params_power` by anujsinha3 in https://github.com/OSOceanAcoustics/echopype/pull/1285
* `apply_mask` changes: Allow multi-channel masks, allow the `depth` dimension, and simplify fill value by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1230
* Update `_calc_Sv_offset` for AZFP parsing by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1304
* Move `compute_Sv_offset` to inside of AZFP parser by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1307
* Support calibration of EK80 broadband complex data from sequential pinging by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1302
* Add `skipna` argument to `compute_MVBS/NASC` to control mean behavior over NaN elements by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1269

Bug fixes and warnings
* Fix error parsing integer in frequency-differencing equation by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1278
* Fix time encoding to use int64 and "nanoseconds since 1970-01-01 00:00:00Z" by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1299
* Add warnings and raised errors for `add_location` by ctuguinay in https://github.com/OSOceanAcoustics/echopype/pull/1296

Others
* One time `dev` merge to `main` to facilitate switching to trunk-based workflow by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1297
* Engaging in Google Summer of Code 2024 program by leewujung https://github.com/OSOceanAcoustics/echopype/pull/1266, https://github.com/OSOceanAcoustics/echopype/pull/1267, https://github.com/OSOceanAcoustics/echopype/pull/1274
* Update Development roadmap by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1262
* Deprecating `echopype.visualize` by leewujung in https://github.com/OSOceanAcoustics/echopype/pull/1310

New Contributors
* anantmittal made their first contribution in https://github.com/OSOceanAcoustics/echopype/pull/1208
* anujsinha3 made their first contribution in https://github.com/OSOceanAcoustics/echopype/pull/1285
* ctuguinay made their first contribution in https://github.com/OSOceanAcoustics/echopype/pull/1230

**Full Changelog**: https://github.com/OSOceanAcoustics/echopype/compare/v0.8.3...v0.8.4

Overview

This release includes a bug fix for changes from the previous release and a few functionality enhancements.

Enhancements
- Add parser support for EK80 MRU1 datagram (1242)
- Add support for `consolidate` subpackage functions to accept both in-memory or stored datasets (1216)
- Add test for ES60 spare field decoding issue (1233)
- Add test for EK80 missing `receiver_sampling_freq` error (1234)

Bug fixes
- Fixed reshape bug in `pad_shorter_ping` that was remnant from `use_swap` full refactoring (1234)

Overview

This release includes a number of important performance enhancements, bug fixes, and under-the-hood refactoring to improve code readability.

Enhancements
- Full refactor of the functionality to directly storing parsed data to zarr before set_groups_* (1185, 1218, 1070)
- This avoids memory expansion during NaN padding across channel
- Improve `compute_MVBS` using flox (1124)
- Revive and improve `compute_NASC` using flox (1167)
- Refactor AZFP XML parser
- Improved XML parsing by using `xml.etree.ElementTree` to be consistent with other XML parsers in echopype (1135)
- Parse more parameters and store in the resulting `EchoData` objects (1135)
- Add partial support for AZFP multiple phase settings (1182)
- Overhaul AZFP Environment group
- Set mandatory variables not in data to NaN (`sounds_speed_indicative`, `absorption_indicative`) (1226)
- Parse and store AZFP pressure data when exist (1189, 1226)
- Add `utils.misc.depth_from_pressure` function for converting pressure to depth (1207)
- Remove unused mechanism to selectively parse some EK datagrams (1214)
- Enhancements of `EchoData.update_platform` method
- Factor out utility functions to a submodule (1209)
- Auto-assign timestamp for fixed-location external data using first `ping_time`(1196)
- Add support for `consolidate` subpackage functions to accept both in-memory or stored datasets (1216) [NOT MERGED]

Bug fixes
- Fix `EchoData.to_zarr` encoding (1128)
- Eliminate preferred chunks
- Ensuring chunk alignment and encoding only when handling dask arrays
- Fix scaling bugs in `compute_NASC` (1167)
- Fix `UnicodeDecodeError` for ES60 files (1215)
- Handle missing `receiver_sampling_frequency` for EK80 data (1219)

Infrastructure and refactoring
- Add support for running individual test files (1166)
- Add module based testing (1180)
- Update CI to barebone python, without conda (1192)

Overview

This release includes a bug fix and a few infrastructure upgrades and enhancements.

Enhancements and bug fix
- More intuitive input syntax for `frequency_differencing` (1106)
- Set EK60/80 Platform and NMEA nan timestamp to first `ping_time` value (1154)
- This avoids warnings due to xarray's new behavior with nan timestamp in coordinate
- Remove automatic version conversion in `open_converted` (1143)

Infrastructure
- Bump testing minimum Python version to 3.9 (1146)
- Experiment with Python 3.11 in testing (1148, 1156)
- Some tests related to `parsed2zarr` fail due to the pinned `more_itertools` version in python 3.11. `parsed2zarr` is being overhauled in 1070 so these are deferred to the next release.
- Remove logging during testing (1141)
- Remove NASC import causing failure (1145)