Neuroconv

Bug Fixes
* Fixed a setup bug introduced in `v0.6.2` where installation process created a directory instead of a file for test configuration file [PR 1070](https://github.com/catalystneuro/neuroconv/pull/1070)
* The method `get_extractor` now works for `MockImagingInterface` [PR 1076](https://github.com/catalystneuro/neuroconv/pull/1076)
* Updated opencv version for security [PR 1087](https://github.com/catalystneuro/neuroconv/pull/1087)
* Solved a bug of `PlexonRecordingInterface` where data with multiple streams could not be opened [PR 989](https://github.com/catalystneuro/neuroconv/pull/989)

Deprecations

Features
* Added chunking/compression for string-only compound objects: [PR 1042](https://github.com/catalystneuro/neuroconv/pull/1042)
* Added automated EFS volume creation and mounting to the `submit_aws_job` helper function. [PR 1018](https://github.com/catalystneuro/neuroconv/pull/1018)
* Added a mock for segmentation extractors interfaces in ophys: `MockSegmentationInterface` [PR 1067](https://github.com/catalystneuro/neuroconv/pull/1067)
* Added a `MockSortingInterface` for testing purposes. [PR 1065](https://github.com/catalystneuro/neuroconv/pull/1065)
* BaseRecordingInterfaces have a new conversion options `always_write_timestamps` that can be used to force writing timestamps even if neuroconv heuristic indicates regular sampling rate [PR 1091](https://github.com/catalystneuro/neuroconv/pull/1091)


Improvements
* Testing on mac sillicon [PR 1061](https://github.com/catalystneuro/neuroconv/pull/1061)
* Add writing to zarr test for to the test on data [PR 1056](https://github.com/catalystneuro/neuroconv/pull/1056)
* Modified the CI to avoid running doctests twice [PR 1077](https://github.com/catalystneuro/neuroconv/pull/#1077)
* Consolidated daily workflows into one workflow and added email notifications [PR 1081](https://github.com/catalystneuro/neuroconv/pull/1081)
* Added zarr tests for the test on data with checking equivalent backends [PR 1083](https://github.com/catalystneuro/neuroconv/pull/1083)

Bug Fixes
* Fixed a bug where `IntanRecordingInterface` added two devices [PR 1059](https://github.com/catalystneuro/neuroconv/pull/1059)
* Fix a bug in `add_sorting_to_nwbfile` where `unit_electrode_indices` was only propagated if `waveform_means` was passed [PR 1057](https://github.com/catalystneuro/neuroconv/pull/1057)

Deprecations
* The following classes and objects are now private `NWBMetaDataEncoder`, `NWBMetaDataEncoder`, `check_if_imaging_fits_into_memory`, `NoDatesSafeLoader` [PR 1050](https://github.com/catalystneuro/neuroconv/pull/1050)

Features
* Make `config_file_path` optional in `DeepLabCutInterface`[PR 1031](https://github.com/catalystneuro/neuroconv/pull/1031)
* Added `get_stream_names` to `OpenEphysRecordingInterface`: [PR 1039](https://github.com/catalystneuro/neuroconv/pull/1039)
* Most data interfaces and converters now use Pydantic to validate their inputs, including existence of file and folder paths. [PR 1022](https://github.com/catalystneuro/neuroconv/pull/1022)
* All remaining data interfaces and converters now use Pydantic to validate their inputs, including existence of file and folder paths. [PR 1055](https://github.com/catalystneuro/neuroconv/pull/1055)


Improvements
* Using ruff to enforce existence of public classes' docstrings [PR 1034](https://github.com/catalystneuro/neuroconv/pull/1034)
* Separated tests that use external data by modality [PR 1049](https://github.com/catalystneuro/neuroconv/pull/1049)
* Added Unit Table descriptions for phy and kilosort: [PR 1053](https://github.com/catalystneuro/neuroconv/pull/1053)
* Using ruff to enforce existence of public functions's docstrings [PR 1062](https://github.com/catalystneuro/neuroconv/pull/1062)
* Improved device metadata of `IntanRecordingInterface` by adding the type of controller used [PR 1059](https://github.com/catalystneuro/neuroconv/pull/1059)

Bug fixes
* Fixed the JSON schema inference warning on excluded fields; also improved error message reporting of which method triggered the error. [PR 1037](https://github.com/catalystneuro/neuroconv/pull/1037)

Deprecations
* Deprecated `WaveformExtractor` usage. [PR 821](https://github.com/catalystneuro/neuroconv/pull/821)
* Changed the `tools.spikeinterface` functions (e.g. `add_recording`, `add_sorting`) to have `_to_nwbfile` as suffix [PR 1015](https://github.com/catalystneuro/neuroconv/pull/1015)
* Deprecated use of `compression` and `compression_options` in `VideoInterface` [PR 1005](https://github.com/catalystneuro/neuroconv/pull/1005)
* `get_schema_from_method_signature` has been deprecated; please use `get_json_schema_from_method_signature` instead. [PR 1016](https://github.com/catalystneuro/neuroconv/pull/1016)
* `neuroconv.utils.FilePathType` and `neuroconv.utils.FolderPathType` have been deprecated; please use `pydantic.FilePath` and `pydantic.DirectoryPath` instead. [PR 1017](https://github.com/catalystneuro/neuroconv/pull/1017)
* Changed the `tools.roiextractors` function (e.g. `add_imaging` and `add_segmentation`) to have the `_to_nwbfile` suffix [PR 1017](https://github.com/catalystneuro/neuroconv/pull/1027)


Features
* Added `MedPCInterface` for operant behavioral output files. [PR 883](https://github.com/catalystneuro/neuroconv/pull/883)
* Support `SortingAnalyzer` in the `SpikeGLXConverterPipe`. [PR 821](https://github.com/catalystneuro/neuroconv/pull/821)
* Added `TDTFiberPhotometryInterface` data interface, for converting fiber photometry data from TDT file formats. [PR 920](https://github.com/catalystneuro/neuroconv/pull/920)
* Add argument to `add_electrodes` that grants fine control of what to do with the missing values. As a side effect this drops the implicit casting to int when writing int properties to the electrodes table [PR 985](https://github.com/catalystneuro/neuroconv/pull/985)
* Add Plexon2 support [PR 918](https://github.com/catalystneuro/neuroconv/pull/918)
* Converter working with multiple `VideoInterface` instances [PR 914](https://github.com/catalystneuro/neuroconv/pull/914)
* Added helper function `neuroconv.tools.data_transfers.submit_aws_batch_job` for basic automated submission of AWS batch jobs. [PR 384](https://github.com/catalystneuro/neuroconv/pull/384)
* Data interfaces `run_conversion` method now performs metadata validation before running the conversion. [PR 949](https://github.com/catalystneuro/neuroconv/pull/949)
* Introduced `null_values_for_properties` to `add_units_table` to give user control over null values behavior [PR 989](https://github.com/catalystneuro/neuroconv/pull/989)


Bug fixes
* Fixed the default naming of multiple electrical series in the `SpikeGLXConverterPipe`. [PR 957](https://github.com/catalystneuro/neuroconv/pull/957)
* Write new properties to the electrode table use the global identifier channel_name, group [PR 984](https://github.com/catalystneuro/neuroconv/pull/984)
* Removed a bug where int64 was casted lossy to float [PR 989](https://github.com/catalystneuro/neuroconv/pull/989)

Improvements
* The `OpenEphysBinaryRecordingInterface` now uses `lxml` for extracting the session start time from the settings.xml file and does not depend on `pyopenephys` anymore. [PR 971](https://github.com/catalystneuro/neuroconv/pull/971)
* Swap the majority of package setup and build steps to `pyproject.toml` instead of `setup.py`. [PR 955](https://github.com/catalystneuro/neuroconv/pull/955)
* The `DeeplabcutInterface` now skips inferring timestamps from movie when timestamps are specified, running faster. [PR 967](https://github.com/catalystneuro/neuroconv/pull/967)
* Improve metadata writing for SpikeGLX data interface. Added contact ids, shank ids and, remove references to shanks for neuropixels 1.0. Also deprecated the previous neuroconv exclusive property "electrode_shank_number` [PR 986](https://github.com/catalystneuro/neuroconv/pull/986)
* Add tqdm with warning to DeepLabCut interface [PR 1006](https://github.com/catalystneuro/neuroconv/pull/1006)
* `BaseRecordingInterface` now calls default metadata when metadata is not passing mimicking `run_conversion` behavior. [PR 1012](https://github.com/catalystneuro/neuroconv/pull/1012)
* Added `get_json_schema_from_method_signature` which constructs Pydantic models automatically from the signature of any function with typical annotation types used throughout NeuroConv. [PR 1016](https://github.com/catalystneuro/neuroconv/pull/1016)
* Replaced all interface annotations with Pydantic types. [PR 1017](https://github.com/catalystneuro/neuroconv/pull/1017)
* Changed typehint collections (e.g. `List`) to standard collections (e.g. `list`). [PR 1021](https://github.com/catalystneuro/neuroconv/pull/1021)
* Testing now is only one dataset per test [PR 1026](https://github.com/catalystneuro/neuroconv/pull/1026)

Deprecations
* The usage of `compression_options` directly through the `neuroconv.tools.audio` submodule is now deprecated - users should refer to the new `configure_backend` method for a general approach for setting compression. [PR 939](https://github.com/catalystneuro/neuroconv/pull/939)
* The usage of `compression` and `compression_opts` directly through the `FicTracDataInterface` is now deprecated - users should refer to the new `configure_backend` method for a general approach for setting compression. [PR 941](https://github.com/catalystneuro/neuroconv/pull/941)
* The usage of `compression` directly through the `neuroconv.tools.neo` submodule is now deprecated - users should refer to the new `configure_backend` method for a general approach for setting compression. [PR 943](https://github.com/catalystneuro/neuroconv/pull/943)
* The usage of `compression_options` directly through the `neuroconv.tools.ophys` submodule is now deprecated - users should refer to the new `configure_backend` method for a general approach for setting compression. [PR 940](https://github.com/catalystneuro/neuroconv/pull/940)
* Removed the option of running `interface.run_conversion` without `nwbfile_path` argument . [PR 951](https://github.com/catalystneuro/neuroconv/pull/951)

Features
* Added docker image and tests for an automated Rclone configuration (with file stream passed via an environment variable). [PR 902](https://github.com/catalystneuro/neuroconv/pull/902)

Bug fixes
* Fixed the conversion option schema of a `SpikeGLXConverter` when used inside another `NWBConverter`. [PR 922](https://github.com/catalystneuro/neuroconv/pull/922)
* Fixed a case of the `NeuroScopeSortingExtractor` when the optional `xml_file_path` is not specified. [PR 926](https://github.com/catalystneuro/neuroconv/pull/926)
* Fixed `Can't specify experiment type when converting .abf to .nwb with Neuroconv`. [PR 609](https://github.com/catalystneuro/neuroconv/pull/609)
* Remove assumption that the ports of the Intan acquisition system correspond to electrode groupings in `IntanRecordingInterface` [PR 933](https://github.com/catalystneuro/neuroconv/pull/933)
* Add ValueError for empty metadata in `make_or_load_nwbfile` when an nwbfile needs to be created [PR 948](https://github.com/catalystneuro/neuroconv/pull/948)

Improvements
* Make annotations from the raw format available on `IntanRecordingInterface`. [PR 934](https://github.com/catalystneuro/neuroconv/pull/943)
* Add an option to suppress display the progress bar (tqdm) in `VideoContext` [PR 937](https://github.com/catalystneuro/neuroconv/pull/937)
* Automatic compression of data in the `LightnignPoseDataInterface` has been disabled - users should refer to the new `configure_backend` method for a general approach for setting compression. [PR 942](https://github.com/catalystneuro/neuroconv/pull/942)
* Port over `dlc2nwb` utility functions for ease of maintenance. [PR 946](https://github.com/catalystneuro/neuroconv/pull/946)