Hdmf

New features
- Add methods for automatic creation of `MultiContainerInterface` classes. bendichter (420, 425)
- Add ability to specify a custom class for new columns to a `DynamicTable` that are not `VectorData`,
`DynamicTableRegion`, or `VocabData` using `DynamicTable.__columns__` or `DynamicTable.add_column(...)`. rly (436)
- Add support for creating and specifying multi-index columns in a `DynamicTable` using `add_column(...)`.
bendichter, rly (430)
- Add capability to add a row to a column after IO. bendichter (426)
- Add method `AbstractContainer.get_fields_conf`. rly (441)
- Add functionality for storing external resource references. ajtritt (442)
- Add method `hdmf.utils.get_docval_macro` to get a tuple of the current values for a docval_macro, e.g., 'array_data'
and 'scalar_data'. rly (446)
- Add `SimpleMultiContainer`, a data_type for storing a `Container` and `Data` objects together. ajtritt (449)
- Support `pathlib.Path` paths in `HDMFIO.__init__`, `HDF5IO.__init__`, and `HDF5IO.load_namespaces`. dsleiter (450)
- Use hdmf-common-schema 1.2.1. See https://hdmf-common-schema.readthedocs.io/en/latest/format_release_notes.html for details.
- Block usage of h5py 3+. h5py>=2.9, <3 is supported. rly (461)
- Block usage of numpy>=1.19.4 due to a known issue with numpy on some Windows 10 systems. numpy>1.16, <1.19.4 is supported.
rly (461)
- Add check for correct quantity during the build process in `ObjectMapper`. rly (463, 492)
- Allow passing `GroupSpec` and `DatasetSpec` objects for the 'target_type' argument of `LinkSpec.__init__(...)`.
rly (468)
- Use hdmf-common-schema 1.3.0. rly, ajtritt (486)
- Changes from hdmf-common-schema 1.2.0:
- Add data type ExternalResources for storing ontology information / external resource references. NOTE:
this data type is in beta testing and is subject to change in a later version.
- Fix missing data_type_inc and use dtype uint for CSRMatrix. It now has data_type_inc: Container.
- Add hdmf-schema-language comment at the top of each yaml file.
- Add SimpleMultiContainer, a Container for storing other Container and Data objects together.

Internal improvements
- Drop support for Python 3.5. ajtritt (459)
- Improve warning about cached namespace when loading namespaces from file. rly (422)
- Refactor `HDF5IO.write_dataset` to be more readable. rly (428)
- Fix bug in slicing tables with DynamicTableRegions. ajtritt (449)
- Add testing for Python 3.9 and using pre-release packages. ajtritt, rly (459, 472)
- Improve contributing guide. rly (474)
- Update CI. rly, dsleiter (481, 493, 497)
- Add citation information to documentation and support for duecredit tool. rly (477, 488)
- Add type checking and conversion in `CSRMatrix`. rly (485)
- Clean up unreachable validator code. rly (483)
- Reformat imports. bendichter (469)
- Remove unused or refactored internal builder functions `GroupBuilder.add_group`, `GroupBuilder.add_dataset`,
`GroupBuilder.add_link`, `GroupBuilder.set_builder`, `BaseBuilder.deep_update`, `GroupBuilder.deep_update`,
`DatasetBuilder.deep_update`. Make `BaseBuilder` not instantiable and refactor builder code. rly (452)

Bug fixes
- Fix development package dependency issues. rly (431)
- Fix handling of empty lists against a spec with text/bytes dtype. rly (434)
- Fix handling of 1-element datasets with compound dtype against a scalar spec with text/bytes dtype. rly (438)
- Fix convert dtype when writing numpy array from `h5py.Dataset`. rly (427)
- Fix inheritance when non-`AbstractContainer` is base class. rly (444)
- Fix use of `hdmf.testing.assertContainerEqual(...)` for `Data` objects. rly (445)
- Add missing support for data conversion against spec dtypes "bytes" and "short". rly (456)
- Clarify the validator error message when a named data type is missing. dsleiter (478)
- Update documentation on validation to indicate that the example command is not implemented dsleiter (482)
- Fix generated docval for classes with a LinkSpec. rly (487)
- Fix access of `DynamicTableRegion` of a `DynamicTable` with column of references. rly (491)
- Fix handling of `__fields__` for `Data` subclasses. rly (441)
- Fix `DynamicTableRegion` having duplicate fields conf 'table'. rly (441)
- Fix inefficient and sometimes inaccurate build process. rly (451)
- Fix garbage collection issue in Python 3.9. rly (496)

New features
- Add ability to get list of tuples when indexing a `DynamicTable`. i.e. disable conversion to `pandas.DataFrame`.
ajtritt (418)

Internal improvements
- Improve documentation and index out of bounds error message for `DynamicTable`. rly (419)

Bug fixes:
- Fix error when constructing `DynamicTable` with `DataChunkIterators` as columns. ajtritt (418)

New features
- Users can now use the `MultiContainerInterface` class to generate custom API classes that contain collections of
containers of a specified type. bendichter rly (399)
- See the user guide
https://hdmf.readthedocs.io/en/stable/tutorials/multicontainerinterface.html for more information.

Internal improvements
- Add ability to pass callable functions to run when adding or removing items from a ``LabelledDict``.
An error is now raised when using unsupported functionality in ``LabelledDict``. rly (405)
- Raise a warning when building a container that is missing a required dataset. rly (413)

Internal improvements
- Add tests for writing table columns with DataIO data, e.g., chunked, compressed data. rly (402)
- Add CI to check for breakpoints and print statements. rly (403)

Bug fixes:
- Remove breakpoint. rly (403)
- Allow passing None for docval enum arguments with default value None. rly (409)
- If a file is written with an orphan container, e.g., a link to a container that is not written, then an
`OrphanContainerBuildError` will be raised. This replaces the `OrphanContainerWarning` that was previously raised.
rly (407)

New features
- Users can now call `HDF5IO.export` and `HDF5IO.export_io` to write data that was read from one source to a new HDF5
file. Developers can implement the `export` method in classes that extend `HDMFIO` to customize the export
functionality. See https://hdmf.readthedocs.io/en/latest/export.html for more details. rly (#388)
- Users can use the new export functionality to read data from one source, modify the data in-memory, and then write the
modified data to a new file. Modifications can include additions and removals. To facilitate removals,
`AbstractContainer` contains a new `_remove_child` method and `BuildManager` contains a new `purge_outdated` method.
rly (388)
- Users can now call `Container.generate_new_id` to generate new object IDs for the container and all of its children.
rly (401)
- Use hdmf-common-schema 1.2.0. ajtritt rly (397)
- `VectorIndex` now extends `VectorData` instead of `Index`. This change allows `VectorIndex` to index other `VectorIndex` types.
- The `Index` data type is now unused and has been removed.
- Fix missing dtype for `VectorIndex`.
- Add new `VocabData` data type.

Breaking changes
- `Builder` objects no longer have the `written` field which was used by `HDF5IO` to mark the object as written. This
is replaced by `HDF5IO.get_written`. rly (381)
- `HDMFIO.write` and `HDMFIO.write_builder` no longer have the keyword argument `exhaust_dcis`. This remains present in
`HDF5IO.write` and `HDF5IO.write_builder`. rly (388)
- The class method `HDF5IO.copy_file` is no longer supported and may be removed in a future version. Please use the
`HDF5IO.export` method or `h5py.File.copy` method instead. rly (388)

Internal improvements
- Add ability to close open links. rly (383)

Bug fixes:
- Fix validation of empty arrays and scalar attributes. rly (377)
- Fix issue with constructing `DynamicTable` with empty array colnames. rly (379)
- Fix `TestCase.assertContainerEqual` passing wrong arguments. rly (385)
- Fix 'link_data' argument not being used when writing non-root level datasets. rly (384)
- Fix handling of ASCII numpy array. rly (387)
- Fix error when optional attribute reference is missing. rly (392)
- Improve testing for `get_data_shape` and fix issue with sets. rly (394)
- Fix inability to write references to HDF5 when the root builder is not named "root". rly (395)