Hdmf

Bug fixes
- Fixed release CI that prevented distribution from being uploaded to PyPI. rly (699)

New features
- Added ``hdmf.container.Row.__str__`` to improve print of rows. oruebel (667)
- Added ``to_dataframe`` method for ``hdmf.common.resources.ExternalResource`` to improve visualization. oruebel (667)
- Added ``export_to_sqlite`` method for ``hdmf.common.resources.ExternalResource``. oruebel (667)
- Added ``reset_parent`` method for ``hdmf.container.Container``. rly (692)

Minor improvements
- Plotted results in external resources tutorial. oruebel (667)
- Added support for Python 3.10. rly (679)
- Updated requirements. rly TheChymera (681)
- Improved testing for `ExternalResources`. mavaylon1 (673)
- Improved docs for export. rly (674)
- Enhanced data chunk iteration speeds through new ``GenericDataChunkIterator`` class. CodyCBakerPhD (672)
- Enhanced issue template forms on GitHub. CodyCBakerPHD (700)

Bug fixes
- Fixed `setup.py` not being able to import `versioneer` when installing in an embedded Python environment. rly (662)
- Fixed broken tests in Python 3.10. rly (664)
- Fixed broken LaTeX PDF build of the docs. oruebel (669)
- Fixed adding containers as a child to a parent container sometimes not marking the parent container as modified. rly
(683)
- Fixed `to_hierarchcial_dataframe` failing when a table contains a `VectorIndex` column as a regular data column.
oruebel (666)
- Stop testing against base Python error messages because they may change in the future. rly (689)

Bug fixes
- Updated the new ``DynamicTableRegion.get_linked_tables`` function (added in 3.1.0) to return lists of ``typing.NamedTuple``
objects rather than lists of dicts. oruebel (660)

New features
- Added several features to simplify interaction with ``DynamicTable`` objects that link to other tables via
``DynamicTableRegion`` columns. oruebel (645)
- Added ``DynamicTable.get_foreign_columns`` to find all columns in a table that are a ``DynamicTableRegion``
- Added ``DynamicTable.has_foreign_columns`` to identify if a ``DynamicTable`` contains ``DynamicTableRegion`` columns
- Added ``DynamicTable.get_linked_tables`` to retrieve all tables linked to either directly or indirectly from
the current table via ``DynamicTableRegion``
- Implemented the new ``get_foreign_columns``, ``has_foreign_columns``, and ``get_linked_tables`` also for
``AlignedDynamicTable``
- Added new module ``hdmf.common.hierarchicaltable`` with helper functions to facilitate conversion of
hierarchically nested ``DynamicTable`` objects via the following new functions:
- ``to_hierarchical_dataframe`` to merge linked tables into a single consolidated pandas DataFrame.
- ``drop_id_columns`` to remove "id" columns from a DataFrame.
- ``flatten_column_index`` to replace a ``pandas.MultiIndex`` with a regular ``pandas.Index``

Bug fixes
- Do not build wheels compatible with Python 2 because HDMF requires Python 3.7. rly (642)
- ``AlignedDynamicTable`` did not overwrite its ``get`` function. When using ``DynamicTableRegion`` to reference ``AlignedDynamicTable`` this led to cases where the columns of the category subtables where omitted during data access (e.g., conversion to pandas.DataFrame). This fix adds the ``AlignedDynamicTable.get`` based on the existing ``AlignedDynamicTable.__getitem__``. oruebel (645)
- Fixed 651 to support selection of cells in an ``AlignedDynamicTable`` via slicing with ``[int, (str, str)]``(and ``[int, str, str]``) to select a single cell, and ``[int, str]`` to select a single row of a category table. oruebel (645)

Minor improvements
- Updated ``DynamicTable.to_dataframe()`` and ``DynamicTable.get`` functions to set the ``.name`` attribute
on generated pandas DataFrame objects. oruebel (645)
- Added ``AlignedDynamicTable.get_colnames(...)`` to support look-up of the full list of columns as the
``AlignedDynamicTable.colnames`` property only includes the columns of the main table for compliance with
``DynamicTable`` oruebel (645)
- Fix documentation for `DynamicTable.get` and `DynamicTableRegion.get`. rly (650)
- Allow passing string column name to `DynamicTableRegion`, i.e., `dtr['col_name']` is a shortcut to
`dtr.table['col_name']`. rly (657)

Bug fixes
- Fixed release CI that prevented distribution from being uploaded to PyPI. rly (641)

New features
- Add support for Python 3.9, drop support for Python 3.6. rly (620)
- Add support for h5py 3. ajtritt (480)
- h5py 3 introduced [breaking changes regarding how strings are handled]
(https://docs.h5py.org/en/latest/whatsnew/3.0.html#breaking-changes-deprecations), specifically that
variable-length UTF-8 strings in datasets are now read as `bytes` objects instead of `str` by default.
To reduce the impact of this change on HDMF users, when HDMF reads a variable-length UTF-8 string
dataset, instead of returning an `h5py.Dataset` that is read as `bytes` objects, HDMF will return a
`hdmf.utils.StrDataset` object that extends `h5py.Dataset` and is read as `str` objects, which preserves
previous behavior. For example, under HDMF 2.x, an HDF5 dataset `d` with data ['a', 'b'] is read as a
`h5py.Dataset` object, and `d[:]` returns `str` objects. Under HDMF 3.x, the same dataset `d` is read
as a `hdmf.utils.StrDataset` object and `d[:]` still returns `str` objects.
- Add RRID to docs. oruebel (633)
- Allow passing ``index=True`` to ``DynamicTable.to_dataframe()`` to support returning `DynamicTableRegion` columns
as indices or Pandas DataFrame. rly (579)
- Improve ``DynamicTable`` documentation. rly (639)
- Updated external resources tutorial. mavaylon1 (611)

Breaking changes and deprecations
- Previously, when using ``DynamicTable.__getitem__`` or ``DynamicTable.get`` to access a selection of a
``DynamicTable`` containing a ``DynamicTableRegion``, new columns with mangled names for the table data referred to
by the ``DynamicTableRegion`` were added to the returned DataFrame. This did not work properly for ragged
``DynamicTableRegion``, multiple levels of nesting, or multiple rows returned.
Now, these methods will by default return columns of indices of the ``DynamicTableRegion``. If ``index=False`` is
passed to ``DynamicTable.get``, then nested DataFrames will be returned, one DataFrame per row of the original
resulting DataFrame. rly (579)

Minor improvements
- Updated requirements and tests. rly (640)

Bug fixes
- Update the validator to allow extensions to data types which only define data_type_inc. dsleiter (609)
- Fix error when validating lazy-loaded datasets containing references. dsleiter (609)
- Fix error when using ``DynamicTable.__getitem__`` or ``DynamicTable.get`` when table has a ragged
``DynamicTableRegion``. rly (579)