This release includes only the writer (raw data to HDF5) portion of the full DL1 Data Handler package. The matching data reader and processor will be included in future releases.
Major Features And Improvements
* Major refactoring of image-extractor into DL1DataWriter class, based on a DL1DataDumper class and the ctapipe EventSource class.
* Now uses on YAML configuration files and run lists for all processing settings
* Python multiprocessing package used to allow parallel processing of runs.
* Image mapping functionality in image.py removed.
* Command line script moved out of main package and into separate write_data.py script.
* generate_runlist.py script added to automatically create run lists from directories of files.
* Telescope type names in output files (table columns and table names) modified from original ctapipe names to be valid Python identifiers (for use when doing selections).
* Telescope optics and camera types added to telescope type table.
* xmax column added to event table.
* Telescope image multiplicity columns added to event table.
Bug Fixes and Other Changes
* .gitignore updated to include build, dist directories.
* TravisCI configuration instructions updated, Python version updated 3.5 -> 3.6.
* AUTHORS list updated.
* run_id renamed to obs_id to match ctapipe container field.
* requirements.txt file reduced by using pipreqs instead of pip freeze.
* Out of date scripts moved to scripts/deprecated.
* Improvements to logging.
* Example config and run list files added.
* Fixed an issue where some tables were not being written correctly before files were closed.
Breaking Changes to the API
* Various name changes to columns and tables (see wiki for full data format specification)
* Telescope type names in output files (table columns and table names) modified from original ctapipe names to be valid Python identifiers (for use when doing selections).
* Telescope optics and camera types added to telescope type table.
* xmax column added to event table
* Telescope image multiplicity columns added to event table
Known Issues
* prod_site_array, prod_site_coord, and prod_site_subarray are included in the output files but defaulted to None, as they are included in the ctapipe MC header container but not supported in event_source as of https://github.com/cta-observatory/ctapipe/commit/5ef63ef2a7a093455f4b0683a68653cc42e84df7.
* Calling file.close() in the destructor of CTAMLDataDumper causes a KeyError on some data files due to an empty NodeManager registry. The cause of this exception is unclear. The call has been removed as it has no effect on the output files.