Major Features And Improvements
* Existing telescope types renamed (SCT -> MSTS) and additional telescope types now supported (LST, MSTF, MSTN, SST1, SSTA, SSTC). 2D image mode still only supported for MSTS tel type
* Trace to image conversion for 2D image mode now handled by pre-generating a mapping table
* Now uses telescope num_pixels * optical_foc_len to identify telescope types
* Optical focal length and number of pixels now stored in the telescope table
* Saving raw image vectors (charge and arrival time) now supported in '1D' image mode
* '1D' image mode now saves raw images as float vectors, not uint vectors
* '1D' image mode now saves a PyTables table containing the charge image vector and arrival time image vector as separate columns
* All image tables now store a blank (all-zero) image at index 0. Index 0 is now used in event indices vectors to indicate a non-existant image, instead of index -1.
* Telescope table renamed to Telescope_Info
* Event table renamed to Event_Info
* Image tables renamed to [tel_type/tel_id]
* Now supports varying image scale factors and image dtypes by telescope type
* Units for event-level and telescope-level parameters now saved in the corresponding table attributes
* Removed reconstructed energy parameter from Event Info table
* Gamma_hadron_label parameter renamed to particle_id
* All float parameters (event-level and telescope-level) now saved as 32-bit floats
* Preselection cuts now handled using a dictionary cuts_dict (to be used in ctapipe) or a pre-generated EventDisplay cuts dictionary ED_cuts_dict
* File-level metadata now stored in root group attributes. Identical metadata fields are now enforced for all events in the output file
* particle_id
* zenith
* azimuth
* ImageExtractor version
* ctapipe version
* runlist
* The following metadata fields are included but reading them from the data is not yet implemented
* CORSIKA version
* Simtel version
* prod site array
* prod site subarray
* prod site B field
* prod site altitude
* spectral index
* E min
* E max
* Config file support removed, along with unnecessary options. Example config file removed. All remaining configuration options, value checking, default values now handled through the ImageExtractor class constructor
* 'mode'(gh_class,energy_recon) removed
* 'storage_mode' (all,mapped) -> storage_mode (tel_id,tel_type)
* 'use_pkl_dict' (True,False), energy_bins, preselection_cuts -> ED_cuts_dict,cuts_dict
* image 'mode' (PIXELS_TIMING_3C, etc.) -> img_mode (1D,2D), img_channels, include_timing
* image 'scale_factor' -> img_scale_factors(dict {tel_type: scale_factor})
* image 'dtype' -> img_dtypes(dict {tel_type: dtype})
* image 'dim_order' -> img_dim_order
* telescope 'type_mode' -> tel_type_list(list [tel_types, ...])
* energy_recon removed
* Spliting the data into train/val/test sets now supported
* Shuffling the event data randomly now supported
* '--split' command line flag now takes 3 command line arguments specifying the split fractions
* '--shuffle' command line flag now takes a random seed argument
* '--bins_cuts_dict' command line argument renamed to '--ED_cuts_dict'
* Command line program now takes a text file runlist as an input instead of a wildcard expression
* Added TravisCI and test coverage monitoring. Related dependencies added
Bug Fixes and Other Changes
* Removed unused import statements
* Collected constants
* Some PEP8 fixes (still ongoing)
* Renamed row_descriptors.py to row_types.py, trace_converter.py to image.py
* Debug command line flag now works properly with logger
* Debug output now gives the total number of events in the output file after each new input file processed
* Binning, shuffling, selecting telescopes, writing metadata all moved to ImageExtractor methods
* Some default ImageExtractor constructor arguments modified
* Example pytest tests added
Breaking Changes to the API
* SCT telescope type renamed to MSTS
* Config file removed and config options moved to ImageExtractor constructor. See above for detailed changes
* Images now stored in tables, not earrays
* Default image dtype changed to float32 from uint16. Images now saved without truncation or rounding
* Charge image vectors and arrival time image vectors now stored separately
* All image tables now save a blank image at index 0. Index 0 now used in the event table to indicate a non-existent image/non-triggered camera
* Telescope table renamed to Telescope_Info
* Event table renamed to Event_Info
* Image tables renamed to [tel_type/tel_id]
* Image scale factors and dtypes now determined by telescope type
* Reconstructed energy parameter removed
* Gamma_hadron_label renamed to particle_id
* All float parameters (event-level and telescope-level) now saved as 32-bit floats
* Command line program now takes a text file runlist as an input instead of a wildcard expression
Known Issues
* Due to the new event.mc.shower_primary_id attribute of the ctapipe MC container being only recently added, the current ctapipe version on anaconda (https://anaconda.org/cta-observatory/ctapipe) does not include it. This functionality will not work with the conda installation of ctapipe. Instead, at the current time it is necessary to install the dev version of ctapipe as described here (https://cta-observatory.github.io/ctapipe/getting_started/index.html#get-the-ctapipe-software).
* Some metadata parameters still not fully implemented