Lsstdesc-gcr-catalogs

Below is the change log since v1.2.0rc2. Please also see the change logs of [v1.2.0rc1](https://github.com/LSSTDESC/gcr-catalogs/releases/tag/v1.2.0rc1) and [v1.2.0rc2](https://github.com/LSSTDESC/gcr-catalogs/releases/tag/v1.2.0rc2) for changes since v1.1.0.

New catalogs
- Public catalog `desc_cosmodc2` is available. Similar to the DC2 public catalogs (i.e., `desc_dc2_run2.2i_dr6_*`), this catalog is intended for non-DESC members who obtain data from the [DESC Data Portal](https://lsstdesc-portal.nersc.gov/). DESC members should continue to use catalogs listed [here](https://github.com/LSSTDESC/gcr-catalogs#available-catalogs).

This version of `GCRCatalogs` will appear in conda-forge and DESC environments shortly.

Bug fix: the `GCRCatalogs.helpers` submodule will now be installed.

This is a pre-release. It includes new catalogs and features needed for the first public release of DESC data as well as other updates.

New Catalogs

- `desc_dc2_run2.2i_dr6_object` DR6 object table for the public release
- `desc_dc2_run2.2i_dr6_truth` DR6 truth table for the public release
- `desc_dc2_run2.2i_dr6_object_with_truth_match` DR6 truth-match table for the public release

**NOTE:** These catalogs are meant for public use and are not accessible using default settings at NERSC or IN2P3 sites. However they are available to DESC members under the following names, respectively:

- `dc2_object_run2.2i_dr6`
- `dc2_object_run2.2i_dr6_with_addons`
- `dc2_truth_run2.2i_summary_tract_partition`

Schema changes

- in schema for dc2_dia_source, dc2_forced_source fields and dc2_source fields `Ixx`, `Iyy`, `Ixy`, `IxxPSF`, `IyyPSF` and `IxyPSF` have been renamed to `Ixx_pixel`, `Iyy_pixel`, etc. Units for the new fields are `pixel^2` (had been incorrectly labeled `asec2` in the old fields)
- in dc2_forced_source and dc2_source the field `psf_fwhm_pixels` has been changed to `psf_fwhm_pixel`.

Updates to Documentation

The [Schema documentation](https://github.com/LSSTDESC/gcr-catalogs/GCRCatalogs/SCHEMA.md) has an updated object table description (column names and in some cases units) to be consistent with new catalogs and a description of truth summary catalogs has been added.

Improvements for Catalog Users

The following helper functions in the new module `helpers.tract_catalogs` return GCRQuery objects used in accomplishing tasks commonly appearing in example noteboooks (and user code):
- `tract_filter(tracts, tract_high=None)`
- `sample_filter(frac, random_state=None)`

Many thanks to yymao for these updates and to yymao and heather999 for work on the plan for public release.

The GCRCatalogs v1.1.0 release introduces the **Run2.2i DR2 WFD** and **SkySim5000** catalogs. In addition, the Run2.2i DR6 catalogs have been renamed (postfix `_wfd` was dropped) for clarity. There is also schema change in the object catalogs. GCRCatalogs v1.1.0 now requires GCR v0.9.2.

See below for a complete list of changes. The ["DC2 Runs Data Products and Add-ons" section of the README](https://github.com/LSSTDESC/gcr-catalogs#dc2-runs-data-products-and-add-ons) has also been updated to reflect the current list of recommended catalogs.

A code of conduct has been added to this repository.

New Catalogs

- `dc2_object_run2.2i_dr2_wfd`: static object catalog for Run2.2i DR2 (up to Year 1; WFD visits only)
- `dc2_object_run2.2i_dr2_wfd_with_addons`: same as `dc2_object_run2.2i_dr2` but with all available add-ons (currently including only truth-match; metacal/photo-z not yet available)
- `skysim5000_v1.1.1`: 5000 sq deg of extragalactic catalogs (also available are `skysim5000_v1.1.1_image` and `skysim5000_v1.1.1_small` for subsets of sky; we are also working on a parquet version of SkySim5000 for faster access)

Changes to DR6 Catalog Names

Please use the following catalogs for Run 2.2i DR6:

- `dc2_object_run2.2i_dr6`: static object catalog for Run2.2i DR6 (up to Year 5; WFD+DDF visits; a small sky region was excluded)
- `dc2_object_run2.2i_dr6_with_addons`: same as `dc2_object_run2.2i_dr6` but with metacal and truth-match add-ons. Photo-z not yet available.

All DR6 catalogs whose names contain `dr6_wfd`, `dr6a`, `dr6b`, or `dr6c` have been deprecated.

Schema change

In DC2 object catalogs, the second moment columns have been renamed so that their unit is clear. In particular, `Ixx`, `Ixy`, `Iyy`, `IxxPSF`, `IxyPSF`, and `IxyPSF` have been renamed to `Ixx_pixel`, `Ixy_pixel`, `Iyy_pixel`, `IxxPSF_pixel`, `IxyPSF_pixel`, and `IxyPSF_pixel`, respectively.

Similarly, `Ixx_<band>`, `Ixy_<band>`, `Iyy_<band>`, `IxxPSF_<band>`, `IxyPSF_<band>`, and `IxyPSF_<band>` have been renamed to `Ixx_pixel_<band>`, `Ixy_pixel_<band>`, `Iyy_pixel_<band>`, `IxxPSF_pixel_<band>`, `IxyPSF_pixel_<band>`, and `IxyPSF_pixel_<band>`, respectively.

A similar schema change will be applied to the DC2 source catalogs in the next release.

Improvements for Catalog Users

- The cosmoDC2 catalog reader now works with h5py v3+ (the version installed in `desc-python-bleed`)
- Performance and memory efficiency for accessing parquet-based catalogs have been improved when the users use arrow >=0.16
- New function `get_available_catalog_names(...)` was introduced as a shorthand for `get_available_catalogs(names_only=True, ...)`
- Users can now set default root dir in their user configs (Note: most DESC members would *not* need to do this)
- Upstream improvements from [GCR v0.9.2](https://github.com/yymao/generic-catalog-reader/releases/tag/v0.9.2) allow `get_quantity_into()` to work better with composite catalogs and quantity aliases

Improvements for Catalog Developers

- Composite catalogs now can be configured with more specific options thanks to [GCR v0.9.0](https://github.com/yymao/generic-catalog-reader/releases/tag/v0.9.0). See the GCR v0.9.0 release note for details.

Many thanks to the whole DC2 team for preparing the data products, and to JoanneBogart heather999 evevkovacs for help in preparing this release. This version should be available in conda-forge soon and in DESC environments next week.

This release will be the first version to appear on conda-forge.

Changelog

- Dependencies clean up

GCRCatalogs v1.0.3 includes:
- one new catalog: `dc2_object_run2.2i_dr6_wfd_matched` (thanks to fjaviersanchez)
- minor edits on the listing of truth catalogs (thanks to JoanneBogart)

This version will soon be available on all DESC environments (thanks to heather999). Use `desc-python-bleed` for immediate use.