Magenpy

Changed

- Updated the logic for `detect_outliers` in phenotype transforms to actually reflect the function
name (before it was returning true for inliers...).
- Updated `quantize` and `dequantize` to minimize data copying as much as possible.
- Updated `LDMatrix.load_rows()` method to minimize data copying.
- Fixed bug in `LDMatrix.n_neighbors` implementation.
- Updated `dask` version in `requirements.txt` to avoid installing `dask-expr`.


Added

- Added `get_peak_memory_usage` to `system_utils` to inspect peak memory usage of a process.
- Placeholder method to perform QC on `SumstatsTable` objects (needs to be implemented still).
- New attached dataset for long-range LD regions.
- New method in SumstatsTable to impute rsID (if missing).
- Preliminary support for matching with CHR+POS in SumstatsTable (still needs more work).
- LDMatrix updates:
- New method to filter long-range LD regions.
- New method to prune LD matrix.
- New algorithm for symmetrizing upper triangular and block diagonal LD matrices.
- Much faster and more memory efficient than using `scipy`.
- New `LDMatrix` class has efficient data loading in `.load_data` method.
- We still retain `load_rows` because it is useful for loading a subset of rows.

Changed

- Fixed `manhattan` plot implementation to support various new features.
- Added a warning when accessing `csr_matrix` property of `LDMatrix` when it hasn't been loaded
previously.

Added

- `reset_mask` method for magenpy `LDMatrix`.
- `Dockerfile`s for both `cli` and `jupyter` modes.
- A helper script to convert LD matrices from old format to new format.

Changed

- Fixed bugs in how covariates are processed in `SampleTable`.
- Fixed bugs / issues in implementation of GWAS with `xarray` backend.
- Streamlined implementation of `manhattan` plotting function.

A large scale restructuring of the code base to improve efficiency and usability.

Changed

- Bug fixes across the entire code base.
- Simulator classes have been renamed from `GWASimulator` to `PhenotypeSimulator`.
- Moved plotting script to its own separate module.
- Updated some method names / commandline flags to be consistent throughout.

Added

- Basic integration testing with `pytest` and GitHub workflows.
- Documentation for the entire package using `mkdocs`.
- Integration testing / automating building with GitHub workflows.
- New implementation of the LD matrix that uses CSR matrix data structures.
- Quantization / float precision specification when storing LD matrices.
- Allow user to specify Compressor / Compressor options for Zarr storage.
- New implementation of `magenpy_simulate` script.
- Allow users to set random seed.
- Now accept `--prop-causal` instead of specifying full mixing proportions.
- Tried to incorporate `genome_build` into various data structures. This will be useful in the
future to ensure consistent genome builds across different data types.
- Allow user to pass various metadata to `magenpy_ld` to save information about dataset
characteristics.
- New sumstats parsers:
- Saige sumstats format.
- plink1.9 sumstats format.
- GWAS Catalog sumstats format.
- Chained transform function for transforming phenotypes.

Changed

- Removed the `--fast-math` compiler flag due to concerns about
numerical precision (e.g. [Beware of fast-math](https://simonbyrne.github.io/notes/fastmath/)).
- Updated implementation of `SumstatsParser` class to allow user to specify `read_csv_kwargs` at the point of instantiation.
- Updated plink executors to propagate the error messages to the user.
- Updated `merge_snp_tables` to allow for merges on columns other than `SNP`.
- Refactored, cleaned, and updated the implementation of the `AnnotationMatrix` class.
- Fixed bug in `GWADataLoader.split_by_samples()`: Need to perform `deepcopy`, otherwise splitting would not work properly.
- Updated `read_annotations` method in `GWADataLoader` to work with the latest `AnnotationMatrix` interfaces.
- Fixed bug in the `manhattan` plotting function.

Added

- Added parsers for functional annotations and annotation files. Mainly support LDSC annotation format for now.
- Added a utility method to `GWADataLoader` called `align_with` to streamline aligning `GWADataLoader` objects across SNP and sample dimensions.
- Added utility methods for flattening the LD matrix in `LDMatrix`.
- Added a method to perform matrix-vector multiplication in `LDMatrix`.
- Added a method to perform block-wise iteration in the `LDMatrix` class.

Changed

- Fixed bug in implementation of `identify_mismatched_snps`.
- Fixed bugs in handling of missing information in LD matrix.
- Fixed bug in handling of covariates in `SampleTable`.
- Updated `README` file to remove line indicators `>>>` from sample code.

Added

- Added the reference allele `A2` to the output of the `true_beta_table`
in `GWASimulator`.