Immunopipe

Highlights

- feat: support `Seurat` v5 (integration is now down by `Seurat::IntegrateLayers`)
- feat: support supervised clustering (mapping cells to reference by `Seurat`)
- feat: support dataset with scRNA-seq data only (no scTCR-seq data)
- feat: support diffusion map calculation (by `ModuleScoreCalculator`)
- feat: support subclassing to cluster subsets of cells (by `SeuratSubClustering`)
- feat: allow to ignore TCR data in `TCellSelection` and pass kmeans arguments
- feat: allow to set multiple resolutions (`envs.FindClusters.resolution`) in `SeuratClustering`/`SeuratClusteringOfTCells`
- change: change unsuperved cluster labels to `c1`, `c2`, ... in `SeuratClustering` by default
- docs: add gallery, which contains real-world examples of datasets from publications

Breaking changes

- change: rename `SeuratMetadataMutater` to `IntegratingTCR`
- change: rename `SeuratClusteringOfTCells` to `SeuratClustering`
- change: rename `TCRClusters2Seurat` to `IntegratingTCRClusters`
- refactor: make `SeuratClustering` (instead of `SeuratClusteringOfAllCells`) work for all cells when all are T cells
- change: move data preparation and integration from `SeuratClustering` to `SeuratPreparing`
- change: default `mode` of `ImmunarchLoading` to `paired` (instead of `single`), which requires both alpha and beta chains (instead of beta chain only) to define a clonotype
- change: default `dbs` for enrichment analysis wherever applies to `KEGG_2021_Human` and `MSigDB_Hallmark_2020`

Changes

- feat: make `TopExpressingGenes` optional
- feat: add `validate_config` to validate configuration schematically

Features

- feat(SeuratPreparing): allow to filter genes directly (by specifying `envs.gene_qc.excludes`)
- feat(SeuratClusterStats): add `ngenes` to plot the number of genes expressed in each cluster
- feat(SeuratClusterStats): add `barplot` for features and allow aggregation of features
- feat(SeuratClusterStats): add `envs.mutaters` to mutate meta data
- feat(SeuratClusterStats): add histograms to plot number of cells against another variable
- feat(SeuratClusterStats): Add `frac_ofall` and `transpose` for stats to calculate fraction within group or against all cells, and transpose ident and group, respectively

Dependencies

- deps: add `r-presto` to conda environment files to support using presto to fastly find markers
- deps: add `bioconductor-destiny` to conda environment file to support add diffusion map components in ModuleScoreCalculator
- deps: add `r-harmony` to support harmony integration by Seurat v5 in conda env file
- deps: add `r-sf` to conda env file
- deps: remove `vdjtools` from conda env files
- deps: bump `pipen-report` to [0.16.3](https://github.com/pwwang/pipen-report/releases/tag/0.16.3)
- deps: bump `biopipen` to [0.23.3](https://github.com/pwwang/biopipen/releases). Hightlight changes:
- scrna.MarkersFinder: Add `envs.use_presto` to use presto to speed up finding markers
- scrna.SeuratPreparing: Set envs.gene_qc.min_cells to 0 by default (instead of 3)
- scrna.ScFGSEA: Allow to ignore small group when fgsea fails due to all NAs for pre-ranks
- scrna.CellsDistribution: Allow to order clusters by envs.cluster_orderby
- scrna.CellsDistribution: Add heatmaps
- tcr.CloneResidency: Make section works in report
- tcr.Immunarch: Support paired chain data for VJ conjuction plots
- tcr.TESSA: Change envs.assay to None to use default assay of Seurat object
- scrna.SeuratClusterStats: Add `avgheatmap` to plot more elegant heatmap for average gene expressions
- scrna.SeuratClusterStats: Fix ident not working for dimplots
- scrna.SeuratClusterStats: Add `cluster_orderby` to order clusters for features
- scrna.SeuratClusterStats: Add `na_group` to keep NA values in `group-by`
- utils.mutate_helpers: Change arguments `id_col` and `compare_col` of `paired` to `id` and `compare`, respectively
- utils.mutate_helpers: Fix that subset can't be an expression for expanded family
- utils.mutate_helpers: Add `top` to select top entities (e.g clones)
- scrna.RadarPlots: Add breakdown and test to break down the cell distribution and run statistic test on the fractions

- docs: move `Immunarch` to the later position in process list
- docs: Use `master` tag in getting-started

- chore: change line length to 88 for flake8
- chore: dismissing warning about wasting columns for `SeuratClusteringOfTCells`
- docs: update CHANGELOG.md with missing changes of last version
- docs: add version of renaming `envs.tcell_indicator` to `envs.tcell_selector`
- docs: remove unused doc files
- docs: add metadata illustration
- deps: bump biopipen to 0.22.8. Highlights:
- deps: bump pipen-board to 0.13.10 (pipen-report to 0.16.2)
- CellsDistribution: Don't add rownames to the output table file
- MarkersFinder (ClusterMarkers/ClusterMarkersOfAllCells): Optimize to use `FindAllMarkers` if `ident.1` is not specified
- SeuratClusterStats: Fix path of expression table file
- CellTypeAnnotation: Allow using `NA` to exclude clusters from output `Seurat` object
- utils.mutate_helpers: Return ids only when subset is true and group is not `NA` for `uniq = TRUE` in `expanded`, `collapsed`, `emerged` and `vanished`

- deps: update biopipen to 0.22.1, highlights:
- add V-J junction circos plots to `Immunarch` process
- add cache option to cache the clustering results if nothing changed except ncores, to `SeuratClustering` process
- add dot plots to `MarkersFinder` (`ClusterMarkersOfAllCells`, `ClusterMarkers`) process
- save exported table with only necessary columns for `CellsDistribution` process
- add `descr` to describe cases cases in report for `CellsDistribution` process
- add `subset` for dimplots in `SeuratClusterStats` process
- use a new palette (`biopipen`) for related processes
- optimize report rendering (using `render_job()` filter from `pipen-report`)
- change metacols to extracols so essential columns get exported for `ImmunarchLoading` process
- add cache option to cache the clustering results if nothing changed except ncores for `SeuratClustering` (`SeuratClusteringOfAllCells`) process
- see more at <https://github.com/pwwang/biopipen/releases/tag/0.22.0> and <https://github.com/pwwang/biopipen/releases/tag/0.22.1>
- deps: update pipen-report to 0.16, highlights:
- scroll anchor into view on the page
- build report page when each process is done, instead of the whole pipeline
- see more at <https://github.com/pwwang/pipen-report/releases/tag/0.16.0>
- change: remove `Immunarch2VDJtools` and `VJUsage` processes (vj usage analysis can be done in `Immunarch` process)
- change: change `tcell_indicator` to `tcell_selector` in `TCellSelection` process
- enhance: provide better error message when none barcode matches from RNA and TCR data for `TCRClustering` process
- docs: add memory usage reduction tips in FAQ
- chore: dismiss warnings of wasted input columns for multiple processes

- chore: update pipeline description to include version in the logs
- fix: add fc-cache command to Dockerfile to solve `Fontconfig error`
- docker: optimize building full image based off the base image

- docker: lock r-matrix version to 1.6_1 for compatibility
- docs: adopt mkdocs-rtd 0.0.10 (add scrollbar to the table of contents)
- deps: bump biopipen to 0.21.1
- use `r-logger` for logging in R scripts
- docs: fix internal references in API docs
- deps: bump pipen-board to 0.13.6
- SampleInfo: refactor data subset logic using `subset` instead of `distinct`
- Immunarch: add `in.metafile` to allow other meta info (i.e. seurat clusters) for future subsetting (22)
- Immunarch: fix empty groups in diversity plot after subsetting
- Immunarch: allow `subset` to subset cells for analyses
- Immunarch: allow `separate_by` also works on other diversity plots
- Immunarch: add `ymin` and `ymax` to align diversity plots by `separate_by`
- Immunarch: add `ncol` to specify columns in the combined plots
- RadarPlots: fix `envs.order` not working
- MarkersFinder: add `overlap` to find overlapping markers between cases (24)
- MarkersFinder: allow `subset` to subset cells for analyses
- MarkersFinder: add dot plots for significant markers
- CellsDistribution: allow multiple columns for `cells_by`
- CellsDistribution: allow `subset` to subset cells for analyses
- utils.mutate_helpers.R: add `include_emerged` for `expanded()` and `include_vanished` for `collapsed()`