Indra

Package structure and dependencies
- The high-throughput reading tools in `indra.tools.reading` have been
moved into an independent repository at
`https://github.com/indralab/indra_reading`.
- The `obonet` package is now a required dependency.

Input processors
- New VirHostNet API and processor in `indra.sources.virhostnet`.
- New Hypothes.is API and processor allowing processing
annotations into INDRA Statements in `indra.sources.hypothesis`.
- Integrated new version of Sparser reader with improved and adapted
processor in `indra.sources.sparser`.
- Refactored Eidos API into separate reader, client and server with
significantly extended usage instructions in `indra.sources.eidos`.
- New endpoints in Eidos API implementing biology-oriented processing, as
well as regrounding with respect to a user-supplied ontology in
`indra.sources.eidos`.
- Overhauled Reach API supporting the latest Reach version and prioritizing
using Reach as a (local) web service over JAR-based reading
in `indra.sources.reach`, also with significantly extended usage
instructions.
- Support for extracting Conversion statements from Reach output in
`indra.sources.reach`.
- ISI reader processor in `indra.sources.isi` no longer requires a Python 2
runtime and the separate nxml2text tool.
- Updated RLIMS-P API in `indra.sources.rlimsp` to work with latest web service at
`https://research.bioinformatics.udel.edu/itextmine/api/data/rlims/`.

Resources and clients
- Overhauled GO resources and client to be based on OBO in
`indra.databases.go_client`.
- MeSH tree numbers, `isa` relationships and determination of
subtree implemented in `indra.databases.mesh_client`.
- Extended mappings added between MeSH and GO, ChEBI, EFO, DOID, HP.
- All resource files updated to latest version in `indra.resources`.

Core assembly modules
- New Pipeline module in `indra.pipeline` allowing building
and running custom assembly pipelines programmatically or from a JSON file.
- UniProt protein features and EFO, DOID, HP ontologies
integrated with name/grounding standardization.
- Overhauled integration of `Gilda` (https://github.com/indralab/gilda) with
`indra.preassembler.grounding_mapper` allowing selective re-grounding
of statements as well as disambiguation.
- New one-statement-per-line JSON output mode in `indra.statements.io` to
support streaming large INDRA Statement JSONs.

Literature
- New bioRxiv/medRxiv client to monitor collections and download PDF/XML/text
content in `indra.literature.biorxiv_client`.
- Significantly improved text extraction from PMC NXML in
`indra.literature.pmc_client`.

Output assemblers
- Refactoried English assembler to allow keeping track of
entity coordinates within each assembled sentence in
`indra.assemblers.english`.
- Improved construction of entity links in HTML assembler
in `indra.assemblers.html`.
- More annotations propagated into PyBEL graph in
`indra.assemblers.pybel`.
- Made the upload and update of CX networks more robust in
`indra.assemblers.cx`.


Explanation
- Improvements to finding all paths of a given length in
`indra.explanation`.

Tools
- Refactored LiveCuration service into a separate module in
`indra.tools.live_curation`.
- Updated outdated assembly approach in GeneNetwork tool in
`indra.tools.gene_network`.

Other
- Updated and extended README, usage examples and tutorials.

Input processors
- New PhosphoELM API and processor in `indra.sources.phosphoelm`.
- PyBEL API and processor updated for PyBEL 0.14 compatibility in `indra.sources.bel`.
- Improvements in regrounding approach with Eidos in `indra.sources.eidos`.
- Bug fixes to offline reading via the REACH API in `indra.sources.reach`.
- Refactor ISI reader API and processor for containerization in `indra.sources.isi`.

Core assembly modules
- Full integration of Gilda disambiguation with grounding mapper both as
a package and a web service in `indra.preassembler.grounding_mapper`.
- Refactor grounding mapper module to separate out standardization functions
in `indra.preassembler.grounding_mapper.standardize`.

Output assemblers
- Many improvements to the HTML assembler including refactoring,
improved grouping and heading generation, and support Influences in
`indra.assemblers.html`.
- Improvements to grounding and introduction of part_of relations in
PyBEL assembler in `indra.assemblers.pybel`.


Resources
- Integrate a set of known MeSH to GO mappings.

Explanation
- Improvements to reporting paths in the explanation module in
`indra.explanation`.

Tools
- Many new features in the Live curation, including actively loading
and saving corpora on S3 in `indra.tools.live_curation`.
- Add new high-throughput reader management for ISI and TRIPS readers
in `indra.tools.reading`
- Significant refactoring of `indra.tools.reading` to separate reader
classes and improve containerization.

Package structure
- Remove the top-level data folder from version control (not part of package)
- Make INDRA compatible with networkx 2.4 (which removed several deprecated
features).

Input processors
- Refactoring of the BEL input API in `indra.sources.bel`.
Access to Selventa corpora is now via https://github.com/cthoyt/selventa-knowledge.
- Multitude of improvements to the CWMS processor in `indra.sources.cwms`,
including extracting migration events.
- Refactoring of INDRA DB REST API in `indra.sources.indra_db_rest` to use
a class-based processor structure for all endpoints.
- Add Gilda grounding option to the ISI reader in `indra.sources.isi`.

Core assembly modules
- Normalization across equivalent and opposite concepts in `indra.preassembler`
based on an ontology.
- Improved custom preassembly functions for location and time matching in
`indra.preassembler.custom_preassembly`.
- Preliminary integration of a Gilda client with Grounding mapper in
`indra.preassembler.grounding_mapper.gilda`.

Output assemblers
- Improved headings and grounding display in the HTML assembler in
`indra.assemblers.html`.
- Generalize INDRA Net assembler to causal influences and associations in
`indra.assemblers.indranet`.
- Make PySB assembler compatible with PySB 1.9 in `indra.assemblers.pysb`.

Resources
- Integrate new WorldModelers ontologies with INDRA in
`indra.preassembler.make_wm_ontologies`.
- Updates to FamPlex and other grounding resources.

Representation
- New MovementContext class to represent movement time/places for Events
- New QuantitativeState class to represent specific quantities associated with Events

Input sources
- Support for extracting Migration statements from CWMS in `indra.sources.cwms`
- Adaptation to changes in Eidos API and output format in `indra.sources.eidos`
- Adaptation to changes in Hume API and output format in `indra.sources.hume`
- Custom processor for migration tables in `indra.sources.eidos.migration_table_processor`
- Support for representing source counts in INDRA DB REST in `indra.sources.indra_db_res`

Internal assembly
- Significantly improved and generalized DB refs name standardization, and preassembly matching/refinement for all entities including chemicals and biological processes
- New custom matches/refinement function feature in Preassembler in `indra.preassembler` and BeliefEngine in `indra.belief`
- Support for new World Modelers ontology
- Grounding mapper refactored with new misgrounding map functionality in `indra.preassembler.grounding_mapper`

Model assemblers
- New IndraNet assembler to assemble custom networks for path finding in `indra.assemblers.indranet`
- New NiceCxAssembler for streamlined CX network assembly in `indra.assemblers.cx`
- Significantly improved HTML assembler with source count badges and support for templates `indra.assemblers.html`
- Refactored index card assembler with extended metadata support in `indra.assemblers.index_card`

Database/literature clients
- New PubChem client in `indra.databases.pubchem_client`
- New miRBase client in `indra.databases.mirbase_client`
- New Experimental Factor Ontology (EFO) client in `indra.databases.efo_client`
- New Phenotype Ontology (HP) client in `indra.databases.hp_client`
- New Disease Ontology (DOID) client in `indra.databases.doid_client`
- New mapping functionalities in HGNC, MeSH and ChEBI clients
- ScienceDirect search feature added in Elsevier client in `indra.literature.elsevier_client`
- Additional article info extraction and support for getting MeSH annotations in PubMed client in `indra.literature.pubmed_client`.


Model analysis
- Refactored ModelChecker with four new subclasses for analysis of specific model/network types in `indra.explanation`

Tools
- S3 corpus upload feature in LiveCuration tool in `indra.tools.live_curation`

Representation
- INDRA Events introduced as arguments of Influences or as standalone statements
- Influence statements are no longer derived from IncreaseAmount
- New object model for representing Delta/QualitativeDelta in Events

Input processors
- Support for standalone Events and Migration events extracted from CWMS in `indra.sources.cwms`
- Support for extracting standalone Events from Eidos in `indra.sources.eidos`
- Support for extracting standalone Events from Hume in `indra.sources.hume`
- Support for extracting standalone Events from Sofia in `indra.sources.sofia`

Database/Literature clients
- Support for determining type of gene in `indra.databases.hgnc_client`
- Support for mapping outdated gene names to current ones in `indra.databases.hgnc_client`
- Improved extraction of text from Elsevier XMLs in `indra.literature.elsevier_client`
- Additional annotations and metadata extraction support in `indra.literature.pubmed_client`

Dependencies
- Use Open JDK instead of Oracle JDK with pyjnius
- `deft` renamed to `adeft`

Assemblers
- All relevant assemblers adapted to new Influence/Event/Concept representation
- Support for generating Kappa influence and contact maps in PySB assembler in `indra.assemblers.pysb`
- Support for Influence statements in CX assembler in `indra.assemblers.cx`

Model analysis
- Additional logic in ModelChecker for Kappa influence map pruning in `indra.explanation.model_checker`

Reading tools
- Refactoring of AWS-based reading tools

Input processors
- New input API and processor for HPRD in `indra.sources.hprd` to access
http://www.hprd.org/
- New input API and processor for TRRUST in `indra.sources.trrust` to access
https://www.grnpedia.org/trrust/
- Major overhaul and improvement of MedScan API and processor in
`indra.sources.medscan`
- Extensions and improvements in RLIMS-P processor including capability to
process dumped result files in `indra.sources.rlimsp`
- Fixes to adapt to latest SIGNOR content in `indra.sources.signor`
- New API endpoint to process Causal Biological Networks from http://causalbionet.com in `indra.sources.bel`

Database clients
- New resources and functions to map ChEBI names to and from IDs in `indra.databases.chebi_client`
- New resources and functions to map MeSH names to and from IDs in `indra.databases.mesh_client`

Model analysis
- Bug fixes and improvements in `indra.explanation.model_checker`

General
- Optimized pickle protocols used to exploit Python 3+ compatibility