Colrev

- Restructure package management, moving dependencies to built-in packages (442)
- Relay prep requirements (529)
- Add GitHub SearchSource (468), Unpaywall SearchSource (469), SpringerLink SearchSource (466), OSF SearchSource (471)
- Refactor other SearchSources
- Replace dacite by pydantic
- Stop Docker containers
- CLI: option to add packages interactively
- Testing and bugfixes in built-in packages (paper-md, files_dir, aisel)
- Update docs (add asciinema demonstration)

- Extend documentation (package development, package summaries, asciinema demo)
- Bugfixes and codebase improvements (e.g., package management and discovery, closing sqlite connections)
- Reduce dependencies (e.g., levenshtein, PyPDF2, pdfminer, daff, psutil)
- Refactor colrev.bibliography_export (add writers)
- Extend tests: cover MacOS and Python 3.12
- Remove unnecessary options (e.g., init --local_pdf_collection)
- Add and test support for GitHub codespaces

- Update CoLRev packages (including interfaces, development docs etc.)
- Refactoring (local-index)
- Implement json-loader
- Make ui_web (dash, blinker) optional to prevent errors in WSL
- Bugfixes

- Refactor and test (dataset, records, provenance, local_index)
- Extract package_manager into a separate internal package
- Use bib-dedupe for matching (instead of simple similarities)
- Update docs

Added

- Add linter `colrev_records_variable_naming_convention`
- Test coverage increased from 71% to 80%

Changed

- Split `records`, `dataset`, created `records` package.
- Extracted `process` as a separate package.
- Implemented loaders as a separate package, created a standard interface. SearchSources now create the specific mapping of IDs, entrytypes and fields.
- Moved field standardization from `load` to SearchSources.
- Extended use of constants
- SearchSourceInterface: renamed `run_search` to `search`, prefer `prep_link_md` over `get_masterdata`
- Renamed and refactored `GeneralOriginFeed` to `SearchAPIFeed`
- Pass record objects instead of dicts (in `local_index` in particular)
- Replaced unnecessary keyword arguments by positional arguments
- Moved `zotero_translation_service` to `bibliography_export` package
- Consolidated code for reference parsing in `tei_parser`
- Upgraded Grobid to 0.8.0

Removed

- Removed dead code
- Dropped `INCONSISTENT_WITH_DOI_METADATA`
- `transitions` dependency

Fixed

- Do not require review_manager for `colrev env -i`
- Fixed `status_stats`, including special cases.
- Repository registration: resolve() and absolute() path

Added

- Separate PDF quality model ([268](https://github.com/CoLRev-Environment/colrev/pull/268))
- `download_from_website` pdf-get package
- Separate loader utilities for nbib, ris, bib
- SearchSources: SemanticScholar ([288](https://github.com/CoLRev-Environment/colrev/pull/288)), Arxiv ([#203](https://github.com/CoLRev-Environment/colrev/pull/203))
- Constants module for Fields, ENTRYTYPES, etc.
- CEP003 for SearchSources
- New default dedupe package based on [bib-dedupe](https://github.com/CoLRev-Environment/bib-dedupe)
- Colrev pandas for Jupyter notebooks
- GitHub actions: pip-install test, make documentation

Changed

- Integrated `colrev.resolve_crossrefs` into `load_utils_bib.py`
- Defect codes can be ignored based on the `IGNORE:` prefix ([269](https://github.com/CoLRev-Environment/colrev/pull/269))
- Documentation for setup (VM, MacOS, WSL)
- Revised interfaces for SearchSources
- Integrated: pdf_dir + video_dir > files_dir
- poetry extras
- Backward search: export of parameters and expected sample sizes
- Replace thefuzz witz rapidfuzz

Removed

- Package based on dedupe-io, including incompatible dependencies
- Crossref resolution package (integrated in bib-loader)