Python-terrier

What's Changed
Significant update that has refactored much of the PyTerrier source code and renamed many classes as we progress towards a PyTerrier 1.0 release.

The most significant changes are:
* `pt.init()` is no longer required 😃. If necessary `pt.java` methods can be used to change Java initialisation
* `pt.BatchRetrieve` is now `pt.terrier.Retriever`, and similar changes for other Terrier indexers and retrievers
* `pt.AnseriniBatchRetrieve` is now in its own separate project, [PyTerrier-Anserini](https://github.com/seanmacavaney/pyterrier-anserini), with various improvements

All changes are backwards compatible in this release - deprecation warnings will guide you how to update your code.

More details below:

Improvements
* Move all Java/JNIUS code into pt.java, move all Terrier code into pt.terrier; remove pt.init() by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/447
* dynamic module loading by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/461
* Incorporate Retrieval Scores into RM3 by mam10eks in https://github.com/terrier-org/pyterrier/pull/453
* pt.apply for making an indexer by cmacdonald in https://github.com/terrier-org/pyterrier/pull/467
* query_toks support for terrier.Retriever by cmacdonald in https://github.com/terrier-org/pyterrier/pull/466
* add save_mode='warn' and save_mode='error' to pt.Experiment (warn as default) by cmacdonald in https://github.com/terrier-org/pyterrier/pull/408


 Refactoring
* Deprecate DFIndexer by cmacdonald in https://github.com/terrier-org/pyterrier/pull/457
* pt.terrier.rewrite revisions - remove Axiomatic, remove terrier-prf by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/472
* shims for deprecated modules by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/476
* text_loader abstraction for pt.text.get_text by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/469
* move Anserini to a separate project by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/473


Documentation
* Add RankVicuna and RankZephyr Plugins by kaustubhdhole in https://github.com/terrier-org/pyterrier/pull/441
* Update tuning.rst by albertoueda in https://github.com/terrier-org/pyterrier/pull/446
* Add PyTerrier_ChatNoir to the plugin section by mam10eks in https://github.com/terrier-org/pyterrier/pull/452
* Remove nptyping dependency to assure numpy 2 compatability by cmacdonald in https://github.com/terrier-org/pyterrier/pull/445

Minor
* change all tests to use new terrier retriever names, but check old names too by cmacdonald in https://github.com/terrier-org/pyterrier/pull/458
* Parallel fixes by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/462
* fix logger error by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/464
* Add comments to requirements.txt by cmacdonald in https://github.com/terrier-org/pyterrier/pull/465
* failing anserini tests due to version 0.36.0, disabling for now by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/468
* remove the writing of a default terrier.properties file by cmacdonald in https://github.com/terrier-org/pyterrier/pull/470
* fix test_maven by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/471
* Python 3.12 in GHA by cmacdonald in https://github.com/terrier-org/pyterrier/pull/459
* Bump most JDK version tested in GHA to 21 by cmacdonald in https://github.com/terrier-org/pyterrier/pull/475
* Update pt.terrier.Retriever __str__ and __repr__ 474

New Contributors
* kaustubhdhole made their first contribution in https://github.com/terrier-org/pyterrier/pull/441
* mam10eks made their first contribution in https://github.com/terrier-org/pyterrier/pull/452

**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.10.1...0.11.0

Minor release with minor improvements and bug fixes.

What's Changed
* Bugfix: Delete baseline pvalue from correction method input by JorgeGabin in https://github.com/terrier-org/pyterrier/pull/440
* Fix: fix msmarco location by cmacdonald in https://github.com/terrier-org/pyterrier/pull/435
* Feature: added corpus_iter for Terrier index by cmacdonald in https://github.com/terrier-org/pyterrier/pull/426
* remove sklearn as required dependency by cmacdonald in https://github.com/terrier-org/pyterrier/pull/410
* Add troubleshoot for installation and certification error by Krissy510 in https://github.com/terrier-org/pyterrier/pull/411
* fix parsing of trecxml topics by lukaszett in https://github.com/terrier-org/pyterrier/pull/414
* paired t-tost by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/420
* read_results optimization by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/421
* pickling QE pipelines to parallelised QE gridsearch by cmacdonald in https://github.com/terrier-org/pyterrier/pull/430
* Require Python 3.8 minimum by cmacdonald in https://github.com/terrier-org/pyterrier/pull/431
* Bump logback from 1.2.0 to 1.2.13 in /terrier-python-helper by dependabot
* improved error message pt.apply.query - from 433 by cmacdonald in https://github.com/terrier-org/pyterrier/pull/434
* Improved testing of FeaturesBatchRetrieve by cmacdonald in https://github.com/terrier-org/pyterrier/pull/437

New Contributors
* Krissy510 made their first contribution in https://github.com/terrier-org/pyterrier/pull/411
* JorgeGabin made their first contribution in https://github.com/terrier-org/pyterrier/pull/440


**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.10.0...0.10.1

What's Changed

New Features
* `Transformer.__call__` now supports both dataframe and iterdicts by cmacdonald in https://github.com/terrier-org/pyterrier/pull/381
* Terrier: Custom stopwords by cmacdonald in https://github.com/terrier-org/pyterrier/pull/372
* Terrier: Access the stemmer of Terrier from PyTerrier by cmacdonald in https://github.com/terrier-org/pyterrier/pull/382
* Terrier: Improved API for loading Terrier indices into memory by cmacdonald in https://github.com/terrier-org/pyterrier/pull/386

Improvements
* added tokenizer as arg for pt.text.sliding by mihirs16 in https://github.com/terrier-org/pyterrier/pull/387
* addresses 367 - include qid in pt.apply Exception by cmacdonald in https://github.com/terrier-org/pyterrier/pull/370
* addresses 377: pt.apply.query() raises exception if the query column does not exist by cmacdonald in https://github.com/terrier-org/pyterrier/pull/380
* let pt.tqdm exist without pt.init() by cmacdonald in https://github.com/terrier-org/pyterrier/pull/399
* deprecate pt.Utils by cmacdonald in https://github.com/terrier-org/pyterrier/pull/384
* removes two warnings by cmacdonald in https://github.com/terrier-org/pyterrier/pull/385
* work on test failure by cmacdonald in https://github.com/terrier-org/pyterrier/pull/401
* Test pyterrier with newer Python versions by cmacdonald in https://github.com/terrier-org/pyterrier/pull/400
* bump supported Anserini version by cmacdonald in https://github.com/terrier-org/pyterrier/pull/406, addresses #404
* Terrier: allow to put term and LexiconEntry into a tuple by cmacdonald in https://github.com/terrier-org/pyterrier/pull/369

Bugs:
* stringify properties and controls, addresses 357 by cmacdonald in https://github.com/terrier-org/pyterrier/pull/358
* fix bug in metadata size warning by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/362

Documentation
* Update pipeline_examples.md by gurcankavakci in https://github.com/terrier-org/pyterrier/pull/359
* Fixed typo by hermlon in https://github.com/terrier-org/pyterrier/pull/364
* Update ltr.rst by Hermi-Mire in https://github.com/terrier-org/pyterrier/pull/371
* Update transformer.rst by albertoueda in https://github.com/terrier-org/pyterrier/pull/383
* clarify docstring for indexing with regards to metadata by lukaszett in https://github.com/terrier-org/pyterrier/pull/394
* Query Rewriting & Expansion by cakiki in https://github.com/terrier-org/pyterrier/pull/402, https://github.com/terrier-org/pyterrier/pull/403

New Contributors
* gurcankavakci made their first contribution in https://github.com/terrier-org/pyterrier/pull/359
* hermlon made their first contribution in https://github.com/terrier-org/pyterrier/pull/364
* Hermi-Mire made their first contribution in https://github.com/terrier-org/pyterrier/pull/371
* lukaszett made their first contribution in https://github.com/terrier-org/pyterrier/pull/394
* cakiki made their first contribution in https://github.com/terrier-org/pyterrier/pull/402
* mihirs16 made their first contribution in https://github.com/terrier-org/pyterrier/pull/387

**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.9.2...0.10.0

Minor release with minor improvements and bug fixes.

What's Changed
* add [sbert](https://www.sbert.net/) example notebook by cmacdonald in https://github.com/terrier-org/pyterrier/pull/344
* Update scikit-learn requirement from the deprecated sklearn, which was causing [build errors at some times](https://github.com/scikit-learn/sklearn-pypi-package).
* adding batching operations to `apply.generic()` and `apply.by_query()` by cmacdonald in https://github.com/terrier-org/pyterrier/pull/351 - thanks to Xun Zhou, University of Michigan via #350
* improve error messages for invalid indexing configurations by cmacdonald in https://github.com/terrier-org/pyterrier/pull/349 -- thanks to maxhenze in #348
* Various empty dataframe fixes by cmacdonald in https://github.com/terrier-org/pyterrier/pull/353 -- thanks to report by [Prithvijit Dasgupta](https://github.com/prithvidasgupta), University of Michigan in #352
* improved error message for add_ranks by cmacdonald in https://github.com/terrier-org/pyterrier/pull/354


**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.9.1...0.9.2

Bugfix release addressing a problem with pretokenised indices on Windows

What's Changed
* Nofifo pretok indexing fixes by cmacdonald in https://github.com/terrier-org/pyterrier/pull/343

**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.9.0...0.9.1

Significant update - refactoring of [public API](https://pyterrier.readthedocs.io/en/latest/transformer.html) (e.g. `pt.transformer.TransformerBase` -> `pt.Transformer`) and support in the Terrier backend for [making indices from pre-tokenised documents](https://pyterrier.readthedocs.io/en/latest/terrier-indexing.html#pretokenised). Python 3.10 is now supported.


What's Changed
* fix error in IRDSDataset when a query field is named "query" by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/303
* Fix type annotation by heinrichreimer in https://github.com/terrier-org/pyterrier/pull/313
* addresses 315 IRDS corpus_iter are not subscriptable by cmacdonald in https://github.com/terrier-org/pyterrier/pull/316
* Missing comma in bm25_qe example by JohnGiorgi in https://github.com/terrier-org/pyterrier/pull/319
* Argument meta should be supplied as dictionary by JohnGiorgi in https://github.com/terrier-org/pyterrier/pull/320
* use Jnius 1.4 by cmacdonald in https://github.com/terrier-org/pyterrier/pull/249
* Python 3.10 support by cmacdonald in https://github.com/terrier-org/pyterrier/pull/322
* Lz4 support for pt.io.autoopen() by cmacdonald in https://github.com/terrier-org/pyterrier/pull/323
* addresses 326 faster version of add_ranks for single queries by cmacdonald in https://github.com/terrier-org/pyterrier/pull/327
* addresses 321 pt.apply.doc_score batching by cmacdonald in https://github.com/terrier-org/pyterrier/pull/325
* IterDictIndexer can index pre-tokenised documents by cmacdonald in https://github.com/terrier-org/pyterrier/pull/328
* Bump logback-core from 1.2.0 to 1.2.9 in /terrier-python-helper by dependabot in https://github.com/terrier-org/pyterrier/pull/336
* documenting BM25F controls and tuning by cmacdonald in https://github.com/terrier-org/pyterrier/pull/296, addresses #294
* 0.9refactor by cmacdonald in https://github.com/terrier-org/pyterrier/pull/314, https://github.com/terrier-org/pyterrier/pull/339, addresses #271
* pt.Experiment() alters the input measures list to drop "mrt" 301
* Expose Termpipelines in Terrier index backend by cmacdonald in https://github.com/terrier-org/pyterrier/pull/338
* pt.rewrite.tokenise() impl by cmacdonald in https://github.com/terrier-org/pyterrier/pull/340 addresses #252 253
* upgraded GitHub actions by cmacdonald in https://github.com/terrier-org/pyterrier/pull/341, https://github.com/terrier-org/pyterrier/pull/342
* fix LTR groupby for xgboost & lightgbm by cmacdonald in https://github.com/terrier-org/pyterrier/pull/284

New Contributors
* heinrichreimer made their first contribution in https://github.com/terrier-org/pyterrier/pull/313
* JohnGiorgi made their first contribution in https://github.com/terrier-org/pyterrier/pull/319

**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.8.1...0.9.0