Python-terrier

What's Changed

A new feature release, demonstrating the new Artifact API, and making a fix in when fields are created in Terrier indexes by IterDictIndexer.

Significant improvements:
* New feature: Artifact API by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/436 - we'll be improving this feature and its documentation in future releases.
* Improvement: terrier.IterDictIndexer doesn't create a field-index when not requested by cmacdonald in https://github.com/terrier-org/pyterrier/pull/525

Minor changes:
* bibliography file in documentation by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/519
* ruff github action by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/523
* Remaining mypy errors by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/524
* RemoteDataset: get one file from a zip by cmacdonald in https://github.com/terrier-org/pyterrier/pull/529
* use decorate_batch properly in Terrier by cmacdonald in https://github.com/terrier-org/pyterrier/pull/528
* saves process name such that jps can shows a name for Pyterrier processes by cmacdonald in https://github.com/terrier-org/pyterrier/pull/527


**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.12.1...0.13.0

Wrapping up various improvements developed in the last few weeks.

New Feature:
* Common prefix pipeline computation in pt.Experiment by cmacdonald, with seanmacavaney and Parry-Parry in https://github.com/terrier-org/pyterrier/pull/514

Improvements:
* PRF compilation by cmacdonald in https://github.com/terrier-org/pyterrier/pull/504
* More compilation improvements by cmacdonald in https://github.com/terrier-org/pyterrier/pull/506
* pt.debug.pdb by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/510
* Guessing batch size for indexing pipelines by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/510

Minor:
* Dataset Decompression Improvements by cmacdonald in https://github.com/terrier-org/pyterrier/pull/512
* support verbose=True on by_query iter by cmacdonald in https://github.com/terrier-org/pyterrier/pull/513
* pass kwargs through to the underlying open functions by cmacdonald in https://github.com/terrier-org/pyterrier/pull/518

Documentation:
* more extension docs by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/511
* Transformer docs by cmacdonald in https://github.com/terrier-org/pyterrier/pull/507

**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.12.0...0.12.1

0.12.0 has an API change for Transformer, making it easier to both implement and call Transformers using iter-dicts rather than DataFrames (DataFrame support is also maintained). This release also completely refreshes the `.compile()` implementation, making it easier to implement pipelines that can be optimised.

Major:
* API change: `Transformer.transform_iter()` returns an iter-dict generator by cmacdonald in https://github.com/terrier-org/pyterrier/pull/481
* `Transformer.compile()` improvements by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/480
* deprecate caching operator `~` by cmacdonald in https://github.com/terrier-org/pyterrier/pull/483
* more extensions with integrated documentation by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/503

Minor:
* addresses 423 IRDS warnings on `pt.list_datasets()` by cmacdonald in https://github.com/terrier-org/pyterrier/pull/485
* bump maven plugin versions by cmacdonald in https://github.com/terrier-org/pyterrier/pull/495
* integrated extension documentation by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/497
* move to pyproject.toml by cmacdonald in https://github.com/terrier-org/pyterrier/pull/490
* Auto-generated Citations from DBLP by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/498
* dont consume arbitrary unused kwargs in TerrierIndexer, update overridden properties by cmacdonald in https://github.com/terrier-org/pyterrier/pull/500
* More types by cmacdonald in https://github.com/terrier-org/pyterrier/pull/484
* Add Apple Silicon GHAs by cmacdonald in https://github.com/terrier-org/pyterrier/pull/478

**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.11.0...0.12.0

What's Changed
Significant update that has refactored much of the PyTerrier source code and renamed many classes as we progress towards a PyTerrier 1.0 release.

The most significant changes are:
* `pt.init()` is no longer required 😃. If necessary `pt.java` methods can be used to change Java initialisation
* `pt.BatchRetrieve` is now `pt.terrier.Retriever`, and similar changes for other Terrier indexers and retrievers
* `pt.AnseriniBatchRetrieve` is now in its own separate project, [PyTerrier-Anserini](https://github.com/seanmacavaney/pyterrier-anserini), with various improvements

All changes are backwards compatible in this release - deprecation warnings will guide you how to update your code.

More details below:

Improvements
* Move all Java/JNIUS code into pt.java, move all Terrier code into pt.terrier; remove pt.init() by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/447
* dynamic module loading by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/461
* Incorporate Retrieval Scores into RM3 by mam10eks in https://github.com/terrier-org/pyterrier/pull/453
* pt.apply for making an indexer by cmacdonald in https://github.com/terrier-org/pyterrier/pull/467
* query_toks support for terrier.Retriever by cmacdonald in https://github.com/terrier-org/pyterrier/pull/466
* add save_mode='warn' and save_mode='error' to pt.Experiment (warn as default) by cmacdonald in https://github.com/terrier-org/pyterrier/pull/408


 Refactoring
* Deprecate DFIndexer by cmacdonald in https://github.com/terrier-org/pyterrier/pull/457
* pt.terrier.rewrite revisions - remove Axiomatic, remove terrier-prf by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/472
* shims for deprecated modules by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/476
* text_loader abstraction for pt.text.get_text by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/469
* move Anserini to a separate project by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/473


Documentation
* Add RankVicuna and RankZephyr Plugins by kaustubhdhole in https://github.com/terrier-org/pyterrier/pull/441
* Update tuning.rst by albertoueda in https://github.com/terrier-org/pyterrier/pull/446
* Add PyTerrier_ChatNoir to the plugin section by mam10eks in https://github.com/terrier-org/pyterrier/pull/452
* Remove nptyping dependency to assure numpy 2 compatability by cmacdonald in https://github.com/terrier-org/pyterrier/pull/445

Minor
* change all tests to use new terrier retriever names, but check old names too by cmacdonald in https://github.com/terrier-org/pyterrier/pull/458
* Parallel fixes by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/462
* fix logger error by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/464
* Add comments to requirements.txt by cmacdonald in https://github.com/terrier-org/pyterrier/pull/465
* failing anserini tests due to version 0.36.0, disabling for now by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/468
* remove the writing of a default terrier.properties file by cmacdonald in https://github.com/terrier-org/pyterrier/pull/470
* fix test_maven by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/471
* Python 3.12 in GHA by cmacdonald in https://github.com/terrier-org/pyterrier/pull/459
* Bump most JDK version tested in GHA to 21 by cmacdonald in https://github.com/terrier-org/pyterrier/pull/475
* Update pt.terrier.Retriever __str__ and __repr__ 474

New Contributors
* kaustubhdhole made their first contribution in https://github.com/terrier-org/pyterrier/pull/441
* mam10eks made their first contribution in https://github.com/terrier-org/pyterrier/pull/452

**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.10.1...0.11.0

Minor release with minor improvements and bug fixes.

What's Changed
* Bugfix: Delete baseline pvalue from correction method input by JorgeGabin in https://github.com/terrier-org/pyterrier/pull/440
* Fix: fix msmarco location by cmacdonald in https://github.com/terrier-org/pyterrier/pull/435
* Feature: added corpus_iter for Terrier index by cmacdonald in https://github.com/terrier-org/pyterrier/pull/426
* remove sklearn as required dependency by cmacdonald in https://github.com/terrier-org/pyterrier/pull/410
* Add troubleshoot for installation and certification error by Krissy510 in https://github.com/terrier-org/pyterrier/pull/411
* fix parsing of trecxml topics by lukaszett in https://github.com/terrier-org/pyterrier/pull/414
* paired t-tost by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/420
* read_results optimization by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/421
* pickling QE pipelines to parallelised QE gridsearch by cmacdonald in https://github.com/terrier-org/pyterrier/pull/430
* Require Python 3.8 minimum by cmacdonald in https://github.com/terrier-org/pyterrier/pull/431
* Bump logback from 1.2.0 to 1.2.13 in /terrier-python-helper by dependabot
* improved error message pt.apply.query - from 433 by cmacdonald in https://github.com/terrier-org/pyterrier/pull/434
* Improved testing of FeaturesBatchRetrieve by cmacdonald in https://github.com/terrier-org/pyterrier/pull/437

New Contributors
* Krissy510 made their first contribution in https://github.com/terrier-org/pyterrier/pull/411
* JorgeGabin made their first contribution in https://github.com/terrier-org/pyterrier/pull/440


**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.10.0...0.10.1

What's Changed

New Features
* `Transformer.__call__` now supports both dataframe and iterdicts by cmacdonald in https://github.com/terrier-org/pyterrier/pull/381
* Terrier: Custom stopwords by cmacdonald in https://github.com/terrier-org/pyterrier/pull/372
* Terrier: Access the stemmer of Terrier from PyTerrier by cmacdonald in https://github.com/terrier-org/pyterrier/pull/382
* Terrier: Improved API for loading Terrier indices into memory by cmacdonald in https://github.com/terrier-org/pyterrier/pull/386

Improvements
* added tokenizer as arg for pt.text.sliding by mihirs16 in https://github.com/terrier-org/pyterrier/pull/387
* addresses 367 - include qid in pt.apply Exception by cmacdonald in https://github.com/terrier-org/pyterrier/pull/370
* addresses 377: pt.apply.query() raises exception if the query column does not exist by cmacdonald in https://github.com/terrier-org/pyterrier/pull/380
* let pt.tqdm exist without pt.init() by cmacdonald in https://github.com/terrier-org/pyterrier/pull/399
* deprecate pt.Utils by cmacdonald in https://github.com/terrier-org/pyterrier/pull/384
* removes two warnings by cmacdonald in https://github.com/terrier-org/pyterrier/pull/385
* work on test failure by cmacdonald in https://github.com/terrier-org/pyterrier/pull/401
* Test pyterrier with newer Python versions by cmacdonald in https://github.com/terrier-org/pyterrier/pull/400
* bump supported Anserini version by cmacdonald in https://github.com/terrier-org/pyterrier/pull/406, addresses #404
* Terrier: allow to put term and LexiconEntry into a tuple by cmacdonald in https://github.com/terrier-org/pyterrier/pull/369

Bugs:
* stringify properties and controls, addresses 357 by cmacdonald in https://github.com/terrier-org/pyterrier/pull/358
* fix bug in metadata size warning by seanmacavaney in https://github.com/terrier-org/pyterrier/pull/362

Documentation
* Update pipeline_examples.md by gurcankavakci in https://github.com/terrier-org/pyterrier/pull/359
* Fixed typo by hermlon in https://github.com/terrier-org/pyterrier/pull/364
* Update ltr.rst by Hermi-Mire in https://github.com/terrier-org/pyterrier/pull/371
* Update transformer.rst by albertoueda in https://github.com/terrier-org/pyterrier/pull/383
* clarify docstring for indexing with regards to metadata by lukaszett in https://github.com/terrier-org/pyterrier/pull/394
* Query Rewriting & Expansion by cakiki in https://github.com/terrier-org/pyterrier/pull/402, https://github.com/terrier-org/pyterrier/pull/403

New Contributors
* gurcankavakci made their first contribution in https://github.com/terrier-org/pyterrier/pull/359
* hermlon made their first contribution in https://github.com/terrier-org/pyterrier/pull/364
* Hermi-Mire made their first contribution in https://github.com/terrier-org/pyterrier/pull/371
* lukaszett made their first contribution in https://github.com/terrier-org/pyterrier/pull/394
* cakiki made their first contribution in https://github.com/terrier-org/pyterrier/pull/402
* mihirs16 made their first contribution in https://github.com/terrier-org/pyterrier/pull/387

**Full Changelog**: https://github.com/terrier-org/pyterrier/compare/0.9.2...0.10.0