New features:
The `Query` object now supports two new methods for easier evaluation of your retrieval pipeline:
- `query.get_runs()` returns a dictionary of TREC-style runs for all the agents that retrieved documents for that query. (the mapping is agent_id -> query_id->document_id->retrieval_score).
- `query.get_qrels()` returns a TREC-style qrels dictionary with the judgement scores assigned by an Evaluator. The mapping is query_id->document_id->relevance).
You can explore how these two methods work in the new [example notebook here](https://github.com/zetaalphavector/RAGElo/blob/master/docs/examples/notebooks/custom_retrieval_prompt.ipynb) that uses the [`ir-measures`](https://ir-measur.es/en/latest/) package.
Another addition (by RodrJ106) is the addition of a new `LLMProvider` for Ollama! Now you can also run RAGElo on locally, without the need to call an external provider. Thanks!
A potential breaking change is that the `retrieved_docs` and the `answers` attributes of the `Query` object are now dictionaries instead of lists (mapping the document id or the agent name, respectively, to the actual object). This was done to better support future changes where RAGElo relies less on CSV files everywhere, but instead saves and serializes its internal state as a dictionary, until the user actually asks for an output as a CSV.
What's Changed
* Add missing f-string to warning. by RodrJ106 in https://github.com/zetaalphavector/RAGElo/pull/38
* Add ollama as new llm provider by RodrJ106 in https://github.com/zetaalphavector/RAGElo/pull/39
* Remove extra domain sentence by din0s in https://github.com/zetaalphavector/RAGElo/pull/40
* Add get_qrels and get_runs for Queries by ArthurCamara in https://github.com/zetaalphavector/RAGElo/pull/41
New Contributors
* RodrJ106 made their first contribution in https://github.com/zetaalphavector/RAGElo/pull/38
* din0s made their first contribution in https://github.com/zetaalphavector/RAGElo/pull/40
**Full Changelog**: https://github.com/zetaalphavector/RAGElo/compare/0.1.7...0.1.8