Breaking Changes
In this release, we re-aligned the groundedness feedback function with other LLM-based feedback functions. It's now faster and easier to define a groundedness feedback function, and can be done with a standard LLM provider rather than importing groundedness on its own. In addition, the custom groundedness aggregation required is now done by default.
Before:
python
from trulens_eval.feedback.provider.openai import OpenAI
from trulens_eval.feedback import Groundedness
provider = OpenAI() or any other LLM-based provider
grounded = Groundedness(groundedness_provider=provider)
f_groundedness = (
Feedback(grounded.groundedness_measure_with_cot_reasons, name = "Groundedness")
.on(Select.RecordCalls.retrieve.rets.collect())
.on_output()
.aggregate(grounded.grounded_statements_aggregator)
)
After:
python
provider = OpenAI()
f_groundedness = (
Feedback(provider.groundedness_measure_with_cot_reasons, name = "Groundedness")
.on(Select.RecordCalls.retrieve.rets.collect())
.on_output()
)
This change also applies to the NLI-based groundedness feedback function available from the Huggingface provider.
Before:
python
from trulens_eval.feedback.provider.openai import Huggingface
from trulens_eval.feedback import Groundedness
from trulens_eval.feedback.provider import Huggingface
huggingface_provider = Huggingface()
grounded = Groundedness(groundedness_provider=huggingface_provider)
f_groundedness = (
Feedback(grounded.groundedness_measure_with_cot_reasons, name = "Groundedness")
.on(Select.RecordCalls.retrieve.rets.collect())
.on_output()
.aggregate(grounded.grounded_statements_aggregator)
)
After:
python
from trulens_eval.feedback import Feedback
from trulens_eval.feedback.provider.hugs = Huggingface
huggingface_provider = Huggingface()
f_groundedness = (
Feedback(huggingface_provider.groundedness_measure_with_nli, name = "Groundedness")
.on(Select.RecordCalls.retrieve.rets.collect())
.on_output()
)
In addition to the change described above, below you can find the full release description.
What's Changed
* update groundedness prompt by bpmcgough in <https://github.com/truera/trulens/pull/1112>
* Default names for rag triad utility by joshreini1 in <https://github.com/truera/trulens/pull/1122>
* Unify groundedness interface by joshreini1 in <https://github.com/truera/trulens/pull/1135>
Bug Fixes
* Fixed bug with trace view initialization when no feedback functions exist by walnutdust in <https://github.com/truera/trulens/pull/1108>
* Remove references to running moderation endpoint on AzureOpenAI by joshreini1 in <https://github.com/truera/trulens/pull/1116>
* swap rag utility (qs)relevance by piotrm0 in <https://github.com/truera/trulens/pull/1120>
* Fix Link in Readme by timbmg in <https://github.com/truera/trulens/pull/1128>
* chore: remove unused code cell by stokedout in <https://github.com/truera/trulens/pull/1113>
* trurails: update to getattr by joshreini1 in <https://github.com/truera/trulens/pull/1130>
* Fix typo in README.md by eltociear in <https://github.com/truera/trulens/pull/1136>
* fix rag triad and awaitable calls by piotrm0 in <https://github.com/truera/trulens/pull/1110>
* Remove placeholder feedback for asynchronous responses by arn-tru in <https://github.com/truera/trulens/pull/1127>
* Stop iteration streams in openai cost tracking by piotrm0 in <https://github.com/truera/trulens/pull/1138>
Examples
* Show OSS models (and tracking) in LiteLLM application by joshreini1 in <https://github.com/truera/trulens/pull/1109>
New Contributors
* stokedout made their first contribution in <https://github.com/truera/trulens/pull/1113>
* timbmg made their first contribution in <https://github.com/truera/trulens/pull/1128>
* bpmcgough made their first contribution in <https://github.com/truera/trulens/pull/1112>
* eltociear made their first contribution in <https://github.com/truera/trulens/pull/1136>
**Full Changelog**: <https://github.com/truera/trulens/compare/trulens-eval-0.28.0...trulens-eval-0.29.0>