Cappr

Breaking changes

None

New features

* `cappr.huggingface` now supports Llama and Llama 2 (chat, raw, GPTQd)

Bug fixes

None

Breaking changes

* `cappr.huggingface` functions only allow `model_and_tokenizer` input, not the string `model` input.

New features

None

Bug fixes

* Correct type hint for `predict_proba_examples` functions to reflect that the 2nd dimension is always an array.

Breaking changes

None

New features

None

Bug fixes

* `cappr.huggingface.classify.predict_proba` and `cappr.huggingface.classify.predict` now accept a `prior` kwarg, as was intended (I just forgot to add it in).

Breaking changes

None

New features

None

Bug fixes

* For OpenAI models, the completion token probabilities should actually be sliced based on the tokenization of `end_of_prompt + completion`, not just `completion`. Based on a few experiments, this change doesn't impact statistical performance. But it should be fixed ofc.

Breaking changes

None

New features

* Allow for pre-computed completion log-probs for the experimental discount feature. Use the newly surfaced function, `cappr.openai.token_logprobs`, to compute them once and re-use them.

Bug fixes

None

Breaking changes

* Deprecate `cappr.utils.classify.agg_log_probs_from_constant_completions`. I doubt anyone was using this. If you were, then use `cappr.utils.classify.agg_log_probs` from now on (it does the exact same thing).

New features

* Highly experimental feature which discounts completions by their marginal probability. See my [updated answer here](https://stats.stackexchange.com/a/606323/337906). The plan is to evaluate this method more thoroughly and discuss it in the user guide. For now, feel free to mess with it.

Bug fixes

* Fix type hint for tokenizer: `AutoTokenizer` to `PreTrainedTokenizer`.