Flexeval

What's Changed
* Set `effective_order=True` for computing sentence-level bleu by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/122
* Implement `RepetitionCount` metric by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/123


**Full Changelog**: https://github.com/sbintuitions/flexeval/compare/v0.9.1...v0.9.2

What's Changed
* Update docstring of `TemplateChatDataset` by butsugiri in https://github.com/sbintuitions/flexeval/pull/118
* Use `smart_open` for jsonl loading by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/119
* Implement `FixedFewShotGenerator` by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/120
* Simplify `prompt_template` in configs by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/121


**Full Changelog**: https://github.com/sbintuitions/flexeval/compare/v0.9.0...v0.9.1

What's Changed
* Introduce `category_key` for `flexeval_reward` by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/107
* Move `vllm` import inside function to avoid `ImportError` by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/106
* Set `self.model.config.pad_token_id` from tokenizer if not set in `SequenceClassificationRewardModel` by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/108
* Add new StringProcessors by teruaki-o in https://github.com/sbintuitions/flexeval/pull/110
* Replace Deprecated parameter `max_tokens` with `max_completion_tokens` for OpenAI API by junya-takayama in https://github.com/sbintuitions/flexeval/pull/112
* Cache jinja2 template by butsugiri in https://github.com/sbintuitions/flexeval/pull/113
* Read prefix in perplexity evaluation by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/114
* Quit sharing `StringProcessor` between lm_output and reference by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/115
* Add `Correlation` metric class by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/116
* Remove `NoopNormalizer` by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/117


**Full Changelog**: https://github.com/sbintuitions/flexeval/compare/v0.8.1...v0.9.0

What's Changed
* Upgrade vLLM from v0.5 series to v0.6 by butsugiri in https://github.com/sbintuitions/flexeval/pull/103
* Make some modules accessible from the top level by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/104
* Debug `SequenceClassificationRewardModel` by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/105


**Full Changelog**: https://github.com/sbintuitions/flexeval/compare/v0.8.0...v0.8.1

What's Changed
* Update reward dataset with `TemplateRewardBenchDataset` by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/96
* Add default values for instance classes by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/97
* Set `require_incremental_response` to `False` by default by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/98
* Fix the default value of template based reward datasets by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/100
* Update `RewardBenchInstance` to handle a list of messages by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/99
* Implement `SequenceClassificationRewardModel` by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/101
* Implement RewardModel based on log probs by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/102
* End of life of Python 3.8 by ryokan0123 in https://github.com/sbintuitions/flexeval/pull/80


**Full Changelog**: https://github.com/sbintuitions/flexeval/compare/v0.7.8...v0.8.0

What's Changed
* EvalSetupの更新 by teruaki-o in https://github.com/sbintuitions/flexeval/pull/92
* hellaswagの few-shot examples を修正 by Ktakuya332C in https://github.com/sbintuitions/flexeval/pull/93
* PIQAを追加 by Ktakuya332C in https://github.com/sbintuitions/flexeval/pull/94
* ARC Challenge を追加 by Ktakuya332C in https://github.com/sbintuitions/flexeval/pull/95

New Contributors
* teruaki-o made their first contribution in https://github.com/sbintuitions/flexeval/pull/92

**Full Changelog**: https://github.com/sbintuitions/flexeval/compare/v0.7.7...v0.7.8