Promptfoo

What's Changed
- [fix: handle null result timestamp when writing to db.](https://github.com/promptfoo/promptfoo/commit/40e1ebfbfd512fea56761b4cbdfff0cd25d61ae1) https://github.com/promptfoo/promptfoo/issues/800

What's Changed
* fix: python prompts break when using whole file by typpo in https://github.com/promptfoo/promptfoo/pull/784
* feat(webui): add --filter-description option to `promptfoo view` by typpo in https://github.com/promptfoo/promptfoo/pull/780
* Langfuse need to compile variables by albertpurnama in https://github.com/promptfoo/promptfoo/pull/779
* chore(webui): display prompt and completion tokens by typpo in https://github.com/promptfoo/promptfoo/pull/794
* chore: include full error response in openai errors by typpo in https://github.com/promptfoo/promptfoo/pull/791
* chore: add logprobs to assertion context by typpo in https://github.com/promptfoo/promptfoo/pull/790
* feat: support var interpolation in function calls by typpo in https://github.com/promptfoo/promptfoo/pull/792
* chore: add timestamp to EvaluateSummary by typpo in https://github.com/promptfoo/promptfoo/pull/785
* fix: render markdown in variables too by typpo in https://github.com/promptfoo/promptfoo/pull/796
* feat(bedrock): add support for embeddings models by typpo in https://github.com/promptfoo/promptfoo/pull/797
* fix(vertex): remove leftover dependency on apiKey by typpo in https://github.com/promptfoo/promptfoo/pull/798


**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.58.1...0.59.0

What's Changed
* fix: improve GradingResult validation by typpo in https://github.com/promptfoo/promptfoo/pull/772
* fix(langfuse): Check runtime type of `getPrompt`, stringify the result by albertpurnama in https://github.com/promptfoo/promptfoo/pull/774
* [fix: update python ProviderResponse error message and docs.](https://github.com/promptfoo/promptfoo/commit/258013080809bc782afe3de51c9309230cb5cdb2) https://github.com/promptfoo/promptfoo/issues/769
* [chore(openai): add gpt-4o models (](https://github.com/promptfoo/promptfoo/commit/ff4655d31d3588972522bb162733cb61e460f36f)https://github.com/promptfoo/promptfoo/pull/776[)](https://github.com/promptfoo/promptfoo/commit/ff4655d31d3588972522bb162733cb61e460f36f)

Breaking
`rouge`-type assertions no longer support multiple reference strings. This is due to an update to the underlying rouge package. To check multiple strings, break them into separate assertions.

What's Changed
* feat: assert-set by mikkoh in https://github.com/promptfoo/promptfoo/pull/765
* feat: add comma-delimited string support for array-type assertion values by typpo in https://github.com/promptfoo/promptfoo/pull/755
* fix: Resolve JS assertion paths relative to configuration file by Arkham in https://github.com/promptfoo/promptfoo/pull/756
* fix: not-equals assertion by EKranjec in https://github.com/promptfoo/promptfoo/pull/763
* fix: upgrade rouge package and limit to strings by typpo in https://github.com/promptfoo/promptfoo/pull/764

New Contributors
* Arkham made their first contribution in https://github.com/promptfoo/promptfoo/pull/756
* EKranjec made their first contribution in https://github.com/promptfoo/promptfoo/pull/763

**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.57.1...0.58.0

What's Changed
* fix: do not serialize js objects to non-js providers by typpo in https://github.com/promptfoo/promptfoo/pull/754

Breaking

The `eval --first-n` option has been renamed to `eval --filter-first-n` to match other new filtering options.

What's Changed
* feat: ability to override provider per test case by typpo in https://github.com/promptfoo/promptfoo/pull/725
* feat: eval tests matching pattern by mikkoh in https://github.com/promptfoo/promptfoo/pull/735
* feat: add `-n` limit arg for `promptfoo list` by typpo in https://github.com/promptfoo/promptfoo/pull/749
* feat: `promptfoo import` and `promptfoo export` commands by typpo in https://github.com/promptfoo/promptfoo/pull/750
* feat: add support for `--var name=value` cli option by typpo in https://github.com/promptfoo/promptfoo/pull/745
* feat: promptfoo eval --filter-failing outputFile.json by mikkoh in https://github.com/promptfoo/promptfoo/pull/742
* fix: eval --first-n arg by typpo in https://github.com/promptfoo/promptfoo/pull/734
* chore: Update openai package to 3.48.5 by matteodepalo in https://github.com/promptfoo/promptfoo/pull/739
* chore: include logger and cache utils in javascript provider context by typpo in https://github.com/promptfoo/promptfoo/pull/748
* chore: add `PROMPTFOO_FAILED_TEST_EXIT_CODE` envar by typpo in https://github.com/promptfoo/promptfoo/pull/751
* docs: Document `python:` prefix when loading assertions in CSV by efung in https://github.com/promptfoo/promptfoo/pull/731
* docs: update README.md by eltociear in https://github.com/promptfoo/promptfoo/pull/733
* docs: Fixes to Python docs by jamesbraza in https://github.com/promptfoo/promptfoo/pull/728
* docs: Update to include --filter-* cli args by mikkoh in https://github.com/promptfoo/promptfoo/pull/747

New Contributors
* efung made their first contribution in https://github.com/promptfoo/promptfoo/pull/731
* eltociear made their first contribution in https://github.com/promptfoo/promptfoo/pull/733
* mikkoh made their first contribution in https://github.com/promptfoo/promptfoo/pull/735
* matteodepalo made their first contribution in https://github.com/promptfoo/promptfoo/pull/739

**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.56.0...0.57.0