Promptfoo

What's Changed
* feat: Add support for huggingface token classification by typpo in https://github.com/promptfoo/promptfoo/pull/574
* feat: Mistral provider support for URL and API key envar by jvert in https://github.com/promptfoo/promptfoo/pull/570
* feat: run assertions in parallel by typpo in https://github.com/promptfoo/promptfoo/pull/575
* feat: support for azure openai assistants by typpo in https://github.com/promptfoo/promptfoo/pull/577
* feat(vertexai): use gcloud application default credentials by typpo in https://github.com/promptfoo/promptfoo/pull/580
* feat: ability to set tags on standalone assertion llm outputs by typpo in https://github.com/promptfoo/promptfoo/pull/581
* feat: add support for claude3 on bedrock by typpo in https://github.com/promptfoo/promptfoo/pull/582
* fix: load file before running prompt function by typpo in https://github.com/promptfoo/promptfoo/pull/583
* fix(selfhost): handle sqlite db in docker image and build by typpo in https://github.com/promptfoo/promptfoo/pull/568
* [fix: broken ansi colors on cli table](https://github.com/promptfoo/promptfoo/commit/bbb0157b09c0ffb5366d3cbd112438ca3d2d61c9)
* [fix: remove duplicate instruction output](https://github.com/promptfoo/promptfoo/commit/fb095617d36102f5b6256e9718e736378c0a5cea)
* chore: better error messages when expecting json but getting text by typpo in https://github.com/promptfoo/promptfoo/pull/576
* chore(deps): bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /site by dependabot in https://github.com/promptfoo/promptfoo/pull/579

New Contributors
* jvert made their first contribution in https://github.com/promptfoo/promptfoo/pull/570

**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.48.0...0.49.0

What's Changed
* feat: migrate filesystem storage to sqlite db by typpo in https://github.com/promptfoo/promptfoo/pull/558
* **When you first run `eval` or `view` with 0.48.0, your saved evals will be migrated from `.json` files to a sqlite db. Please open an issue if you run into problems.**
* Restoration: By default, the migration process runs on the promptfoo output directory `~/.promptfoo/output`. This directory is backed up at `~/.promptfoo/output-backup-*` and you can restore it and use a previous version by renaming that directory back to `output`
* feat: Add anthropic:messages and replicate:mistral as default providers to web ui by matt-hendrick in https://github.com/promptfoo/promptfoo/pull/562
* feat(csv): add support for `__description` field by typpo in https://github.com/promptfoo/promptfoo/pull/556
* feat: add label field to provider options by typpo in https://github.com/promptfoo/promptfoo/pull/563
* fix(azureopenai): add support for `max_tokens` and `seed` by typpo in https://github.com/promptfoo/promptfoo/pull/561
* docs: adjust configuration for python provider by romaintoub in https://github.com/promptfoo/promptfoo/pull/565
* chore: db migration and cleanup by typpo in https://github.com/promptfoo/promptfoo/pull/564

New Contributors
* matt-hendrick made their first contribution in https://github.com/promptfoo/promptfoo/pull/562

**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.47.0...0.48.0

Breaking
Multiline inline python assertions don't rely on `print` statements anymore. You can simply `return` the value instead. `print` can be used for debugging.

If this breaks your multiline inline python assertion, the fix is simple: replace statements like `print(json.dumps(result))` with `return result`

What's Changed
* feat: improve python inline asserts to not require printing by typpo in https://github.com/promptfoo/promptfoo/pull/542
* feat: add tools and tool_choice config parameters to azure openai provider by heartyguy in https://github.com/promptfoo/promptfoo/pull/550
* feat: Add support for Claude 3 Haiku by streichsbaer in https://github.com/promptfoo/promptfoo/pull/552
* fix(replicate): support non-array outputs by typpo in https://github.com/promptfoo/promptfoo/pull/547
* fix: validate custom js function return values by typpo in https://github.com/promptfoo/promptfoo/pull/548
* fix: dedupe prompts from combined configs by typpo in https://github.com/promptfoo/promptfoo/pull/554

New Contributors
* heartyguy made their first contribution in https://github.com/promptfoo/promptfoo/pull/550

**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.46.0...0.47.0

Breaking

Self-hosted instances no longer require the trailing `/api` in `NEXT_PUBLIC_PROMPTFOO_REMOTE_API_BASE_URL`. Here's an example build command:


docker build --build-arg NEXT_PUBLIC_PROMPTFOO_REMOTE_API_BASE_URL=http://localhost:3000 -t promptfoo-ui .


See further documentation [here](https://promptfoo.dev/docs/usage/sharing#self-hosting)

What's Changed
* feat(self-host): run evals via web ui by typpo in https://github.com/promptfoo/promptfoo/pull/540
* feat(self-host): Persist changes on self-deployed UI without sharing a new link by typpo in https://github.com/promptfoo/promptfoo/pull/538
* feat: add support for calling specific functions for python prompt by typpo in https://github.com/promptfoo/promptfoo/pull/533
* feat(webui): ability to change eval name by typpo in https://github.com/promptfoo/promptfoo/pull/537
* fix(anthropic): wrap text if prompt supplied as json by typpo in https://github.com/promptfoo/promptfoo/pull/536
* fix: openai tools and function checks handle plaintext responses by typpo in https://github.com/promptfoo/promptfoo/pull/541


**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.45.2...0.46.0

What's Changed
* fix: python provider handles relative script paths correctly by typpo in https://github.com/promptfoo/promptfoo/pull/535


**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.45.1...0.45.2

What's Changed
* fix: json and yaml vars files by typpo in https://github.com/promptfoo/promptfoo/pull/531
* fix(python): deserialize objects from json by typpo in https://github.com/promptfoo/promptfoo/pull/532


**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.45.0...0.45.1