Promptfoo

What's Changed

* feat: add image classification example and xml assertions by mldangelo in https://github.com/promptfoo/promptfoo/pull/1153
* feat(redteam): specify the default number of test cases to generate per plugin by mldangelo in https://github.com/promptfoo/promptfoo/pull/1154

* fix(redteam): fix dynamic import paths by mldangelo in https://github.com/promptfoo/promptfoo/pull/1162

**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.71.0...0.71.1

What's Changed

Features
- **Eval picker for web UI** by [typpo](https://github.com/typpo) in [#1143](https://github.com/promptfoo/promptfoo/pull/1143)
- **Update default model providers to Claude 3.5** by [mldangelo](https://github.com/mldangelo) in [#1157](https://github.com/promptfoo/promptfoo/pull/1157)
- **Allow provider customization for dataset generation** by [mldangelo](https://github.com/mldangelo) in [#1158](https://github.com/promptfoo/promptfoo/pull/1158)
- **Predict Redteam injectVars** by [mldangelo](https://github.com/mldangelo) in [#1141](https://github.com/promptfoo/promptfoo/pull/1141)

Fixes
- **Fix JSON prompt escaping in HTTP provider and add LM Studio example** by [mldangelo](https://github.com/mldangelo) in [#1156](https://github.com/promptfoo/promptfoo/pull/1156)
- **Fix poor performing harmful test generation** by [mldangelo](https://github.com/mldangelo) in [#1124](https://github.com/promptfoo/promptfoo/pull/1124)
- **Update overreliance grading prompt** by [mldangelo](https://github.com/mldangelo) in [#1146](https://github.com/promptfoo/promptfoo/pull/1146)
- **Move multiple variables warning to before progress bar** by [typpo](https://github.com/typpo) in [#1160](https://github.com/promptfoo/promptfoo/pull/1160)

Docs
- **Add contributing guide** by [mldangelo](https://github.com/mldangelo) in [#1150](https://github.com/promptfoo/promptfoo/pull/1150)

Refactoring
- **Refactor and optimize injection and iterative methods** by [mldangelo](https://github.com/mldangelo) in [#1138](https://github.com/promptfoo/promptfoo/pull/1138)
- **Update plugin base class to support multiple assertions** by [mldangelo](https://github.com/mldangelo) in [#1139](https://github.com/promptfoo/promptfoo/pull/1139)
- **Structural refactor, abstract plugin and method actions** by [mldangelo](https://github.com/mldangelo) in [#1140](https://github.com/promptfoo/promptfoo/pull/1140)
- **Move CLI commands into individual files** by [mldangelo](https://github.com/mldangelo) in [#1155](https://github.com/promptfoo/promptfoo/pull/1155)
- **Update Jest linter rules** by [mldangelo](https://github.com/mldangelo) in [#1161](https://github.com/promptfoo/promptfoo/pull/1161)

Build and Dependencies
- **Bump openai from 4.52.4 to 4.52.5** by [dependabot](https://github.com/dependabot) in [#1137](https://github.com/promptfoo/promptfoo/pull/1137)
- **Bump aws-sdk/client-bedrock-runtime from 3.613.0 to 3.614.0** by [dependabot](https://github.com/dependabot) in [#1136](https://github.com/promptfoo/promptfoo/pull/1136)
- **Bump openai from 4.52.5 to 4.52.7** by [dependabot](https://github.com/dependabot) in [#1142](https://github.com/promptfoo/promptfoo/pull/1142)
- **Update documentation and MUI dependencies** by [mldangelo](https://github.com/mldangelo) in [#1152](https://github.com/promptfoo/promptfoo/pull/1152)
- **Update Drizzle dependencies and configuration** by [mldangelo](https://github.com/mldangelo) in [#1151](https://github.com/promptfoo/promptfoo/pull/1151)
- **Bump dependencies with patch and minor version updates** by [mldangelo](https://github.com/mldangelo) in [#1159](https://github.com/promptfoo/promptfoo/pull/1159)

**Full Changelog**: [0.70.1...0.71.0](https://github.com/promptfoo/promptfoo/compare/0.70.1...0.71.0)

What's Changed

Fixes
- **provider**: put provider in outer loop to reduce model swap by typpo in [1132](https://github.com/promptfoo/promptfoo/pull/1132)
- **evaluator**: ensure unique prompt handling with labeled and unlabeled providers by mldangelo in [1134](https://github.com/promptfoo/promptfoo/pull/1134)
- **eval**: validate --output file extension before running eval by mldangelo in [1135](https://github.com/promptfoo/promptfoo/pull/1135)

Build
- **deps-dev**: bump aws-sdk/client-bedrock-runtime from 3.609.0 to 3.613.0 by dependabot in [1126](https://github.com/promptfoo/promptfoo/pull/1126)

Tests
- fix pythonCompletion test by mldangelo in [1133](https://github.com/promptfoo/promptfoo/pull/1133)

**Full Changelog**: [0.70.0...0.70.1](https://github.com/promptfoo/promptfoo/compare/0.70.0...0.70.1)

What's Changed
* feat: Add `promptfoo redteam init` command by typpo in https://github.com/promptfoo/promptfoo/pull/1122
* fix(redteam): use final prompt in moderation instead of original by typpo in https://github.com/promptfoo/promptfoo/pull/1117
* chore: refactor eval and generate commands out of main.ts by typpo in https://github.com/promptfoo/promptfoo/pull/1121
* build(deps): bump openai from 4.52.3 to 4.52.4 by dependabot in https://github.com/promptfoo/promptfoo/pull/1118
* refactor(redteam): relocate harmful and pii plugins from legacy directory by mldangelo in https://github.com/promptfoo/promptfoo/pull/1123
* refactor(redteam): Migrate harmful test generators to plugin-based architecture by mldangelo in https://github.com/promptfoo/promptfoo/pull/1116


**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.69.2...0.70.0

What's Changed
* feat: add support for nested grading results by typpo in https://github.com/promptfoo/promptfoo/pull/1101
* fix: issue that caused harmful prompts to not save by typpo in https://github.com/promptfoo/promptfoo/pull/1112
* fix: resolve relative paths for prompts by typpo in https://github.com/promptfoo/promptfoo/pull/1110
* ci: compress images in PRs by typpo in https://github.com/promptfoo/promptfoo/pull/1108
* site: landing page updates by typpo in https://github.com/promptfoo/promptfoo/pull/1096


**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.69.1...0.69.2

New Features
- **feat**: Add Zod schema validation for providers in `promptfooconfig` by mldangelo in [1102](https://github.com/promptfoo/promptfoo/pull/1102)

Fixes
- **fix**: Re-add provider context in prompt functions by mldangelo in [1106](https://github.com/promptfoo/promptfoo/pull/1106)
- **fix**: Add missing `gpt-4-turbo-2024-04-09` by aloisklink in [1100](https://github.com/promptfoo/promptfoo/pull/1100)

Chores
- **chore**: Update minor and patch versions of several packages by mldangelo in [1107](https://github.com/promptfoo/promptfoo/pull/1107)
- **chore**: Format Python code and add check job to GitHub Actions workflow by mldangelo in [1105](https://github.com/promptfoo/promptfoo/pull/1105)
- **chore**: Bump version to 0.69.1 by mldangelo

Documentation
- **docs**: Add example and configuration guide for using `llama.cpp` by mldangelo in [1104](https://github.com/promptfoo/promptfoo/pull/1104)
- **docs**: Add Vitest integration guide by mldangelo in [1103](https://github.com/promptfoo/promptfoo/pull/1103)

Full Changelog: [0.69.0...0.69.1](https://github.com/promptfoo/promptfoo/compare/0.69.0...0.69.1)