Promptfoo

What's Changed

Features
* feat(redteam): add base path to CLI state for redteam generate by mldangelo in [1464](https://github.com/promptfoo/promptfoo/pull/1464)
* feat(eval): add global pass rate threshold by mldangelo in [1443](https://github.com/promptfoo/promptfoo/pull/1443)

Chores
* chore: check config.redteam instead of config.metadata.redteam by mldangelo in [1463](https://github.com/promptfoo/promptfoo/pull/1463)
* chore(deps): bump openai from 4.55.9 to 4.56.0 by dependabot in [1466](https://github.com/promptfoo/promptfoo/pull/1466)
* chore: Add vscode settings for prettier formatting by sklein12 in [1469](https://github.com/promptfoo/promptfoo/pull/1469)
* chore(deps): replace rouge with js-rouge by QuarkNerd in [1420](https://github.com/promptfoo/promptfoo/pull/1420)

Build
* build: add defaults for supabase environment variables by sklein12 in [1468](https://github.com/promptfoo/promptfoo/pull/1468)

Fixes
* fix: smarter caching in exec provider by typpo in [1467](https://github.com/promptfoo/promptfoo/pull/1467)

Documentation
* docs: display consistent instructions for npx vs npm vs brew by typpo in [1465](https://github.com/promptfoo/promptfoo/pull/1465)

New Contributors
* sklein12 made their first contribution in [1469](https://github.com/promptfoo/promptfoo/pull/1469)
* QuarkNerd made their first contribution in [1420](https://github.com/promptfoo/promptfoo/pull/1420)

**Full Changelog**: [0.78.2...0.78.3](https://github.com/promptfoo/promptfoo/compare/0.78.2...0.78.3)

What's Changed
* feat: multi-turn jailbreak by typpo in https://github.com/promptfoo/promptfoo/pull/1459
* feat: plugin aliases for owasp, nist by typpo in https://github.com/promptfoo/promptfoo/pull/1410
* refactor(redteam): aliase `generate redteam` to `redteam generate`. by mldangelo in https://github.com/promptfoo/promptfoo/pull/1461
* chore: strongly typed envars by typpo in https://github.com/promptfoo/promptfoo/pull/1452
* chore: further simplify redteam onboarding by typpo in https://github.com/promptfoo/promptfoo/pull/1462
* docs: strategies by typpo in https://github.com/promptfoo/promptfoo/pull/1460


**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.78.1...0.78.2

What's Changed

Features
- **feat:** Helicone integration by maamalama in [1434](https://github.com/promptfoo/promptfoo/pull/1434)

Fixes
- **fix:** is-sql assertion `databaseType` not `database` by typpo in [1451](https://github.com/promptfoo/promptfoo/pull/1451)
- **fix(redteam):** Improve iterative tree-based red team attack provider by mldangelo in [1458](https://github.com/promptfoo/promptfoo/pull/1458)

Chores
- **chore:** Use temporary file for Python interprocess communication by enkoder in [1447](https://github.com/promptfoo/promptfoo/pull/1447)
- **chore:** Redteam onboarding updates by typpo in [1453](https://github.com/promptfoo/promptfoo/pull/1453)
- **chore(deps):** Update various dependencies by mldangelo in [1442](https://github.com/promptfoo/promptfoo/pull/1442)
- **chore(deps):** Bump `aws-sdk/client-bedrock-runtime` from 3.629.0 to 3.631.0 by dependabot in [1448](https://github.com/promptfoo/promptfoo/pull/1448)
- **chore(deps):** Bump `aws-sdk/client-bedrock-runtime` from 3.631.0 to 3.632.0 by dependabot in [1455](https://github.com/promptfoo/promptfoo/pull/1455)
- **chore(deps):** Bump `anthropic-ai/sdk` from 0.25.2 to 0.26.0 by dependabot in [1449](https://github.com/promptfoo/promptfoo/pull/1449)
- **chore(deps):** Bump `anthropic-ai/sdk` from 0.26.0 to 0.26.1 by dependabot in [1456](https://github.com/promptfoo/promptfoo/pull/1456)
- **chore(deps):** Bump `openai` from 4.55.7 to 4.55.9 by dependabot in [1457](https://github.com/promptfoo/promptfoo/pull/1457)

Others
- **site:** Add blog post by typpo in [1444](https://github.com/promptfoo/promptfoo/pull/1444)

New Contributors
- **enkoder** made their first contribution in [1447](https://github.com/promptfoo/promptfoo/pull/1447)
- **maamalama** made their first contribution in [1434](https://github.com/promptfoo/promptfoo/pull/1434)

**Full Changelog:** https://github.com/promptfoo/promptfoo/compare/0.78.0...0.78.1

What's Changed

Features
- **Web UI**: Added ability to choose prompt/provider column in report view by typpo in [1426](https://github.com/promptfoo/promptfoo/pull/1426)
- **Eval**: Support loading scenarios and tests from external files by mldangelo in [1432](https://github.com/promptfoo/promptfoo/pull/1432)
- **Redteam**: Added language support for generated tests by mldangelo in [1433](https://github.com/promptfoo/promptfoo/pull/1433)
- **Transform**: Support custom function names in file transforms by mldangelo in [1435](https://github.com/promptfoo/promptfoo/pull/1435)
- **Extension Hook API**: Introduced extension hook API by aantn in [1249](https://github.com/promptfoo/promptfoo/pull/1249)

Fixes
- **Report**: Hide unused plugins in report by typpo in [1425](https://github.com/promptfoo/promptfoo/pull/1425)
- **Memory**: Optimize memory usage in `listPreviousResults` by not loading all results into memory by typpo in [1439](https://github.com/promptfoo/promptfoo/pull/1439)

Documentation
- **TypeScript**: Added TypeScript `promptfooconfig` example by mldangelo in [1427](https://github.com/promptfoo/promptfoo/pull/1427)

Refactors
- **Tests**: Moved `evaluatorHelpers` tests to a separate file by mldangelo in [1437](https://github.com/promptfoo/promptfoo/pull/1437)

Dependencies
- **Dev**: Bumped `aws-sdk/client-bedrock-runtime` from 3.624.0 to 3.629.0 by dependabot in [1428](https://github.com/promptfoo/promptfoo/pull/1428)
- **SDK**: Bumped `anthropic-ai/sdk` from 0.25.1 to 0.25.2 by dependabot in [1429](https://github.com/promptfoo/promptfoo/pull/1429)
- **SDK**: Bumped `openai` from 4.55.4 to 4.55.7 by dependabot in [1436](https://github.com/promptfoo/promptfoo/pull/1436)

New Contributors
- **aantn** made their first contribution in [1249](https://github.com/promptfoo/promptfoo/pull/1249)

**Full Changelog**: [0.77.0...0.78.0](https://github.com/promptfoo/promptfoo/compare/0.77.0...0.78.0)

What's Changed
* feat: ssrf plugin by typpo in https://github.com/promptfoo/promptfoo/pull/1411
* feat(assertions): add option to disable AJV strict mode by mldangelo in https://github.com/promptfoo/promptfoo/pull/1415
* feat: `basic` strategy to represent raw payloads only by typpo in https://github.com/promptfoo/promptfoo/pull/1417
* refactor: transform function by mldangelo in https://github.com/promptfoo/promptfoo/pull/1423
* fix: suppress docker lint by typpo in https://github.com/promptfoo/promptfoo/pull/1412
* fix(redteam): plugins respect config-level numTest by typpo in https://github.com/promptfoo/promptfoo/pull/1409
* fix: update eslint config and resolve unused variable warnings by mldangelo in https://github.com/promptfoo/promptfoo/pull/1413
* fix: handle retries for harmful generations by typpo in https://github.com/promptfoo/promptfoo/pull/1422
* chore(deps): bump openai from 4.55.3 to 4.55.4 by dependabot in https://github.com/promptfoo/promptfoo/pull/1418
* docs: add plugin documentation by typpo in https://github.com/promptfoo/promptfoo/pull/1421
* docs(faq): expand and restructure FAQ content by mldangelo in https://github.com/promptfoo/promptfoo/pull/1416


**Full Changelog**: https://github.com/promptfoo/promptfoo/compare/0.76.1...0.77.0