Inspect-ai

Latest version: v0.3.81

Safety actively analyzes 723217 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 14

0.3.81

- Requirements: Temporarily upper-bound `rich` to < 14.0.0 to workaround issue.

0.3.80

- Google: Compatibility with httpx client in `google-genai` >= 1.8.0 (which is now required).
- Mistral: Compatibility with tool call schema for `mistralai` >= v1.6.0 (which is now required).
- Inspect View: Correctly parse NaN values (use JSON5 for all JSON parsing)

0.3.79

- Google: Compatibility with v1.7 of google-genai package (create client per-generate request)
- Bugfix: Properly record scorer and metrics when there are multiple tasks run in an eval.

0.3.78

- OpenAI: Ensure that assistant messages always have the `msg_` prefix in responses API.

0.3.77

- New [think()](https://inspect.aisi.org.uk/tools-standard.html#sec-think) tool that provides models with the ability to include an additional thinking step.
- OpenAI: Support for the new [Responses API](https://inspect.ai-safety-institute.org.uk/providers.html#responses-api) and [o1-pro](https://platform.openai.com/docs/models/o1-pro) models.
- OpenAI: Remove base64-encoded audio content from API call JSON in ModelEvent.
- AzureAI: Support for use of native [OpenAI](https://inspect.ai-safety-institute.org.uk/providers.html#openai-on-azure) and [Mistral](https://inspect.ai-safety-institute.org.uk/providers.html#mistral-on-azure-ai) clients using service qualifiers (e.g. `openai/azure/gpt-4o-mini` or `mistral/azure/Mistral-Large-2411`).
- OpenRouter: Handle "error" field in response object and retry for empty responses.
- Added `--metadata` option to eval for associating metadata with eval runs.
- Task display: Show reasoning tokens for models that report them.
- Anthropic: Include reasoning tokens in computation of total tokens
- Inspect View: Properly wrap tool input for non-code inputs like `think`.

0.3.76

- [bash_session()](https://inspect.ai-safety-institute.org.uk/tools-standard.html#sec-bash-session) tool for creating a stateful bash shell that retains its state across calls from the model.
- [text_editor()](https://inspect.ai-safety-institute.org.uk/tools-standard.html#sec-text-editor) tool which enables viewing, creating and editing text files.
- Structured Output: Properly handle Pydantic BaseModel that contains other BaseModel definitions in its schema.
- OpenAI: Support for .wav files in audio inputs for gpt-4o-audio-preview.
- OpenAI: Strip 'azure' prefix from model_name so that model type checks all work correctly.
- OpenAI: Don't send `reasoning_effort` parameter to o1-preview (as it is not supported).
- Inspect View: Fix error sorting numeric or categorical score results.
- Inspect View: Properly wrap model API call text in the transcript.
- Bugfix: Only initialise display in eval_set if it wasn't initialised from the CLI
- Bugfix: Set the global log level based on the specified Inspect log level.
- Bugfix: Resolve issue when deserialising a SubtaskEvent from a log file which does not have a completed time.
- Bugfix: Fix unnecessary warnings about task arguments.
- Bugfix: When a task does not take a kwargs argument, only warn if the provided argument is not valid.

Page 1 of 14

© 2025 Safety CLI Cybersecurity Inc. All Rights Reserved.