Swebench

Latest version: v2.1.7

Safety actively analyzes 688619 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 3

2.0.12

* Minor naming changes
* 186 fix: correct some typings and a incorrect function call
* 183 Fix timeout
* 178 Add schema version to report card
* 177 Fix run live scripts

2.0.9

* 176 Move inference to swebench.inference sub-package
* 175 Fix link in collect README.md

2.0.8

* Add `cutoff_date`, `max_pulls` arguments to collection pipeline
* Minor Django issue comment parsing logic
* Rewritten `extract_patches` logic
* Remove `MAP_REPO_TO_TEST_FRAMEWORK` symbol

2.0.4

* 173 Fix: Allow to set GH token from env var in collect/print_pulls
* 171 Don't let tox install a virtualenv during evaluation
* 169 Handle failures because of None/empty patches

2.0.3

* 149 Interface fix: run_id is required
* 151 Fix: Support JSON datasets (avoid loading json twice)
* 152 Add very simple CI
* 153 Various nitpicks
* 155 Fix link to collection tutorial
* 161 Fix path to image in docs
* 162 Fix evaluation hanging issue and improve patch apply
* 164 Fix so it doesn't crash when no env imgs to build
* 166 Fix newline outputs for django's log parser
* 168 Update reporting and skip empty model patch predictions

2.0.0

Major release - the SWE-bench evaluation harness has been upgraded to incorporate containerized, sandboxed execution environments based on Docker. There are several chances to the API resulting from this:
* Removal of the `swebench.metrics` module
* Updates to the API of `swebench.harness` functionality
* Significant modifications to underlying evaluation logic
* Minor updates to installation specifications for different repos + versions.

Read the full report [here](https://github.com/princeton-nlp/SWE-bench/tree/main/docs/20240627_docker)

Page 1 of 3

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.