Garak

* Probes can now be selected by MISP tag, e.g. owasp:llm01
* garak now automatically creates an HTML report on completion
* HTML reports can be grouped by module but also by probe tag category, so you can see e.g. top-level scores for prompt injection, hallucination, and so on
* logs now go to a dedicated log dir by default, to keep things clean
* new buffs: encoding.Base64, encoding.CharCode
* new generator: [NeMo guardrails](https://github.com/NVIDIA/NeMo-Guardrails)
* new probe: [AutoDAN](https://arxiv.org/abs/2310.04451)
* RealToxicityProbes now only loads local lists, much faster
* update OpenAI models list
* fix attempt parameter stability
* better logging of config params
* atk is now atkgen

Contributions from erickgalinkin , drazvan . Enjoy & Happy holidays! 🎅🎄

What's Changed
* Attempt no longer uses mutable type defaults by leondz in https://github.com/leondz/garak/pull/360
* Add NeMoGuardrails generator (WIP). by drazvan in https://github.com/leondz/garak/pull/345
* add test for mutable defaults bug in attempt.Attempt by leondz in https://github.com/leondz/garak/pull/362
* refresh openai model name list by leondz in https://github.com/leondz/garak/pull/363
* speed up realtoxicityprompts loading by leondz in https://github.com/leondz/garak/pull/364
* Feature/digest report 231212 by leondz in https://github.com/leondz/garak/pull/365
* Autodan by erickgalinkin in https://github.com/leondz/garak/pull/367
* Auto-reporting by leondz in https://github.com/leondz/garak/pull/368
* add guardrails doc connection by leondz in https://github.com/leondz/garak/pull/369
* Feature/digest plugin descrs by leondz in https://github.com/leondz/garak/pull/370
* Add Base64 and CharCode buffs by erickgalinkin in https://github.com/leondz/garak/pull/372
* tidy buffs, add test for buff config loading by leondz in https://github.com/leondz/garak/pull/376
* Feature/tag selection by leondz in https://github.com/leondz/garak/pull/383
* set default for probe_tags in core config; use this as default cli arg by leondz in https://github.com/leondz/garak/pull/386
* hitlogs should use same paths as other reporting. add test for this by leondz in https://github.com/leondz/garak/pull/387
* Feature/reporting categories by leondz in https://github.com/leondz/garak/pull/389

New Contributors
* drazvan made their first contribution in https://github.com/leondz/garak/pull/345

**Full Changelog**: https://github.com/leondz/garak/compare/v0.9.0.9...v0.9.0.10

* Added GCG jailbreak probe (probes.gcg.GCG_Probe)
* Add support for NVIDIA Optimum (generators.huggingface.OptimumPipeline)
* Add OWASP tags to probes
* Add fast & slow paraphrase buffs (buffs.paraphrase.Fast, buffs.paraphrase.PegasusT5)
* Support for config files: there's a core config, site config, and a CLI config, and all can be used to set system, run, and plugin parameters
* Supply some sample config files for a few different styles of garak run
* Progress bar for buffs
* Added debugging REST server for dev
* Move RealToxicityPrompts resources to their own subdir

Thanks to erickgalinkin drazvan DavidLee528

* Rename ART to AG (Attack Generator)
* Add generator support for NeMo LLM
* Add generator support for OctoML
* Add generic REST connector, with configs
* Add option to parallelise requests
* Add option to parallelise attempts
* Include AutoDAN probe
* Added "interactive mode", where you get a garak CLI 🎉
* Fix continuation probe trigger alignment
* Fix RTP prompts to be aggressive
* Add support for langchain LLM interface
* Upgrade in avidtools
* Improve checking for detector names in probes
* Turn-by-turn visual indicator on attack generator probe

* tests, tests, tests
* docstrings in many classes, also in the documentation (https://reference.garak.ai/)
* improved package hallucination probe prompts
* speedup on package hallucination detector scan

New in garak!

* **integrated vulnerability reporting:** vulnerabilities found with garak can now be directly reported to [AVID](https://avidml.org/) shubhobm
* **package hallucination:** added a probe for detecting [package hallucination](https://vulcan.io/blog/ai-hallucinations-package-risk)
* **docs are up:** reference guide is here, https://reference.garak.ai/
* **primary/extended detectors:** it's now possible to designate a primary detector for a probe (when using the default probewise harness)
* **multiple payloads for encoding module:** as well as the default option, there's slurs and xss injection attempts; access them with `--probe_options '{"encoding.options": ["default", "slurs", "xss"]}'` (adjust to taste)
* **fine-tune perspective api backoff for bandwidth:** never wait sixty seconds, the window use to determine rate limit
* **doc fixes:** mkonxd
* **hitlog entries now more self-contained:** store how many generations were targeted with that prompt
* **remove shortnames:** from probes and detectors
* **move encoding injection module to use triggers:** finer-grained detection, means fewer false positives

New in `garak`

* enable reporting of vulnerabilities into [AVID](https://avidml.org/)
* de-prefix prompt from LLM output by default
* add a data leakage/replay attack probe
* add a glitch token detection probe
* enable narrow-format CLI output
* extra payloads (secret level!) in encoding probe