Factscore

Latest version: v0.2.0

Safety actively analyzes 681812 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

0.2.0

This update fixes a few bugs in our custom retrieval corpus support, and adds the ACL anthology as an example (see 24).

In other news, FActScore was recently accepted as a main conference paper in EMNLP 2023! This release additionally updates our [PyPI webpage](https://pypi.org/project/factscore) to reflect this.

0.1.7

This update adds a few new important features to our PIP package (thanks again to our beta users!):

- (https://github.com/shmsw25/FActScore/pull/21) To penalize very short generations, we have added a length penalty similar to the BLEU scores. This is enabled by default and can be customized using the `--gamma` flag. See [here](https://github.com/shmsw25/FActScore#running-factscore-using-a-command-line) for more details.
- (https://github.com/shmsw25/FActScore/pull/20) Users can now customize their abstain detection workflow using the `--abstain_detection` flag. We have added two abstain detection functions (`generic`, `perplexity_ai`), but disabled them by default. See [here](https://github.com/shmsw25/FActScore#running-factscore-using-a-command-line) for more details.
- (d768d818f7fcdeeb61e90a92491b08c1a0e640c6) bug fixes in custom corpus retrieval

0.1.5

This update to our PIP package contains a number of new features requested by our beta users:

- You can now use Instruct LLAMA as well besides ChatGPT! Set `--model_name retrieval+llama+npm` to use it. The `factscore.download_data` script has been fully updated to also install Instruct LLAMA if `--llama_7B_HF_path` is set as shown in the README.

- We have add a lot more information into our logging, such as alternative metrics (respond ratio, of generated facts) and OpenAI API cost estimation. We have also moved to the `logging` package and have a few flags to ensure a nicer logging experience (`--print_rate_limit_error` and `--verbose`).

- More customization of cache is now possible, with `--model_dir` and `--data_dir` directories.

Links

Releases

Has known vulnerabilities

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.