Fmeval

What's Changed
* docs: update telemetry-related info in README and docstrings by danielezhu in https://github.com/aws/fmeval/pull/265
* feat: fetch log probability jmespath from JS metadata by keerthanvasist in https://github.com/aws/fmeval/pull/267
* build(deps): bump jinja2 from 3.1.3 to 3.1.4 by dependabot in https://github.com/aws/fmeval/pull/272
* build(deps): bump tqdm from 4.66.2 to 4.66.3 by dependabot in https://github.com/aws/fmeval/pull/271
* fix: update pinned sagemaker python sdk version and get_user_agent_extra util function by danielezhu in https://github.com/aws/fmeval/pull/274
* build: bump fmeval version to 1.0.3 by danielezhu in https://github.com/aws/fmeval/pull/275


**Full Changelog**: https://github.com/aws/fmeval/compare/v1.0.2...v1.0.3

What's Changed
* chore: simplify botocore/boto3-related util code by danielezhu in https://github.com/aws/fmeval/pull/256
* docs: create Github Actions workflow for generating docs via pdoc by danielezhu in https://github.com/aws/fmeval/pull/260
* test: update matplotlib version and figure cell init test by oyangz in https://github.com/aws/fmeval/pull/259
* chore: update lib versions based on dependabot recommendation by keerthanvasist in https://github.com/aws/fmeval/pull/258
* docs: add syntax highlighting by connorads in https://github.com/aws/fmeval/pull/261
* feat: add fmeval-specific user agent header to botocore config for telemetry purposes by danielezhu in https://github.com/aws/fmeval/pull/262
* fix: fix patching in unit tests by danielezhu in https://github.com/aws/fmeval/pull/264
* build: bump fmeval version to 1.0.2 by danielezhu in https://github.com/aws/fmeval/pull/263

New Contributors
* connorads made their first contribution in https://github.com/aws/fmeval/pull/261

**Full Changelog**: https://github.com/aws/fmeval/compare/v1.0.1...v1.0.2

* fix: fix s3 uri validation for built-in datasets
* docs: update README with details on contributing new eval algos
* fix: update output record key validation logic in validate_call

What's Changed
* chore: Update readme with installation tips by danielezhu in https://github.com/aws/fmeval/pull/181
* fix: readme by polaschwoebel in https://github.com/aws/fmeval/pull/196
* docs: add troubleshooting item for OOM errors by keerthanvasist in https://github.com/aws/fmeval/pull/198
* fix: add data for example notebook by polaschwoebel in https://github.com/aws/fmeval/pull/203
* fix: update terminology in README and source code by danielezhu in https://github.com/aws/fmeval/pull/208
* feat: implement Transform and TransformPipeline classes for modular redesign by danielezhu in https://github.com/aws/fmeval/pull/209
* feat: implement helper models used by evaluation algorithms by danielezhu in https://github.com/aws/fmeval/pull/210
* feat: implement transforms for summarization accuracy metrics by danielezhu in https://github.com/aws/fmeval/pull/211
* docs: update README to include information about Windows support by danielezhu in https://github.com/aws/fmeval/pull/213
* fix: update the default prompt templates for the built-in datasets by jmikko in https://github.com/aws/fmeval/pull/212
* feat: update implementation of SummarizationAccuracy to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/214
* feat: implement transforms for semantic perturbations by danielezhu in https://github.com/aws/fmeval/pull/215
* refactor: update Transform API by danielezhu in https://github.com/aws/fmeval/pull/216
* feat: add prompt template to report by oyangz in https://github.com/aws/fmeval/pull/217
* feat: update various transforms to accept multiple input keys by danielezhu in https://github.com/aws/fmeval/pull/218
* chore: change PromptComposer.PLACEHOLDER from "feature" to "model_input" by danielezhu in https://github.com/aws/fmeval/pull/219
* feat: update GetModelResponse transform to support multiple model invocations on the same input by danielezhu in https://github.com/aws/fmeval/pull/220
* feat: update implementation of GeneralSemanticRobustness to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/222
* fix: update GetModelResponse transform to work with any ModelRunner by danielezhu in https://github.com/aws/fmeval/pull/228
* fix: restore semantic perturbation constants to their original values by danielezhu in https://github.com/aws/fmeval/pull/229
* feat: example notebook for comparative plotting by polaschwoebel in https://github.com/aws/fmeval/pull/223
* refactor: move repeated code in evaluate method into util functions and simplify the EvalAlgorithmInterface method signatures by danielezhu in https://github.com/aws/fmeval/pull/224
* feat: updated docstrings by polaschwoebel in https://github.com/aws/fmeval/pull/225
* chore: restore evaluate_sample and evaluate signatures in EvalAlgorithmInterface by danielezhu in https://github.com/aws/fmeval/pull/231
* refactor: update evaluate_dataset to take in a dataset instead of dataset config by danielezhu in https://github.com/aws/fmeval/pull/232
* feat: update implementation of SummarizationAccuracySemanticRobustness to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/233
* feat: update implementation of QAAccuracy to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/234
* feat: update implementation of QAAccuracySemanticRobustness to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/235
* feat: update implementation of ClassificationAccuracy to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/236
* feat: update implementation of ClassificationAccuracySemanticRobustness to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/237
* Updating third party attributions by malhotra18 in https://github.com/aws/fmeval/pull/239
* feat: update implementation of FactualKnowledge to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/238
* feat: update implementation of PromptStereotyping to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/240
* fix: set default region for boto3 client to access built-in datasets by oyangz in https://github.com/aws/fmeval/pull/242
* feat: update implementation of Toxicity to use Transform-based approach by danielezhu in https://github.com/aws/fmeval/pull/241
* build: bump fmeval version to 1.0.0 by danielezhu in https://github.com/aws/fmeval/pull/243

New Contributors
* jmikko made their first contribution in https://github.com/aws/fmeval/pull/212

**Full Changelog**: https://github.com/aws/fmeval/compare/v0.4.0...v1.0.0

What's Changed
* feat: make sm/br runners easier to subclass by franluca in https://github.com/aws/fmeval/pull/159
* chore: update example notebooks to pip install the fmeval package by danielezhu in https://github.com/aws/fmeval/pull/158
* fix(pre-launch science review): correcting categories for toxicity da… by franluca in https://github.com/aws/fmeval/pull/136
* fix: replace add_column with map in _generate_prompt_column by danielezhu in https://github.com/aws/fmeval/pull/161
* Update f1 score in QA accuracy eval by bilalaws in https://github.com/aws/fmeval/pull/166
* feat: added the precision and recall metrics for QA accuracy by bilalaws in https://github.com/aws/fmeval/pull/157
* Strip text when computing precision and recall. by bilalaws in https://github.com/aws/fmeval/pull/172
* fix: create single source of truth for dataset column names by danielezhu in https://github.com/aws/fmeval/pull/171
* fix: update Ray to version 2.9.0 by danielezhu in https://github.com/aws/fmeval/pull/173
* chore: update devtool all to install first, lint after by keerthanvasist in https://github.com/aws/fmeval/pull/174
* feat: stringify dataset column contents during data loading by danielezhu in https://github.com/aws/fmeval/pull/168
* fix: unblock release pipeline by xiaoyi-cheng in https://github.com/aws/fmeval/pull/176
* fix: update scores description by xiaoyi-cheng in https://github.com/aws/fmeval/pull/177
* fix: split text by any newline and spaces by franluca in https://github.com/aws/fmeval/pull/178
* fix: load detoxify model from state dict and upgrade transformers version by oyangz in https://github.com/aws/fmeval/pull/180
* fix: Fix example notebook unit tests by danielezhu in https://github.com/aws/fmeval/pull/188
* chore: Update Ray to 2.9.1 by danielezhu in https://github.com/aws/fmeval/pull/189
* chore: remove xsum dataset and update gigaword description by xiaoyi-cheng in https://github.com/aws/fmeval/pull/191
* chore: remove XSUM dataset from example notebook and integration tests by danielezhu in https://github.com/aws/fmeval/pull/192
* feat: add support for non-deterministic models in GeneralSemanticRobustness and add BERTScore Dissimilarity by bilalaws in https://github.com/aws/fmeval/pull/184
* fix: add bert_score_dissimilarity description by oyangz in https://github.com/aws/fmeval/pull/193
* fix: Toxicity evaluate_sample error message by xiaoyi-cheng in https://github.com/aws/fmeval/pull/185
* build(deps): bump aiohttp to fix vulnerability by xiaoyi-cheng in https://github.com/aws/fmeval/pull/194
* build: bump fmeval version to 0.4.0 by xiaoyi-cheng in https://github.com/aws/fmeval/pull/195

New Contributors
* bilalaws made their first contribution in https://github.com/aws/fmeval/pull/166

**Full Changelog**: https://github.com/aws/fmeval/compare/v0.3.0...v0.4.0

What's Changed
* fix: add proper capitalization for 'SageMaker' by oyangz in https://github.com/aws/fmeval/pull/153
* fix: fix f1_score by xiaoyi-cheng in https://github.com/aws/fmeval/pull/152
* fix: remove s3fs from data-loading code and use boto3 instead by danielezhu in https://github.com/aws/fmeval/pull/155
* feat: support inference component by xiaoyi-cheng in https://github.com/aws/fmeval/pull/156


**Full Changelog**: https://github.com/aws/fmeval/compare/v0.2.1...v0.3.0