Textflint

New Features

Add 6 Chinese NLP tasks support

This update adds preprocessing and transformations for 6 Chinese NLP tasks, including **Machine Reading Comprehension**, **Semantic Matching**, **Named Entity Recognition**, and **Sentiment Analysis**.

It provides **15** universal transformations and **12** specific transformations.

Add 3 English NLP task support

Now support transformations of **Neural Machine Translation** transformation between English and German.

Now support transformations of **Word Sense Disambiguation**.

Now support transformations of **the Winograd Schema Challenge**.

Fix

Update requirements.

Update tutorial docs to synchronize with toolset version.

Performance
------------
1> Update README: provide more tutorial docs and relate links

Fix
----------
1> Fix bug of pos tagging components which was not initialized;
2> Fix CSV load bug of NER sample;
3> Fixed some bugs for FlintmodelNER, and the tutorial for that is updated.

Performance
-----------
1> Optimize the installation , remove the textattack in the requirements. Because textattack relies on too many packages which may cause the failure of installation. It is recommended to install the package manually for adversarial attack.

2> Speed up the loading process of textflint from 1 minute to 3 seconds.

Features

1. Add command supports
2. Reconstruct Engine interfaces


Fix

1. Barchat incomplete display
2. UT sample is_legal bug
3. Specify importlib-metadata lib version
4. Flintmodel load bug

***Input layer:*** receives textual datasets and models as input, represented as `Dataset` and `FlintModel` separately.

- **`DataSet`**: a container for `Sample`, provides efficiently and handily operation interfaces for `Sample`. `Dataset` supports loading, verification, and saving data in Json or CSV format for various NLP tasks.
- **`FlintModel`**: a target model used in an adversarial attack.

***Generation layer:*** there are mainly four parts in generation layer:

- **`Subpopulation`**: generates a subset of a `DataSet`.
- **`Transformation`**: transforms each sample of `Dataset` if it can be transformed.
- **`AttackRecipe`**: attacks the `FlintModel` and generate a `DataSet` of adversarial examples.
- **`Validator`**: verifies the quality of samples generated by `Transformation` and `AttackRecipe`.

***Report layer:*** analyzes model testing results and provides robustness report for users.