Arelight

[:blue_book: Complete list of updates](https://github.com/nicolay-r/ARElight/issues/131)

Main Changes

Support of the new UI
It is expected to be a separated UI for the ARElight, which is sourced there:
https://github.com/nicolay-r/ARElight-server

- Support bulk-translate 162
- Inference argument JSON exporter function 142
- ARElight operations -- logger 143
- ARElight inference -- support dynamic log writing 141
- 🔥 Support SQlite filtering operation for relation pairs. 144
- 🔥 🐛 MLTextTranslatorPipelineItem -- causes issues with the indices in tranlation (batching mode) 146
- 🔴 Translate: optional source. 153

Refactored features

- Drop support for the SQLiteService from AREkit 161
- Improve DeepPavlov NER performance by adopting batching 140
- Support new pipelines concept from AREkit with optional batching 136 🔥
- Add SQLite support for sampling 129
- Support prediction writing in SQLite 130
- Refactoring SQLite impemenation for OpenNRE 134
- OpenNRE implement SentenceREDataset based on SQLite storage 132
- * SentenceRELoader create a custom implementation (torch 🔥 dependency)
- * SentenceREDataset implement, fix the original issue of storing the whole data in memory! (torch 🔥 dependency )
- AREkit version of 0.25.0 (c8fb0bb)

Minor
- Writers relocation from AREkit 165
- CustomTermsSplitterPipelineItem -- remove that class since it only complicates NER pipeline 135
- Appropriate formatting of unit tests (https://github.com/nicolay-r/ARElight/blob/main/test/test_translation.py)
- 🔧 Predict header formatter -- sync 137
- 🔧 Sync samples storing path between different pipeline elements 138
- Fix httpx package 📦 related issues
- 🐛 SQLite3Writer -- extension is missed for the results of BERT application 145
- torch -- take control over num_workers parameter 149
- 🔥 🐛 MLTextTranslatorPipelineItem -- causes issues with the indices in tranlation (batching mode) 146
- ❌ save_demo_page method is no longer applicable. 151
- Clarifying cuda/ cpu parameter choices 156
- 🐛 MystemStemmer -- bring back wrapper for Mystem 163
- 🐛 BaseSerializerPipelineItem -- legacy, move from AREkit earlier version. 164


**Full Changelog**: https://github.com/nicolay-r/ARElight/compare/0.24.0...0.25.0

[:blue_book: The complete list of changes](https://github.com/nicolay-r/ARElight/issues/76)

Simplified usability / Flexibility
- 🔧 SampleRowProvider -- Enable Crop Mode 125
- input_data for the pipeline -- design mechanism for supporting an output independing of the sequence of elements 90
- Refactoring installation process and usage as library 61
- D3JS templates and the related pipeline provider 77
- 🔧 Fix launch of the tranformer-based models 78
- ❌ Drop training support (43cac8c)
- Make home folder for states downloading 95
- tqdm -- for inference we may use defined progress bar since we know the total amount of rows 96
- Stemmer -- provide paramer options that allow to enable/disable stemming 97
- googletrans -- Integrate Text Translator (transfer learning) ✨ 108
- Torch and Huggingface
- infer_bert -- switch to LM teminology, support pre-trained states as parameters 70
- ❓ Support of OpenNRE with torch and transformers [NIVTS project findinds] 63
- Remove DeepPavlov and switch to the transformers 69
- Library
- Make it even simpler -- a single cmd script for everything 71
- 🔧 Switch to AREkit==0.24.0
- 🔧 Support new DeepPavlov==1.2.0
- 📦 Make it as a package
- DATA_DIR dependencies -- refactoring 84
- Remove Downloads 85

Minor fixes
- Remove data and predefined text dir 101
- Entities Formatters -- simplify structure, place them in the same infer.py script ✨ 103
- DataFrame processing -- text column customization via cmd parameter ✨ 107
- BERT -- pretrained state might be None if checkpoint is cached. ✨ 106
- Entities Formatters -- simplify structure, place them in the same infer.py script ✨ 103
- Remove pandas dependency 117
- graphs_operations_weighted results in empty graph (no nodes) 112
- Generalization -- settings for label provider/mapping [OpenNRE inference] 119
- from-files -- use DirectoryFilesDocOperation 121
- Setup the correct dependency from the OpenNRE 0.1.1 (ca76a3e)

Change Log

**Full Changelog**: https://github.com/nicolay-r/ARElight/compare/0.23.1...0.24.0

Increased performance x2 due to support streaming from AREkit-0.23.1 :fire:
Simplified API
Stop docking support
Updated project description

Updates:
https://github.com/nicolay-r/ARElight/issues/62

0.23.0-p0
Improved stabillty and added new functionality

* Fixed problem with long-sequences of NER: 57
* Provided support of the **dataframes** in input :58
* Generalization of NER, support of English language: 53
* Fixed output formatting, sync `prediction` and `sample` file names: 54
* Support sentence splitting selection algo: 55

* Support of the AREkit 0.23.0
* Removed support of Neural Networks
* Simplified API

0.22.1-p2

Changes

Sync with AREkit-0.22.1 API
- Removed support of collections serlalization
- Simplified implementations
- Proved unit testing scripts with `html` logs

Embeddings:
- `news_mystem_skipgram_1000_20_2015.bin.gz` now is a part of the AREkit-0.22.1