Txtai

This release adds the following new features, improvements and bug fixes.

New Features
--------------------------
- Enhance Labels pipeline to support standard text classification models (95)
- Add Trainer pipeline (96)
- Modularize txtai install (97)
- Evaluate if faiss-cpu can be used as default across all platforms (98)
- Add vector method for sentence-transformers (101)

Improvements
--------------------------
- Add book search example application (91)
- Add wiki search example application (92)
- Change tokenization to default to false for TransformerVectors (99)
- Infer vector method using path (100)
- Improve performance when running models through transformers (102)
- Update notebooks and example applications (103)

Bug Fixes
--------------------------
- Clear workflow batch during processing bug (90)

This release adds the following new features:

- Add support for update/delete embeddings index operations (86)
- Add Embeddings Cluster component (87)
- Switch default backend on Windows to Hnswlib (88)
- Add notebook covering distributed embedding clusters (89)

Summary of txtai features:

- 🔎 Large-scale similarity search with multiple index backends ([Faiss](https://github.com/facebookresearch/faiss), [Annoy](https://github.com/spotify/annoy), [Hnswlib](https://github.com/nmslib/hnswlib))
- 📄 Create embeddings for text snippets, documents, audio and images. Supports transformers and word vectors.
- 💡 Machine-learning pipelines to run extractive question-answering, zero-shot labeling, transcription, translation, summarization and text extraction
- ↪️️ Workflows that join pipelines together to aggregate business logic. txtai processes can be microservices or full-fledged indexing workflows.
- 🔗 API bindings for [JavaScript](https://github.com/neuml/txtai.js), [Java](https://github.com/neuml/txtai.java), [Rust](https://github.com/neuml/txtai.rs) and [Go](https://github.com/neuml/txtai.go)
- ☁️ Cloud-native architecture that scales out with container orchestration systems (e.g. Kubernetes)

New Features
--------------------------
- Add Docker file for API (59)
- Require Faiss 1.7.0 (60)
- Add summary pipeline (65)
- Add text extraction pipeline (66)
- Add transcription pipeline (67)
- Add translation pipeline (68)
- Add workflow framework (69)
- Add additional pipeline abstraction layer for tensor frameworks (70)
- Add tests for new v3 functionality (71)
- Add notebooks covering new v3 functionality (73)
- Add Pipeline Factory (76)
- Add API extensions (77)
- Add workflow builder application (80)
- Add text segmentation pipeline (81)
- Add workflow to API (82)
- Add service workflow task (83)
- Add object storage workflow task (84)
- Add URL workflow task (85)

Improvements
--------------------------
- Refactor code into smaller components and modules (63)
- Modify pipeline to accept GPU device id (64)
- Allow direct download of sentence-transformer models (72)
- Update documentation, add site through GitHub pages (75)
- Modularize the API (78)
- Add default truncation to pipelines (79)

Bug Fixes
--------------------------
- Non intuitive behaviour of Tokenizer (61)
- [Python 3.9, Mac OS] Code hangs while building embedding index (62)
- embeddings.index Truncation RuntimeError: The size of tensor a (889) must match the size of tensor b (512) at non-singleton dimension 1 (74)

In addition to Python, txtai has API support for [JavaScript](https://github.com/neuml/txtai.js), [Java](https://github.com/neuml/txtai.java), [Rust](https://github.com/neuml/txtai.rs) and [Go](https://github.com/neuml/txtai.go).

New Features
--------------------------
- [BREAKING CHANGES] Make API definitions consistent (54)
- Zero-shot similarity pipeline (21, 49)
- Add batch support for all modules (18, 53)
- Add example notebook integrating Hugging Face Datasets (26)
- Add example notebook that adds semantic search to existing system (57)

Improvements
--------------------------
- Add API tests, increase test coverage (42)
- Refactor pipeline component (44)
- Upgrade to Transformers 4.x (45)
- Review, organize and update example notebooks (52)
- Allow setting ANN index parameters (55)
- Modify API add method to stream data (56)

Bug Fixes
--------------------------
- Fix language support issues (39, 43)

This release adds the following enhancements and bug fixes:

- Refresh example notebooks and add notebook on labeling (40)
- Enhance API to fully support all txtai functionality (41)

This release adds the following enhancements and bug fixes:

- Split extractor embedding query and QA calls (35)
- Upgrade to Faiss 1.6.4 (36)
- Migrate build to GitHub Actions (38)