Esperanto

This release updates the Google SDK implementation to use google-genai v1.8.0, replacing the deprecated google-generativeai package.

- Improved dependency error message to show specific missing package

Changes in this release

- Fixed Voyage embedding provider to handle new response format
- Updated tests to test both old and new response formats
- Bumped version to 1.0.0

Fixes

- Added missing test dependencies:
- pytest-mock
- pytest-asyncio
- pytest-cov
- responses
- Updated dev dependency group with correct versions
- All tests now passing with 100% coverage for Voyage AI provider

Installation

For development:
bash
pip install "esperanto[dev]"

Improvements

- Enhanced transformers embedding provider documentation
- Added detailed examples for basic and advanced usage
- Documented tokenizer configuration options
- Added example with multilingual model (Portuguese BERT)
- Explained different pooling strategies and their use cases
- Fixed tokenizer warning by adding default max_length

Configuration Options

Added clear documentation for tokenizer configuration:
- max_length: Default 512 (BERT standard)
- padding: True by default
- truncation: True by default

Pooling Strategies

Added detailed explanations for each pooling strategy:
- mean: Average of all token embeddings (default, best for semantic similarity)
- max: Maximum value across token embeddings (good for key feature extraction)
- cls: [CLS] token embedding (suited for sentence classification)

New Features

- Added transformers embedding provider for local Hugging Face models
- Support for BERT and other transformer models
- Multiple pooling strategies (mean, max, cls)
- Device configuration (CPU, CUDA, MPS) with auto-detection
- Optional model quantization (4-bit and 8-bit)
- Model caching configuration
- Batch processing for efficient embedding generation
- Both synchronous and asynchronous methods
- Comprehensive test coverage