Languagemodels

Changed

- Pin Llama 3.2 model versions
- Decrease repetition penalty for Llama 3.2 models

Added

- Support SmolLM2
- Add `embed` function
- Support Llama 3.1 8B instruct
- Use models directly from Huggingface with config.use_hf_model()
- Add "echo" config option to allow streaming tokens to stdout as they are generated

Changed

- Skip checking for model updates
- Download entire model upfront even if we only need the tokenizer initially
- Use most recent version of CTranslate2
- Add per-model repetition penalties

Added

- Support Llama 3.2 1B and 3B
- Support Danube3
- Support SmolLM

Changed

- Add new separators to document chunking heuristic

Fixed

- Allow missing query prefixes for embedding models

Added

- Support Phi-3-mini-4k-instruct
- Support GIST-small-Embedding-v0 embedding model
- Store model runtime stats to improve benchmarking and analysis

Added

- Support Meta-Llama-3-8B-Instruct
- Support gemma-2b-it
- Support h2o-danube2-1.8b-chat
- Support WizardLM-2-7B

Fixed

- Correct issue causing `choices` to be scored improperly

Added

- CUDA 12 support