Ctransformers

Changes

- Skip evaluating tokens that are evaluated in the past. This can significantly speed up prompt processing in chat applications that prepend previous messages to prompt.
- Deprecate `LLM.reset()` method. Use high-level API instead.
- Add support for batching and beam search to 🤗 model.
- Remove universal binary option when building for AVX2, AVX on macOS.

Changes

- Add support for 🤗 Transformers

Changes

- Add support for GGUF v2
- Add CUDA support for Falcon GGUF models
- Add ROCm support
- Add low-level API for `add_bos_token`, `bos_token_id`

Changes

- Add GGUF format support for Llama and Falcon models
- Add support for Code Llama models

Changes

- Add `mmap` and `mlock` parameters for LLaMA and Falcon models
- Add `revision` option for models on Hugging Face Hub

Changes

- Add experimental CUDA support for StarCoder, StarChat models
- Add `gpt_bigcode` as model type for StarCoder, StarChat models
- Fix loading GPTQ models from a local path