Mistralrs

Highlights
- UQFF
- FLUX model
- Llama 3.2 Vision model

MSRV

The MSRV of this release is 1.79.0.

What's Changed
* Enable automatic determination of normal loader type by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/742
* Add the `ForwardInputsResult` api by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/745
* Implement Mixture of Quantized Experts (MoQE) by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/747
* Bump quinn-proto from 0.11.6 to 0.11.8 by dependabot in https://github.com/EricLBuehler/mistral.rs/pull/748
* Fix f64-f32 type mismatch for Metal/Accelerate by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/752
* Nicer error when misconfigured PagedAttention input metadata by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/753
* Update deps, support CUDA 12.6 by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/755
* Patch bug when not using PagedAttention by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/759
* Fix `MistralRs` Drop impl in tokio runtime by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/762
* Use nicer Candle Error APIs by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/767
* Support setting seed by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/766
* Fix Metal build error with seed by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/771
* Fix and add checks for no kv cache by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/776
* UQFF: The uniquely powerful quantized file format. by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/770
* Add `Scheduler::running_len` by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/780
* Deduplicate RoPE caches by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/787
* Easier and simpler Rust-side API by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/785
* Add some examples for AnyMoE by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/788
* Rust API for sampling by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/790
* Our first Diffusion model: FLUX by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/758
* Fix build bugs with metal, NSUInteger by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/792
* Support weight tying in Llama 3.2 GGUF models by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/801
* Implement the Llama 3.2 vision models by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/796


**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.3.0...v0.3.1

Highlights
- New model topology feature: ISQ and device mapping
- 🔥Faster FlashAttention support when batching
- Removed `plotly` and associated JS dependencies
- φ³ Support Phi 3.5, Phi 3.5 vision, Phi 3.5 MoE
- Improved Rust API ergonomics
- Support multiple (shaded) GGUF files

MSRV
The Rust MSRV of this version is 1.79.0

What's Changed
* Fixes for auto dtype selection with RUST_BACKTRACE=1 by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/690
* Add support multiple GGUF files by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/692
* Refactor normal and vision loaders by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/693
* Fix `split.count` GGUF duplication handling by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/695
* Batching example by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/694
* Some fixes by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/697
* Improve vision rust examples by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/698
* Add ISQ topology by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/701
* Add custom logits processor API by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/702
* Add Gemma 2 PagedAttention support by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/704
* Faster RmsNorm in Gemma/Gemma2 by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/703
* Fix bug in Metal ISQ by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/706
* Support GGUF BF16 tensors by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/691
* Better support for FlashAttention: real batching + sliding window + softcap by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/707
* Remove some usages of `pub` in models by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/708
* Support the Phi 3.5 V model by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/710
* Implement the Phi 3.5 MoE model by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/709
* Device map topology by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/717
* Implement DRY penalty by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/637
* Remove plotly and just output CSV loss file by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/700
* Using once_cell to reduce MSRV by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/724
* Fixes for Windows build by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/729
* Even more phi3.5moe fix attempts by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/731
* Add example for Phi 3.5 MoE by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/733
* Add Phi 3.5 chat template by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/734
* Patch ISQ for Mixtral by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/730
* Gracefully handle Engine Drop with termination request by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/735
* feat(vision): add support for proper file and data image URLs by Schuwi in https://github.com/EricLBuehler/mistral.rs/pull/727
* Add new parsing to Python API by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/737
* Remove test and add custom error type to Python API by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/738
* Update kernels for metal bf16 by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/719
* Better `Response` Result API by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/739
* More Metal quantized kernel fixes by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/740
* [Breaking] Bump version to v0.3.0 by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/736
* Final changes for v0.3.0 by EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/741

New Contributors
* Schuwi made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/727

**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.2.5...v0.3.0

| File | Platform | Checksum |
|--------|----------|----------|
| [mistralrs-server-aarch64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.5/mistralrs-server-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.5/mistralrs-server-aarch64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.5/mistralrs-server-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.5/mistralrs-server-x86_64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-unknown-linux-gnu.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.5/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz) | x64 Linux | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.5/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz.sha256) |

| File | Platform | Checksum |
|--------|----------|----------|
| [mistralrs-server-aarch64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-aarch64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-x86_64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-unknown-linux-gnu.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz) | x64 Linux | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz.sha256) |

| File | Platform | Checksum |
|--------|----------|----------|
| [mistralrs-server-aarch64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-aarch64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-x86_64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-unknown-linux-gnu.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz) | x64 Linux | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz.sha256) |