Openvino-genai

Please check out the latest documentation pages related to the new `openvino_genai` package!

Please check out the latest documentation pages related to the new `openvino_genai` package!

- [How to install](https://docs.openvino.ai/2024/get-started/install-openvino/install-openvino-genai.html)
- [How to use](https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/genai-guide.html)

What's Changed
* Bump OV version to 2024.4.1 by akladiev in https://github.com/openvinotoolkit/openvino.genai/pull/894
* Update requirements.txt and add requirements_2024.4.txt by wgzintel in https://github.com/openvinotoolkit/openvino.genai/pull/893

**Full Changelog**: https://github.com/openvinotoolkit/openvino.genai/compare/2024.4.0.0...2024.4.1.0

Please check out the latest documentation pages related to the new `openvino_genai` package!

- [How to install](https://docs.openvino.ai/2024/get-started/install-openvino/install-openvino-genai.html)
- [How to use](https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/genai-guide.html)

What's Changed
* Support chat conversation for StaticLLMPipeline by TolyaTalamanov in https://github.com/openvinotoolkit/openvino.genai/pull/580
* Prefix caching. by popovaan in https://github.com/openvinotoolkit/openvino.genai/pull/639
* Allow to build GenAI with OpenVINO via extra modules by ilya-lavrenov in https://github.com/openvinotoolkit/openvino.genai/pull/726
* Simplified partial preemption algorithm. by popovaan in https://github.com/openvinotoolkit/openvino.genai/pull/730
* Add set_chat_template by Wovchena in https://github.com/openvinotoolkit/openvino.genai/pull/734
* Detect KV cache sequence length axis by as-suvorov in https://github.com/openvinotoolkit/openvino.genai/pull/744
* Enable u8 KV cache precision for CB by ilya-lavrenov in https://github.com/openvinotoolkit/openvino.genai/pull/759
* Add test case for native pytorch model by wgzintel in https://github.com/openvinotoolkit/openvino.genai/pull/722
* Prefix caching improvements by popovaan in https://github.com/openvinotoolkit/openvino.genai/pull/758
* Add USS metric by wgzintel in https://github.com/openvinotoolkit/openvino.genai/pull/762
* Prefix caching optimization by popovaan in https://github.com/openvinotoolkit/openvino.genai/pull/785
* Transition to default int4 compression configs from optimum-intel by nikita-savelyevv in https://github.com/openvinotoolkit/openvino.genai/pull/689
* Control KV-cache size for StaticLLMPipeline by TolyaTalamanov in https://github.com/openvinotoolkit/openvino.genai/pull/795
* [2024.4] update optimum intel commit to include mxfp4 conversion by eaidova in https://github.com/openvinotoolkit/openvino.genai/pull/828
* [2024.4] use perf metrics for genai in llm bench by eaidova in https://github.com/openvinotoolkit/openvino.genai/pull/830
* Update Pybind to version 13 by mryzhov in https://github.com/openvinotoolkit/openvino.genai/pull/836
* Introduce stop_strings and stop_token_ids sampling params [2024.4 base] by mzegla in https://github.com/openvinotoolkit/openvino.genai/pull/817
* StaticLLMPipeline: Handle single element list of prompts by TolyaTalamanov in https://github.com/openvinotoolkit/openvino.genai/pull/848
* Fix Meta-Llama-3.1-8B-Instruct chat template by pavel-esir in https://github.com/openvinotoolkit/openvino.genai/pull/846
* Add GPU support for continuous batching [2024.4] by sshlyapn in https://github.com/openvinotoolkit/openvino.genai/pull/858


**Full Changelog**: https://github.com/openvinotoolkit/openvino.genai/compare/2024.3.0.0...2024.4.0.0

Please check out the latest documentation pages related to the new `openvino_genai` package!

- [How to install](https://docs.openvino.ai/2024/get-started/install-openvino/install-openvino-genai.html)
- [How to use](https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/genai-guide.html)

Please check out the latest documentation pages related to the new `openvino_genai` package!

- [How to install](https://docs.openvino.ai/2024/get-started/install-openvino/install-openvino-genai.html)
- [How to use](https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/genai-guide.html)