Inference

🚀 Added
👀 Gaze detection in Workflows

<table>
<tr>
<td width="35%"><img src="https://github.com/user-attachments/assets/5e4d1a8f-1b74-495c-afa3-be48d9696d48" /></td>
<td width="65%"><img src="https://github.com/user-attachments/assets/26c65bf3-e44a-404e-8348-318654cf5520" /></td>
</tr>
</table>

joaomarcoscrs (as part of hist first contribution 🏅) introduced Gaze detection model into Workflows.

Don't know what Gaze Detection is?

Gaze detection is a method to determine where a person is looking by analyzing their eye movements and gaze direction. It typically uses cameras or sensors to track eye position and orientation, identifying the point of focus in real time.

It is commonly used in areas like:

* **Human-Computer Interaction**: Controlling devices with eye movements.
* **Behavioral Analysis:** Understanding attention and interest.
* **Marketing Research:** Measuring what catches a person's attention.

Now - you can apply Workflows in such use-cases. Check out Gaze Detection block [📖 documentation](https://inference.roboflow.com/workflows/blocks/gaze_detection/) to find more information.

> [!NOTE]
> The block is currently not supported on Roboflow Hosted Platform. Check out how to [send requests to localhost `inference` server](https://inference.roboflow.com/workflows/modes_of_running/#http-api-request)

🏋️‍♂️ New experimental Workflows blocks enabling new capabilities

yeldarby prepared whole series of blocks to open-up new capabilities for Workflows, including:
* [Workflows Buffer Block](https://inference.roboflow.com/workflows/blocks/buffer/) in https://github.com/roboflow/inference/pull/894
* [Workflows Grid Visualization Block](https://inference.roboflow.com/workflows/blocks/grid_visualization/) in https://github.com/roboflow/inference/pull/895
* Workflow Cache [Get](https://inference.roboflow.com/workflows/blocks/cache_get/)/[Set Blocks](https://inference.roboflow.com/workflows/blocks/cache_set/) in https://github.com/roboflow/inference/pull/893
* Workflows [Outlier Detection Block](https://inference.roboflow.com/workflows/blocks/identify_outliers/) in https://github.com/roboflow/inference/pull/896


💪 Improved

`Florence 2` runs now up to 3x faster
🧙‍♂️ isaacrob-roboflow did some magic 🪄 and now, all of the sudden `Florence2` models deployed in `inference` could run up to 3x faster 🤯 ❗
See details in https://github.com/roboflow/inference/pull/885


🔧 Fixed

Security vulnerability in landing page
We've fixed security issue in `inference` server landing page: https://github.com/roboflow/inference/pull/890

Issue description
If a Next.js application is performing authorization in middleware based on pathname, it was possible for this authorization to be bypassed.
This issue was patched in Next.js `14.2.15` and later.

> [!CAUTION]
> We advise all users of older versions of `inference` server to migrate to version `0.32.0`

Other fixes
* Add fix for the problem with inference-cli workflows predictions saving by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/891
* Improvements in blocks descriptions' by EmilyGavrilenko (https://github.com/roboflow/inference/pull/898) and casmwenger in (https://github.com/roboflow/inference/pull/897)
* Fix usage collector fps by grzegorz-roboflow in https://github.com/roboflow/inference/pull/903

🚧 What's Changed
* Add test to detect blocks with missing __init__.py by grzegorz-roboflow in https://github.com/roboflow/inference/pull/883
* Cache CLIP Text Embeddings in Workflow Block by yeldarby in https://github.com/roboflow/inference/pull/892
* Allow using video metadata for rate limiter on recorded video by yeldarby in https://github.com/roboflow/inference/pull/887
* Serialized owlv2 model by probicheaux in https://github.com/roboflow/inference/pull/889
* Skip additional test by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/902

🏅 New Contributors
* joaomarcoscrs made their first contribution in https://github.com/roboflow/inference/pull/888

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.31.1...v0.32.0

🔧 Fixed

* Fix inference 0.30.0 release by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/882 - just we forgot `__init__.py`


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.31.0...v0.31.1

🚀 Added

📏 Easily create embeddings and compare them in Workflows

Thanks to yeldarby, we have [Clip Embedding](https://inference.roboflow.com/workflows/blocks/clip_embedding_model/) and [Cosine Similarity](https://inference.roboflow.com/workflows/blocks/cosine_similarity/) Workflows blocks. Just take a look what would now be possible.

💡 Application ideas
* **Visual Search:** Match text queries (e.g., "red shoes") to the most relevant images without training a custom model.
* **Image Deduplication:** Identify similar or duplicate images by calculating embeddings and measuring cosine similarity.
* **Zero-Shot Classification:** Classify images into categories by comparing their embeddings to pre-defined text labels (e.g., "cat," "dog").
* **Outliers detection:** Check which images do not match to general trend

<div align="center"><img width="50%" src="https://github.com/user-attachments/assets/33dc9e75-bb65-4697-ba9a-28ae6a5c9265" /></div>

✨ `gemini-2.0-flash` 🤝 Workflows

Check out [model card](https://deepmind.google/technologies/gemini/flash/) and start using new model, simply pointing new model type in [Google Gemini](https://inference.roboflow.com/workflows/blocks/google_gemini/) Workflow block 😄 All thanks to EmilyGavrilenko

<div align="center"><img width="50%" src="https://github.com/user-attachments/assets/06f591a6-ef25-48bc-8743-52dab271fbdb" /></div>

🔥 Recent [`supervision`](https://github.com/roboflow/supervision) versions are now supported

For a long time we had issue with not supporting _up-to-date_ [`supervision`](https://github.com/roboflow/supervision) releases. This is **no longer the case** thanks to LinasKo and his contribution https://github.com/roboflow/inference/pull/881 🙏

🐕‍🦺 React on changes in Workflows

We have new [Delta Filter](https://inference.roboflow.com/workflows/blocks/delta_filter/) block that optimizes workflows by triggering downstream steps only when input values change, reducing redundant processing.

📊 Key Features:
* **Value Changes Detection:** Triggers actions only on value changes.
* **Flexibility:** Hooks up to changes in numbers, strings, and more.
* **Per-Video Caching:** Tracks changes using - changes for each video stream or batch element would be traced separately

💡 Use Case:
* Detect changes (e.g., people count) in video analysis and trigger downstream actions efficiently.

🔧 Fixed
* `confidence` threshold was not applied for `multi-label` classification models. grzegorz-roboflow fixed the problem in https://github.com/roboflow/inference/pull/873
* Active Learning Data collection finally works for `multi-label` classification models - see grzegorz-roboflow work in https://github.com/roboflow/inference/pull/874
* Fixed `model_id` bug with InferenceAggregator block by robiscoding in https://github.com/roboflow/inference/pull/876
* Security issue: `nanoid` from 3.3.7 to 3.3.8 - see https://github.com/roboflow/inference/pull/878
* Fix measurement logic for segmentations in measurement block by NickHerrig in https://github.com/roboflow/inference/pull/872


🚧 Changed
* Improve is_mergeable workflow speed by grzegorz-roboflow in https://github.com/roboflow/inference/pull/846
* Additional DetectionsSelection Operations by EmilyGavrilenko in https://github.com/roboflow/inference/pull/879
* Add Block Copy updates and Spellcheck by casmwenger in https://github.com/roboflow/inference/pull/856
* Updated the OPC UA Writer Block Descriptions and Examples by chandlersupple in https://github.com/roboflow/inference/pull/868

New Contributors
* casmwenger made their first contribution in https://github.com/roboflow/inference/pull/856

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.30.0...v0.31.0

🚀 Added

✨ Paligemma2 support!
Enhanced model support: We’re excited to introduce Paligemma2 integration, a next-generation model designed for more flexible and efficient inference. This upgrade facilitates smoother handling of multi-modal inputs like images and captions, offering better versatility in machine learning applications. Check out the implementation details and examples in [this script](https://github.com/roboflow/inference/blob/main/examples/paligemma/paligemma_client.py) to see how to get started.

Change added by probicheaux in https://github.com/roboflow/inference/pull/864

Remaining changes
* Unpin `httpx` by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/861
* Fix docs builder CI by grzegorz-roboflow in https://github.com/roboflow/inference/pull/863
* Add poison pill to connectionstatechange in webrtc by grzegorz-roboflow in https://github.com/roboflow/inference/pull/862
* Bump version of rich by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/867
* Kapa: remove duplicate initialization by LinasKo in https://github.com/roboflow/inference/pull/869
* Add model id output param by robiscoding in https://github.com/roboflow/inference/pull/857

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.29.2...v0.30.0

`ultralytics` security issue fixed

> [!CAUTION]
> Ultralytics maintainers notified the community, that code in the `ultralytics` wheel `8.3.41` is not what's in GitHub and appears to invoke mining. Users of ultralytics who install 8.3.41 will unknowingly execute an xmrig miner.
> Please see [this issue](https://github.com/ultralytics/ultralytics/issues/18027) for more details

Remaining fixes
* python 3.12 support by hansent in https://github.com/roboflow/inference/pull/841
* Pin ultralytics version by bigbitbus in https://github.com/roboflow/inference/pull/858


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.29.1...v0.29.2

🛠️ Fixed

`python-multipart` security issue fixed

> [!CAUTION]
> We are **removing** the following vulnerability detected recently in `python-multipart` library.
>
> **Issue summary**
> When parsing form data, python-multipart skips line breaks (CR `\r` or LF `\n`) in front of the first boundary and any tailing bytes after the last boundary. This happens one byte at a time and emits a log event each time, which may cause excessive logging for certain inputs.
>
> An attacker could abuse this by sending a malicious request with lots of data before the first or after the last boundary, causing high CPU load and stalling the processing thread for a significant amount of time. In case of ASGI application, this could stall the event loop and prevent other requests from being processed, resulting in a denial of service (DoS).
>
> **Impact**
> Applications that use python-multipart to parse form data (or use frameworks that do so) are affected.
>
> **Next steps**
> We advise all `inference` clients to migrate to version `0.29.1`, especially when `inference` docker image is in use. Clients using
> older versions of Python package may also upgrade the vulnerable dependency in their environment:
> bash
> pip install "python-multipart==0.0.19"
>
>
> **Details of the change:** https://github.com/roboflow/inference/pull/855

Remaining fixes
* Fix problem with docs rendering by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/854
* Remove piexif dependency by iurisilvio in https://github.com/roboflow/inference/pull/851


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.29.0...v0.29.1