Inference

🚀 Added

Llama Vision 3.2 🤝 other VLMs supported in Workflows

<table>
<tr>
<td><img src="https://github.com/user-attachments/assets/ff59504a-57d6-4024-a40b-5862a31e4848" /></td>
<td width="50%" align="center"><img src="https://github.com/user-attachments/assets/61f5b741-5963-422e-a449-187f0704ec71" /></td>
<td><img src="https://github.com/user-attachments/assets/327a1808-5752-4888-8837-02332cf743c8" /></td>
</tr>
</table>

We welcome new block bringing [Llama Vision 3.2](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/) to workflows ecosystem!

Llama 3.2 is a new generation of vision and lightweight models that fit on edge devices, tailored for use cases that require more private and personalized AI experiences.

Brought by AHB102 in https://github.com/roboflow/inference/pull/866

Related changes:
* Fix/onboarding llama 3.2 by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/927
* Tests for LLama Vision 3.2 by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/928

MQTT Writer Enterprise Workflow Block (added in https://github.com/roboflow/inference/pull/930)

This block enables our enterprise users to publish messages to an MQTT broker through Workflows.

MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol designed for low-bandwidth, high-latency, or unreliable networks. It's widely used in applications where devices need to communicate with minimal overhead, such as the Internet of Things (IoT).

With this change workflows can communicate with the world through MQTT!

Change introduced by chandlersupple

Plc EthernetIP Enterprise Workflow Block (added in https://github.com/roboflow/inference/pull/905)

This block enables our enterprise users to interface with PLC.
A Programmable Logic Controller (PLC) is an industrial computer specifically designed to automate machinery and processes in manufacturing and other industries. It monitors inputs (e.g., sensors), processes data based on a programmed logic, and controls outputs (e.g., actuators) to perform tasks.
This block is utilizing `pylogix` library over Ethernet/IP. Block supports three modes of operation:
- read: Reads specified tags from the PLC.
- write: Writes specified values to the PLC tags.
- read_and_write: Performs both read and write operations in a single execution.

This change brings vision capabilities into real-world industrial plants!

Change introduced by reedajohns

💪 Improved

Documentation improvements

yeldarby transforms Inference docs with streamlined navigation, styling, and instant rendering!

* [Refresh README](https://github.com/roboflow/inference/pull/908)
* [Update Docs Styling & Nav](https://github.com/roboflow/inference/pull/912)
* [Add Side Nav to Blocks and Kinds Gallery Page](https://github.com/roboflow/inference/pull/910)
* [Add Instant Rendering for Docs](https://github.com/roboflow/inference/pull/917)
* [Docs: Update Links](https://github.com/roboflow/inference/pull/919)
* [Update Landing Page](https://github.com/roboflow/inference/pull/920)

More contributions enhancing Inference documentation:
* better version handling in generated block pages by hansent in https://github.com/roboflow/inference/pull/922
* Update README.md by ThatOrJohn in https://github.com/roboflow/inference/pull/921

Improvements to CI by alexnorell

* [Add readiness endpoint for dedicated deployments](https://github.com/roboflow/inference/pull/909)
* [Support forks](https://github.com/roboflow/inference/pull/924)
* [Remove explicit checkout ref](https://github.com/roboflow/inference/pull/925)


Other changes
* Add env-injectable headers to RF API requests by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/932
* Pass roboflow workflow ID as usage_workflow_id if available by grzegorz-roboflow in https://github.com/roboflow/inference/pull/926
* Collect usage after execution of decorated methods by grzegorz-roboflow in https://github.com/roboflow/inference/pull/931
* Improved the SizeMeasurementBlock Docs by chandlersupple in https://github.com/roboflow/inference/pull/916


New Contributors
* AHB102 made their first contribution in https://github.com/roboflow/inference/pull/866
* ThatOrJohn made their first contribution in https://github.com/roboflow/inference/pull/921

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.32.0...v0.33.0

🚀 Added
👀 Gaze detection in Workflows

<table>
<tr>
<td width="35%"><img src="https://github.com/user-attachments/assets/5e4d1a8f-1b74-495c-afa3-be48d9696d48" /></td>
<td width="65%"><img src="https://github.com/user-attachments/assets/26c65bf3-e44a-404e-8348-318654cf5520" /></td>
</tr>
</table>

joaomarcoscrs (as part of hist first contribution 🏅) introduced Gaze detection model into Workflows.

Don't know what Gaze Detection is?

Gaze detection is a method to determine where a person is looking by analyzing their eye movements and gaze direction. It typically uses cameras or sensors to track eye position and orientation, identifying the point of focus in real time.

It is commonly used in areas like:

* **Human-Computer Interaction**: Controlling devices with eye movements.
* **Behavioral Analysis:** Understanding attention and interest.
* **Marketing Research:** Measuring what catches a person's attention.

Now - you can apply Workflows in such use-cases. Check out Gaze Detection block [📖 documentation](https://inference.roboflow.com/workflows/blocks/gaze_detection/) to find more information.

> [!NOTE]
> The block is currently not supported on Roboflow Hosted Platform. Check out how to [send requests to localhost `inference` server](https://inference.roboflow.com/workflows/modes_of_running/#http-api-request)

🏋️‍♂️ New experimental Workflows blocks enabling new capabilities

yeldarby prepared whole series of blocks to open-up new capabilities for Workflows, including:
* [Workflows Buffer Block](https://inference.roboflow.com/workflows/blocks/buffer/) in https://github.com/roboflow/inference/pull/894
* [Workflows Grid Visualization Block](https://inference.roboflow.com/workflows/blocks/grid_visualization/) in https://github.com/roboflow/inference/pull/895
* Workflow Cache [Get](https://inference.roboflow.com/workflows/blocks/cache_get/)/[Set Blocks](https://inference.roboflow.com/workflows/blocks/cache_set/) in https://github.com/roboflow/inference/pull/893
* Workflows [Outlier Detection Block](https://inference.roboflow.com/workflows/blocks/identify_outliers/) in https://github.com/roboflow/inference/pull/896


💪 Improved

`Florence 2` runs now up to 3x faster
🧙‍♂️ isaacrob-roboflow did some magic 🪄 and now, all of the sudden `Florence2` models deployed in `inference` could run up to 3x faster 🤯 ❗
See details in https://github.com/roboflow/inference/pull/885


🔧 Fixed

Security vulnerability in landing page
We've fixed security issue in `inference` server landing page: https://github.com/roboflow/inference/pull/890

Issue description
If a Next.js application is performing authorization in middleware based on pathname, it was possible for this authorization to be bypassed.
This issue was patched in Next.js `14.2.15` and later.

> [!CAUTION]
> We advise all users of older versions of `inference` server to migrate to version `0.32.0`

Other fixes
* Add fix for the problem with inference-cli workflows predictions saving by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/891
* Improvements in blocks descriptions' by EmilyGavrilenko (https://github.com/roboflow/inference/pull/898) and casmwenger in (https://github.com/roboflow/inference/pull/897)
* Fix usage collector fps by grzegorz-roboflow in https://github.com/roboflow/inference/pull/903

🚧 What's Changed
* Add test to detect blocks with missing __init__.py by grzegorz-roboflow in https://github.com/roboflow/inference/pull/883
* Cache CLIP Text Embeddings in Workflow Block by yeldarby in https://github.com/roboflow/inference/pull/892
* Allow using video metadata for rate limiter on recorded video by yeldarby in https://github.com/roboflow/inference/pull/887
* Serialized owlv2 model by probicheaux in https://github.com/roboflow/inference/pull/889
* Skip additional test by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/902

🏅 New Contributors
* joaomarcoscrs made their first contribution in https://github.com/roboflow/inference/pull/888

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.31.1...v0.32.0

🔧 Fixed

* Fix inference 0.30.0 release by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/882 - just we forgot `__init__.py`


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.31.0...v0.31.1

🚀 Added

📏 Easily create embeddings and compare them in Workflows

Thanks to yeldarby, we have [Clip Embedding](https://inference.roboflow.com/workflows/blocks/clip_embedding_model/) and [Cosine Similarity](https://inference.roboflow.com/workflows/blocks/cosine_similarity/) Workflows blocks. Just take a look what would now be possible.

💡 Application ideas
* **Visual Search:** Match text queries (e.g., "red shoes") to the most relevant images without training a custom model.
* **Image Deduplication:** Identify similar or duplicate images by calculating embeddings and measuring cosine similarity.
* **Zero-Shot Classification:** Classify images into categories by comparing their embeddings to pre-defined text labels (e.g., "cat," "dog").
* **Outliers detection:** Check which images do not match to general trend

<div align="center"><img width="50%" src="https://github.com/user-attachments/assets/33dc9e75-bb65-4697-ba9a-28ae6a5c9265" /></div>

✨ `gemini-2.0-flash` 🤝 Workflows

Check out [model card](https://deepmind.google/technologies/gemini/flash/) and start using new model, simply pointing new model type in [Google Gemini](https://inference.roboflow.com/workflows/blocks/google_gemini/) Workflow block 😄 All thanks to EmilyGavrilenko

<div align="center"><img width="50%" src="https://github.com/user-attachments/assets/06f591a6-ef25-48bc-8743-52dab271fbdb" /></div>

🔥 Recent [`supervision`](https://github.com/roboflow/supervision) versions are now supported

For a long time we had issue with not supporting _up-to-date_ [`supervision`](https://github.com/roboflow/supervision) releases. This is **no longer the case** thanks to LinasKo and his contribution https://github.com/roboflow/inference/pull/881 🙏

🐕‍🦺 React on changes in Workflows

We have new [Delta Filter](https://inference.roboflow.com/workflows/blocks/delta_filter/) block that optimizes workflows by triggering downstream steps only when input values change, reducing redundant processing.

📊 Key Features:
* **Value Changes Detection:** Triggers actions only on value changes.
* **Flexibility:** Hooks up to changes in numbers, strings, and more.
* **Per-Video Caching:** Tracks changes using - changes for each video stream or batch element would be traced separately

💡 Use Case:
* Detect changes (e.g., people count) in video analysis and trigger downstream actions efficiently.

🔧 Fixed
* `confidence` threshold was not applied for `multi-label` classification models. grzegorz-roboflow fixed the problem in https://github.com/roboflow/inference/pull/873
* Active Learning Data collection finally works for `multi-label` classification models - see grzegorz-roboflow work in https://github.com/roboflow/inference/pull/874
* Fixed `model_id` bug with InferenceAggregator block by robiscoding in https://github.com/roboflow/inference/pull/876
* Security issue: `nanoid` from 3.3.7 to 3.3.8 - see https://github.com/roboflow/inference/pull/878
* Fix measurement logic for segmentations in measurement block by NickHerrig in https://github.com/roboflow/inference/pull/872


🚧 Changed
* Improve is_mergeable workflow speed by grzegorz-roboflow in https://github.com/roboflow/inference/pull/846
* Additional DetectionsSelection Operations by EmilyGavrilenko in https://github.com/roboflow/inference/pull/879
* Add Block Copy updates and Spellcheck by casmwenger in https://github.com/roboflow/inference/pull/856
* Updated the OPC UA Writer Block Descriptions and Examples by chandlersupple in https://github.com/roboflow/inference/pull/868

New Contributors
* casmwenger made their first contribution in https://github.com/roboflow/inference/pull/856

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.30.0...v0.31.0

🚀 Added

✨ Paligemma2 support!
Enhanced model support: We’re excited to introduce Paligemma2 integration, a next-generation model designed for more flexible and efficient inference. This upgrade facilitates smoother handling of multi-modal inputs like images and captions, offering better versatility in machine learning applications. Check out the implementation details and examples in [this script](https://github.com/roboflow/inference/blob/main/examples/paligemma/paligemma_client.py) to see how to get started.

Change added by probicheaux in https://github.com/roboflow/inference/pull/864

Remaining changes
* Unpin `httpx` by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/861
* Fix docs builder CI by grzegorz-roboflow in https://github.com/roboflow/inference/pull/863
* Add poison pill to connectionstatechange in webrtc by grzegorz-roboflow in https://github.com/roboflow/inference/pull/862
* Bump version of rich by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/867
* Kapa: remove duplicate initialization by LinasKo in https://github.com/roboflow/inference/pull/869
* Add model id output param by robiscoding in https://github.com/roboflow/inference/pull/857

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.29.2...v0.30.0

`ultralytics` security issue fixed

> [!CAUTION]
> Ultralytics maintainers notified the community, that code in the `ultralytics` wheel `8.3.41` is not what's in GitHub and appears to invoke mining. Users of ultralytics who install 8.3.41 will unknowingly execute an xmrig miner.
> Please see [this issue](https://github.com/ultralytics/ultralytics/issues/18027) for more details

Remaining fixes
* python 3.12 support by hansent in https://github.com/roboflow/inference/pull/841
* Pin ultralytics version by bigbitbus in https://github.com/roboflow/inference/pull/858


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.29.1...v0.29.2