Inference

result = client.infer_from_yolo_world(
inference_input=YOUR_IMAGE,
class_names=["dog", "cat"],
)


Active Learning 🤝 `workflows`
Active Learning data collection made simple with `workflows` 🔥 Now, with just a little bit of configuration you can start data collection to improve your model over time. Just take look how easy it is:

<div align="center">
<video src="https://github.com/roboflow/inference/assets/146137186/06e0b355-51f3-486d-8a5b-07123284b0e9" />
</div>

Key features:
* works for all models supported at Roboflow platform, including the ones from Roboflow Universe - making it trivial to use off-the-shelf model during project kick-off stage to collect dataset while serving meaningful predictions
* combines well with multiple `workflows` blocks - including `DetectionsConsensus` - making it possible to sample based on predictions of models ensemble 💥
* Active Learning block may use project-level config of Active Learning or define Active Learning strategy directly in the block definition (refer to [Active Learning documentation 📖 ](https://github.com/roboflow/inference/blob/main/inference/core/active_learning/README.md) for details on how to configure data collection)

See [documentation 📖 ](https://github.com/roboflow/inference/tree/main/inference/enterprise/workflows#activelearningdatacollector) of new `ActiveLearningDataCollector` to find detailed info.

🌱 Changed
`InferencePipeline` now works with all models supported at Roboflow platform 🎆
For a long time - `InferencePipeline` worked only with object-detection models. This is no longer the case - from now on, other type of models supported at Roboflow platform (including stubs - like `my-project/0`) work under `InferencePipeline`. No changes are required in existing code. Just put `model_id` of your model and the pipeline should work. Sinks suited for detection-only models were adjusted to ignore non-compliant formats of predictions and produce warnings notifying about incompatibility.

🔨 Fixed
* Bug in `yolact` model in https://github.com/roboflow/inference/pull/266

🏆 Contributors
paulguerrie (Paul Guerrie), probicheaux (Peter Robicheaux), PawelPeczek-Roboflow (Paweł Pęczek)


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.9.10...v0.9.11

* **New Kind**: A [secret](https://inference.roboflow.com/workflows/kinds/secret/) kind for credentials is now available. No action needed for existing blocks, but future blocks should use it for secret parameters.

* **Serialization Fix**: Fixed a bug where non-batch outputs weren't being serialized in v1.3.0.

* **Execution Engine Fix**: Resolved an issue with empty inputs being passed to downstream blocks. This update ensures smoother workflow execution and may fix previous issues without any changes needed.

See [full changelog](https://inference.roboflow.com/workflows/execution_engine_changelog/#execution-engine-v140-inference-v0290) for more details.

🚧 Changed

Open Workflows on Roboflow Platform

We are moving towards shareable Workflow Definitions on Roboflow Platform - to reflect that yeldarby made the `api_key` optional in Workflows Run requests in https://github.com/roboflow/inference/pull/843

⛑️ Maintenance
* Update Docker Tag Logic by alexnorell in https://github.com/roboflow/inference/pull/840
* Make check_if_branch_is_mergeable.yml to succeed if merging to main by grzegorz-roboflow in https://github.com/roboflow/inference/pull/848
* Add workflow to check mergeable state executed on pull request by grzegorz-roboflow in https://github.com/roboflow/inference/pull/847

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.28.2...v0.29.0

> [!TIP]
> Changes introduced in Execution Engine `v1.3.0` are non breaking, but we shipped couple of nice extensions and we **encourage** contributors to adopt them.
>
> Full details of the changes and migration guides available [here](https://inference.roboflow.com/workflows/execution_engine_changelog/#execution-engine-v130-inference-v0270).


⚙️ [Kinds](https://inference.roboflow.com/workflows/kinds/) with dynamic serializers and deserializers
* Added serializers/deserializers for each kind, enabling integration with external systems.
* Updated the Blocks Bundling page to reflect these changes.
* Enhanced `roboflow_core` kinds with suitable serializers/deserializers.

See our [updated blocks bundling guide](https://inference.roboflow.com/workflows/blocks_bundling/#serializers-and-deserializers-for-kinds) for more details.


🆓 Any data can be now a Workflow input

We've added new Workflows input type `WorkflowBatchInput` - which is capable of accepting any `kind`, unlike our previous inputs like `WorkflowImage`. What's even nicer - you can also specify dimensionality level for `WorkflowBatchInput` - basically **making it possible to break down each workflow into single-steps executed in *debug* mode**.

Take a look at [📖 docs](https://inference.roboflow.com/workflows/definitions/#generic-batch-oriented-inputs) to learn more

🏋️ Easier blocks development

We got tired wondering if specific field in block manifest should be marked with `StepOutputSelector`, `WorkflowImageSelector`,
`StepOutputImageSelector` or `WorkflowParameterSelector` type annotation. That was **very confusing** and was effectively increasing the difficulty of contributions.

Since the selectors type annotations are **required** for the Execution Engine that block define *placeholders* for data of specific *kind* we could not eliminate those annotations, but we are making them easier to understand - introducing generic annotation called `Selector(...)`.

`Selector(...)` no longer tells Execution Engine that the block accept batch-oriented data - so we replaced old `block_manifest.accepts_batch_input()` method with two new:
* `block_manifest.get_parameters_accepting_batches()` - to return list of params that `WorkflowBlock.run(...)` method
accepts to be wrapped in `Batch[X]` container
* `block_manifest.get_parameters_accepting_batches_and_scalars()` - to return list of params that `WorkflowBlock.run(...)` method
accepts either to be wrapped in `Batch[X]` container or provided as stand-alone scalar values.

> [!TIP]
> To adopt changes while creating new block - visit our updated [blocks creation](https://inference.roboflow.com/workflows/create_workflow_block/) guide.
>
> To migrate existing blocks - take a look at [migration guide](https://inference.roboflow.com/workflows/execution_engine_changelog/#execution-engine-v130-inference-v0270).

🖌️ Increased JPEG compression quality
`WorkflowImageData` has a property called `base64_image` which is auto-generated out from `numpy_image` associated to the object. In the previous version of `inference` - default compression level was `90%` - we increased it to `95%`. We expect that this change will generally improve the quality of images passed between steps, yet there is no guarantee of better results from the models (that depends on how models were trained). Details of change: https://github.com/roboflow/inference/pull/798

> [!CAUTION]
> Small changes in model predictions are expected due to this change - as it may happen that we are passing slightly different JPEG images into the models. If you are negatively affected, please let us know via GH Issues.

🧠 Change in Roboflow models blocks
We've changed the way on how Roboflow models blocks work on Roboflow hosted platform. Previously they were using `numpy_image` property of `WorkflowImageData` as an input to `InferenceHTTPClient` while executing remote calls - which usually caused that we are serialising numpy image to JPEG and then to `base64`, whereas usually on Roboflow hosted platform, we had `base64` representation of image already provided, so effectively we were:
* slowing down the processing
* artificially decreasing the quality of images

This is no longer the case, so we do only transform image representation (and apply lossy compression) when needed. Details of change: https://github.com/roboflow/inference/pull/798.

> [!CAUTION]
> Small changes in model predictions are expected due to this change - as it may happen that we are passing slightly different JPEG images into the models. If you are negatively affected, please let us know via GH Issues.

🗒️ New kind `inference_id`

We've diagnosed the need to give a semantic meaning for inference identifiers that are used by external systems as correlation IDs.
That's why we introduce new kind - [`inference_id`](https://inference.roboflow.com/workflows/kinds/inference_id/).
We encourage blocks developer to use new kind.

🗒️ New field available in `video_metadata` and `image` kinds

We've added new optional field to video metadata - `measured_fps` - take a look at [📖 docs](https://inference.roboflow.com/workflows/internal_data_types/#videometadata)


🏗️ Changed
* Disable telemetry when running YOLO world by grzegorz-roboflow in https://github.com/roboflow/inference/pull/800
* Pass webrtc TURN config as request parameter when calling POST /inference_pipelines/initialise_webrtc by grzegorz-roboflow in https://github.com/roboflow/inference/pull/801
* Remove reset from YOLO settings by grzegorz-roboflow in https://github.com/roboflow/inference/pull/802
* Pin all dependencies and update to new versions of libs by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/803
* bumping owlv2 version and putting cache size in env by isaacrob-roboflow in https://github.com/roboflow/inference/pull/813

🔧 Fixed
* Florence 2 - fixing model caching by probicheaux in https://github.com/roboflow/inference/pull/808
* Use measured fps when fetching frames from live stream by grzegorz-roboflow in https://github.com/roboflow/inference/pull/805
* Fix issue with label visualisation by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/811 and PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/814


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.26.1...v0.27.0

🚀 Added
👀 Gaze detection in Workflows

<table>
<tr>
<td width="35%"><img src="https://github.com/user-attachments/assets/5e4d1a8f-1b74-495c-afa3-be48d9696d48" /></td>
<td width="65%"><img src="https://github.com/user-attachments/assets/26c65bf3-e44a-404e-8348-318654cf5520" /></td>
</tr>
</table>

joaomarcoscrs (as part of hist first contribution 🏅) introduced Gaze detection model into Workflows.

Don't know what Gaze Detection is?

Gaze detection is a method to determine where a person is looking by analyzing their eye movements and gaze direction. It typically uses cameras or sensors to track eye position and orientation, identifying the point of focus in real time.

It is commonly used in areas like:

* **Human-Computer Interaction**: Controlling devices with eye movements.
* **Behavioral Analysis:** Understanding attention and interest.
* **Marketing Research:** Measuring what catches a person's attention.

Now - you can apply Workflows in such use-cases. Check out Gaze Detection block [📖 documentation](https://inference.roboflow.com/workflows/blocks/gaze_detection/) to find more information.

> [!NOTE]
> The block is currently not supported on Roboflow Hosted Platform. Check out how to [send requests to localhost `inference` server](https://inference.roboflow.com/workflows/modes_of_running/#http-api-request)

🏋️‍♂️ New experimental Workflows blocks enabling new capabilities

yeldarby prepared whole series of blocks to open-up new capabilities for Workflows, including:
* [Workflows Buffer Block](https://inference.roboflow.com/workflows/blocks/buffer/) in https://github.com/roboflow/inference/pull/894
* [Workflows Grid Visualization Block](https://inference.roboflow.com/workflows/blocks/grid_visualization/) in https://github.com/roboflow/inference/pull/895
* Workflow Cache [Get](https://inference.roboflow.com/workflows/blocks/cache_get/)/[Set Blocks](https://inference.roboflow.com/workflows/blocks/cache_set/) in https://github.com/roboflow/inference/pull/893
* Workflows [Outlier Detection Block](https://inference.roboflow.com/workflows/blocks/identify_outliers/) in https://github.com/roboflow/inference/pull/896


💪 Improved

`Florence 2` runs now up to 3x faster
🧙‍♂️ isaacrob-roboflow did some magic 🪄 and now, all of the sudden `Florence2` models deployed in `inference` could run up to 3x faster 🤯 ❗
See details in https://github.com/roboflow/inference/pull/885


🔧 Fixed

Security vulnerability in landing page
We've fixed security issue in `inference` server landing page: https://github.com/roboflow/inference/pull/890

Issue description
If a Next.js application is performing authorization in middleware based on pathname, it was possible for this authorization to be bypassed.
This issue was patched in Next.js `14.2.15` and later.

> [!CAUTION]
> We advise all users of older versions of `inference` server to migrate to version `0.32.0`

Other fixes
* Add fix for the problem with inference-cli workflows predictions saving by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/891
* Improvements in blocks descriptions' by EmilyGavrilenko (https://github.com/roboflow/inference/pull/898) and casmwenger in (https://github.com/roboflow/inference/pull/897)
* Fix usage collector fps by grzegorz-roboflow in https://github.com/roboflow/inference/pull/903

🚧 What's Changed
* Add test to detect blocks with missing __init__.py by grzegorz-roboflow in https://github.com/roboflow/inference/pull/883
* Cache CLIP Text Embeddings in Workflow Block by yeldarby in https://github.com/roboflow/inference/pull/892
* Allow using video metadata for rate limiter on recorded video by yeldarby in https://github.com/roboflow/inference/pull/887
* Serialized owlv2 model by probicheaux in https://github.com/roboflow/inference/pull/889
* Skip additional test by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/902

🏅 New Contributors
* joaomarcoscrs made their first contribution in https://github.com/roboflow/inference/pull/888

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.31.1...v0.32.0

🔧 Fixed

* Fix inference 0.30.0 release by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/882 - just we forgot `__init__.py`


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.31.0...v0.31.1

🚀 Added

📏 Easily create embeddings and compare them in Workflows

Thanks to yeldarby, we have [Clip Embedding](https://inference.roboflow.com/workflows/blocks/clip_embedding_model/) and [Cosine Similarity](https://inference.roboflow.com/workflows/blocks/cosine_similarity/) Workflows blocks. Just take a look what would now be possible.

💡 Application ideas
* **Visual Search:** Match text queries (e.g., "red shoes") to the most relevant images without training a custom model.
* **Image Deduplication:** Identify similar or duplicate images by calculating embeddings and measuring cosine similarity.
* **Zero-Shot Classification:** Classify images into categories by comparing their embeddings to pre-defined text labels (e.g., "cat," "dog").
* **Outliers detection:** Check which images do not match to general trend

<div align="center"><img width="50%" src="https://github.com/user-attachments/assets/33dc9e75-bb65-4697-ba9a-28ae6a5c9265" /></div>

✨ `gemini-2.0-flash` 🤝 Workflows

Check out [model card](https://deepmind.google/technologies/gemini/flash/) and start using new model, simply pointing new model type in [Google Gemini](https://inference.roboflow.com/workflows/blocks/google_gemini/) Workflow block 😄 All thanks to EmilyGavrilenko

<div align="center"><img width="50%" src="https://github.com/user-attachments/assets/06f591a6-ef25-48bc-8743-52dab271fbdb" /></div>

🔥 Recent [`supervision`](https://github.com/roboflow/supervision) versions are now supported

For a long time we had issue with not supporting _up-to-date_ [`supervision`](https://github.com/roboflow/supervision) releases. This is **no longer the case** thanks to LinasKo and his contribution https://github.com/roboflow/inference/pull/881 🙏

🐕‍🦺 React on changes in Workflows

We have new [Delta Filter](https://inference.roboflow.com/workflows/blocks/delta_filter/) block that optimizes workflows by triggering downstream steps only when input values change, reducing redundant processing.

📊 Key Features:
* **Value Changes Detection:** Triggers actions only on value changes.
* **Flexibility:** Hooks up to changes in numbers, strings, and more.
* **Per-Video Caching:** Tracks changes using - changes for each video stream or batch element would be traced separately

💡 Use Case:
* Detect changes (e.g., people count) in video analysis and trigger downstream actions efficiently.

🔧 Fixed
* `confidence` threshold was not applied for `multi-label` classification models. grzegorz-roboflow fixed the problem in https://github.com/roboflow/inference/pull/873
* Active Learning Data collection finally works for `multi-label` classification models - see grzegorz-roboflow work in https://github.com/roboflow/inference/pull/874
* Fixed `model_id` bug with InferenceAggregator block by robiscoding in https://github.com/roboflow/inference/pull/876
* Security issue: `nanoid` from 3.3.7 to 3.3.8 - see https://github.com/roboflow/inference/pull/878
* Fix measurement logic for segmentations in measurement block by NickHerrig in https://github.com/roboflow/inference/pull/872


🚧 Changed
* Improve is_mergeable workflow speed by grzegorz-roboflow in https://github.com/roboflow/inference/pull/846
* Additional DetectionsSelection Operations by EmilyGavrilenko in https://github.com/roboflow/inference/pull/879
* Add Block Copy updates and Spellcheck by casmwenger in https://github.com/roboflow/inference/pull/856
* Updated the OPC UA Writer Block Descriptions and Examples by chandlersupple in https://github.com/roboflow/inference/pull/868

New Contributors
* casmwenger made their first contribution in https://github.com/roboflow/inference/pull/856

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.30.0...v0.31.0