Inference

result = client.infer_from_yolo_world(
inference_input=YOUR_IMAGE,
class_names=["dog", "cat"],
)


Active Learning 🤝 `workflows`
Active Learning data collection made simple with `workflows` 🔥 Now, with just a little bit of configuration you can start data collection to improve your model over time. Just take look how easy it is:

<div align="center">
<video src="https://github.com/roboflow/inference/assets/146137186/06e0b355-51f3-486d-8a5b-07123284b0e9" />
</div>

Key features:
* works for all models supported at Roboflow platform, including the ones from Roboflow Universe - making it trivial to use off-the-shelf model during project kick-off stage to collect dataset while serving meaningful predictions
* combines well with multiple `workflows` blocks - including `DetectionsConsensus` - making it possible to sample based on predictions of models ensemble 💥
* Active Learning block may use project-level config of Active Learning or define Active Learning strategy directly in the block definition (refer to [Active Learning documentation 📖 ](https://github.com/roboflow/inference/blob/main/inference/core/active_learning/README.md) for details on how to configure data collection)

See [documentation 📖 ](https://github.com/roboflow/inference/tree/main/inference/enterprise/workflows#activelearningdatacollector) of new `ActiveLearningDataCollector` to find detailed info.

🌱 Changed
`InferencePipeline` now works with all models supported at Roboflow platform 🎆
For a long time - `InferencePipeline` worked only with object-detection models. This is no longer the case - from now on, other type of models supported at Roboflow platform (including stubs - like `my-project/0`) work under `InferencePipeline`. No changes are required in existing code. Just put `model_id` of your model and the pipeline should work. Sinks suited for detection-only models were adjusted to ignore non-compliant formats of predictions and produce warnings notifying about incompatibility.

🔨 Fixed
* Bug in `yolact` model in https://github.com/roboflow/inference/pull/266

🏆 Contributors
paulguerrie (Paul Guerrie), probicheaux (Peter Robicheaux), PawelPeczek-Roboflow (Paweł Pęczek)


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.9.10...v0.9.11

* **New Kind**: A [secret](https://inference.roboflow.com/workflows/kinds/secret/) kind for credentials is now available. No action needed for existing blocks, but future blocks should use it for secret parameters.

* **Serialization Fix**: Fixed a bug where non-batch outputs weren't being serialized in v1.3.0.

* **Execution Engine Fix**: Resolved an issue with empty inputs being passed to downstream blocks. This update ensures smoother workflow execution and may fix previous issues without any changes needed.

See [full changelog](https://inference.roboflow.com/workflows/execution_engine_changelog/#execution-engine-v140-inference-v0290) for more details.

🚧 Changed

Open Workflows on Roboflow Platform

We are moving towards shareable Workflow Definitions on Roboflow Platform - to reflect that yeldarby made the `api_key` optional in Workflows Run requests in https://github.com/roboflow/inference/pull/843

⛑️ Maintenance
* Update Docker Tag Logic by alexnorell in https://github.com/roboflow/inference/pull/840
* Make check_if_branch_is_mergeable.yml to succeed if merging to main by grzegorz-roboflow in https://github.com/roboflow/inference/pull/848
* Add workflow to check mergeable state executed on pull request by grzegorz-roboflow in https://github.com/roboflow/inference/pull/847

**Full Changelog**: https://github.com/roboflow/inference/compare/v0.28.2...v0.29.0

> [!TIP]
> Changes introduced in Execution Engine `v1.3.0` are non breaking, but we shipped couple of nice extensions and we **encourage** contributors to adopt them.
>
> Full details of the changes and migration guides available [here](https://inference.roboflow.com/workflows/execution_engine_changelog/#execution-engine-v130-inference-v0270).


⚙️ [Kinds](https://inference.roboflow.com/workflows/kinds/) with dynamic serializers and deserializers
* Added serializers/deserializers for each kind, enabling integration with external systems.
* Updated the Blocks Bundling page to reflect these changes.
* Enhanced `roboflow_core` kinds with suitable serializers/deserializers.

See our [updated blocks bundling guide](https://inference.roboflow.com/workflows/blocks_bundling/#serializers-and-deserializers-for-kinds) for more details.


🆓 Any data can be now a Workflow input

We've added new Workflows input type `WorkflowBatchInput` - which is capable of accepting any `kind`, unlike our previous inputs like `WorkflowImage`. What's even nicer - you can also specify dimensionality level for `WorkflowBatchInput` - basically **making it possible to break down each workflow into single-steps executed in *debug* mode**.

Take a look at [📖 docs](https://inference.roboflow.com/workflows/definitions/#generic-batch-oriented-inputs) to learn more

🏋️ Easier blocks development

We got tired wondering if specific field in block manifest should be marked with `StepOutputSelector`, `WorkflowImageSelector`,
`StepOutputImageSelector` or `WorkflowParameterSelector` type annotation. That was **very confusing** and was effectively increasing the difficulty of contributions.

Since the selectors type annotations are **required** for the Execution Engine that block define *placeholders* for data of specific *kind* we could not eliminate those annotations, but we are making them easier to understand - introducing generic annotation called `Selector(...)`.

`Selector(...)` no longer tells Execution Engine that the block accept batch-oriented data - so we replaced old `block_manifest.accepts_batch_input()` method with two new:
* `block_manifest.get_parameters_accepting_batches()` - to return list of params that `WorkflowBlock.run(...)` method
accepts to be wrapped in `Batch[X]` container
* `block_manifest.get_parameters_accepting_batches_and_scalars()` - to return list of params that `WorkflowBlock.run(...)` method
accepts either to be wrapped in `Batch[X]` container or provided as stand-alone scalar values.

> [!TIP]
> To adopt changes while creating new block - visit our updated [blocks creation](https://inference.roboflow.com/workflows/create_workflow_block/) guide.
>
> To migrate existing blocks - take a look at [migration guide](https://inference.roboflow.com/workflows/execution_engine_changelog/#execution-engine-v130-inference-v0270).

🖌️ Increased JPEG compression quality
`WorkflowImageData` has a property called `base64_image` which is auto-generated out from `numpy_image` associated to the object. In the previous version of `inference` - default compression level was `90%` - we increased it to `95%`. We expect that this change will generally improve the quality of images passed between steps, yet there is no guarantee of better results from the models (that depends on how models were trained). Details of change: https://github.com/roboflow/inference/pull/798

> [!CAUTION]
> Small changes in model predictions are expected due to this change - as it may happen that we are passing slightly different JPEG images into the models. If you are negatively affected, please let us know via GH Issues.

🧠 Change in Roboflow models blocks
We've changed the way on how Roboflow models blocks work on Roboflow hosted platform. Previously they were using `numpy_image` property of `WorkflowImageData` as an input to `InferenceHTTPClient` while executing remote calls - which usually caused that we are serialising numpy image to JPEG and then to `base64`, whereas usually on Roboflow hosted platform, we had `base64` representation of image already provided, so effectively we were:
* slowing down the processing
* artificially decreasing the quality of images

This is no longer the case, so we do only transform image representation (and apply lossy compression) when needed. Details of change: https://github.com/roboflow/inference/pull/798.

> [!CAUTION]
> Small changes in model predictions are expected due to this change - as it may happen that we are passing slightly different JPEG images into the models. If you are negatively affected, please let us know via GH Issues.

🗒️ New kind `inference_id`

We've diagnosed the need to give a semantic meaning for inference identifiers that are used by external systems as correlation IDs.
That's why we introduce new kind - [`inference_id`](https://inference.roboflow.com/workflows/kinds/inference_id/).
We encourage blocks developer to use new kind.

🗒️ New field available in `video_metadata` and `image` kinds

We've added new optional field to video metadata - `measured_fps` - take a look at [📖 docs](https://inference.roboflow.com/workflows/internal_data_types/#videometadata)


🏗️ Changed
* Disable telemetry when running YOLO world by grzegorz-roboflow in https://github.com/roboflow/inference/pull/800
* Pass webrtc TURN config as request parameter when calling POST /inference_pipelines/initialise_webrtc by grzegorz-roboflow in https://github.com/roboflow/inference/pull/801
* Remove reset from YOLO settings by grzegorz-roboflow in https://github.com/roboflow/inference/pull/802
* Pin all dependencies and update to new versions of libs by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/803
* bumping owlv2 version and putting cache size in env by isaacrob-roboflow in https://github.com/roboflow/inference/pull/813

🔧 Fixed
* Florence 2 - fixing model caching by probicheaux in https://github.com/roboflow/inference/pull/808
* Use measured fps when fetching frames from live stream by grzegorz-roboflow in https://github.com/roboflow/inference/pull/805
* Fix issue with label visualisation by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/811 and PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/814


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.26.1...v0.27.0

What's Changed
* Fix issue with detections filter and perspective transformation by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/992
* Apply changes to filtering by parent class by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/993


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.36.0...v0.36.1

🚀 Added

🏋 Workflows Batch Processing in Roboflow cloud

We’re thrilled to introduce *early access* for **Workflows Batch Processing in Roboflow Cloud**! Now, you can process entire directories of images or video files using your Workflows - no coding required!

⚡ Feature characteristics
* **Asynchronous batch processing** – You kick of a job that run in the background, and you can retrieve results later
* **High throughput** - Feature is designed to process a lot of data - so if your machine is not capable enough to process the data you have - this solution may be for you 🎯
* **Fully managed** – Runs on our cloud, so you don’t have to worry about infrastructure.
* **No UI yet, but CLI ready** – Use the updated `inference-cli` to access the feature.

📺 Check out the demo to see it in action!

This is an alpha release, and we’re inviting a small group of early adopters to test it out. Let us know if you’d like to be among the first to try it! 🚀

https://github.com/user-attachments/assets/03553826-458c-4a01-89a8-9bc25d464ba7


🔍 Select Bounding Boxes Contained Within a Specific Class

We’ve heard your feedback! A common challenge in object detection is distinguishing objects contained within another object - and we’ve got you covered.

For example:
* 🚗 People inside a car vs. standing on the street
* 📦 Products on a store shelf vs. misplaced items
* 🔍 Defects within a specific component vs. background noise

We’re excited to announce a new Workflows operation that lets you filter bounding boxes based on spatial containment within a selected class. This makes it easier than ever to refine detections and extract the insights that matter most!


https://github.com/user-attachments/assets/7ad65705-7bde-40fb-86f7-960964a25c77


Check the details [here](https://github.com/roboflow/inference/pull/981)

*Auto* mode in Stitch OCR Detections Block

Previously, users needed to manually specify the direction of text (e.g., `left-to-right`, `top-to-bottom`) before stitching. With the new "auto" mode, text direction is automatically detected as either left-to-right or top-to-bottom.


Check the details of chandlersupple change [here](https://github.com/roboflow/inference/pull/990)

💪 Improved

* Handle paligemma through new getWeights endpoint by grzegorz-roboflow in https://github.com/roboflow/inference/pull/980
* Add optional language_hints input parameter to be passed as imageContext.languageHints by grzegorz-roboflow in https://github.com/roboflow/inference/pull/982
* Prevent pipeline manager from entering infinite loop if workflow could not be parsed by grzegorz-roboflow in https://github.com/roboflow/inference/pull/983
* When stream output received in camera streaming payload is not WorkflowImageData, select first available WorkflowImageData output by grzegorz-roboflow in https://github.com/roboflow/inference/pull/984
* Add 'longer' and 'shorter' keys to the output of size_measurement block by grzegorz-roboflow in https://github.com/roboflow/inference/pull/987
* When running bounding_rect update mask and xyxy of source sv.Detections so results can be visualized by grzegorz-roboflow in https://github.com/roboflow/inference/pull/986
* remove print from cli docs generation script by hansent in https://github.com/roboflow/inference/pull/978
* Extend query_language DetectionsProperty to cover sv.Detections data keys produced by blocks by grzegorz-roboflow in https://github.com/roboflow/inference/pull/985
* Simplify dedicated deployments authorizer middleware to rely on workspace ID match by grzegorz-roboflow in https://github.com/roboflow/inference/pull/988
* Fix platform tests by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/991
* HotFix for `aiortc` release breaking the build by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/989


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.35.0...v0.36.0

🚀 Added

PLC Modbus TCP Block

This new block brings Modbus TCP communication to Workflows. The block uses the [pymodbus](https://pypi.org/project/pymodbus/) library to read from and write to PLC registers over Modbus TCP. It supports three modes of operation:

read: Reads specified registers from a PLC.
write: Writes specified values to PLC registers.
read_and_write: Performs both reading and writing operations in one execution.

Adding Modbus TCP support allows workflows to interact with a broader range of PLCs and industrial devices that use the Modbus protocol.

* Change by reedajohns in https://github.com/roboflow/inference/pull/951

Velocity block

New analytics block was added to Workflows to calculate and embed velocities and speeds of tracked objects across video frames.

Key Features:

- Velocity Calculation: Computes raw and smoothed velocities based on object movements between frames.
- Speed Calculation: Determines the magnitude of velocity vectors to obtain speed.

This block supports smoothing of velocity measurements using an exponential moving average with configurable `smoothing_alpha`.
Incorporated unit conversion from pixels to meters via the `pixels_per_meter parameter`

* Change by reedajohns in https://github.com/roboflow/inference/pull/754

Add support to run ResNet Classification Model in Inference

The recent update to the Roboflow Inference repository introduces support for ResNet classification models, as described in the seminal paper *"Deep Residual Learning for Image Recognition"* (He et al., 2015, [[arXiv:1512.03385](https://arxiv.org/abs/1512.03385)](https://arxiv.org/abs/1512.03385)). This integration enables users to leverage the powerful ResNet architecture for image classification tasks, enhancing the model options available within the inference engine and expanding its utility for diverse computer vision applications.

* Change by Matvezy in https://github.com/roboflow/inference/pull/959

💪 Improved
Multiple changes enhancing Inference documentation!
* Docs: /start by yeldarby in https://github.com/roboflow/inference/pull/936
* Block gallery refactor by hansent in https://github.com/roboflow/inference/pull/955
* fix requirement command in doc build workflow by hansent in https://github.com/roboflow/inference/pull/960
* Docs: Fix Typo by yeldarby in https://github.com/roboflow/inference/pull/963
* Change "fire" to "run" in rate limiter description by capjamesg in https://github.com/roboflow/inference/pull/968

Improvements to Workflows Blocks
* Detection Offset - Use bbox for percent padding by sberan in https://github.com/roboflow/inference/pull/970
* Fix/perspective correction input accept output of dynamic zones by grzegorz-roboflow in https://github.com/roboflow/inference/pull/972
* Fix Google Vision Block bounding box parsing by brunopicinin in https://github.com/roboflow/inference/pull/961

Security improvements
* Added robots noindex meta tag by bigbitbus in https://github.com/roboflow/inference/pull/964
* Fix/improve security in the inference server start command by bigbitbus in https://github.com/roboflow/inference/pull/940

Improvements to webcam workflow preview
* Send raw image over WebRTC if no visualization is available by grzegorz-roboflow in https://github.com/roboflow/inference/pull/967
* Handle wildcard outputs in webrct by grzegorz-roboflow in https://github.com/roboflow/inference/pull/979

Other changes
* Add changes to enable indexing video processing results in workflows CLI by PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/962
* send api key for model registry to make it work on dedicated deployments by hansent in https://github.com/roboflow/inference/pull/974
* Add logging when workflow ID is known and internal workflow ID was not passed by grzegorz-roboflow in https://github.com/roboflow/inference/pull/965
* Copy internal workflow id to specification before it's cached by grzegorz-roboflow in https://github.com/roboflow/inference/pull/966
* Suppress ONNX providers warning by grzegorz-roboflow in https://github.com/roboflow/inference/pull/969


**Full Changelog**: https://github.com/roboflow/inference/compare/v0.34.0...v0.35.0