v0.24.0

🚀 Added

🎥 Data analysis and export in Workflows

We’re excited to introduce a suite of new blocks to supercharge your video processing and streamline your workflow exports!

✨ What’s New?

• Enhanced Video Analytics: Easily track object counts over time or within specific zones. Want to know how many items appear in a unit of time? ✅ Now you can!
• Custom Notifications: Send automatic emails 📧 when a detected object enters a defined area. Stay informed without manual checks!
• REST API Integration: Effortlessly export your workflow results 🌐. Connect to any REST API and deliver your data wherever you need it.

Our new blocks make your Workflows more powerful and versatile than ever before. Start building smarter workflows today! 🚀

video_analysis_with_workflows.mp4

New blocks

• Data Aggregator: Collects and processes Workflow data to create time-based analytics. 📊 Supports custom aggregation strategies, making it easy to summarize data streams efficiently.
• CSV Formatter: Formats the data into CSV files which can be saved or send as an attachment to in notification block 📚
• Email Notification: Send email notifications 📧 in Workflows.
• Local File Sink: Saves data generated in Workflow runtime into local file 📁
• Webhook SInk: Enables users to integrate their Workflows with REST API to export data or analysis results

Important

At the moment of release, Workflows UI lack some capabilities to display new blocks, but stay tuned - we will fix that shortly

🏷️ Tracking stabiliser block

Ever experienced flickering detections and loosing tracker id? We know this pain - that's why @grzegorz-roboflow prepared Detections Stabilizer block.

376666040-4a259335-ac24-4af0-a1c5-46b4bc27f093.1.mp4

💻 Improvements in stream processing

Each week we are closer and closer to enabling full-blown video processing features - this week @grzegorz-roboflow and @hansent pushed us forward:

• WebRTC streams are now faster - #751
• we have better management of InferencePipelines in inference server - #751

🔧 Fixed

• Loosed typer requirements in #746 fixing the problem raised in #738
• Fixed Google Vision OCR icon by @brunopicinin in #749

🌱 Changed

• Adding more inference metrics to the prometheus scraper by @bigbitbus in #750

Full Changelog: v0.23.0...v0.24.0

v0.23.0

🚀 Added

🕵️ Predictions difference visualization in Workflows 🔎

Ever felt like a detective trying to uncover the subtle differences between model predictions? It used to be hard—sifting through data, struggling to spot where models disagreed. But not anymore! 🔦

The Model Comparison Visualization block makes comparing models effortless by visually highlighting the differences between two models’ predictions.

Don't you think that was a great idea by @yeldarby to create the block?

✨ Just take a look how it works

multiple_models_comparison.mp4

🐎 Compilation cache in Workflows Execution Engine 🚀

If you're familiar with programming, you know the trade-off between compiled and interpreted languages: interpreted ones offer flexibility, but interpreting every line of code can slow things down. To overcome this, interpreters often cache code to speed up execution.

We realized that the Workflows Execution engine was adding unnecessary graph interpretation overhead, even when running the same workflow multiple times. 🕒 In the quest for speed, we’ve introduced compiled Workflow caching, which drastically reduces execution time for repeated workflows! ⚡

When you will be able to see the speed-up? If you send multiple requests to inference server asking to run the same Workflow - you will be impacted 😄 Now your workflows will run faster than ever! 🎯

Full details and benchmark results are available here

📐 Distance measurement in Workflows 📏

Introducing the Distance Measurement block, now available in Workflows! This block calculates the distance between two bounding boxes on a 2D plane using a camera positioned perpendicularly.

🔍 Key Features:

• Measure distance in centimeters or pixels with ease.
• Use either a reference object of known size or a pixel-to-centimeter ratio for accurate scaling.
• Whether you're analyzing footage or comparing object placements, this block gives you precise, scalable distance measurements in your workflows. Try it out and streamline your 2D distance calculations! 📐

Everything thanks to @ediardo 🏅

📊 inference server telemetry

Thanks to @robiscoding, inference server gained ability to export detailed metrics that can be pooled in runtime to ensure better service observability. Check out details.

🗞️ Exciting news

🎥 Video processing with Workflows

The Roboflow team is pushing the boundaries of video processing within Workflows, laying the foundation for powerful new capabilities. You can catch a glimpse of the preliminary changes we've introduced thanks to @grzegorz-roboflow: #719, #743,

Stay tuned as we continue to build and enhance video processing features, making it easier than ever to integrate video into your workflows. 📹✨

🌱 Changed

Deprecation of video_metadata kind

The video_metadata kind has been deprecated. We recommend switching to the updated image kind, which now supports optional video_metadata. We plan to remove video_metadata kind in Execution Engine v2.0.0

While this is non-breaking, some older blocks may become incompatible with future video processing blocks.

With the updated internal representation of image kind, two new methods, WorkflowImageData.copy_and_replace(...) and WorkflowImageData.create_crop(...), have been added for easier metadata manipulation. See the updated usage guide.

From now one, we will also publish list of changes in Workflows Execution Engine in a changelog 📖

Remaining changes

• Add yolov11 aliases to inference_sdk by @probicheaux in #730
• CI changes by @alexnorell in #740, #700
• Update license reference by @emmanuel-ferdman in #741
• Retrieve device stats for docker containers by @robiscoding in #739

🏅 New Contributors

• @alexnorell made their first contribution in #740
• @emmanuel-ferdman made their first contribution in #741
• @ediardo made their first contribution in #731

Full Changelog: v0.22.2...v0.23.0

v0.22.2

What's Changed

• Relax openvino version by @probicheaux in #727
• Bump version by @probicheaux in #728

Full Changelog: v0.22.1...v0.22.2

v0.22.1

Bumps ONNX Runtime requirements to ensure our users can run the new yolov11 models. Also adds some yolov11 documentation to the aliases page.

What's Changed

• yolov11 improvements by @probicheaux in #726

Full Changelog: v0.22.0...v0.22.1

v0.22.0

🚀 Added

🔥 YOLOv11 in inference 🔥

We’re excited to announce that YOLOv11 has been added to inference! 🚀 You can now use both inference and the inference server to get predictions from the latest YOLOv11 model. 🔥

All thanks to @probicheaux and @SolomonLake 🏅

skateboard_yolov11.mov

Try the model ininference Python package

import cv2
from inference import get_model

image = cv2.imread("<your-image>")
model = get_model("yolov11n-640")
predictions = model.infer(image)

print(predictions)

💪 Workflows update

Google Vision OCR in workflows

Thanks to open source contribution from @brunopicinin we have Google Vision OCR integrated into workflow ecosystem. Great to see open source community contribution 🏅

google_vision_ocr.mp4

See 📖 documentation of the new block to explore it's capabilities.

Images stitch Workflow block

📷 Your camera is not able to cover the whole area you want to observe? Don't worry! @grzegorz-roboflow just added the Workflow block which would be able to combine the POV of multiple cameras into a single image that can be further processed in your Workflow.

image 1	image 2	stitched image

📏 Size measurement block

Thanks to @chandlersupple, we can now measure actual size of objects with Workflows! Take a look at 📖 documentation to discover how the block works.

Workflows profiler and Execution Engine speedup 🏇

We've added Workflows Profiler - ecosystem extension to profile the execution of your workflow. It works for inference server requests (both self-hosted and on Roboflow platform) as well as for InferencePipeline.

The cool thing about profiler is that it is compatible with chrome://tracing - so you can easily grab profiler output and render it in Google Chrome browser.

To profile your Workflow execution use the following code snippet - traces are saved in ./inference_profiling directory by default.

from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="https://detect.roboflow.com",
    api_key="<YOUR-API-KEY>"
)
results = client.run_workflow(
    workspace_name="<your-workspace>",
    workflow_id="<your-workflow-id>",
    images={
        "image": "<YOUR-IMAGE>",
    },
    enable_profiling=True,
)

See detailed report regarding speed optimisations in the PR #710

❗ Important note

As part of speed optimisation we enabled server-side caching for workflows definitions saved on Roboflow Platform - if you frequently change and your Workflow, to see results immediately you need to specify use_cache=False parameter of client.run_workflow(...) method

🔧 Fixed

• Fix prometheus scraping by @robiscoding in #712
• Fix the problem with VLMs on batch inference by @PawelPeczek-Roboflow in #718

🌱 Changed

• Docker auto-reload configuration by @EmilyGavrilenko in #703
• Multi-Label Classification UQL Operations by @EmilyGavrilenko in #714
• Add port forward to Notebook Landing Page message by @hansent in #711
• Add optional descriptions to dynamic blocks by @EmilyGavrilenko in #702
• Add descriptions to task types in VLM as Detector by @PawelPeczek-Roboflow in #704
• Add tests for Google Vision OCR by @PawelPeczek-Roboflow in #715
• Improvements regarding custom python blocks by @PawelPeczek-Roboflow in #716

🏅 New Contributors

We do want to honor @brunopicinin who made their first contribution to inference in #709 as a part of Hacktoberfest 2024. We invite other open-source community members to contribute 😄

Full Changelog: v0.21.1...v0.22.0

v0.21.1

What's Changed

• Improvements in contribution guides by @PawelPeczek-Roboflow in #691
• Fix issue with SAM2 producing segmentation masks as points and lines by @PawelPeczek-Roboflow in #697

Full Changelog: v0.21.0...v0.21.1

v0.21.0

🚀 Added

👩‍🎨 Become an artist with Workflows 👨‍🎨

Ever wanted to be an artist but felt like you lacked the skills? No worries! We’ve just added the StabilityAI Inpainting block to the Workflows ecosystem. Now, you can effortlessly add whatever you envision into your images! 🌟🖼️

Credits to @Fafruch for origin idea 💪

inpainting_stability_ai_demo.mp4

📖 docs

• Model 📖 docs
• Workflow block 📖 docs

🤯 Workflows + video + inference server - Experimental feature preview 🔬

Imagine creating a Workflow in our UI, tuning it to understand what happens in your video. So far, video processing with InferencePipeline required a bit of glue code and setup in your environment. We hope that soon, you won’t need any custom scripts for video processing! You’ll be able to ship your Workflow directly to the inference server, just pointing which video source to process.

We're thrilled to announce that we’ve taken the first step toward making this idea a reality! Check out our experimental feature for video processing, now controlled by the inference server with a user-friendly REST API for easy integration.

video_processing_behind_api.mp4

🔍 We encourage you to try it out! The feature is available in the inference server Docker images that you can self-host. Please note that this feature is experimental, and breaking changes are to be expected. Check out our 📖 docs to learn more.

🙃 Flips, Rotations and Resizing in Workflows

Tired of dealing with image orientation problems while building demos with Workflows? Whether it's resizing, rotating, or flipping, those headaches end today with our new features for seamless image adjustments!

All thanks to @EmilyGavrilenko and PR #683

✨ Ensure Your Tracked Objects Stay on Course! 🛰️

Wondering if the objects you're tracking follow the path you intended? We’ve got you covered in Workflows! Thanks to @shantanubala, we now offer Fréchet Distance Analysis as a Workflow block. Simply specify the desired trajectory, and Workflow calculates the deviation for each tracked box. 📊
See details: #682

What’s Fréchet Distance?
It’s a mathematical measure that compares the similarity between two curves—perfect for analyzing how closely your tracked objects follow the path you’ve set.

🆕 Background removal in Dynamic Crop and updated UI for VLMs

Let’s be honest—VLMs in Workflows still had room for improvement, especially when integrating model outputs with other blocks. Well, we've made it better! 🎉 Now, each model task comes with a clear description and a suggested parser to follow the block, helping you get the most out of your model predictions with ease. 🛠️

Additionally, you can now remove background while performing Dynamic Crop on Instance Segmentation model results 🤯

CleanShot.2024-09-26.at.18.38.33.mp4

🔧 Fixed

• Fix lora by @probicheaux in #676
• Add ability to parametrise class remap UQL operation by @PawelPeczek-Roboflow in #674
• Fix license link in docs by @NickHerrig in #680
• Add transformers extra to dev setup py by @PawelPeczek-Roboflow in #678

🌱 Changed

• gstreamer backend in inference by @PawelPeczek-Roboflow in #646
• Add workflow schema api docs by @NickHerrig in #677
• Add Florence-2 📖 documentation by @capjamesg in #626

🏅 New Contributors

• @shantanubala made their first contribution in #682

Full Changelog: v0.20.1...v0.21.0

v0.20.1

What's Changed

• Fix workflows gallery previews by @PawelPeczek-Roboflow in #668
• Add option to reset timer when detection falls outside of zone by @grzegorz-roboflow in #671
• Line counter improvements by @EmilyGavrilenko in #670

Full Changelog: v0.20.0...v0.20.1

v0.20.0

🚀 Added

🌟 Florence 2 🤝 Workflows

Thanks to @probicheaux, the Workflows ecosystem just got better with the addition of the Florence 2 block. Florence 2, one of the top open-source releases this year, is a powerful Visual Language Model capable of tasks like object detection, segmentation, image captioning, OCR, and more. Now, you can use it directly in your workflows!

Florence 2 and SAM 2 - zero shot grounded segmentation

Ever wished for precise segmentation but didn’t have the data to train your model? Now you don’t need it! With Florence 2 and SAM 2, you can achieve stunning segmentation results effortlessly — without a single annotation.

Discover how to combine these powerful models and get top-tier segmentation quality for free!

florence2_and_sam2.mp4

Florence 2 as OCR model

Need Text Layout Detection and OCR? Florence 2 Has You Covered!

florence2_with_ocr.mp4

Zero-shot object detection needed?

Do not hesitate and try out Florence 2 as object detection model - the quality of results is surprisingly good 🔥

florence2_object_detection.mp4

🔔 Additional notes

• Florence 2 requires either Roboflow Dedicated Deployment or self-hosted inference server - it is not available on Roboflow Hosted Platform
• To discover full potential of Florence 2 - read the paper
• Visit 📖 documentation of Florence 2 Workflow block

New version of SIFT block

Tired using SIFT descriptors calculation block followed by SIFT comparison? This is no longer needed. Check out SIFT Comparison v2 block. PR: #657

Workflows UQL extended with new operations

You may not even be aware, but Universal Query Language powers Workflows operations that can be fully customised in UI. There are two new features shipped:

• selecting prediction by confidence by @EmilyGavrilenko in #655
• class names remapping operation by @NickHerrig in #656

Instance Segmentation ⏩ oriented rectangle

Thanks to @chandlersupple, Instance Segmentation results can be turned into oriented bounding boxes - check out 📖 docs

🔧 Fixed

• Broken links removed from docs in #663
• Fixes to release 0.19.0:
  • broken visualisation blocks got repaired: by @grzegorz-roboflow in #662
  • proper defaults brought into line counter and time in zone analytics blocks by @grzegorz-roboflow in #658

🌱 Changed

• Control what properties are visible in the UI by default by @EmilyGavrilenko in #659
• E2E tests for describe workflow and examples by @PawelPeczek-Roboflow in #652
• Add secrets to integration tests by @PawelPeczek-Roboflow in #667

Full Changelog: v0.19.0...v0.20.0

v0.19.0

🚀 Added

🎥 Video processing in workflows 🤯

We’re excited to announce that, thanks to the contributions of @grzegorz-roboflow, our Workflows ecosystem now extends to video processing! Dive in and explore the new possibilities:

dwell_time_demo.mp4

New blocks:

• Time In Zone to analyse dwell time
• Line Counter to detect objects passing line
• Visualisation for zone and line counter

We've introduced minimal support for video processing in the Workflows UI, with plans to expand to more advanced features soon. To get started, you can create a Python script using the InferencePipeline, similar to the provided example.

Video source YT | Karol Majek

🔥 OWLv2 🤝 inference

Thanks to @probicheaux we have OWLv2 model in inference. OWLv2 was primarily trained to detect objects from text. The implementation in Inference currently only supports detecting objects from visual examples of that object.

You can use model in inference server - both CPU and GPU, as well as in Python package. Visit our 📖 docs to learn more.

Screen.Recording.2024-09-19.at.21.36.13.mov

👓 TROCR 🤝 inference

@stellasphere shipped TROCR model to expand OCR models offering in inference 🔥

You can use model in inference server - both CPU and GPU, as well as in Python package. Visit our 📖 docs to learn more.

🧑‍🎓 Workflows - endpoint to discover interface

Guessing the data format for Workflow inputs and outputs was a challange as for now, but thanks to @EmilyGavrilenko this is no longer the case. We offer two new endpoints (for workflows registered on the platform and for workflows submitted in payload). Details in #644.

🔔 Example response

{
    "inputs": {
        "image": ["image"],
        "model_id": ["roboflow_model_id"],
    },
    "outputs": {
        "detections": ["object_detection_prediction"],
         "crops": ["image"],
         "classification": {
            "inference_id": ["string"],
            "predictions": ["classification_prediction"],
         },
    },
    "typing_hints": {
        "image": "dict",
        "roboflow_model_id": "str",
        "object_detection_prediction": "dict",
        "string": "str",
        "classification_prediction": "dict",
    },
    "kinds_schemas": {
        "image": {},
        "object_detection_prediction": {"dict": "with OpenAPI 3.0 schema of result"},
        "classification_prediction": {"dict": "with OpenAPI 3.0 schema of result"}
    }
}

🌱 Changed

• Bring back transformers extras by @probicheaux in #639
• Move xformers to paligemma code by @probicheaux in #641
• Cache plan details by @grzegorz-roboflow in #636
• Bump next from 14.2.7 to 14.2.12 in /inference/landing by @dependabot in #649

🔧 Fixed

• Fixed bug with Workflows Execution Engine causing bug when conditional execution discards inputs of a step that changes dimensionality - see details in #645

♻️ Removed

• Remove unmaintained device management code by @robiscoding in #647

Full Changelog: v0.18.1...v0.19.0