v0.9.14
PawelPeczek-Roboflow
released this
23 Feb 17:40
·
2648 commits
to main
since this release
🚀 Added
LMMs (GPT-4V and CogVLM) 🤝 workflows
Now, with Roboflow workflows
LMMs integration is made easy 💪 . Just look at the demo! 🤯
lmms_in_workflows.mp4
As always, we encourage you to visit workflows
docs 📖 and examples.
This is how to create a multi-functional app with workflows
and LMMs:
inference server start
from inference_sdk import InferenceHTTPClient
LOCAL_CLIENT = InferenceHTTPClient(
api_url="http://127.0.0.1:9001",
api_key=ROBOFLOW_API_KEY,
)
FLEXIBLE_SPECIFICATION = {
"version": "1.0",
"inputs": [
{ "type": "InferenceImage", "name": "image" },
{ "type": "InferenceParameter", "name": "open_ai_key" },
{ "type": "InferenceParameter", "name": "lmm_type" },
{ "type": "InferenceParameter", "name": "prompt" },
{ "type": "InferenceParameter", "name": "expected_output" },
],
"steps": [
{
"type": "LMM",
"name": "step_1",
"image": "$inputs.image",
"lmm_type": "$inputs.lmm_type",
"prompt": "$inputs.prompt",
"json_output": "$inputs.expected_output",
"remote_api_key": "$inputs.open_ai_key",
},
],
"outputs": [
{ "type": "JsonField", "name": "structured_output", "selector": "$steps.step_1.structured_output" },
{ "type": "JsonField", "name": "llm_output", "selector": "$steps.step_1.*" },
]
}
response_gpt = LOCAL_CLIENT.infer_from_workflow(
specification=FLEXIBLE_SPECIFICATION,
images={
"image": cars_image,
},
parameters={
"open_ai_key": OPEN_AI_KEY,
"lmm_type": "gpt_4v",
"prompt": "You are supposed to act as object counting expert. Please provide number of **CARS** visible in the image",
"expected_output": {
"objects_count": "Integer value with number of objects",
}
}
)
🌱 Changed
- developer friendly theming aka dark theme by @onuralpszr in #270 (thanks for contribution 🥇 )
YoloWorld
docs @capjamesg in #276- @ryanjball made their first contribution in #271 with his cookbook for RGB anomaly detection
🔨 Fixed
- turn off instant page for load to cookbook page properly by @onuralpszr in #275 (thanks for contribution 🥇 )
- bug in
workflows
that made cropping in multi-detection set-up
Full Changelog: v0.9.13...v0.9.14