Note: The list below is currently in progress
A collection of useful documentation, tools, code snippets and articles for writing Gstreamer applications. Anyone deep enough in the gstreamer ecosystem knows how difficult it can be to find up to date documentation or examples on specific plugins/tools. Hopefully this guide can give you a hand navigating this space.
⚙️ Feel free to update the list with any resources you might have with a PR ⚙️
- Start Here
- Primary Sources
- Common Questions
- Observability
- Debugging Pipelines
- Profiling and Optimisation
- Testing
- Nvidia Deepstream
- IDE / Environment
Todo
Prepare to read a lot of plugin implementations and examples. Although there is a fair bit of documentation, it won't answer a lot of questions you will have about specifics. My usual workflow for findings answers is:
95% of your questions can be answered by these
For high level design overviews and core/custom plugin documentation. This should usually be your first point of call. Useful docs:
- Basic concepts (pads, elements, pipelines etc...)
- Architecture and design overview
- Tutorial (in c) or rust port
- API reference
- Plugin reference
Command line access to plugin documentation and allows you to query your own custom plugins to check:
- Description
- Pad capabilities
- Properties
- Signals
This will only work for plugins you have installed (see: installing a custom plugin in rust or in c).
This repository, as well as gst-plugins-rs are, imo the most useful references for writing your own plugins. @sdroege has done a momentous job building these bindings and writing excellent tutorials/documentation. I frequently visit the plugin examples and find the following resources extremely useful for newcomers and experienced gst devs alike:
- Examples of gst api usage from rust
- Installing the bindings
- Gstreamer tutorial code rewritten in rust
- Rust docs
Contains endless plugin examples, a great tutorial and is constantly active. Core maintainers are very reponsive if you have any questions not covered in their examples/docs.
Messages are passed directly to the applications message bus. Events are passed up/down the pipeline and can trigger events.
For example, a QOS event send out by a sink at the end of your pipeline, could be received by a decoder who would send a Message to the message bus with stats on how many frames it has processed and dropped.
Tracers are extremely useful for monitoring pipeline throughput, element latency, cpu usage and a variety of other metrics.
-
- Not great but has some good examples of usage. These will work directly with a rust application.
-
- Better formatted logs using
fmttracing chrometracerfor pipeline profiling. Use a tool like perfetto to visualise the results
- Better formatted logs using
-
- Set of plugins that extend/replace some of the core gstreamer ones. Very well documented.
-
- have been unable to get this working
- [todo - write up on other tools]
- Measure framerate, bitrate and CPU usage
-
Define
GST_DEBUG_DUMP_DOT_DIRenvironment variable -
Upcast pipeline to
gst::Elementthen callelement.debug_to_dot_file(). Example usage from gstreamer-rs playbin example:
let bin_ref = playbin.downcast_ref::<gst::Bin>().unwrap();
bin_ref.debug_to_dot_file(gst::DebugGraphDetails::all(), "PLAYING");
Easy pipeline image generation
- todo: check works in non-gst-launch environment
Used to set the gstreamer specific log levels. Can log everything from basic errors/info logs to every event and message on the event bus.
These can be easily viewed using the debug-viewer. A small app for filtering and sorting through GST logs.
TODO
todo - debugging todo - gstd todo - gstd2
"GStreamer Daemon is gst-launch on steroids"
todo
A lot of your time will be spent reading, compiling and testing c/c++. I would suggest setting up your IDE to enable quick definition/ reference jumping (the promise of rust bindings doesn't let you escape cpp completely unfortunately).
todo: vscode devcontainer setup for gstreamer/nvidia todo: nvim container setup