Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve torchcodec's performance for small video sizes #192

Open
ahmadsharif1 opened this issue Aug 26, 2024 · 1 comment
Open

Improve torchcodec's performance for small video sizes #192

ahmadsharif1 opened this issue Aug 26, 2024 · 1 comment
Assignees

Comments

@ahmadsharif1
Copy link
Contributor

I ran some benchmarks with smaller videos and found torchcodec's sampler is slower than torchvision's sampler. I then did some improvements that got me a 5x or so performance improvement over the current code. I will be submitting PRs for the next few weeks but I wanted to document inefficiencies here:

  1. filtergraph takes a while to setup -- we could either share the filtergraph across instances or use libswscale to do the color conversion.
  2. Batch decoding is slow because we incur an extra frame memcpy after doing the actual decoding.
  3. Temp IO buffer is too big -- we could get away with a few KB of allocation, not MBs like we currently do.
  4. For use cases that need approximate frames we don't need to scan the file. We can just trust the header and get approximate frames that way using the core API.

With all these improvements I got the per-batch sampler time to go from about 200ms per batch to about 37ms per batch when sampling with frames_per_clip=4, clips_per_video=1, dilation=2 and video sizes were all under 2MB each.

image

(Original time was about 200ms per frame -- so the total gain is more than 5x for short videos).

@ahmadsharif1 ahmadsharif1 self-assigned this Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
@ahmadsharif1 and others