Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could you check if I'm implementing it correctly? #22

Open
ksw29zz opened this issue May 30, 2024 · 1 comment
Open

Could you check if I'm implementing it correctly? #22

ksw29zz opened this issue May 30, 2024 · 1 comment

Comments

@ksw29zz
Copy link

ksw29zz commented May 30, 2024

Thank you always for your amazing work! :)

I have a few questions, so I wanted to talk to you about them.

  1. The mouth seems to have a low resolution. I wonder if this is something I need to train to improve.
  2. When small and large faces appear alternately in the video, it looks like the video is glitching. I wonder if I'm doing something wrong here.
  3. I have attached a video, and in this particular video, it can't seem to detect the face. Could you check it out for me?
    video_and_audio.zip
@Inferencer
Copy link
Owner

  1. I will add up-scaling of reference frames as an option and make conversions and encodings slower to increase quality I could train to double the resolution but that's a long way away.
    To be clear though this tool works phenomenally with the correct source footage as it was trained on studio lighting with static cameras so news readers are what works best but if you search for celebs reading mean tweets you will find some good vids as the lighting is great, the background is blue and the resolution of the teeth is good due to the high quality recording. however they do look down at their laptops a lot so will be best to clip them so the face is looking at the camera at all times.

  2. I recently changed the crop radius to be consistent so when the face is overlapped onto the original video there is no flickering but this can cause issues if the face is moving back and fourth or the camera is too much, in your circumstance of small and large faces appearing alternately n the source footage.. I will implement a scene change detection so we can use different crop radius's for each scene as well as different custom ref frames.

  3. The vid you shared appears to be ok in terms of the face in each frame however I wonder if the fps is the cause of the issue as when it converts to 25fps it maybe doing something it shouldn't so I will investigate that.
    Looking for a good source vid should take up a longer time than inference so in the case of Taylor I would have used
    https://www.youtube.com/watch?v=XnbCSboujF4
    with the time stamps of 6:47 to 6:51

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants