-
Notifications
You must be signed in to change notification settings - Fork 243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
About the Equation 5 for Full Surround Monodepth from Multiple Cameras #218
Comments
This is my code I implement, I can not promise it's right
` |
@hjxwhy Thanks a lot. I believe this is right. By the way, have you eval implement spatio-temporal loss in FSM? I cannot achieve the same improvement (even decrease) as Table.3 in FSM paper. Maybe there are some problem in my implementation. I implement spatial-wise pe loss as below, as Equation 3 in FSM paper. ` def spatial_wise_pe_loss(self, batch, output, return_logs=False, progress=0.0):
|
The implementation looks alright to me. Some things that have helped other people achieving similar results:
|
Hi, Thank you! |
@VitorGuizilini-TRI Thanks a lot ! Your suggestion is very helpful. I tried focal length scaling and it works. I am tryining start from a pretrained model without the spatio-temporal constraints now. @hurjunhwa Hi, I implement focal length scaling by scale the output depth by a constant, i.e., focal length. This focal length comes from the intrinsics input. Because I do not have the camera parameters, e.g., dx and dy. So I simply take the f_x in intrinsics as focal length to scale the depth. I tried this trick on DDAD and it works. Hope this can be helpful. |
@LionRoarRoar My STC implement is the same as you,but the result also degrade. You have try to scale the depth by focal length, which means that the every camera output multiple focal length or divide focal length? By the way, as my test, the input image with self occlusion cause the RMSE larger than front camera only, Have you faced this problem? |
I scale each camera output with its corresponding focal length. All other cameras get all worse results than front camera in my experiments. Only RMSE larger than front camera seems unreasonable? Maybe you have wrong normalization layer in last output layer. |
@LionRoarRoar Thanks for your reply. I have an experiment that train only front camera and CAMERA_8 seperate, the CAMERA_8 is worse than front camera in all metrics, so I guess it's cause by the self occlusion in image in CAMERA_8. But I'm not sure because the paper seems don't have this problem. Do you plan to do this experiment? I'm sorry for ask again, the scale depth means inverse depth multiple focal length? |
@hjxwhy A2: You should scale depth map instead of inverse depth |
@LionRoarRoar THANKS, I will try again. If I have some new results I will share with you here. Best wishes! |
Updates: 2、I also tried spatial-wise loss with a larger min_depth start from a pre-trained model without the spatio-temporal constraints. And the performance drops. |
rot_loss += torch.sum((X_ba.mat2vec()[:, :3] - pose.mat2vec()[0, :3]).pow(2)),I think the pose here should use cam1_pose supervise |
Have you reached the accuracy of the paper? I can't reproduce it |
@abing222 No. Only Self-oclussion mask work. STC and Pose consistency loss does not work. |
In my experiment, Only Self-oclussion mask absrel did not decline as much as the paper |
At present, I can obtain the absolute scale through spatio, the accuracy decreases slightly. After adding STC, the accuracy increases a little |
You mean spatial-wise constraints not work but STC works? That is interesting. Could you please provide more implement details about your STC?Such as loss weight, how to warp |
Self-oclussion mask
spatial-wise is useful, provide absolute scale, but the accuracy decreased. I changed code on the basis of monodepth2 repo code without using packnet repo |
I also cannot obtain the absolute scale with the spatio photometric loss. Do you use any pretrained model? Or change the min_depth parameter in monodepth2 repo? |
Hi, weiyi. I am also try to implement this work. Maybe we can add wechat for discussion. My wechat: zhuhaow_ |
Hi, thank your for your works. I am trying to reproduce your pose consistency loss. This loss constraints the predicted pose from other camera to be consistent with the front camera after transformation. However, It is hard to understand how the coordinate transforms to other coordinate by Equ.5 . Could you please provide more explanation clues or detail code? Thanks.
The text was updated successfully, but these errors were encountered: