About the test on First-Person Hand Action Benchmark

[ZYzhouya]

Hi,
Thanks for your great work!
I would like to test the model on First-Person Hand Action Benchmark and calculate the error. I wonder how you transform the labels of the benchmark? Since the output joint position is in the wrist as the origin of coordinates. What is the tranform relationship of the two representation？How to align them?

Sincerely thank you very much!

[hassony2]

Hi !

All the results in the paper are reported in hand-centric coordinates, so centered on the hand root joint.
Therefore I do not provide a way to bring back the hand in pixel coordinates for the models released as part of the paper code release.

However, if you predict additionally a scale and a translation in pixel space, you can reproject the predicted MANO on the image. This can be done with the current code using the AbsoluteBranch see this line where the scale and translation branch are added to the model and this line where it predicts translation and scale in the forward pass.
For this branch to be initialized you need to set --mano_lambda_joints2d some_value

You can take a look at what the options in release_models/hands_only/opt.txt to see how to set the different supervision losses at training time when also training with reprojection.

python image_demo.py --resume release_models/hands_only/checkpoint.pth.tar will give you an example with reprojected joints on the 2d image.

Best,

Yana

[MedlarTea]

@hassony2 Hi, Thanks for your excellent work! If I want to get the real 3d coordinates of the hand in the camera-based coordinate, what should I do?