I guess line 794 in g2otypes.cpp is wrong, which calculates the jacobian of reproject error w.r.t. dPwb, while theannotation(line 793) is right.
793 //Matrix3d J_Pc_dPwb = -Rcb;
794 Matrix<double,2,3> JdPwb = - Jpi * (-Rcb*Rwb.transpose());
not (-Rcb*Rwb.transpose()) but -Rcb
I guess line 794 in g2otypes.cpp is wrong, which calculates the jacobian of reproject error w.r.t. dPwb, while theannotation(line 793) is right.
793 //Matrix3d J_Pc_dPwb = -Rcb;
794 Matrix<double,2,3> JdPwb = - Jpi * (-Rcb*Rwb.transpose());
not (-Rcb*Rwb.transpose()) but -Rcb