EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation.

In this session, Matthes Krull will be leading the conversation. In the 2022 paper EPro-PnP, the authors showed how their continuous PnP layer can beat other 6DoF approaches, without the use of a fancy network design and we will dig deeper into the reasons why. The algorithm is highly mathematical, but every step they take is well reasoned and crucial to its outstanding performance, as they’ve shown in the ablation study.

In brief, several algorithms are combined and applied on top of a backbone, which outputs: A) 2D-3D correspondences B) weights for those correspondences. First, a decoupled PnP solution is found by a Levenberg-Marquardt (LM) inspired solver. Second, this intermediate solution is used in a Monte-Carlo approach to find a continuous pose distribution around this “local” solution (which is parameterized by our 2D-3D correspondences).

Here, they also use the Adaptive Multiple Importance Sampling (AMIS) algorithm to iteratively refine that pose distribution. Finally and third, this distribution is used to optimize the network by randomly sample N poses from it and calculating the gradient according to their reprojection errors.

Here are the slides and videos showed in the session:

Info

Videos: