I discussed Jabri, Owens and Efros (2020) Space-Time Correspondence as a Contrastive Random Walk published at NeurIPS at the March 31st session of the PINLab Reading Group. It proposes a self-supervised method (constructing palindromic video cycles) for learning representations for visual correspondences across time, which can then be used for labelling e.g. of objects, semantic labels or pose keypoints.

My slides are available for download here

Visualisation of the Video Graph from Space-Time Correspondence as a Contrastive Random Walk

Two Sentence Paper Summary: The paper proposes a self-supervised method (constructing palindromic video cycles) for learning representations for visual correspondences across time, which can then be used for labelling e.g. of objects, semantic labels or pose keypoints. The authors represent video as graphs of frame patches where directed edges connect the same patches across time steps (i.e. across frames) and minimise the cross-entropy loss of nodes ending up at their starting position (true by construction in the palindromic video sequences) whilst using edge dropout as a regularizer.

Paper: https://proceedings.neurips.cc/paper/2020/file/e2ef524fbf3d9fe611d5a8e90fefdc9c-Paper.pdf

Code: https://github.com/ajabri/videowalk/

Website: https://ajabri.github.io/videowalk/