Title: Improved Baselines with Momentum Contrastive Learning
Authors: Xinlei Chen, Haoqi Fan, Ross Girshick, Kaiming He
Published: 9th March 2020 (Monday) @ 17:56:49
Link: http://arxiv.org/abs/2003.04297v1
Abstract
Contrastive unsupervised learning has recently shown encouraging progress, e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the effectiveness of two of SimCLRâs design improvements by implementing them in the MoCo framework. With simple modifications to MoCo---namely, using an MLP projection head and more data augmentation---we establish stronger baselines that outperform SimCLR and do not require large training batches. We hope this will make state-of-the-art unsupervised learning research more accessible. Code will be made public.
Suggests some improvements borrowed from MoCo to reduct the computational and memory requirements of training according to SimCLRv1