大家好，我是对白。

最近对比学习真的太火了，已然成为各大顶会争相投稿的一个热门领域，而它火的原因也很简单，就是因为它解决了有监督训练标注数据有限这个典型问题（这个问题在工业界非常滴常见）。所以对比学习的出现，给CV、NLP和推荐都带来了极大的福音，具体来说：

1、在CV领域，解决了“在没有更大标注数据集的情况下，如何采用自监督预训练模式，来从中吸取图像本身的先验知识分布，得到一个预训练模型”的问题；

2、在NLP领域，验证了”自监督预训练使用的数据量越大，模型越复杂，那么模型能够吸收的知识越多，对下游任务效果来说越好“这样一个客观事实；

3、在推荐领域，解决了以下四个原因：数据的稀疏性、Item的长尾分布、跨域推荐中多个不同的view聚合问题以及增加模型的鲁棒性或对抗噪音，感兴趣地可以看我写的这篇文章：推荐系统中不得不学的对比学习（Contrastive Learning）方法

因此为了更加清楚的掌握对比学习的前沿方向与最新进展，我为大家整理了最近一年来各大顶会中对比学习相关的论文，一共涵盖：ICLR2021,SIGIR2021,WWW2021,CVPR2021,AAAI2021,NAACL2021,ICLR2020,NIPS2020,CVPR2020,ICML2020,KDD2020共十一个会议60多篇论文。本次整理以long paper和research paper为主，也包含少量的short paper和industry paper。由于工作量巨大，难免有疏漏，欢迎大家在评论区指正。

本文整理的论文列表已经同步更新到GitHub，GitHub上会持续更新顶会论文，欢迎大家关注和star~

https://github.com/coder-duibai/Contrastive-Learning-Papers-Codes

分成九类

Awesome Contrastive Learning Papers&Codes。

我将60多篇论文和它们的代码，分到了九个类别里。

1.Computer Vision

第一类是计算机视觉，也是内容最饱满的章节，有19篇论文的代码。

不乏最近非常著名的模型，例如何恺明提出的MoCo和MoCo v2以及Geoffrey Hinton提出的SimCLR和SimCLR v2便属于这一类。

1. [PCL] Prototypical Contrastive Learning of Unsupervised Representations.ICLR2021. Authors:Junnan Li, Pan Zhou, Caiming Xiong, Steven C.H. Hoi. paper code

2. [BalFeat] Exploring Balanced Feature Spaces for Representation Learning. ICLR2021.

Authors:Bingyi Kang, Yu Li, Sa Xie, Zehuan Yuan, Jiashi Feng. paper

3. [MiCE] MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering. ICLR2021. Authors:Tsung Wei Tsai, Chongxuan Li, Jun Zhu. paper code

4. [i-Mix] i-Mix: A Strategy for Regularizing Contrastive Representation Learning. ICLR2021.

Authors:Kibok Lee, Yian Zhu, Kihyuk Sohn, Chun-Liang Li, Jinwoo Shin, Honglak Lee. paper code

5. Contrastive Learning with Hard Negative Samples.ICLR2021.

Authors:Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, Stefanie Jegelka. paper code

6. [LooC] What Should Not Be Contrastive in Contrastive Learning. ICLR2021.

Authors:Tete Xiao, Xiaolong Wang, Alexei A. Efros, Trevor Darrell. paper

7. [MoCo] Momentum Contrast for Unsupervised Visual Representation Learning. CVPR2020.

Authors:Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick. paper code

8. [MoCo v2] Improved Baselines with Momentum Contrastive Learning.

Authors:Xinlei Chen, Haoqi Fan, Ross Girshick, Kaiming He. paper code

9. [SimCLR] A Simple Framework for Contrastive Learning of Visual Representations. ICML2020. Authors:Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton. paper code

10. [SimCLR v2] Big Self-Supervised Models are Strong Semi-Supervised Learners. NIPS2020.

Authors:Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, Geoffrey Hinton. paper code

11. [BYOL] Bootstrap your own latent: A new approach to self-supervised Learning.

Authors:Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H, etc.

12. [SwAV] Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. NIPS2020. Authors:Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, Armand Joulin. paper code

13. [SimSiam] Exploring Simple Siamese Representation Learning. CVPR2021.

Authors:Xinlei Chen, Kaiming He. paper code

14. Hard Negative Mixing for Contrastive Learning. NIPS2020.

Authors:Yannis Kalantidis, Mert Bulent Sariyildiz, Noe Pion, Philippe Weinzaepfel, Diane Larlus. paper

15. Supervised Contrastive Learning. NIPS2020. Authors:Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, Dilip Krishnan. paper

16. [LoCo] LoCo: Local Contrastive Representation Learning. NIPS2020.

Authors:Yuwen Xiong, Mengye Ren, Raquel Urtasun. paper

17. What Makes for Good Views for Contrastive Learning?. NIPS2020.

Authors:Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, Phillip Isola. paper

18. [ContraGAN] ContraGAN: Contrastive Learning for Conditional Image Generation. NIPS2020.

Authors:Minguk Kang, Jaesik Park. paper code

19. [SpCL] Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID. NIPS2020.

Authors:Yixiao Ge, Feng Zhu, Dapeng Chen, Rui Zhao, Hongsheng Li. paper code

2.Audio

第二类是音频，有1篇论文，wav2vec 2.0

1. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. Authors:Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael Auli. paper code

3.Videos and Multimodal

第三类是视频和多模态，主要包含ICLR2021和NIPS2020的论文，包含少量CVPR2020，有12篇论文的实现。

1. Time-Contrastive Networks: Self-Supervised Learning from Video.

Authors: Pierre Sermanet; Corey Lynch; Yevgen Chebotar; Jasmine Hsu; Eric Jang; Stefan Schaal; Sergey Levine. paper

2. Contrastive Multiview Coding.

Authors:Yonglong Tian, Dilip Krishnan, Phillip Isola. paper code

3. Learning Video Representations using Contrastive Bidirectional Transformer.

Authors:Chen Sun, Fabien Baradel, Kevin Murphy, Cordelia Schmid. paper

4. End-to-End Learning of Visual Representations from Uncurated Instructional Videos.CVPR2020.

Authors:Antoine Miech, Jean-Baptiste Alayrac, Lucas Smaira, Ivan Laptev, Josef Sivic, Andrew Zisserman. paper code

5. Multi-modal Self-Supervision from Generalized Data Transformations.

Authors:Mandela Patrick, Yuki M. Asano, Polina Kuznetsova, Ruth Fong, João F. Henriques, Geoffrey Zweig, Andrea Vedaldi. paper

6. Support-set bottlenecks for video-text representation learning. ICLR2021.

Authors:Mandela Patrick, Po-Yao Huang, Yuki Asano, Florian Metze, Alexander Hauptmann, João Henriques, Andrea Vedaldi. paper

7. Contrastive Learning of Medical Visual Representations from Paired Images and Text. ICLR2021.

Authors:Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D. Manning, Curtis P. Langlotz. paper

8. AVLnet: Learning Audio-Visual Language Representations from Instructional Videos.

Authors:Andrew Rouditchenko, Angie Boggust, David Harwath, Brian Chen, Dhiraj Joshi, Samuel Thomas, Kartik Audhkhasi, Hilde Kuehne, Rameswar Panda, Rogerio Feris, Brian Kingsbury, Michael Picheny, Antonio Torralba, James Glass. paper

9. Self-Supervised MultiModal Versatile Networks. NIPS2020.

Authors:Jean-Baptiste Alayrac, Adrià Recasens, Rosalia Schneider, Relja Arandjelović, Jason Ramapuram, Jeffrey De Fauw, Lucas Smaira, Sander Dieleman, Andrew Zisserman. paper

10. Memory-augmented Dense Predictive Coding for Video Representation Learning.

Authors:Tengda Han, Weidi Xie, Andrew Zisserman. paper code

11. Spatiotemporal Contrastive Video Representation Learning.

Authors:Rui Qian, Tianjian Meng, Boqing Gong, Ming-Hsuan Yang, Huisheng Wang, Serge Belongie, Yin Cui. paper code

12. Self-supervised Co-training for Video Representation Learning. NIPS2020.

Authors:Tengda Han, Weidi Xie, Andrew Zisserman. paper

4.NLP

第四类是自然语言处理，主要包含ICLR2021和NAACL2021的论文，有14项研究的实现。

1. [CALM] Pre-training Text-to-Text Transformers for Concept-centric Common Sense. ICLR2021. Authors:Wangchunshu Zhou, Dong-Ho Lee, Ravi Kiran Selvam, Seyeon Lee, Xiang Ren. papercode

2. Residual Energy-Based Models for Text Generation. ICLR2021.

Authors:Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, Marc’Aurelio Ranzato. paper

3. Contrastive Learning with Adversarial Perturbations for Conditional Text Generation. ICLR2021.

Authors:Seanie Lee, Dong Bok Lee, Sung Ju Hwang. paper

4. [CoDA] CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding. ICLR2021.

Authors:Yanru Qu, Dinghan Shen, Yelong Shen, Sandra Sajeev, Jiawei Han, Weizhu Chen. paper

5. [FairFil] FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders. ICLR2021.

Authors:Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, Lawrence Carin. paper

6. Towards Robust and Efficient Contrastive Textual Representation Learning. ICLR2021.

Authors:Liqun Chen, Yizhe Zhang, Dianqi Li, Chenyang Tao, Dong Wang, Lawrence Carin. paper

7. Self-supervised Contrastive Zero to Few-shot Learning from Small, Long-tailed Text data. ICLR2021.

Authors:Nils Rethmeier, Isabelle Augenstein. paper

8. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. ICLR2021.

Authors:Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, Arnold Overwijk. paper

9. Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents. NAACL2021.

Authors:Mohammad Kachuee, Hao Yuan, Young-Bum Kim, Sungjin Lee. paper

10. SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency. NAACL2021.

Authors:Sameer Dharur, Purva Tendulkar, Dhruv Batra, Devi Parikh, Ramprasaath R. Selvaraju. paper

11. Supporting Clustering with Contrastive Learning. NAACL2021.

Authors:Dejiao Zhang, Feng Nan, Xiaokai Wei, Shangwen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew Arnold, Bing Xiang. paper

12. Understanding Hard Negatives in Noise Contrastive Estimation. NAACL2021.

Authors:Wenzheng Zhang, Karl Stratos. paper

13. Contextualized and Generalized Sentence Representations by Contrastive Self-Supervised Learning: A Case Study on Discourse Relation Analysis. NAACL2021. Authors:Hirokazu Kiyomaru, Sadao Kurohashi. paper

14. Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach. NAACL2021.

Authors:Yue Yu, Simiao Zuo, Haoming Jiang, Wendi Ren, Tuo Zhao, Chao Zhang. paper

5.Language Contrastive Learning

第五类是语言模型，在这个方向上有5篇论文。

1. Distributed Representations of Words and Phrases and their Compositionality. 2013NIPS.

Authors:Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean. Paper

2. An efficient framework for learning sentence representations.

Authors:Lajanugen Logeswaran, Honglak Lee. Paper

3. XLNet: Generalized Autoregressive Pretraining for Language Understanding.

Authors:Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, Quoc V. Le. Paper

4. A Mutual Information Maximization Perspective of Language Representation Learning.

Authors:Lingpeng Kong, Cyprien de Masson d’Autume, Wang Ling, Lei Yu, Zihang Dai, Dani Yogatama. Paper

5. InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training.

Authors:Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou. Paper

6.Graph

第六类是图与对比学习的结合，有4项研究的实现。

1. [GraphCL] Graph Contrastive Learning with Augmentations. NIPS2020.

Authors:Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen. paper

2. Contrastive Multi-View Representation Learning on Graphs. ICML2020.

Authors:Kaveh Hassani, Amir Hosein Khasahmadi. Paper

3. [GCC] GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training. KDD2020.

Authors:Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, Jie Tang. Paper

4. [InfoGraph] InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. ICLR2020.

Authors:Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, Jian Tang. Paper

7.Adversarial Learning

第七类是对抗训练+对比学习，目前只有1篇论文。

1. Contrastive Learning with Adversarial Examples. NIPS2020.

Authors:Chih-Hui Ho, Nuno Vasconcelos. paper

8.Recommendation

第八类是推荐系统结合对比学习，解决点击数据的稀疏性或增加模型的鲁棒性，有3篇论文。

1. Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation. AAAI2021.

Authors:Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, Xiangliang Zhang. paper code

2. Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation. WWW2021. Authors:Junliang Yu, Hongzhi Yin, Jundong Li, Qinyong Wang, Nguyen Quoc Viet Hung, Xiangliang Zhang. paper code

3. Self-supervised Graph Learning for Recommendation. SIGIR2021.

Authors:Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, Xing Xie. paper code

9.Applications

第九类是对比学习在图像-图像翻译中的应用，有1篇论文。

1. Contrastive Learning for Unpaired Image-to-Image Translation.

Authors:Taesung ParkAlexei A. Efros, Richard ZhangJun-Yan Zhu. paper

对比学习算法交流群

已建立对比学习算法交流群！想要进交流群学习的同学，可以直接加我的微信号：duibai996。加的时候备注一下：昵称+学校/公司。群里聚集了很多学术界和工业界大佬，欢迎一起交流算法心得，日常还可以唠嗑~

关于我

你好，我是对白，清华计算机硕士毕业，现大厂算法工程师，拿过8家大厂算法岗SSP offer（含特殊计划），薪资40+W-80+W不等。

高中荣获全国数学和化学竞赛二等奖。

本科独立创业五年，两家公司创始人，拿过三百多万元融资(已到账)，项目入选南京321高层次创业人才引进计划。创业做过无人机、机器人和互联网教育，保研清华后退居股东。

我每周至少更新三篇原创，分享人工智能前沿算法、创业心得和人生感悟。我正在努力实现人生中的第二个小目标，上方关注后可以加我微信交流。

期待你的关注，我们一起悄悄拔尖，惊艳所有~

机器学习新宠：对比学习论文实现大合集，60多篇分门别类，从未如此全面相关推荐

年终盘点！2022顶会论文代码大合集！
我整理了2022年各大顶会顶刊的论文.代码.数据集合集.按会议合集.细分方向合集整理如下: 1.2022顶会论文代码合集会议论文合集 CVPR 2022论文&代码 ECCV 2022论文&a ...
肝了1个月！2022 顶会论文代码大合集！
我整理了2022年各大顶会顶刊的论文.代码.数据集合集.按会议合集.细分方向合集整理如下: 1.2022顶会论文代码合集会议论文合集 CVPR 2022论文&代码 ECCV 2022论文&a ...
大合集！80 篇 CVPR2020 论文分方向整理: 目标检测/图像分割/姿态估计等（附链接＆下载）...
来源:极市平台本文约2500字,建议阅读5分钟本文整理和分类80篇CVPR2020论文. 标签:深度学习 CVPR2020在2月24日公布了所有接受论文ID,从论文ID公布以来,我们一直在对CVP ...
推特800赞：图网络论文实现大合集，70多篇分门别类，从未如此全面 | 已过1k星...
方栗子发自凹非寺量子位出品 | 公众号 QbitAI △ 右边是不规则的自从科学家发现,图神经网络 (GNN) 能处理不规则数据.攻克从前难解的问题,后每每出现图网络的资源,便广受人类的喜爱 ...
[深度学习论文笔记]医学图像分割U型网络大合集
[深度学习论文笔记]医学图像分割U型网络大合集 2015 U-Net: Convolutional Networks for Biomedical Image Segmentation (MICCAI ...
对比学习论文综述(part1、2)
对比学习论文综述(part1|2) 18年MOCO开始对比学习在CV领域发展历程大概可以分为四个阶段 1.百花齐放 InstDisc(instance discrimination) CPC CMC ...
年度盘点！必看AI顶会论文、Github高星项目大合集（附链接）
目前,国外计算机界评价学术水平主要是看在顶级学术会议上发表的论文,特别是在机器学习.计算机视觉和人工智能领域,顶级会议才是王道.本期我们盘点了2019年人工智能领域的顶会,如最核心的顶会AAAI.IJ ...
独家 | 年度盘点！必看AI顶会论文、Github高星项目大合集（附链接）
目前,国外计算机界评价学术水平主要是看在顶级学术会议上发表的论文,特别是在机器学习.计算机视觉和人工智能领域,顶级会议才是王道.本期我们盘点了2019年人工智能领域的顶会,如最核心的顶会AAAI.IJ ...
网络安全中机器学习大合集 Awesome
网络安全中机器学习大合集 from:https://github.com/jivoi/awesome-ml-for-cybersecurity/blob/master/README_ch.md#-da ...

机器学习新宠：对比学习论文实现大合集，60多篇分门别类，从未如此全面

关于我

机器学习新宠：对比学习论文实现大合集，60多篇分门别类，从未如此全面相关推荐

最新文章

热门文章