# Contrastive_Learning_Summary

**Repository Path**: mingliangbai/Contrastive_Learning_Summary

## Basic Information

- **Project Name**: Contrastive_Learning_Summary
- **Description**: Awesome Contrastive Learning
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 1
- **Created**: 2021-07-04
- **Last Updated**: 2022-09-26

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Awesome Contrastive Learning

This repo is a collection of AWESOME things about contrastive learning now. It is also a summary of my own research. In addition, we will update relative research such as self-supervised learning, domain generalization, including papers, code, etc. Feel free to star and fork.

# Contents
- [Awesome Contrastive Learning](#awesome-contrastive-learning)
- [Contents](#Contents)
- [Survey](#Survey)
- [Theory](#Theory)
- [Related Work (Basic)](#Related-work-basic)
- [Contrastive Learning (Discriminanation)](#Contrastive-Learning-Discriminanation)
  - [Cluster-Based](#Cluster-based)
  - [Context-Instance](#context-Instance)
  - [Instance-Instance](#Instance-Instance)
- [Generative Learning (Generation)](#Generative-Learning)
- [Summay Blogs](#summary-blogs)
- [Reference](#Reference)

# Survey
- A Survey on Contrastive Self-Supervised Learning [[31 Oct 2020]](https://arxiv.org/abs/2011.00362)
- Self-supervised Learning: Generative or Contrastive [[15 Jun 2020]](https://arxiv.org/abs/2006.08218)
- Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey [[16 Feb 2019]](https://arxiv.org/abs/1902.06162)
- A survey on Semi-, Self- and Unsupervised Learning for Image Classification [[20 Feb 2020]](https://arxiv.org/abs/2002.08721)

# Theory
- A Theoretical Analysis of Contrastive Unsupervised Representation Learning [[25 Feb 2019]](https://arxiv.org/abs/1902.09229)
- How Useful is Self-Supervised Pretraining for Visual Tasks? [[31 Mar 2020]](https://arxiv.org/abs/2003.14323)
- What Makes for Good Views for Contrastive Learning? [[NeurIPS 2020]](https://arxiv.org/abs/2005.10243)
- Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere [[ICML 2020]](https://arxiv.org/abs/2005.10242)

# Related work (basic)
- (**First Contrastive Loss**) Dimensionality Reduction by Learning an Invariant Mapping [[CVPR 2006]](https://ieeexplore.ieee.org/document/1640964)
- (N-pair Loss) Improved Deep Metric Learning with
Multi-class N-pair Loss Objective [[NeurIPS 2016]]( https://www.nec-labs.com/uploads/images/Department-Images/MediaAnalytics/papers/nips16_npairmetriclearning.pdf)
- (Triplet Loss) FaceNet: A Unified Embedding for Face Recognition and Clustering [[CVPR 2015]](https://arxiv.org/abs/1503.03832)
- (CPC : InfoNCE Loss ) Representation Learning with Contrastive Predictive Coding [[ 10 Jul 2018]]( https://arxiv.org/abs/1807.03748)
- (Memory Bank) Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination [[CVPR 2018 ]](https://arxiv.org/abs/1805.01978)
# Contrastive Learning (Discriminanation)

## Cluster-Based
- (DeepCluster) Deep Clustering for Unsupervised Learning of Visual Features [[ECCV 2018]](https://arxiv.org/abs/1807.05520)
- (SwAV) Unsupervised Learning of Visual Features by Contrasting Cluster Assignments [[NeurIPS 2020]](https://arxiv.org/abs/2006.09882)[[PyTorch]](https://github.com/facebookresearch/swav)

## Context-Instance
- (CPC) Representation Learning with Contrastive Predictive Coding [[ 10 Jul 2018]]( https://arxiv.org/abs/1807.03748)
- (Deep Infomax) Learning deep representations by mutual information estimation and maximization [[ ICLR 2019]](https://arxiv.org/abs/1808.06670)

## Instance-Instance
- 
- (MoCo : **Civilian weapon**) Momentum Contrast for Unsupervised Visual Representation Learning [[CVPR 2020]](https://arxiv.org/abs/1911.05722)[[PyTorch]](https://github.com/facebookresearch/moco)
- (SimCLR : **Richer Game**) A Simple Framework for Contrastive Learning of Visual Representations [[ICML 2020]](https://arxiv.org/abs/2002.05709)[[TensorFlow]](https://github.com/google-research/simclr)
- (MoCo_v2) Improved Baselines with Momentum Contrastive Learning [[9 Mar 2020]](https://arxiv.org/abs/2003.04297)
- (SimCLR_v2) Big Self-Supervised Models are Strong Semi-Supervised Learners [[NeurIPS 2020]](https://arxiv.org/abs/2006.10029)[[TensorFlow]](https://github.com/google-research/simclr)
- (BYOL : **Dispel superstition**) Bootstrap your own latent: A new approach to self-supervised Learning [[ 13 Jun 2020 ]](https://arxiv.org/abs/2006.07733)
- (SimSiam : **Make simpler**) Exploring Simple Siamese Representation Learning [[CVPR 2021]](https://github.com/facebookresearch/simsiam)
- (Barlow Twins : **Back to Start**) Barlow Twins: Self-Supervised Learning via Redundancy Reduction [[ICML 2021]](https://arxiv.org/abs/2103.03230)
# Generative Learning (Generation)
- updating...

# Summay Blogs
- contrastive self-supervised learning[[Jan 26 2020]](https://ankeshanand.com/blog/2020/01/26/contrative-self-supervised-learning.html)
- The Illustrated Self-Supervised Learning [[Amit Chaudhary 2020]](https://amitness.com/2020/02/illustrated-self-supervised-learning/)
- The Illustrated SimCLR Framework [[Amit Chaudhary 2020]](https://amitness.com/2020/03/illustrated-simclr/)

# Reference
- [[Awesome Self-Supervised learning Github]](https://github.com/jason718/awesome-self-supervised-learning)