In this post we explore the benefits of applying self-supervised learning to the image classification problem in computer vision.

在这篇文章中，我们探讨了将自我监督学习应用于计算机视觉中图像分类问题的好处。

If you don’t have a clear idea of what self supervised learning is, see my short introduction to the concept here.

如果您不清楚什么是自我监督学习， 请在此处查看我对该概念的简短介绍 。

If you just want to access the (dirty) code, you can find the jupyter notebook here.

如果您只想访问(脏)代码， 则可以在此处找到jupyter笔记本。

“实验”设置 (“Experiment” setup)

• Dataset — CIFAR10

  数据集 — CIFAR10

• Neural Network Architecture — ResNet adapted from David Yangs implementation

  神经网络架构 -ResNet改编自David Yangs的实现

• Self-Supervised pretext task — Image rotation

  自我监督的借口任务 —图像旋转

程序 (Procedure)

We create an augmented version of the cifar10 dataset with all images randomly rotated 0,90,180 or 270 degrees. Using this rotated dataset we train a ResNet based neural network to classify images into their appropriate levels of rotation. Below are the graphs of our loss and classification accuracy.

我们创建cifar10数据集的增强版本，所有图像随机旋转0,90,180或270度。 使用这个旋转的数据集，我们训练了一个基于ResNet的神经网络，将图像分类为适当的旋转级别。 以下是我们的损失和分类准确性的图表。

Notably, we don’t achieve a very high classification accuracy for this pretext task, which is curious. It seems that the rotation pretext task might be more difficult than our downstream task.

值得注意的是，我们对此借口任务没有达到很高的分类精度，这很好奇。 轮换借口任务似乎比我们的下游任务更加困难。

Having created our pre-trained model using self-supervision we now re-purpose it to solve our downstream task of image classification. We do this by replacing the the last layer of the model with a new task specific layer of 10 nodes, as we are now predicting one in 10 image classes, not one in four rotation variants as we did previously.

使用自我监督创建了我们的预训练模型后，我们现在将其重新用于解决图像分类的下游任务。 为此，我们将模型的最后一层替换为10节点的新任务特定层，因为我们现在正在预测十分之一的图像类别，而不是像以前那样四分之一的旋转类别。

Then we proceed to train ResNet models with and without the self-supervised feature extractor on different portions of the data, to see how potential performance improvements depend on the amount of labeled data available. Below are the graphs of our loss and classification accuracy vs number of samples of labeled data used.

然后，我们继续在数据的不同部分上训练带有和不带有自监督特征提取器的ResNet模型，以查看潜在的性能改进如何取决于可用的标记数据量。 下面是我们的损失和分类准确性与所用标记数据样本数量的关系图。

What we find is that the less labeled data we use the larger the performance gain from self-supervision. When using only 1000 labeled images we see a large performance gain in classification accuracy ~19 %, and a relative gain of 37%, however as we approach 10000 labeled samples, or 1000 labeled samples per class, the performance gain is only about 2 %.

我们发现，我们使用的标签数据越少，自我监督的性能提升就越大。 当仅使用1000张带标签的图像时，我们看到的分类精度约为19％，相对性能为37％，但是当我们接近10000个带标签的样本或每类1000个带标签的样本时，性能增益仅为2％ 。

Although the diminishing returns of self-supervision are a bit disappointing, this trend is expected. For the purpose of self-supervision is to learn general features from a large distribution of data that cannot be attained from a small labeled but non-representative subset of said data. However, as that subset gets larger and larger, it becomes more and more representative and thus more conducive to learning general features that boost performance on unseen data.

尽管自我监督的收益递减有点令人失望，但这种趋势是可以预期的。 出于自我监督的目的，是从数据的大量分布中学习一般特征，而这些特征是从所述数据的较小标记但非代表性子集中无法获得的。 但是，随着该子集变得越来越大，它变得越来越具有代表性，因此更有利于学习可提高看不见数据性能的通用功能。

One final point: We have used a vanilla self-supervision setup, and despite our notable performance gains, far more profound improvements have been found when employing more advanced methods, in particular contrastive learning. If you want to read more about this I suggest looking at this paper.

最后一点 ：我们使用了一种香草自我监督设置，尽管我们取得了显着的性能提升，但在采用更高级的方法(尤其是对比学习)时，发现了更为深刻的进步。 如果您想了解更多有关此的内容，建议您阅读本文。

Thanks!

谢谢！

And here’s the jupyter notebook.

这是jupyter笔记本。