本文重新审视了计算机视觉中用于视觉识别任务的标准预训练-微调范例。通常情况下，最先进的基础模型使用具有数十亿张图像的大规模（弱）监督数据集进行预训练。我们引入了一个额外的预-预训练阶段，它采用自监督的MAE技术来初始化模型。尽管MAE仅被证明与模型大小有关，但我们发现它也与训练数据集的大小有关。因此，我们基于MAE的预-预训练可同时适用于模型和数据大小，适用于训练基础模型。预-预训练始终提高了各种模型规模（百万到亿级参数）和数据集大小（百万到亿级图像）的模型收敛性和下游转移性能。我们在涵盖图像分类、视频识别、目标检测、少样本分类和零样本识别等10种不同视觉识别任务上度量了预-预训练的有效性。我们的最大模型在iNaturalist-18（91.3％）、1-shot ImageNet-1k（62.1％）和Food-101的零样本转移（96.0％）方面实现了新的最先进结果。我们的研究表明，即使进行数十亿张图像的Web规模预处理，模型初始化也发挥了重要作用。
标题：The effectiveness of MAE pre-pretraining for billion-scale pretraining
论文：https://arxiv.org/pdf/2303.13496.pdf
作者：Mannat Singh, Quentin Duval, Kalyan Vasudev Alwala, Haoqi Fan, Vaibhav Aggarwal, Aaron Adcock, Armand Joulin（研究主任）, Piotr Dollár, Christoph Feichtenhofer, Ross Girshick, Rohit Girdhar, Ishan Misra
正文完
可以使用微信扫码关注公众号（ID：xzluomor）
ufabet
มีเกมให้เลือกเล่นมากมาย: เกมเดิมพันหลากหลาย ครบทุกค่ายดัง
tornado crypto mixer
Discover the power of privacy with TornadoCash! Learn how this decentralized mixer ensures your transactions remain confidential.
ดูบอลสด
Very well presented. Every quote was awesome and thanks for sharing the content. Keep sharing and keep motivating others.
ดูบอลสด
Pretty! This has been a really wonderful post. Many thanks for providing these details.
ดูบอลสด
Pretty! This has been a really wonderful post. Many thanks for providing these details.
ดูบอลสด
Hi there to all, for the reason that I am genuinely keen of reading this website’s post to be updated on a regular basis. It carries pleasant stuff.
Obrazy Sztuka Nowoczesna
Thank you for this wonderful contribution to the topic. Your ability to explain complex ideas simply is admirable.
ufabet
Hi there to all, for the reason that I am genuinely keen of reading this website’s post to be updated on a regular basis. It carries pleasant stuff.
ufabet
Very well presented. Every quote was awesome and thanks for sharing the content. Keep sharing and keep motivating others.
