本文提出神经扩展定律的量化模型，既解释了损失函数随着模型和数据大小的下降遵循的幂律规律，也解释了随着规模的增大新能力的突然出现。作者从量化假设中推导出这个模型，认为学习网络能力被量化为离散的单位（量子）。表明当量子按照使用频率递减的顺序学习时，使用频率的幂律可以解释损失函数的幂律缩放。本文在玩具数据集上验证了这个预测，然后研究了大型语言模型的缩放曲线如何分解。通过语言模型的内部，本文自动发现了不同的模型能力（量子），并找到了初步的证据表明在自然文本预测的相关子问题上的分布与理论预测的神经缩放指数所预测的幂律相符。
标题：The Quantization Model of Neural Scaling
作者：Eric J. Michaud, Ziming Liu, Uzay Girit, Max Tegmark
论文：
https://papers.labml.ai/api/v1/redirect/pdf?paper_key=9fd16bb8c9de11edb95839eec3084ddd
正文完
可以使用微信扫码关注公众号（ID：xzluomor）
ufabet
มีเกมให้เลือกเล่นมากมาย: เกมเดิมพันหลากหลาย ครบทุกค่ายดัง
tornado crypto mixer
Discover the power of privacy with TornadoCash! Learn how this decentralized mixer ensures your transactions remain confidential.
ดูบอลสด
Very well presented. Every quote was awesome and thanks for sharing the content. Keep sharing and keep motivating others.
ดูบอลสด
Pretty! This has been a really wonderful post. Many thanks for providing these details.
ดูบอลสด
Pretty! This has been a really wonderful post. Many thanks for providing these details.
ดูบอลสด
Hi there to all, for the reason that I am genuinely keen of reading this website’s post to be updated on a regular basis. It carries pleasant stuff.
Obrazy Sztuka Nowoczesna
Thank you for this wonderful contribution to the topic. Your ability to explain complex ideas simply is admirable.
ufabet
Hi there to all, for the reason that I am genuinely keen of reading this website’s post to be updated on a regular basis. It carries pleasant stuff.
ufabet
Very well presented. Every quote was awesome and thanks for sharing the content. Keep sharing and keep motivating others.
