青源TALK120期：扩散模型的可重复性与一致性

​

扩散模型作为一种强大的生成模型，在众多应用领域如图像生成、逆向问题解决及文本至图像转换中展现出卓越性能。这些模型通过逆向扩散过程，将随机噪声输入转换成新的数据内容（例如图像）。在本研究中，我们发现了扩散模型相较于大多数其他生成模型所呈现的独特现象，我们称之为“一致性与可重复性”。更具体地，我们的大量实验显示，当从相同的初始噪声输入出发，并采用非随机的求解器进行采样时，扩散模型倾向于生成几乎一致的结果。

这种一致性并不随模型架构或训练过程的变化而改变。此外，我们的研究还揭示，这种模型的可重复性在两个不同的区域中均有显现：一是“记忆区域”，在该区域，模型在较小的数据集上训练并过度参数化，通过记忆训练数据来实现可重复性；二是“泛化区域”，模型在更大的数据集上训练，其可重复性随着模型的泛化能力而显现。我们对“记忆区域”下模型可重复性的理论基础进行了分析。此外，我们的研究还表明，这种重要的可重复性特性同样适用于多种扩散模型的变体，包括条件扩散模型、用于解决逆问题的扩散模型以及经过微调的扩散模型。深入理解这一现象将有助于我们提高基于扩散模型的数据生成过程的可解释性和可控性。

论文地址： https://arxiv.org/abs/2310.05264

Recently, diffusion models have emerged as powerful deep generative models, showcasing cutting-edge performance across various applications such as image generation, solving inverse problems, and text-to-image synthesis. These models generate new data (e.g., images) by transforming random noise inputs through a reverse diffusion process. In this work, we uncover a distinct and prevalent phenomenon within diffusion models in contrast to most other generative models, which we refer to as “consistent model reproducibility”. To elaborate, our extensive experiments have consistently shown that when starting with the same initial noise input and sampling with a deterministic solver, diffusion models tend to produce nearly identical output content. This consistency holds true regardless of the choices of model architectures and training procedures. Additionally, our research has unveiled that this exceptional model reproducibility manifests in two distinct training regimes: (i) “memorization regime,” characterized by a significantly overparameterized model which attains reproducibility mainly by memorizing the training data; (ii) “generalization regime,” in which the model is trained on an extensive dataset, and its reproducibility emerges with the model’s generalization capabilities. Our analysis provides theoretical justification for the model reproducibility in “memorization regime”. Moreover, our research reveals that this valuable property generalizes to many variants of diffusion models, including conditional diffusion models, diffusion models for solving inverse problems, and fine-tuned diffusion models. A deeper understanding of this phenomenon has the potential to yield more interpretable and controllable data generative processes based on diffusion models.

密歇根大学安娜堡分校的博士生张挥杰，导师是曲庆。科研方向包括生成式模型和扩散模型，具体包括提升扩散模型的训练效率，理解扩散模型的可重复性与一致性，利用扩散模型实现领域自适应等方向。

​Read More