Diffusion Models

Diffusion models define an easy forward process that gradually corrupts data and learn a reverse process that reconstructs samples from noise.

Forward process

With a variance schedule $\beta_t$ and $\alpha_t=1-\beta_t$, the Markov transition is

$$q(x_t\mid x_{t-1}) = \mathcal{N}(\sqrt{\alpha_t}x_{t-1},\,\beta_t I).$$

The useful closed form is

$$x_t = \sqrt{\bar\alpha_t}x_0 + \sqrt{1-\bar\alpha_t}\,\epsilon, \qquad \epsilon \sim \mathcal{N}(0,I),$$

where $\bar\alpha_t=\prod_{s=1}^{t}\alpha_s$. It lets us sample any noisy level directly without simulating every earlier step.

Learning to denoise

A common objective trains a network $\epsilon_\theta(x_t,t)$ to predict the injected noise:

$$\mathcal{L}_{\mathrm{simple}} = \mathbb{E}_{x_0,t,\epsilon} \left[\lVert \epsilon - \epsilon_\theta(x_t,t) \rVert_2^2\right].$$

At generation time, repeated reverse updates transform Gaussian noise into a sample. Faster samplers reduce the number of evaluations by changing the numerical path while retaining the learned field.

Evaluation is multi-dimensional

No single metric fully describes a generative model. Fidelity, diversity, prompt adherence, memorization, failure severity and human preference answer different questions. Hallucination analysis should therefore make its definition of “error” explicit before choosing a statistic.