Diffusion: Reverse process

3 minute read

Published:

Quick recap of the forward process

One noisy step:

\[q(x_t|x_{t-1}) := \mathcal{N}(x_t; \sqrt{1-\beta_t}\,x_{t-1}, \beta_t I)\]

Jump to an arbitrary time step \(t\):

\[q(x_t|x_0) := \mathcal{N}(x_t; \sqrt{\bar\alpha_t}\,x_0, (1-\bar\alpha_t)I)\]

Complete forward process:

\[\begin{aligned} &q(x_1,\cdots,x_T|x_0) = q(x_{1:T} | x_0) = \\ &q(x_1 | x_0) q(x_2 | x_1, x_0) \cdots q(x_T | x_{T-1}, \cdots, x_0) = \\ &q(x_1 | x_0) q(x_2 | x_1) \cdots q(x_T | x_{T-1}) = \\ &\prod_{t=1}^T q(x_t | x_{t-1}) \end{aligned}\]

where \(x_0\) is the observed data, \(x_1,\cdots,x_{T-1}\) form a sequence of latent noisy states, and \(x_T\) follows the terminal distribution, chosen to be approximately standard Gaussian: \(x_T \sim \mathcal{N}(0, I)\).

One denoising step

The reverse process is the opposite of the forward process. It performs denoising to remove the noise from the data step by step:

\[x_T \rightarrow x_{T-1} \rightarrow \cdots \rightarrow x_1 \rightarrow x_0\]

Let’s assume there is a probability density function \(p_\theta\), called approximate posterior, that allows us to reverse the individual noising steps: \(x_{t} \rightarrow x_{t-1}\).

While the forward process is defined by a conditional Gaussian distribution q(x_t \rvert x_{t-1}) with mean \(\sqrt{1-\beta_t}\,x_{t-1}\) and variance \(\beta_t I\), the reverse process depends on the learnable parameters \(\theta\), which are optimized during training to effectively remove the noise. We can define a single denoising step p_\theta(x_{t-1} \rvert x_t) as a Gaussian distribution.

\[p_\theta(x_{t-1} | x_t) := \mathcal{N}(x_{t-1}; \mu_\theta(x_t, t), \Sigma_\theta(x_t, t))\]


The mean of this Gaussian distribution is \(\mu_\theta(x_t, t)\) and its variance is \(\Sigma_\theta(x_t, t)\). Both are functions parameterized by \(\theta\) and depend on the current noisy state \(x_t\) and time step \(t\).

Complete reverse process

Similar to the forward process, the complete reverse process describes a trajectory containing all the intermediate denoised states \(x_T,\cdots,x_1\) starting at \(x_T\). We represent this trajectory using a joint distribution \(p_\theta(x_T,\cdots,x_0)\). To determine this joint distribution, we use the chain rule of probability to decompose it into individual conditional distributions. We then assume a first-order Markov chain, in which each noising step depends only on the previous state.

\[p_\theta(x_T,\cdots,x_0) = p_\theta(x_{0:T}) := p(x_T)\prod_{t=1}^T p_\theta(x_{t-1}|x_t)\]


How to optimize parameters \(\theta\)?

To find the best \(\theta\) that explains the observed data \(x_0\), we need to minimize the negative log-likelihood (or maximize the log-likelihood) computed by marginalizing over all possible intermediate noisy states \(x_1, \ldots, x_T\):

\[-logp_\theta(x_0) = -log\int p_\theta(x_{0:T})dx_{1:T}\]

In other words, over all possible latent trajectories connecting \(x_T\) to \(x_0\). Of course, trajectories must follow a specific Markov factorization for both

complete forward process (red path):

\[q(x_{1:T} | x_0) := \prod_{t=1}^T q(x_t | x_{t-1}) \qquad q(x_t|x_{t-1}) := \mathcal{N}(x_t; \sqrt{1-\beta_t}\,x_{t-1}, \beta_t I)\]

and complete reverse process (blue path):

\[p_\theta(x_{0:T}) := p(x_T)\prod_{t=1}^T p_\theta(x_{t-1}|x_t) \qquad p_\theta(x_{t-1} | x_t) := \mathcal{N}(x_{t-1}; \mu_\theta(x_t, t), \Sigma_\theta(x_t, t))\]

With many diffusion steps, this high-dimensional marginalization becomes intractable. The likelihood requires integrating over the high-dimensional continuous latent trajectory.

So, instead of directly maximizing the exact likelihood, we find and optimize a tractable lower bound on it, called the Evidence Lower Bound, or ELBO:

\[logp_\theta(x_0) \geq \mathbb{E}_{q(x_{1:T} | x_0)}\left[log \frac{p_\theta(x_{0:T})}{q(x_{1:T} | x_0)}\right]\]