The identities behind VAEs, GANs, and diffusion: reparameterize a Gaussian, the closed-form KL, reconstruction plus beta, discriminator BCE, transposed-conv size, DDPM noising and its inverse, classifier-free guidance, and a U-Net skip concat.
Learning Objectives
→Reparameterize a diagonal Gaussian as z = mu + sigma * eps so gradients survive the sample
→Evaluate the closed-form KL of a diagonal Gaussian to N(0, I), using variance, not std
→Assemble a VAE loss as mean reconstruction plus a weighted KL
→Compute discriminator binary cross-entropy with a clip before the log
→Read transposed-convolution output size from stride, pad, and kernel
→Write the DDPM closed-form noising step and invert it to recover x0
→Apply classifier-free guidance as an extrapolation past the conditional prediction