A variational autoencoder learns more than a direct mapping from one image to another. Its encoder represents each input as a probability distribution in a continuous latent space, and its decoder learns to turn samples from that space back into images. Once trained, the model can generate new examples rather than only reconstructing images it has seen.
For this project, I trained a ResNet-style conditional VAE in PyTorch on the CelebA face dataset. The aim was to explore both unconditional sampling and simple, attribute-guided changes to generated faces.
A residual conditional VAE
The convolutional encoder compresses a face into the parameters of a latent distribution. A differentiable reparameterization step draws a sample, and a residual decoder expands that sample into a reconstructed face. Training balances reconstruction quality with a regularization term that keeps the latent distributions close to a shared prior, making it possible to sample coherent faces after training.
The residual blocks make it easier to train a deeper encoder and decoder while preserving information across layers. The resulting samples capture broad facial structure and variation, although fine details remain soft—a familiar tradeoff for likelihood-based VAEs compared with sharper adversarial models.
Conditioning on facial attributes
CelebA includes forty binary attributes, such as glasses, facial hair, and expression. I concatenated those attributes with the latent representation so the decoder could model the requested characteristics alongside the underlying identity and appearance.
I also built a small attribute-toggling tool that changes one condition at a time while holding the remaining input fixed. This provides an intuitive way to probe what the model learned, but it does not imply perfect disentanglement: correlated training attributes can cause a single toggle to change several visible features.
Takeaway
The project made the core VAE tradeoff concrete. A structured latent space supports smooth sampling and controllable experiments, while the reconstruction objective tends to average uncertain high-frequency details. It was also a useful lesson in evaluating conditional generative models beyond a few attractive samples: control, diversity, reconstruction quality, and attribute leakage all matter.
Explore the source code, training instructions, and sample-generation tools on GitHub.