Learning VAEs and World Gen

What I worked on

Training a VAE to generate new environments using game screenshots, specifically from GTA5.

What I noticed

• A VAE can be used to generate new data from learned latent representations
• Game screenshots provide good structured input data
• Procedural world generation can benefit from the ability to interpolate between environments

”Aha” Moment

That VAEs don’t just reconstruct but can generate new versions of what they’ve learned from the latent space.

What still feels messy

Still unclear how to shape the dataset and how much data is enough for the VAE to learn meaningful spatial and color patterns.

Next step

Train a simple VAE using the GTA5 dataset and test how it reconstructs versus generates novel images.