Tomorrow Tuesday — October 2026 (Issue #13)
Welcome to another issue of Tomorrow Tuesday, where we cover the revolution running through film, video games, music and the creative industries.
This week we’re going for another deep dive. Today: the state of the art of AI for 3D.
The 3D world has come a long way these past few years, and AI opened up 3 paths for creating 3D elements and environments on a computer:
- Mesh generation with AI
- Gaussian Splatting
- World Models
- Bonus: agents for 3D creation
A newsletter by SOUTS, where we explore what happens when artificial intelligence stops being just a tool and starts changing the way we create and consume stories.
Intro to 3D graphics
How do we make a chair exist inside a computer? Our requirement is that we want to be able to look at it from different angles, turn it around, edit it.
Historically, the main way to do it was to build the shape out of polygons. That’s the foundation of the modeling used in video games, animation and interactive graphics.
For our chair example, the first thing we do is place points in space, where the edges of the chair would be. These are the vertices.
Step two is connecting those points with lines, the edges.
But now, where the lines connect, triangles form. These are our polygons. With polygons we make faces.
Simplifying a lot, this is the foundation of 3D modeling on computers. Everything else is optimizations and constructions built on top of it.
For example, to make our chair we start with the easiest shape there is: a square.
We can extend its sides until it becomes a cube.
We can split one face of the cube and move it to form the backrest.
And from there we keep going with this process of altering vertices, edges and faces, and we have a chair.
In this newsletter we’re getting into modeling. After that there are countless other areas: textures, materials, animation, rigging, lighting, sound, optimizations, etc etc etc.
Enter AI
AI breaks the paradigm of creating 3D graphics on a computer a little bit.
We have 3 (+1) ways of doing it:
| How does it work? | What do we get? | |
|---|---|---|
| Mesh generation | AI creates geometry from images. | An editable 3D object. |
| Gaussian Splatting | Reconstructs the appearance using spheres in space. | A 3D representation we can move through with a camera. |
| World Models | AI generates environments and how they respond to actions. | Generated interactive experiences. |
| Agents for 3D | An agent operates tools like Blender. | A 3D project we can keep working on. |
Mesh generation
The first path for AI in 3D is the same one: create an object with geometry that we can take into a 3D program.
Only instead of building our chair from a cube, we hand it an image. The model is trained to predict where each of the vertices, edges and faces goes.
The problem with this kind of generation: a front-facing photo is missing a lot of info.
The system has to fill in what the image looks like from the side, from behind, from below. It can produce a plausible backrest, but it has no way of knowing if it’s the real backrest. That’s why references from several angles matter.
If you’re inventing a chair, that ability to fill in the gaps is part of the fun. But if you’re trying to recreate exactly the chair you sell, it’s a problem.
Where the advances of the last year were:
- Better meshes. The models understand images better and better, and they’re better at creating the 3D mesh.
- Editing. Now you can tell the models “make the backrest with bigger ornaments” and they adjust the model for you.
- Better topology. Topology is how the polygons underneath build up a face. Clean topology looks really nice, messy topology looks bad. This doesn’t affect the model itself, but it matters a lot when we go to edit or animate it.
Tools to explore
- Meshy: generation from text or images, good for organic shapes and for 3D printing.
- Tripo: generation of 3D objects with good topology.
- Hunyuan3D, by Tencent: an alternative with local models in ComfyUI.
Gaussian Splatting
Back to our chair. Now we want to look at it from different angles and have it look exactly like it does in real life.
What we can do is walk around it with a camera, take lots of photos, and use those images to build a representation we can move through.
The new AI way of doing this is Gaussian Splatting.
Instead of building surfaces with polygons, it places thousands of tiny blobs in space. Each one has a position, a size, an orientation and a color. Seen together from the right angle, they reconstruct the image of the chair. The nice part is that the color of these blobs depends on where the camera is, so when we move, those blobs show us a different view. Useful for reflections, for example.
It sounds counterintuitive but it works really well. The result is a scene we can move through with a virtual camera.
It works for reconstructing objects and also for spaces.
For film this has a direct application: you capture a location and then change where you put the camera afterwards.
The complication with gaussian splatting shows up when you want to edit the chair.
The system never learned the parts of each object, only how they look in the scene. The analogy would be that a mesh is like a vector image where you can move the points without breaking anything, and gaussian splatting is an AI-generated image: you can move things, but the chances of something breaking go up.
This is advancing a lot. There are providers that let you edit objects in splats, apply effects, animations and lighting adjustments.
Tools to explore:
- Postshot: 3D scene reconstruction from photos or videos, processed locally.
- SuperSplat: editing, viewing and publishing splats in the browser.
- Brush: open source tool for training and viewing Gaussian splats on Mac, Windows and Linux.
World Models
World models are the big promise of several AI labs. We already talked about this in another issue: it’s one of those terms every company uses to describe something a little bit different.
The general idea is to learn what an environment is like and how it changes when we take an action.
What many of today’s world models actually do is reconstruct a 3D scene using Gaussian Splatting.
The real promise of these models is this: if I can get a video model to predict 100% accurately what a setting looks like and how physics behaves inside it, why do I need to know the polygons?
In other words, imagine a simulator where when you move a chair it drags along the floor, when you ask it to change the backrest it does, when you throw something gravity makes it fall. It doesn’t matter how it does it, as long as the user experiences it as real.
That’s the idea behind, for example, Runway GWM Worlds 2. The system generates video and audio in real time, responding to camera movements and instructions about characters and the environment.
The chair doesn’t ‘exist’ beyond the weights of a neural network, but who cares. For the uses we’re after, it’s the same thing.
This is super experimental, but if they pull it off, it’s the paradigm shift that’s been promised for a while.
The big use for these models is robotics. By building simulated environments that respond exactly like reality, we can put simulated robots in them to learn about the world much faster and more safely.
Tools to explore
- Marble, by World Labs: generation of environments exportable as splats or meshes.
- Atlas, by World Labs: generation and reconstruction with camera control.
- Project Genie, by Google DeepMind: creation and exploration of interactive worlds.
- GWM Worlds 2, by Runway: interactive video and audio in real time, research only for now.
Bonus: agents for 3D creation
There’s another way to get our chair: give Claude access to Blender and ask it to build it.
AI can operate the tools we would use ourselves. Create objects, modify geometry, assign materials, place lights and cameras.
The latest editions of GPT and Claude have gotten very, very good at building in 3D using Blender or Three.js.
Which is probably why your social feeds filled up with content like this:
The agent can build with traditional geometry, bring in an existing model, or combine several tools.
I asked it:
“Build me an interrogation room. A table, two different chairs, a fluorescent tube light overhead. I want to test three camera positions before deciding on the framing.”
This lowers a huge barrier to entry. I had done very little with 3D and now I play with it all the time. It can also save someone who already knows 3D a ton of work.
What do I want all this for?
The interesting part is that we have more and more ways to build a space and work inside it.
For a film, it can help us walk through a location before deciding the shots. For a video game, put together a prototype to test a mechanic. For an interactive experience, let the audience explore something they could previously only look at.
It also changes what we have to design.
When we make a film we choose what to show. We can have a spectacular set that ends exactly where the frame ends. If we want the audience to be able to turn around, we have to think about what’s on the other side.
3D is key in the space between film and video games that we’ve been exploring at SOUTS.
That was Tomorrow Tuesday, the weekly newsletter. If you want to subscribe, subscribe!
At SOUTSwe try to always stay on top of what’s happening, and every time there’s something relevant we’ll bring it to you in this newsletter.
See you next Tomorrow Tuesday.
Yves
