Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I think this approach (separate parts) is the right call. This is how human artists build models, and once the model is built and segmented you can decide which parts go together, and then then group, remesh, UV-map and bake those parts. All of which are hard problems too but are getting closer to being automated.by joeld42
- Agreed. I think you'll be interested in seeing this: https://app.nova3d.xyz/showcaseby baigy
- I was thinking about this approach, and this paper basically validates it.
The inference cost must be extremely high compared to diffusion based approaches. I wonder if this would ever be useful for more organic sculpting type workflows. E.g. for organic non-hard surface models, use a diffusion model for generation, and leverage this LLM codegen and tool call approach for retopology and cleanup.
by LarsDu88 - My belief is it'll get better over time with organic shapes as LLMs improve their ability to synthesize higher order differentials. I also believe the future may not be coded 3D everywhere, but a mix of code and dumb 3D. But I'm not a huge fan of inverse code recovery; I found that to be extremely lossy.
Do you want to connect over discord or linkedin or something, to cross-pollinate ideas?
by baigy - Very cool.
"Asset as a service" is something I've been kicking around for a while. I did some work on something similar at Apple.
The idea was to create "Assets as a service" where the generation system can decide how much configurability remains live at run time, and how much is "compiled away" at asset generation time.
https://patentimages.storage.googleapis.com/43/19/69/c4c2dce... https://patentimages.storage.googleapis.com/d3/4a/df/d329bb5... https://patentimages.storage.googleapis.com/d2/31/6d/123b055...
by forelle2 - Whoa super cool, you guys got the patent too. I hope there's no cease and desist in my future lol (kidding..... not kidding).by baigy
- @baigy Huh. Interesting. I was just doing the final clean-up for something convergent to this research that I had been working on for the past few months. I think I arrived at your thesis (code first semantics from a different direction in CAD, so I think it'd be interesting for us to compare notes.
Have you formalized this into a compiler infrastructure yet? I think Python on its own would be too slow to build complex parts, especially since for triangle mesh, accuracy inversely correlates to performance.
Vision is generally not the most reliable form of checks for LLMs, even on GPT 5.6 Sol, so a recommendation I would have is to instead emit JSON or CSV of the color/topology data for the LLM to inspect directly, and this is the instance where ray query for topology checking will greatly improve accuracy in general. SDFs are a bit more complicated right now, I have a full implementation designed for 3D analysis
My own experimental compiler generated mesh suffers from the spiderweb effect: it's very polygon efficient but not very friendly towards UV unwrapping in general, and I'm struggling to find the correct approach for that. If you have any suggestions, I'd love if you can point me towards the correct approach.
Definitely very interesting though.
by YuechenLi - Would love to compare notes. It's compiler-shaped rn. The generated code is the source and Blender is the current build target. Python directs Blender's native geometry operations only. I also already generate UVs from the coded model before GLB export, while its part structure is still available, which helps avoid recovering everything from one fused spiderweb mesh. Visual checks aren’t the only option either. Having the code gives us direct structural checks too. Btw curious to see your SDF approach.by baigy
- I co-authored this paper. It's a new technique to generate 3D graphics as source code instead of a point cloud.
Under the hood, it generates 3D objects with separate, sophisticated internal assembly, producing an editable "kit of parts" (instead of monolithic blobs).
E.g. imagine you generated a 3D washing machine via this approach. It's not merely going to be just "geometry" that looks like a washing machine. We actually know that there is a `Door`, `Drum`, `Control_panel` etc. Which things belong to which assemblies. What moves and where its pivot is. And eventually what those components are supposed to do.
Most current 3D GenAI cannot do this since it generates "monolithic blobs" that look good, but are unusable in downstream workflows (e.g. game engines). I.e. if you generate a 3D bicycle using traditional approaches, it's basically a blob. When you need the wheels to turn, a human (or another AI) must spend time cutting the blob into parts, naming them, placing pivots and rigging joints. I.e. you need post-generation segmentation workflows of some sort.
The paper breaks down the whole technique, and there's a showcase (+ github repo) you can play around with: https://nova3d.xyz/
by baigy - So fucking cool.by qarl2
- Did I read this right? The objects are still just surfaces, not solids?by chrisjj
- How far are we from speaking a GameCube-era game into existence as a pastime?by bckr
- Cool. Do the individual parts still use point clouds? Or are they meshes or CSG?by RobotToaster
- How does this compare to parametric modeling tools like Fusion/Solidworks/ProE ? Is it more about the integration with game specific tools?by zactato
- This looks super interesting. I'm trying out the hosted app using "bring your own key", I've added an OpenAI key but it doesn't seem to let me generate a 3d model. It's still saying I need credits. Is this expected?by WilcoKruijer
- Have you explored optimizing the assets to be game-ready?
This kind of decomposition works if you have a single object on screen, and it's super artist + programmer friendly. But the generated assets have ~50 mesh parts, which means importing just a couple of these into a scene and you've blown your entire draw call budget for a shippable game; once you start adding things like shadowing it's game over. It's the brick wall every gamedev realizes after trying to make a scene out of easy-to-work-with primitives. You just can't hit a playable frame rate like this unless your entire game consists of just a few objects.
Have you experimented with atlasing, mesh fusion, baking animations, standardizing PSO's to a scene budget, etc? Because if this can't be automated, I've found it really limits the utility of such freeform generation techniques, since the approach is fundamentally incompatible with performance on today's graphics stack.
by avaer - > there's a showcase (+ github repo) you can play around with: https://nova3d.xyz/
Before anyone else bothers giving them your Google account, there's zero free generations, something they conveniently don't disclose until after funneling you to sign up.
by RobotToaster