Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I hope this kind of stuff puts the idea to rest that we're close to actual AGI. Outsourcing this kind of basic stuff which a real intelligence would be able to do "internally" is a hack which works for this specific case but would prevent further generalizations of the task at hand.

    But I'm forseeing the opposite. This kind of tool use will soon be integrated and hidden such that people will eventully say "see we solved the problem that AI can't do 123+456, now we are really really close to AGI. Yeah no, with an AGI, it would have been the AGI itself that would have come up with needing at tool, building the tool and then using the tool. But that's not what LLMs are. They are statistical machines to predict tokens. They are very good at it, but that's not an AGI.

    by teiferer
  • This seems analogous to how a human would do it accurately. If you asked an artist to paint stones in a large circular arrangement with the numbers in order in one shot, with no fixes or sketching allowed, it wouldn't be surprising to end up with problems in the arrangement.
    by petercooper
  • Love the concluding note : it works, but not really.

    So LLM/GenAI crave. An entire article to show that it's nearly there, yet it's not, despite convoluted effort to make it just so on a very very niche example.

    by utopiah
  • But if it works part of the time, it's useful. It's easy for a human to check that the numbers are correct, and if they aren't, just regenerate the image. Orders of magnitude easier than creating the image from scratch without the model.
    by Al-Khwarizmi
  • I was thinking about doing the opposite for the common task of "SVG of a pelican riding a bike". Obviously, directly spitting out the SVG is gonna be bad. But image gen can produce a really stunning photorealistic image easily. Probably a good way to get an LLM to produce a decent bike-pelican SVG is to generate an image first and then get the model to trace it into an SVG. After all, few human beings can generate SVG works of art by just typing out numbers into Notepad. At the core of it, we still rely on looking at it and thinking about it as an image.
    by dllu
  • Inpainting/guiding from a sketch is how I've always used diffusion models. I thought everyone did that, or at least everyone who wasn't just trying to get some arbitrary filler material without much care of what the output looked like.
    by nullc
  • We've been doing this for a long time now, it's similar to using a depth map or a line drawing to control the silhouette.
    by Geonode
  • Yeah, I’ve used a similar technique to build a "pizza clock" before (where the number of slices corresponds to the hours).
    by usagisushi
  • The standard objection: if the LLM is supposedly intelligent, why can’t it figure out on its own that this two-step process would achieve a better result?
    by xigoi
  • They are not, in fact, intelligent.
    by airstrike
  • Nobody asked it to!
    by nine_k
  • You don’t know what you don’t know
    by pyrolistical
  • Because the LLM is more or less hardcoded to just pass "create image" style prompts to a separate model, possibly with some embellishment.
    by Sharlin
  • Part of the problem is that it isn't the LLM making the image directly itself, it's the LLM repeatedly prompting edits for a separate edit diffusion model. The Gemini reasoning summary shows part of this. The style of some of the images makes it also clear that it uses an Imagen 4 derived diffusion model underneath.
    by cubefox
  • Because image models at the basic level are just text tokens in, image tokens out. You'd need an agentic process on top to come up with a strategy, review output, try again, and so on.

    I believe Nano Banana and gpt-image-2 have a little of this going on, but it's like asking a model to one-shot some code vs having an agentic harness with tools do it. Even the most basic agent can produce better code than ChatGPT can.

    by petercooper
  • This hack definitely falls in the “duh, why didn’t I think of that” category of tricks, but glad to now have it next time imagegen comes up short
    by sparuchuri
  • Even the original stable diffusion app had image 2 image. It just didn’t work as well. I‘m not sure why this is supposed to be novel.
    by manmal
  • This is just img2img where first image with correct structure was generated by code.
    by smusamashah
  • Right, but you can use a different (codegen) model to make that code.
    by philsnow
  • Pretty much what the author said- just gave some context for the uninitiated
    by jasonjmcghee
  • Yup, that’s exactly what this is. If you’ve been using generative models since the early Stable Diffusion days, it’s a pretty common (and useful!) technique: using a sketch (SVG, drawn, etc) as an ad-hoc "controlnet" to guide the generative model’s output.

    Example: In the past I'd use a similar approach to lay out architectural visualizations. If you wanted a couch, chair, or other furniture in a very specific location, you could use a tool like Poser to build a simple scene as an approximation of where you wanted the major "set pieces". From there, you could generate a depth map and feed that into the generative model, at the time SDXL, to guide where objects should be placed.

    by vunderba
  • I found a simple technique to get reliable text and numbers in AI generated images.

    I’m surprised the image models aren’t already doing this, so wanted to share since I’m finding this so useful

    by samcollins
  • TLDR: use SVG to outline image correctly first, then send that image with your text prompt to get Gemini 3.0 Pro to render with correct numbers and text
    by samcollins
  • Very impressive, simple, and reliable. I'm sure it will be picked up by image generation labs soon.
    by jere
  • Isn’t this sort of just “chain of thought” (i.e. the seminal https://arxiv.org/abs/2201.11903 ) where the user is helping the model 1-shot or k-shot the solution instead of 0-shot? I’ve used a similar technique to great effect. I feel things are so new / moving so fast that it’s hard to have common lingo. So very helpful to have a blog / example! But I wonder if the phenomena has been seen / understood before and just in smaller circles / different name.
    by choppaface
  • In some ways, this is similar to use of a Control Net. I've been doing this same technique for a while, using only SVGs as the base image. Works well.
    by bsenftner
  • > Transform this image into a photographed claymation diorama of assorted artisan chocolates and candies […] viewed from a low-angle

    Side note: whenever I read prompts for image generation, I notice very specific details which the model obviously ignored. Here the chocolates / candies in the last two images look anything but artisanal. They look very "sterile" and mass-produced. The viewing angle is also not accurate.

    Why do we even bother writing such elaborate prompts, when the model ignores most of it anyway?

    by IdiotSavage
  • I wonder how long it took to come up with all this?

    Because if I wanted a spiral of little "buttons" like the last one at the end (and they don't look very much like sweets) I'd be able to knock that out in Blender in an afternoon, and I'm not very good at Blender.

    by ErroneousBosh
  • There might be an overlap between people who use AI enough to write such posts, and people who don’t respect craftsmanship. The output looks fine to them because they never trained their eye to look closer. They vaguely hear music but never listen to the notes.
    by nicbou
  • I believe most detailed prompts are AI generated.
    by ryanthedev