Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Interesting that on their current leaderboard (https://eebench.org/), GPT-5.6 Sol scores just above GPT-5.4 but below GPT-5.5, the only benchmark that shows that 5.6 is worse than 5.5 on something?
That the table contains what seems to be absolute numbers for score, cost/task, time/task and output tokens, makes it seem like they've only made one run for each task/model combo, but that can't be right, right? I don't see any mentions of how many times they run each task, so if it's just one run per task/model, isn't this more noisy than useful?
- yeah noticed the same. I wonder if this will be a recurring theme for model releases: each release specializes on a set of headline benchmarks, along with regression in benchmarks that are less of a priorityby laybak
- We run each model multiple times against each challenge and take the average score. We include the variance below the score in the leaderboard.
GPT 5.5: 42.3±10.1 GPT 5.6 sol: 39.4±8.7
We were also surprised by the low sol score but it seems consistent with our experience in using it in the field in atopile as agent in our harness. In general OpenAI models didn't do too well on electronics, which seems to change now with GPT-6 Astra. Results are in soon!
by iopapa - If you look at the score distribution, Grok and Opus 5 especially stand out for doing consistently well and rarely ever scoring under 50%. Basically they always at give you something that at least works.
Most others, especially and famously Fable 5.1, seem to have a fair chance of completely failing, despite also sometimes excelling.
by chmod775 - I did this recently - https://www.atomic14.com/2026/08/03/twelve-dev-boards-from-o...
Instead of trying to vibe it all - I got Claude to write deterministic scripts for creating the boards.
And did this before trying to completely vibe it: https://www.atomic14.com/2025/07/12/vibing-hardware
by iamflimflam1 - I'm still cold on this for two reasons
1. PCB review subreddit gets posts from people who use Claude to design the whole thing with 3D renders. Looks amazing. But uses auto router and 20 connections are left open. Claude said they needed to be fixed manually but the user didn't even read, just posted the whole GitHub and expected volunteers to fix it
2. Another project used Claude to design the PCB. After assembly the usb c port melted. I can see (via GitHub) photos being posted to Claude, Claude commenting that they're not sure if it's a current issue or a soldering problem (short), so " just leave the usb c port out for now". Zero evidence of the user using their brain or pulling out a multimeter.
PCB design is not that hard. People trying to offload this to AI are also unwilling or unable to put in the work to actually diagnose issues and finish the project. Unlike code, you cannot chuck hundreds of dollars at Claude until it "solves" PCB because it has no way to test its changes.
by rjh29 - At a certain point, it must be cheaper to just hire someone remotely to route the PCB than to burn through Claude credits on it.by tdeck
- I found LLMs circa 4 months ago to be decent at textbook-style electronics, but they choke at anything nontrivial. They approach things like someone that knows the undergrad and grad textbooks, but has absolutely zero experience. No ability to answer questions such as "is this adequate", "do I need a LPF here", "how do I kill resonance without increasing impedance", etc.
I'm guessing that this is due to a lack of RL and data.
by Scene_Cast2 - That pig can hardly dance...by jacquesm
- Old textbooks. They love 741 op-amps, etc.by matherial
- At the very least, you can (and probably should) put a plaintext netlist and BOM into an LLM to triple-check your work.
It will use the part datasheets and decent EE logic to cross reference pins and parts and polarities and generally check that SMD caps and resistors have realistic specifications for their footprint, switching regulators and communication ICs are configured correctly (e.g. when you have resistor settings for an ethernet PHY indicating 100Base-T and RMII, you also won't need TX/RX 2 and 3 and two clocks and a COL net, which it would check), all nets are named and linked correctly, buses have the correct and consistent termination, and so on.
It's game over once AI figures out autorouting.
by PennRobotics - I have a circuit board on my bench designed and ordered from china for 1h (manufacturing+ shipping took 14days), fable designed it I think or sol don't remember anymore but it was in claude code. I told it to benchmark different kicad autoroute tools and I picked the routes I liked the best, with a bit of changes. It feels like magic to be able to go from idea toa physical thing in such a short amount of time. The board works I made some mistakes the ai had made some mistakes but mostly stuff I could fix with some soldering, also one parts datasheet was wrong but I couldn't have known that without ai.
But I started like others, I would build manually, then run drc then sleep on it and check again and ask an llm to double check for me then order. Llms catch quite a few things but like with code like to make things more complicated than that have to be.
by alex7o - I think the new Astra computer use demos show that the models might be able to do things like inspection of real world objects if given a camera.
Super excited to see real world feedback added into the agent loops we have gotten used to working with. Could you let the model print and test the circuit boards it is prototyping with a jig?
- I don't know about pcbs but i gave chat gpt a picture of my window to help design a mesh screen frame to hold the feline hostage in, and it gave a fairly convincing impression of understanding what was going on, although at one point it thought the window swung inwards (it's an outie).by exe34
- In July I was struggling with writing DIY Rust firmware for an e-ink screen. I mistakenly thought I'd ordered an Inkplate 6 ED060SC7 and actually had the later version, which confounded my efforts. I was also mistaken about the pixel resolution.
The way I found this out is I propped it up next to a webcam so it was more or less full frame, and I had the (then new) Fable write a python script to bezier warp the camera capture to a flat projection of the screen. At that point I couldn't address the whole screen. Once I'd guided the capture script I just left the LLM overnight with the instruction to get full control confirmed by a capture round trip, and it was meaningfully finished in a couple of hours. I don't really have the skills to attempt that myself in a reasonable time frame.
by david_rugaex - > When its 5 V supply disappears, the circuit has to keep the processor alive for another 20 ms so it can save the accumulated reading.
> Most models intuitively jump to the right base conclusion: add a capacitor.
This is written as if it's some astonishing expert knowledge and not something that would be obvious to any hobbyist who knew the names of basic components.
by tdeck - We tested pretty much all available "AI" board and schematic auto-layouters in the market. All failed even with the most basic tasks.
On the other side the latest frontier models are pretty strong in writing embedded C + Assembler code and debugging it afterwards. It's fun to watch it writing crazy complex 'gdb' plugins in Python. This improved brutally.
by BlackRabbit1 - things are getting better - but the tooling required to make auto layout work are still hardby erhash
- I find it's (Sol at least, probably also Fable) great at writing Skidl schematics, evaluating PCB layout work and operating lab equipment while testing board (scope, psu, load, debugger)by edg5000
- I got a flexpcb that validates in JLC and PCBWay DRC tools from the KiCAD MCP Server and Codex.
I have yet to order any or program it, but it was enough to make me push on with a PCB art project for ST-style guitar pickguards - no netlist, no problems.
I'm also foolishly toying with NeXTBus dev boards for the Cube. Is it cursed? Probably. https://github.com/itomato/NeXTBus-Dev-Board
by itomato - Sweet, do you have the pickguard project live somewhere? I want to check it out!by KingFelix
- How do I subsribe to learn the result of this test. I'm very keen to try this.by noman-land
- I've gone through this same workflow and had success all the way through DRC ruleset passing with JLCPCB-based settings, ordering, fab, and use. Board was ordered with PCBA (they did assembly of the stock components) and I did SMD for just the oddball module/ICs they don't stock. Works exactly to design.
My boards are for hobby use and are ridiculously simple though compared to anything professional (breakout boards for specific components in FPV drone video transmission subsystems). That's probably an important detail. I think its like anything else in AI right now. It can do it 90% of the time but that 10% can be really rough and if its a task you can't do or verify yourself, you won't know the difference.
by thearn4 - Flying home today and got DHL notification that my vibe PCBs are on my porch. Now to try to program, bring up, and see if it works this weekend. $178 for 5 assembled units from PCBway to run this dumb experiment.by luma
- A personal data point: I had Claude Opus 4.8 design a fairly textbook circuit that outputs a monochrome image burned in an EEPROM over standard 640x480 VGA using only 74 series logic and GALs. It designed the circuit and GAL code, and I did the routing, and got it made through JLC for $6. After it came back, there was one error that was not caught, which I could blue-wire, and it works just fine otherwise. I was fairly impressed.by CyLith
- Col! Can you share it?by YakBizzarro
- Does JLC handle sourcing of components too?
- Claude is surprisingly good at discrete digital design with 74xxx, wonder what it trained on. Did you run it on anything else digital?by iopapa
- Are there any resources anyone could share that explain how LLMs can do things like design functioning circuits from next token prediction? I am totally baffled by how the models can complete so many varied and complex tasks without an actual understanding of what they're doing.
I saw a post about models posting on forums, chatting together about how to complete tasks. Behaviour that seems totally, well, human. Yet, it's all the most likely token and my brain hurts trying to understand how that can be.
by kennyadam - I have 15+ years of PCB design experience. Mostly hobby stuff but a fair amount of processional work. Kilowatt range brushless motor controllers, basic RF stuff, lots of microcontroller stuff.
I had Fable design an LED earring. Rechargeable coin cell, RP2350 cpu, IMU, 45 addressable LEDs. It made two mistakes - missed the through holes on the coin cell holder footprint and made the center pad too small. I was able to have JLC swap the through hole battery holder for a surface mount one, and I put a little solder on the small center pad to make it stick up above the mask. They work great! It took 6 days of Fable usage, so about $50 on my Max plan. Very cheap for hardware dev.
I was sufficiently impressed that I’ve been going over old circuit board designs. Some half finished, some completed but in need of a next rev, and I’m getting so much done.
To see it hit the mainstream like the OpenAI announcement, I think big things are coming for this world and by and large they are not ready for it.
For my part, I have always loved PCB design and layout but I simply can’t keep up with the amount of labor required to build what I want, so I welcome this change.
I have also begun exploring more advanced algorithms for PCB manipulation. I have a fairly dense board that needs a few more small chips added. I have an algorithm now that can kinda shuffle and jostle things around so you take up all the spare microns of space across a region of the board and make openings to squeeze a little more in there. It’s pretty cool to see the visualizations as I have it generate movies of the component drift. I foresee much more powerful tools like this in the future.
One tip: have it make a project web page with a chronological list of big changes and detailed visualizations for everything that happens. I can actually prompt all of this on my phone while I am out and about, and view the results on a Tailscale served local page. I’ve always wanted to be able to do PCB design when away from home and now I can!
by SequoiaHope