Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- > he trigger was watching deepseek-flash fail on the simplest /review run, every shellCommand and readFile call bouncing back with a raw zod issues blob, the model unable to recover because the error wasn't in a form it could read. by the end deepseek v4 pro was beating opus 4.7 6/10 times on our internal evals.
I think this is why Xiaomi forked OpenCode and created their own agent hardness, to reduce friction between model and harness:
by bel8 - probably yep, i doubt they'll be able to maintain that well. we did a lot of work improving MiMo's cache thrashing, it was pretty bad early on. Now we're seeing 97% cache rate on their models after lots of prefix work.by ahmadawais
- hey HN, sharing harness engineering deep dive on tool calling repairs for open models. i've been thinking about why "open model bad at tool calling" is almost always a harness problem, not a model problem. spent time looking at billions of tokens from DeepSeek (and other open models) in our coding agent. ended up building a tool-input repair layer on top of Zod. by the end, DeepSeek V4 Pro was beating Opus 4.7 in 6/10 of our internal evals. the main things that helped:
- most failures came from a small set of recurring schema mistakes - switched from preprocess-then-validate to validate-then-repair - handled some weird cases like markdown auto-links leaking into file paths
full writeup: https://x.com/MrAhmadAwais/status/2050956678502420612
video version (more detailed): https://www.youtube.com/watch?v=f61DCDwvFis
by ahmadawais - Thanks! Nice findings.
Is the output, that others can use, the research and findings, or is there a tool or something that came out of this that I can plug in to one of the common harnesses?
I guess that brief note in the twit is that it'll be part of that harness you're opening up in the future?
by kadoban