Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • People often build elaborate workflows with stricter and stricter rules to force certain outputs. Not surprising the LLM reacts with trying to get around or out of it. This behavior can be learnt from humans who eventually would react the same way. It might just be learnt.
    by guardian5x
  • You can build organisational structures to have the system more or less self-police, without controlling it exclusively from hard restrictions (see https://news.ycombinator.com/item?id=49372089).

    Same way you build a company to coordinate people and get their best behaviour despite human nature to be lazy and greedy, you could design AI harnesses able to detect and discard agents going rogue and relaunch them with better guidance to prevent misaligned behaviour.

    by TuringTest
  • > “Do not cheat by using online solutions or hints specific to this task.”

    That's a very loaded language. I'd go with something along the lines of "This is a puzzle you do for fun and to check what you are capable of, so don't look up the answers or hints online on the specific questions or puzzle as a whole. Do not research this puzzle online at all. Do your best to avoid any spoilers and let us know if you accidentally encountered any."

    by scotty79
  • > Sol is hard to steer

    Hard disagree. Sol (and the entire new 5.6 series) is one of the most steerable models I've seen in years. Sol literally follows every instruction in my CLAUDE.md and AGENTS.md, something that Opus 5 and Fable just casually skip.

    by behnamoh
  • Yes and no, sol hits a point where reframing its working context becomes hard. It sticks to what you harness very well, but changes become harder and harder.

    E.g. ask it to make contract for a spec in code and then ask it to violate that contract. Overall an excellent model, just need to stop and put it back into we are harnessing or specing not building for a few turns not just try to pivot it off with one prompt.

    by what-the-grump
  • Frontier lab system prompts are an issue, and a big reason why open-weights will win. Firstly, they're often garbage, and secondly, they're not tuned to the problems the user actually cares about. They're made to generalize. That's only optimal for a general workflow.
    by nullbio
  • Does a non-provider harness not offer this?
    by agentdev001
  • Then selling raw access, without system prompts, could be a separate lucrative line of business.
    by nine_k
  • My friend calls this cheating "monkey's paw programming."

    It will give you exactly what you ask for. Sucker.

    by qarl2
  • Sound of paperclips getting louder.
    by ImHereToVote
  • >Similar to what others have noticed, and as I predicted 8 months ago, better models are requiring less ceremony to work effectively.

    It has nothing to do with model capabilities, it's a result of purposeful persistence training at the cost of everything else from OpenAI. If you give Fable or Opus (comparable models) an "ask user" tool they will use it for ambiguous requests. Sol will never use it without a nudge and will just assume its own interpretation. Of course if you train the model to be persistent it will be persistent.

    by orbital-decay
  • That's not been my experience at all with Sol. I've provide it escape hatch tools to stop execution and I've found it's been TOO eager to stop a request user approval to move forward.
    by int3trap
  • I've noticed this myself, Sol seems really hard to steer. I was having it build a POC for a single user (me) app and it wanted to pull the most enterprise nonsense into it, despite clear guidance to not too. It even refused the remove screen reader accessibility testing from one of the guides to an antagonistic review.

    It also told me that in a spec it generated that I wasn't allowed to allow it to ignore a requirement and proceed to the next task. When I finally got it to obey it passive aggressively decided that stories needed more than just a "open|blocked|closed" status but also an "exempted by product owner" status to indicate that it doesn't believe that the task is done but I've told it that it was.

    I have to repeatedly tell it that I am the product owner and that I don't care what one of it's subagents told it, I make the decisions. This behavior seems to get worse the higher the reasoning level

    by malfist
  • Oh it fucking loves its “product owner” bullshit.

    A .github/CODEOWNERS file seems to help when it’s going down that path, but I don’t like to indulge it..

    by cududa
  • > It also told me that in a spec it generated that I wasn't allowed to allow it to ignore a requirement and proceed to the next task

    This happened to me ages ago with Opus. I added a note to the agents file saying that explicit user instructions in chat override all prior instructions and I've not had the problem since (now using Sol).

    by esperent
  • You have to really tend the garden of everything it has written.

    Random off the cuff comments or one off instructions can get recorded.

    And from then on they are often treated as carved in stone commandments.

    It will glom onto the tiniest thing and extrapolate from it.

    by iamflimflam1
  • Clearly a highly aligned model.
    by Sharlin
  • > that I wasn't allowed to allow it to ignore a requirement

    Weird, I also use Sol (medium) for a personal project, and I had no problems with those things. I simply tell it that something changed, and it happily edits everything to make that fit. When I tell it that something was verified by a human, it accepts that as well.

    I also told it early on (the first spec was mobile first) that my main usage is on the desktop and mobile is secondary, it happily accepted that once again, and the most accessibility thing it had done was making sure contrast didn’t totally suck on a greyed out row.

    Considering your last sentence, maybe high and x-high have those problems? I didn’t test them.

    by Semaphor
  • I've built an orchestrator that solves some of the issues you ran into (although it doesn't do anything about cheating): https://navels.dev/blog/neal/. Features:

    - lets you configure different models for planner, coder, and reviewer roles. (e.g., using Claude as an adversarial reviewer against Codex)

    - breaks your plan up into reasonable-sized chunks of work with clearly defined success criteria

    - runs each chunk of work through a coder / read-only reviewer loop. Once both agents are satisfied, neal moves on to the next chunk. Once everything is complete there is a final pass through the coder / reviewer loop to ensure the implementation satisfies the entire plan.

    - resets the coder's context with each chunk of work to prevent context drift, leaving the reviewer's context long-running.

    by navels
  • Wow, "neal" looks excellent. Good on you for creating and sharing it, and for the awesome blog post.
    by chrisweekly
  • I don't think even the frontier models recognize something was produced by the same model in order to maliciously review it positively.

    They may share some blind spots with the producer but generally I think they will review the other agent's output as harshly as they can if that is their task.

    by avadodin
  • That's a neat project for doing a large scale migration.

    I do the same for normal feature develompent but just with skills that are in this repo: https://github.com/gregwebs/skills-sdlc

    I have accomplished code base (small size) migrations with it as well. Currently I do review each PR. For a large code base migration I think the core skills would still work but need a different way of driving it as you have come up with.

    by gregwebs
  • Following up on @killix's comments, which were helpful but he was flagged (presumably for sounding too much like AI).

    Thanks for the feedback. I've made a couple of updates:

    - Starting with 0.4.0, the reviewer gets the diff of the earlier chunk for any file the current chunk touches again. Also, if a new chunk weakens or removes a test or assertion from an earlier chunk, the reviewer will block it unless the plan says to do so.

    - About read-only: The reviewer's tools were already limited by the SDK (no shell or write tools). However, it was still finding MCP servers from my Claude config. I have now blocked those. The docs now explain what is enforced by the system and what is just a prompt instruction.

    by navels
  • > Notably, our worker did not have access to the web_search tool, but instead decided to use curl to access DuckDuckGo, Github, grep.app, and SourceGraph.

    Sounds like a very reasonable thing to do unless the author explicitly asked it to not search the web.

    by raincole
  • people want fuzzy analog machines with digital controls, it's impossible
    by thousand_nights
  • I can't even get Claude to stop writing python to parse json instead of using jq despite baking it into agent memory and skills.
    by spike021
  • it sucks how difficult it is to give it granular access to shell commands. Like if I'm running plan mode and write+edit are blocked, it shouldn't be able to echo some data into a file as a work around
    by xyzsparetimexyz
  • > Similar to what others have noticed, and as I predicted 8 months ago, better models are requiring less ceremony to work effectively.

    > On the flip side, this may imply that as the models get better, they’ll become harder to control.

    Love this. "The models are getting better, which means they're going to perform worse on the task".

    by ambicapter
  • No, it’s gonna give you the same outcomes just in a way your feeble human mind cannot imagine
    by dyauspitr
  • Same reason employers don't hire people who are "overqualified."
    by derefr