How do you guys stop coding agents from acting out of line?

How do you guys stop coding agents from acting out of line?

3 pointsby vivekyyy2 comments

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

  • Hacker News
  • Have you consider asking your agent to set up basic permission deny rules to block dangerous commands? Just ask it and request that it shows you the config before applying it.
  • I can add some basic configs, but it seems kinda provider dependent, so I'm gonna need to do this differently for Claude, Cursor, and Codex. It will also require a custom pre-tool use plugin to allow something like "AI can generate DB migrations via CLI but cannot handwrite them", which is pretty annoying when I need a separate one per model
  • I use a lot of agent hooks to force deterministic guardrails in the moment for faster feedback loops, like someone else mentioned. Others like pre-commit hooks and CI checks are still great for things like enforcing conventional commits, linting, etc.

    What framework do you use for generating migrations? I can help share an example of what it might look like to write a hook for it. Or even your top 3 pain points.

  • We use alembic for it, kinda curious what a hook config would look like here. Alembic files are pretty nice because they basically specify what sections of the revision should and shouldn't be manually edited, but agents routinely just ignore that lol
  • Telling llms what not to do actually makes things worse. Frame things in positive language where possible, if not possible I don't mention it at all. The biggest fix for me is using better models. You can't rely on guardrails to make dumber models act smarter.
  • It's often easier to catch and scold them than to prevent it. We have rule for example that agents (or anyone) can't touch autogenerated code. They still routinely do, but there is a CI job that re-generates everything and fails if there is a diff. Which the agents then see and fix. We have also use hooks for the same thing since they run locally and the feedback cycles are shorter.
  • This is possible, but I just kinda feel like at that point I should just generate the migrations in CI. The point of having them out of CI (for me at least) was so we can manually write in upgrade/downgrade logic when the autogenerated migration is a little off