Halo – open-source, tamper-evident runtime evidence for AI agents

Join the discussion

• Hacker News
• Looks very slop, but it’s a good idea. The main difficulty is that no big name is hosting a witness.
  by nf-x
• Fair point - I am a marketer by training and built this with some help from Claude! But appreciate the love.

  If you find something/have feedback, please let me know and I'll gladly fix it.

  Separately - 100% agree on the witness - only someone/thing outside the vendor can prove nothing was omitted. Who do you think fills that gap - is it an audit firm (my world), a standards body, or something else?

  by brian_kuan
• Why a Python library instead of a completions API proxy?
  by solarkraft
• Proxies are blind to sensitive things enterprises might worry about (eg. the agent read a database, wrote a file, sent an email, etc) as those happen in the agent's own code. That said, if you already run a gateway, its logs can be ingested into the same chain.
  by brian_kuan
• Could I monitor something like Claude Code or Codex with this?
  by rmonvfer
• Yes to both!
  by brian_kuan
• BTW I've added an example to the README, same setup I run on my own machine: https://github.com/bkuan001/halo-record#record-your-coding-a...
  by brian_kuan
• are you thinking co signing to a third party or something like a transparency log style?
  by gawkdev
• > may have built observability dashboards and audit logs, but those are editable and partisan

  Why would these be editable?

  by ambicapter
• Because they live in infrastructure the vendor itself controls - there's some conflict of interest there. Things like retention, rotation, deletion, what gets included/excluded, etc are all decisions the vendor makes. Same reason why compliance requires audits!

  The hash chain doesn't make the log unwritable, it makes any edit detectable.

  by brian_kuan
• I've been working in the same lane. The vendor uses the same technique that academic preregistration uses in experiments. No audit firm/institution/organization needed. The customer just compares what was delivered against what was pre-committed. This way if something is off, it doesn't just show up as nothing... it shows up as a gap.
  by mdellison
• Very cool! Heard on all of that - IMO you didn’t remove the witness, but instead distributed it so that every customer becomes their own witness. The catch is pre-committing catches declared-but-missing, but not the thing nobody put on the list, which is the case security teams fear most.

  Would love to hear more about what you’re building!

  by brian_kuan
• I'm currently working on an agent framework that has auditability as one of its core promises. I'm glad to see others are working in this domain!

  I've seen other products/apps in this space farther up the stack at the API boundary.

  What frameworks does your package work with? How does it handle intercept?

  by all2
• There's no interception - it runs in-process, so it works with any Python code (with or without a framework). There's a TS version too if your stack is in JS.

  Would love to hear more about your agent framework!

  by brian_kuan
• PS: please try to break it - if you find that the report does not catch a deleted line, changed number, or modified record, I'd love to know!

  And to start a discussion: if you sell or buy AI agent products, what do security reviews ask about them?

  by brian_kuan
• > Disclaimer: this proves integrity, not completeness (as a self-held chain proves nothing was edited but does NOT prove that nothing was omitted).

  Or as the page puts it in more detail:

  > A self-held chain proves integrity: nothing was edited or reordered after the fact. It cannot prove completeness: the operator of a recorder can delete the bad day and re-seal the chain, or never write a record at all, and the chain stays internally consistent.

  I don't get this. If I "hold" the "chain" (I hate the jargon agents invent), why can't I edit or reorder and then "re-seal" it?

  by derdi
• Yeah, if you edit the log you could make a new chain when you fully control the chain. And if you send the log to someone else, they can hash it themselves to see if you tamper with it later. I don't get it too. What is the chain good for?
  by jona-f