Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- It's my first time hearing of program analysis but the core seems related to what I studied at college and never used again, which was prolog.
Could anyone with more experience give feedback on whether this approach would be useful for business rules? Particularly for debugging a data pipeline where dozens if not hundred of different values at different points in time may have different implications.
Would it be useful to provide this tools for Business Analysts so they have something more solid for they analysis?
by anktor - This is the fact I’ve been struggling with for quite some time. It’s not because it forgets the facts, it’s because the invalidation doesn’t propagate.
My way of handling that is a decision log. For every project since I started doing that it’s working great. My CLAUDE.md instruct the agent to store my every decision to the file with a metadata when I made this decision and what was the context. The agent is using this file as an index of decisions and rarely lose a track. It also helps team members to find out more about the development phases.
Does your system invalidate the parts of the memory if these are not valid or relevant anymore or just store/retrieve?
by coder-pm - Something of this capacity would be useful in investigating obscure hardware failures in the logs that I couldn't confirm because the problem was not being observed while the device was in my shop. the problem was surfacing in another location probably due to some set of circumstances in the software that I could recreate, or some particular peripherals that were attached.
I ran into the very same problem of the LLM forgetting that we ruled out a conclusion that was verified not to be the cause as it came up further in the conversation history while I was exploring possibilities.
I had to keep reminding we ruled out that conclusion prior.. I just carried on with having the LLM capture some of the supporting sources of other people experiencing the same problem and kept having to refine those sources because it was focused only on summaries, but eventually i got the sources to a point where they were good enough hypothesis that we could formulate a better conclusion on what the potential cause was.
by trinsic2 - I encountered this with trying to have LLMs populate facts about electoral campaigns. Like when a candidate drops out, when endorsements happen, but also if a candidate is un-endorsed or drops and rejoins. It also needed to handle if any of these facts were incorrect.
I settled on a knowledge graph in Postgres and downloading/storing the source documents so it could iterate on past results without more scraping or network calls.
This blog post helped me understand security analysis in this context! A lot of the important systems around malware analysis or large scale system security (the parts people really care about) clicked for me. So thanks for writing it.
Anyways I hope we can find some pattern to converge on with this wrt "facts management" since I feel this is currently something a lot of people and LLMs are struggling with. In practice current LLMs working with episodic memory feels similar to a grandparent with dementia scrawling things down in notebooks, crossing things out, and getting very confused.
by jarboot - This really matches up to my experience on long research projects with Claude.
It’s very hard to remove information - Claude has a habit of recording things all over the place and will happily treat things as facts even after they’ve been disproved.
What is currently true can get easily contaminated with old “facts”.
by iamflimflam1 - Very cool. I recall an HN submission (which I can't find offhand unfortunately) that did something similar -- it used an LLM to decompose articles into a set of statements which were used to construct an entity-relationship graph of facts and events. It then queried that using conventional graph query methods, much like DataLog / Lemmalog is doing here. I remember it was particularly effective at answering timeline-based queries that LLMs (back then) sucked at.
(See also Cyc: https://en.wikipedia.org/wiki/Cyc)
I think approaches like this are going to be (or maybe already are?) the basis of effective grounding of LLM responses in authoritative data sources. It should be possible to pinpoint any error to an incorrect traversal or an incorrect "fact." This would work best for concrete, unambiguous facts, however; fuzzy, ambiguous or opinion-based information will probably remain the purview of LLMs.
by keeda - So he's using an LLM to generate data stored in an "is_a" representation. That's so classic AI.
Soon, he'll discover that he needs quantifiers. Then that "for all" is too strong sometimes, and he needs "for most". That way lies Cyc.
It's not a bad idea. But it does have a history.
by Animats - I reached a similar conclusion: LLMs should only really sit at the terminals of request fulfilment.
1. User request understanding: natural language -> a more rigorous representation, in my case Datalog.
2. Result interpretation: facts and derived facts -> natural language.
Between those terminals, the work should be mechanical reasoning over some ontology or formal knowledge structure.
That connects to another principle I've been thinking about, which I call Weathering: useful reasoning should change the shape of the system. If an LLM has already had to infer a relation, mapping, rule, or abstraction, repeated use should wear that inference into the system so that the next similar request doesn't require discovering it again from scratch.
With continued use, a weathering-capable system should therefore require less and less probabilistic intelligence for recurring work. Put another way, there should be a declining marginal cost of cognition since the products of intelligence harden into structure that can subsequently be reused and evaluated mechanically.
by sim04ful