Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I also like the index feature form https://maki.sh Source code has a lot of structure, using a real parser instead of grepping and reading files can potentially save a lot of tokens
    by wrxd
  • Interesting, will have a look at this, thanks for sharing.
    by Bibabomas
  • Related read (not from me): https://entire.io/blog/improving-agentic-search-in-coding-ag...

    > The clearest result was that faster search alone only modestly helps, while better-ranked results improve first-query retrieval and help agents find the right code sooner.

    Their tool "pgr" is a research preview only, so it'd be interesting to see semble vs pgr.

    I'm also collecting other tools that are similar, most notably is probably Morph's WarpGrep (has a free tier too). Apart from that, there is codemogger (https://github.com/glommer/codemogger), cs (the author also commented in this HN post).

    In the similar area, but not fully related, the author of fff is also pretty involved in any thread that goes into that direction (see e.g. https://x.com/neogoose_btw/status/2052161471296225710). Similar to colGREP is also mgrep (by mixedbread) and osgrep (but they seem to predate colGREP). I also found codedb on X (https://codegraff.com/blog/codedb-code-intelligence), the post reads well, but haven't tried.

    by kurtextrem
  • LSPs are already miles better than grep-like tools. This was true for humans as much as it is for LLMs. It's a shame that Claude still treats it as a second-class citizen (both in the app itself and in the training). A simple "remember to use LSP instead of grep" is usually enough to get it on the right track.
    by jelder
  • Better than grep obviously, but how does this compare to existing LSPs?
    by AussieWog93
  • Or tools like `ck`: https://beaconbay.github.io/ck/
    by CharlesW
  • Semantic code search seems like a useful tool for a human too. Not just for agents.
    by singpolyma3
  • Yeah I agree. I have used semble to quickly index a large monorepo and just ask a question about it, it surfaced the right files pretty quickly. Although without an IDE, it's difficult to display them in nice way
    by stephantul
  • Nice, this sounds great. I want to mention a related issue here, which is that on small codebases, Claude spends a lot of time looking for stuff when it could have just dumped the whole codebase into the context in one go and used very little tokens.

    I found a nice workaround which is that you can just dump the whole directory into context, as a startup hook. So then Claude skips the "fumble around blindly in the dark" portion of every task. (I've also seen a great project that worked on bigger repos where it'll give the model an outline with stubs, though I forget what it was called.)

    by andai
  • This is true, agents just don't know a lot about the things they're looking at, e.g., the number of files, file sizes, etc.

    Although for small codebases it also holds that whatever you would like to find it easy to find, so search still might help you with cost

    by stephantul
  • Maybe aider? https://aider.chat/2023/10/22/repomap.html
    by mackenney
  • I know this tool was meant for AI, but I am more interested in using it myself when exploring new code bases or even my own, when I want to refactor something and want an overview of where to change stuff. LSPs do that, but this tool sounds like it can go one step further.
    by adelks
  • Seems like a cool idea so I decided to play with it a bit. The test I ran was in the browsercode (https://github.com/browser-use/browsercode) repo with the following prompt:

    "Answer this question by only using the `semble` CLI (docs below):

    > What tools does Browsercode provide to the agent other than the base OpenCode tools? Provide the exact schema for tool input and tool output and briefly summarize what they do and how they work

    ---

    [the AGENTS.md snippet provided from https://github.com/MinishLab/semble#bash-integration]"

    And the equivalent for the non-Semble test:

    "Answer this question by only using the `rg` and `fd` CLIs:

    > What tools does Browsercode provide to the agent other than the base OpenCode tools? Provide the exact schema for tool input and tool output and briefly summarize what they do and how they work"

    In both cases, I used Pi with gpt-5.4 medium and a very minimal setup otherwise. (And yes, I did verify that either instance only used rg & fd, or only used semble.)

    Without Semble, it used 10.9% of the model context and used $0.144 of API credits (or, at least, that's what Pi reported - I used this with a Codex sub so cannot be sure). With Semble, it used 9.8% of the model context and $0.172 of API credits. The resulting responses were also about the same. Very close!

    I tried one more test in the OpenCode repo. The question was > Trace the path from 1) the OPENCODE_EXPERIMENTAL_EXA env var being set to to 1 to 2) the resulting effects in the system prompt or tool provided to the OpenCode agent.

    And I included the same instructions/docs as above. The non-Semble version was a bit more detailed -- it went into whether the tool call path invoked Exa based on whether Exa or Parallel was enabled for the web search provider -- but w.r.t. actually answering the question, both versions were accurate. The Semble version used 14.7% context / $0.282 API cost, while the non-Semble version used 19.0% / $0.352. Clearly a win for Semble for context efficiency, but note that the non-Semble version finished about twice as fast as the Semble version.

    Of course this is just me messing around. ymmv.

    by aadishv
  • Wow awesome, thanks for sharing! This is really useful and very much like the experiments we want to be doing in the near future
    by stephantul
  • I did some evals with pi and GPT 5.5. I tested RTK on / headroom on / both on / both off (all with the standard pi system instructions and no AGENTS.md).

    I forget the exact tests I used (a couple of the standard agent evals that people use, one python and one typescript because those are what I use).

    I don't claim it was an exhaustive test, or even a good one. It's possible I could have spent a day or so tuning my AGENTS.md and the pi system prompt/tool instructions and gotten better results, because if there's one thing running evals taught me it's that subtle differences there can change the results a lot.

    However, I got clearly better results with both off, enough to convince me to stop the tests immediately after 3 rounds.

    The problem was that while context use did go down (sometimes), the number of turns to complete went up so the overall cost of the conversation was higher.

    It's made me very aware of one thing: so many people are sharing these kind of tools, but either with zero evals (or suspiciously hard to reproduce), or in the case of this one, extensive benchmarks testing the wrong thing.

    I'm sure this tool does use fewer tokens than grep, and the benchmarks prove it, but that's not what matters here. What matters is, does an agent using it get the same quality of work done more quickly and for lower cost?

    by esperent
  • Have you tried https://github.com/mksglu/context-mode? I prefer it over rtk / headroom (at least in Cursor). The stop hooks seem to work well / prevent the agent from blowing up its context window.
    by kurtextrem
  • with AI the "they could so they never wondered if they should" will be a very frequent thing.
    by zobzu
  • There's an industry wide deficit of tests for AI right now. It's not just this tool, it's everything you add to your code base or your development flow that uses AI. Nobody had tests for "how fast/well was this developed" before AI and they haven't added them now.
    by forgotusername6
  • I'd be interested in seeing actual agent benchmarks (eg CC or Copilot CLI with grep removed and this tool instead).

    For example, I have explored RTK and various LSP implementations and find that the models are so heavily RL'd with grep that they do not trust results in other forms and will continually retry or reread, and all token savings are lost because the model does not trust the results of the other tools.

    by jerezzprime
  • I think the best bet is to use some kind of proxy so when the model calls grep, you intercept the call, use other tool to search and give back the results to the model.
    by DeathArrow
  • >so heavily RL'd with grep

    At least codex listens to me telling it to use rg instead of grep, cause grep is often so slow. But when adding rtk it uses grep through rtk which is kind of annoying.

    by carlmr
  • Token savings is more and more important, but it also important if the agent trusts the result and stops searching. it should measure the full agent loop instead of just the search output
    by Riany
  • I forced Claude to have a global memory for RTK and my own AI memory system (GuardRails) which it happily uses both, the only times it doesnt use GuardRails is if I dont mention it at all, otherwise it always uses RTK unless RTK falls apart running a tool it does not support.
    by giancarlostoro
  • Yeah we're also interested in doing this, it's on the roadmap together with optimization of the prompt and descriptions so that models have an easier time using it.

    Perhaps anecdotally: we do use this tool ourselves of course, and it's been working pretty well so far. Anthropic models call it and seem to trust the results.

    by stephantul
  • Codex CLI is quite happy running RTK. Well with GPT 5.5 xhigh anyway

    One thing that irks me is that when it doesn't support eg. a cli flag of find, it gives an error message rather than sending the full output of the command instead. Then the agent wastes tokens retrying, or worse, doesn't even try because the prompting may make them afraid to not run commands without rtk

    by nextaccountic
  • I just put something in my global CLAUDE.md (under ~/.Claude) asking it to use the LSP instead of grep and have never had this issue since.
    by AussieWog93
  • Interesting. I too have been working in this space, though I took a different approach. Rather than building an index, I worked on making a "smarter grep" by offering search over codebases (and any text content really) with ranking and some structural awareness of the code. Most of my time was spend dealing with performance, and as a result it runs extremely quickly.

    I will have to add this as a comparison to https://github.com/boyter/cs and see what my LLMs prefer for the sort of questions I ask. It too ships with MCP, but does NOT build an index for its search. I am very curious to see how it would rank seeing as it does not do basic BM25 but a code semantic variant of it.

    This seems to work better for the "how does auth work" style of queries, while cs does "authenticate --only-declarations" and then weighs results based on content of the files, IE where matches are, in code, comments and the overall complexity of the file.

    Have starred and will be watching.

    by boyter
  • Nice! Let us know if you have any feedback or results to share, would be happy to do the same.
    by Bibabomas
  • I was going to share a link to this. Thank you for making `cs`, I use it both with LLMs and directly in the terminal, despite not performing indexing it's pretty fast for my needs. Also definitely planning to try out semble.
    by zikani_03
  • What I have personally observed with such tools is that they make the AI's dumb, similar to how it makes coders dumb when relying more on AI tools.

    These agentic AI's are already smart enough to figure out a highly optimized path to code exploration or search. But, with these tools, they just go very aggressive, partly because the search results from these tools almost in 100% of the cases do not furnish full details, but, just the pointers.

    To confirm this behaviour, I did a small test run. This is in no way conclusive, but, the results do align with what I been observing:

    ---

    Task: trace full ingestion and search paths in some okayish complex project. Harness is Pi.

    1. With "codebase-memory-mcp": 85k/4.4k (input/output tokens).

    2. With my own regular setup: 67k/3.2k.

    3. Without any of these: 80k/3.2k.

    As we see, such a tool made it worse (not by much, but, still). The outputs were same in quality and informational content.

    ---

    Now, what my "regular setup" mentioned above is?:

    Just one line in AGENTS.md and CLAUDE.md: "Start by reading PROJECT.md" .

    And PROJECT.md contains just following: 2-3 line description of the project, all relevant files and their one-line description, any nuiances, and finally, ends with this line:

        ## To LLM
    Update this file if the changes you have done are worth updating here. The intent of this file is to give you a rough idea of the project, from where you can explore further, if needed.
    by freakynit