Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • This is so interesting.

    I'm unlocking savings (in both speed and costs) but using a different way of thinking by integrating Decision models:

    In short, LLMs spend so many turns on discovery, triage, and safety checks for everyday tasks. A decision model cuts those turns by preparing the relevant context before the prompt and handling the simple judgments itself, so the LLM only does the real work.

    Here is the full article I published yesterday with in-house benchmark: https://tawfeks.github.io/decisionkit/

    What do you think, guys? I need feedback on this.

    by respectattentio
  • not a fan of README's where there's no clear way to run the program, like a getting started or how to build. i think this is the opposite of a docu-monster where AI documents everything, but clearly more documentation is better than lack of.
    by westoque
  • OK, but isn't this just programmatic tool calling, which is beneficial in some cases but not others?

    This is available in DSH via PTC mode.

    Basically, instead of the completion API returning a sequence of tool calls, it returns a program that invokes the tools.

    I think most are returning Typescript program

    With ordinary tool calling, you may have to wait for a tool to return its result to the completion API before the model can continue and issue the next tool call.

    With PTC, the generated program can invoke independent tools asynchronously, collect their results, and dispatch the relevant results back to the completion API in a single turn.

    Claude and Codex added this as well, but there are some papers on this that show it's not good across all tasks, and models need to be trained for this specific technique, which is what AI labs are doing now.

    by faangguyindia
  • Oh very nice, can you also compare it to https://maki.sh?

    Would be interesting to compare to a harness optimizing for cost reduction too.

    by tontinton
  • Funny, I solved this problem by having Claude write a pi extension

    https://github.com/Pyrolistical/pi-notify

    Now my pi agent setups its own trigger to notify itself when a background process is done

    by pyrolistical
  • Sounds like a trademark issue when Epic ships a wildly popular Unreal Engine
    by tapoxi
  • The headline graph is kind of bizarre.

    For some reason they're comparing their harness running on Astra xhigh to Codex with Astra max?

    ---

    Also worth noting that OpenAI just added support for async tool calling to their harness, which isn't 1:1 with this approach, but is slowly ramping up in being able to provide something similar.

    A big part of why Codex uses so many tokens is that it basically hot loops on polling tasks it starts for... absolutely no good reason: https://www.reddit.com/r/codex/comments/1wdlp7q/weve_discove...

    I fixed it on my fork of Codex too, also back in Jan/Feb – I keep this patch rebased, for anyone who wants it: https://github.com/tekacs/codex/commit/9ffcf8db9078eae43d411...

    It results in token savings similar in scale to those displayed here by Unreal.

    ---

    My harness has used a slightly fancier version of the approach that Unreal is using since ~Feb, and... it definitely works excellently, but it's also assuredly smoother with Astra and other recent models that are more aware of async tool calling.

    by tekacs
  • I think this space is very untapped. Models are interesting, but I am absolutely obsessed with some things I've been researching/working on for the past few years:

    Fractal tool discovery: tool taxonomy where an agent can "drill deeper" to find what specific tool it's looking for. Helps if/when polluting context with a zillion (mostly unnecessary) tools.

    Leveraging splay trees: this is my favorite data structure and I think relatively unused in the context of agents/harnesses. A lot of times, recently-used workflows/tool-chains will be used again, so having those at the top of the search hierarchy is an awesome optimization.

    Virtual containerized notebooks: models working in sandboxed (WASI) Python notebooks is incredible. Even local models (if given enough time) will usually converge on a good solution. Being able to mount tools/resources/fs is again, imo quite untapped. Some problems here are running native things (thing numpy/pandas) in containers is a nightmare (or impossible).

    Anyway, happy to see other folks seriously doing stuff in this space. If anyone wants to collaborate on anything don't hesitate to reach out :) I'm also actively looking for a job or some contract gigs.

    Fun times ahead.

    by dvt

Explore Birbla archives