Birbla

Discussion summary

DeepSeek's verification loop increased its Web-Bench score from 20 to 80, matching Opus at a seventh of the cost. The method involves iterative skill-based testing to improve LLM performance.

What the discussion says

  • Some see it as marketing for a SaaS product.
  • Others note it mimics techniques used by Anthropic and OpenAI.
  • The approach uses multiple skills like debugging and visual testing.
“Verification loop took DeepSeek from ~20 to ~80 on Web-Bench.”
— sozal
“It uses skills in a loop to help LLMs solve tasks better.”
— bel8

Comments

Hacker News

Seems interesting, but buried under a mountain of written slop.

by _aavaa_

Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.

by sozal

It is marketing for some SaaS thing. However, I think something like this, "verification loops with some sort of tricks", is what Anthropic and "OpenAI" do behind the scene with their interfaces.

by rightbyte

From what I understood, the gist of it is that uses a bunch of skills in a loop to help LLMs solve the tasks better via browser-testing, debugging, visual-testing, etc..

The skills can be found here:

https://github.com/ironbee-ai/ironbee-devtools-skills/tree/m...

Question that was left unanswered:

How much longer does a 4x DeepSkeep loop took compared to Opus?

by bel8

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Seems interesting, but buried under a mountain of written slop.
    by _aavaa_
  • Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.
    by sozal
  • It is marketing for some SaaS thing. However, I think something like this, "verification loops with some sort of tricks", is what Anthropic and "OpenAI" do behind the scene with their interfaces.
    by rightbyte
  • From what I understood, the gist of it is that uses a bunch of skills in a loop to help LLMs solve the tasks better via browser-testing, debugging, visual-testing, etc..

    The skills can be found here:

    https://github.com/ironbee-ai/ironbee-devtools-skills/tree/m...

    Question that was left unanswered:

    How much longer does a 4x DeepSkeep loop took compared to Opus?

    by bel8

Related stories

SWE-1.7 Reach Near GPT 5.5 and Opus Intelligence

cognition.com · 7 points · 0 comments

PostgreSQL performance and cost across 23 EC2 instance types

postgres.saneengineer.com · 88 points · 16 comments

Foreman, a self-hosted LLM gateway for cost aware model routing

github.com · 2 points · 0 comments

Decoding the obfuscated bash script on a Uniqlo t-shirt

tris.sherliker.net · 1147 points · 189 comments

StreetComplete: Fixing OpenStreetMap, one tiny quest at a time

streetcomplete.app · 761 points · 182 comments

Every new car sold in the European Union must include a driver monitoring camera

allaboutcookies.org · 745 points · 986 comments