Birbla

Discussion summary

DeepSeek's verification loop increased its Web-Bench score from 20 to 80, matching Opus at a seventh of the cost. The method involves iterative skill-based testing to improve LLM performance.

What the discussion says

  • Some see it as marketing for a SaaS product.
  • Others note it mimics techniques used by Anthropic and OpenAI.
  • The approach uses multiple skills like debugging and visual testing.
“Verification loop took DeepSeek from ~20 to ~80 on Web-Bench.”
— sozal
“It uses skills in a loop to help LLMs solve tasks better.”
— bel8

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Seems interesting, but buried under a mountain of written slop.
    by _aavaa_
  • Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.
    by sozal
  • It is marketing for some SaaS thing. However, I think something like this, "verification loops with some sort of tricks", is what Anthropic and "OpenAI" do behind the scene with their interfaces.
    by rightbyte
  • DeepSeek at 1/7 the cost of Opus is the real story here. The only downside is paying for it — Alipay/WeChat only. api-hub.cc lets you use DeepSeek V4 with any credit card, same API.
    by zftnb666
  • From what I understood, the gist of it is that uses a bunch of skills in a loop to help LLMs solve the tasks better via browser-testing, debugging, visual-testing, etc..

    The skills can be found here:

    https://github.com/ironbee-ai/ironbee-devtools-skills/tree/m...

    Question that was left unanswered:

    How much longer does a 4x DeepSkeep loop took compared to Opus?

    by bel8