Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • It appears that the only available endpoint (as of this writing) requires enabling "Allow paid endpoints that train on request data" in the OpenRouter privacy settings. I hope additional paid providers will become available that don't require training on data.
    by eshack94
  • That is likely because Deepseek themselves is the only host.

    In 24-48 hours there will be other options I presume

    by cdolan
  • Their privacy policy doesn't forbid them from just straight up publishing your raw prompts as training data.

    My threat model is that anything I POST to DeepSeek I treat as public to the web, as much as a public GitHub repo is.

    by jubilanti
  • @dang - Pls merge this with https://news.ycombinator.com/item?id=49274018
    by indigodaddy
  • Email hn@ycombinator.com with anything you want HN mods to see. They're incredibly responsive.
    by CharlesW
  • Benchmarks:

        | Benchmark                | DS-V4-Pro | DS-V4-Flash | DS-V4-Pro | DS-V4-Flash | GLM-5.2   | Kimi-K3   | Opus-4.8  | Fable 5       |
    |                          | 0813      | 0731        | Preview   | Preview     |           |           |           | (w/ fallback) |
    |--------------------------|-----------|-------------|-----------|-------------|-----------|-----------|-----------|---------------|
    | HLE (wo/w tools)         | 42.7/60.0 | 37.8/51.5   | 37.7/48.2 | 34.8/45.1   | 40.5/54.7 | 43.5/56.0 | 49.8/57.9 | 53.3/63.0     |
    | Terminal Bench 2.1       | 87.9      | 82.7        | 72.1      | 61.8        | 81.0      | 88.3      | 85.0      | 88.0          |
    | NL2Repo                  | 61.5      | 54.2        | 38.5      | 39.4        | 48.9      | -         | 69.7      | -             |
    | Cybergym                 | 83.3      | 76.7        | 52.7      | 38.7        | -         | 80.0      | 78.3      | 83.1          |
    | DeepSWE                  | 62.7      | 54.4        | 12.8      | 7.3         | 46.2      | 67.5      | 58.0      | 70.0          |
    | Toolathlon-Verified      | 74.1      | 70.3        | 55.9      | 49.7        | 59.9      | 76.5      | 76.2      | 77.9          |
    | Agents' Last Exam        | 25.7      | 25.2        | 16.5      | 15.8        | 23.8      | 27.6      | 25.7      | -             |
    | AutomationBench (Public) | 31.8      | 25.1        | 12.8      | 10.8        | 12.9      | 30.8      | 27.2      | 29.1          |
    | DSBench-FullStack        | 71.1      | 68.7        | 41.8      | 37.0        | 61.8      | 73.7      | 71.6      | 77.2          |
    | DSBench-Hard             | 67.2      | 59.6        | 31.1      | 25.8        | 54.5      | 63.0      | 71.7      | 68.3          |
    Source: https://reddit.com/r/LocalLLaMA/comments/1vmi0fg/deepseek_v4...
    by scrlk
  • In classic reddit fashion the post you linked to is now deleted
    by NietTim
  • The most interesting part of this is how Flash scores almost as well on all of them.

    Haven't tried the new DeepSeek models but I'm assuming the difference is more than these numbers show!

    by andai
  • IMO the HLE scores without tools seem to align better with real world performance of the models.

    To me it feels like the difference between "RL performance" and the pretraining / base "knowledge".

    Yes you can RL terminal bench to the moon but does the model hold up on out of distribution tasks?

    Kind of like trying to navigate a dark room with a laser light, vs a flashlight. Laser is going to go a lot farther much more efficiently but only if you are already pointing it at the right place.

    by myworkaccount2
  • Geometric mean of all these benchmarks :

    * GPT-5.6 Sol: 65.5

    * Fable 5 (w/ fallback): 64.5

    * Opus 5: 64.0

    * DS-V4-Pro 0813: 62.5

    * Kimi-K3: 62.3

    * DS-V4-Flash 0731: 55.8

    * GLM-5.2: 47.3

    by goldenarm
  • So it's a Fable class LLM?

                                 DSV4Pro vs Fable5
    HLE w tools              60.0 vs 63.0
    Terminal Bench 2.1       87.9 vs 88.0
    Cybergym                 83.3 vs 83.1
    DeepSWE                  62.7 vs 70.0
    Toolathlon-Verified      74.1 vs 77.9
    AutomationBench (Public) 31.8 vs 29.1
    DSBench-FullStack        71.1 vs 77.2
    DSBench-Hard             67.2 vs 68.3
    by bel8
  • The timing looks like they are trying to take the wind out of Qwen's sails by releasing this on the same day that Qwen released the weights of Qwen3.8-max. Or maybe it's coincidence...

    For comparison I looked at Qwen's claimed benchmarks for Qwen3.8-max (https://qwen.ai/blog?id=qwen3.8). Assuming each published set of benchmarks is believable, it looks like v4 Pro 0813 is better on average but overall performance is comparable. Pro 0813 is much cheaper. If you don't need vision capabilities then you don't have much reason to use Qwen3.8-max.

    - 43.6 on HLE (Presumably without tools). Pro 0813 is a little worse.

    - 86.6 on Terminal Bench 2.1. Pro 0813 is better.

    - 55.9 on NL2Repo. Pro 0813 is better.

    - 27 on Agent's Last Exam. Pro 0813 is a little worse.

    - 72.5 on Toolathon-Verified. Pro 0813 is better.

    - 56.6 on DeepSWE 1.1. If the DeepSWE listed for Pro 0813 is the same version, then Pro is better.

    - 27.3 on AutomationBench. If the AutomationBench (Public) listed for Pro 0813 is the same, then Pro is better.

    I guess we do need to wait to see if the upcoming DS pricing increase is enough to change the value proposition. As it is now, they could double or triple prices and it still would be a better value to use DS. I bet they know that.

    by parsimo2010
  • Currently burning money quickly on official deepseek api. They are also increasing pricing starting today. V4 Flash 0731 still feels like the most outstanding model of the past few months and probably to come.
    by Gecko4072
  • Just use opencode go, you get more bang for your buck. Same api
    by killingtime74
  • What's the new pricing?

    The prices on OpenRouter still look the same.

    by Eueudhsbsj32
  • What is the new price through?
    by Jsttan
  • yup :)

    i'm doing opencode <-> openrouter <-> official deepseek api (i don't get the opencode hate, i like it)

    how are you doing it?

    am also using Kimi K3 via kimi-code

    and also GLM 5.2 via ZCode

    happy with all three, they're trailing frontier but i figure if i'm running GNU/Linux then i ought to favour open weights models with my €s -- reduced my usage of claude/gpt to the ~$20 tier just to keep abreast of claude_code/codex developments

    by igravious
  • Deepseek seems to have gotten too cheap. I have been using it for a long time and it's at a point now where my credits balance barely moves even at max setting.
    by sschueller
  • The Deepseek official API is good with excellent caching.

    But their privacy policy is unusually bad - they can train off your prompts and completions.

    by eli
  • DeepSeek V4 Flash is the "too cheap to meter" of AI. And you can run the full unquantized model locally for $8000 (2x DGX Spark) at full 1M context and decent speeds: https://github.com/elsung/dgx-spark-deepseek-v4-flash#-long-...
    by nolist_policy
  • Based on my experience so far, compared to previous models, DeepSeek V4 Pro achieves results equal to or even better than before, but at a lower cost.
    by Hardd
  • Sounds like something a DeepSeek V4 Pro bot would say
    by thunfischtoast
  • Tested both DS v4 pro 0813 and Grok 4.6 (all from openrouter) on Codex cli. Worked on a same new feature development on my project.

    Deepseek 4 pro: Worked for 12m 02s - cost $0.12 - has bug.

    Grok 4.6: Worked for 3m 18s - cost $ 1.41 - no bug.

    by jklmnopqrstuvw
  • I thought it was impossible to downvote posts?
    by NooneAtAll3
  • Nullius in verba
    by hugmynutus
  • It's the third link on the front page right now?
    by Zetaphor
  • Repeat the test like 5 times for each model and see the results.
    by computerex
  • Why are people giving these n=1 comparisons like they mean anything? The worst offender is that pelican guy. These are non-deterministic systems and a single trial should not update your priors much at all.

    Of course it's significant that your response had a bug and took four times longer, but if you're only going to try once, this isn't real science, it's just vibes.

    by bigmadshoe
  • Have been letting it spin pretty hard (~$12.50 for 2B, 50% cache hits) on my traffic simulator/distributed physics engine all day, it's found some pretty significant gains without introducing any new problems.

    I'm happy

    by monster_truck
  • Can you explain how you used 12 billion tokens to do useful work?
    by fooblaster