Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- The DeepSWE benchmark they report (59.3%) overlaps with the confidence interval of 5.6-Sol Medium (61% +/- 2%), but likely at 1/18th the cost (they did not report the DeepSWE benchmark cost, but v4-flash had this cost ratio against Sol Medium).
Interestingly, v4-flash performed several points worse on DeepSWE at 53% +/- 4%. Assuming this result is verified by DeepSWE officially, it would mark a significant advance in Pareto cost/performance on software engineering tasks.
by ttul - The closer comparison would be 5.6-Luna. On DeepSWE at Xhigh it's 57% at 1/6 the cost of Sol M, on Max it's 67% at 1/3rd the cost.
Still an advance, I just thought it worthy to note Sol isn't nearly as impressive on the cost/performance frontier as discounted Luna.
by paytonjjones - I just ran my image recognition benchmark on it ("is this XXX public landmark"?) and it misses a lot that bytedance seed 2.1 turbo gets right; for example: Asked "Is this Salisbury Cathedral" and supplied a picture of Wells Cathedral, it answers "Yes, the west facade of Salisbury Cathedral". Bytedance seed 2.1 turbo correctly says no. Similar results for a picture of Manhattan Bridge sent as Brooklyn Bridge, Chartres Cathedral sent as Notre Dame, etc. I have a benchmark of 12 such images and seed gets 11/12 and deepseek only gets 6/12.by jerkstate
- Do you publish your benchmark somewhere? What's the best vision (captioning) AI you've come across?
- I have zero interest in world knowledge for my LLMs but this got me wondering : are there RAGs for that kind of data ? How could a LLM like DeepSeek-v4-flash-vision-exp accurately answer you question with an indexed database of labeled landmark pictures (or even 3D models ?).by wolfgangK
- This is a fairly small model for coding and agentic work.
Training it on images like yours would just make it worse in other areas.
by throwa356262 - Is there a way to test it online so that one doesn't have to resort to getting an API key and python code ?by wiz21c
- You can use the playground on openrouter. Still needs an account and some money, but it's one of the more useful accounts to have sitting around with a $5 of balance. Great for one-off experiments with various modelsby wongarsu
- 800x800 is 640,000 pixels, or 0.64 Megapixels. That is less than the resolution of computer screens from 1995, Super VGA which has around 0.79 MPs.
This is useful for a reasonable amount of use-cases, but I think the watershed rez will be around triple that, ~1080p, which is enough for almost anything, except small text and subtle details.
by gozucito - I typically provide small screenshots to llms so this seems fine for that usecase, providing an entire screens context seems cause confusion with a lot of llms.by dakolli
- You'd expect a tool-enabled model to leverage crop and zoom tools to inspect and validate what it thinks it's seeing, though.by barrkel
- Interesting. Wasn't Deepseek's founder saying that they had explicitly decided not to focus on multimodal models at all and were going text-only because they believed it was enough to achieve AGI?by v9v
- Worth noting that deepseek has had a separate vision-capable model for some time, which also powers their chat interface's vision modeby swiftcoder
- I think you're thinking of Dario saying this about image generation.by dakolli
- It was explicitly said that they are pursuing multimodal support. A quote from the meeting transcript: https://github.com/demo-zexuan/liang-wenfeng-investor-meetin...
Earlier, the following was said, which might match more what you had in mind.Nevertheless, as a component, we will undoubtedly implement multimodal support — and we are already doing so. We plan to develop relevant models, ensuring that versions like V4 and subsequent iterations will natively support multimodal functionality.
It is difficult to tell who said what, since the speaker ids are missing.Achieving excellence in AI training does not require a global model or even multimodal approaches—by narrowing the scope of AI training and eliminating multimodality, certain tasks may remain unachievable without compromising the algorithm's validity. Multimodal approaches ultimately need to be implemented.by johndough - I've been using v4-flash without vision for this months — it's my go-to for code tasks. Now with vision, I'm wondering: if this model can do everything the text-only version does (plus see images), why keep the text-only one around?
Is it just cost/latency? Or is there something text-only does better?
by cjg007 - Yes, it adds vision to the already capable text-only LLM according to DS:
> This experimental multimodal model matches DeepSeek-V4-Flash on text capabilities—including agents, reasoning, and world knowledge.
by bel8 - Congratulations! DeepSeek has finally gained eyes — the dark days are about to be behind us.by BrucecarlL
- Or about to start. Depending on which life philosophy you desire to believe.by doublerabbit
- News announcement with benchmarks: https://api-docs.deepseek.com/news/news260821/
- > Larger images are scaled down while preserving their aspect ratio, so that the total pixel count after resizing is roughly that of an 800×800 image.
It's useful but for OCR and a lot of other applications it needs to be a bit higher (eg: putting in a full A4 / Letter sized page)
by zmmmmm - Can split and feed?by mkagenius
- I've heard that DeepSeek v4 Flash 0731 has frequently assumed that it has vision capabilities and then resorts to inventing text-based image analysis tools when it finds that it actually can't see. In that case, this is a great upgrade for the model.
Anecdotally, I had to tell 0731 to refrain from viewing screenshots since it kept breaking its sessions by trying to read images.
by LorenDB - I've mitigated this by giving it a "skill" that just means the harness using a different model.by trollbridge
- Yeah I've seen it a lot. It goes through the effort, unasked, of pulling screenshots off a connected device and then it's like... Oh shit yeah I can't see.by mavamaarten
- It tried to recreate vision by analyzing pixels on 3 separate projects I had it working on.
- It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.png
I asked it "what time does the clock show?" (both on reasoning: high)
DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes. * *Second hand (blue, medium):* Pointing at the *9*, which represents 45 seconds.
Qwen answered: The clock shows *8:10* (with the red second hand on the 5, i.e. *8:10:25*).
- *Hour hand* (short, blue) → 8 - *Minute hand* (long, green) → 2 (10 minutes) - *Second hand* (thin, red) → 5 (25 seconds)
Correct answer is 08:09:25.
by leumon - most likely a preview. they often release the preview via API, get more training data, post train some more then release the weight. i would expect to see it perform better in a few weeks or a month.by segmondy
- I was wasting hours yesterday trying to get DeepSeek V4 Flash (with Qwen 3.8 27b as the vision agent, actually) to read sheet music to pass a Terminal Bench 3 benchmark and none of it was working... nothing... I changed models to gemma 31b, I tried OCR models... nothing could get it...
And then I realized, wait a second... you're testing the harness not only against a difficult benchmarking problem, but it's one you're literally never going to use the coding harness for either, lol. I don't write programs that read or interact with sheet music and I never will.
tl;dr Being frustrated that a "state of the art" vision model doesn't have perfect vision is a fools errand.
It can read and extract information from screenshots and PDFs just fine (my setup). No need to worry about edge cases.
- Gemini 3.7 Flash and 5.6-Sol (on all reasoning levels) also answer 8:10:25. The new "stealth" Ox Alpha also replies with the same. Opus 5 replies with 8:10 (no seconds). Not sure why this is so hard for them; Gemini is especially good at vision and I would have expected better from it.by ComputerGuru