Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Why would any website want to put extra effort to "give" their content as markdown to AI and get nothing in return ?
- A question repeatedly asked by different people throughout these comments, but nobody answers.by Semiapies
- An obvious example is if my product has an SDK and I'd like my customers to be able to script my product via their LLMs then I want to make my docs as AI-friendly as possible, since if their LLM picks my product's SDK rather than the competition, I make $$$.by gampleman
- Interesting...
A cherry.jpg image/jpeg file is shown as an image by the browser
A logo in svg like https://news.ycombinator.com/y18.svg too
So browsers should show readme.md as text/markdown and clicking on a link inside that file should also redirect to another markdown file, instant wiki
Browsers should implement that asap and we will follow
by Kuyawa - It depends on one question: does Markdown support <script> tags?by meindnoch
- Even without AI this would be a nice feature.by maurelius2
- Hello … AI company that wants easier access to databy LunicLynx
- Seeing all the questions in this thread, I strongly recommend Dachary Carey's post on the topic of agent-ready docs: https://dacharycarey.com/2026/02/18/agent-friendly-docs/by theletterf
- hoping for this to get mainstream so that I can just view the pages without any ads, js and bloatby kaangiray26
- Markdown ads?by jiehong
- Yeah, that is why this will not get popular.by qznc
- > But imagine for a second they did do that. You get the API, all the info is there.
> Why do you need AI then?
(from this HN comment, worth a read in full: https://news.ycombinator.com/item?id=43679585)
by Elfener - I feel like I am going insane. Who in their right mind would feed an LLM raw HTML in the first place??
HTML is a markup language. User agents present it in a way that makes sense for the user—visually, or through assistive technology. Nothing about adding LLM “users” should change this—their user agent, the harness, should present HTML in a way they can natively understand, by translating it to Markdown.
We are not going to rewrite the entire web because harness developers are too lazy or stupid to pull an HTML to markdown package from npm. If some sites want to do so anyway, good for them, in many cases I’d love to skip the CSS/JS and read the markdown directly (or better, nicely formatted). But don’t blame website authors for your harness wasting your tokens.
by joshum97 - Who in their right mind would feed any random LLM crawler their website's content?by Semiapies
- I think you are operating under the misconception that software developers are in their right mindby 0xbadcafebee
- i have built 2 html to document pipelines and read it back with pdftotext, and I can say that "just convert it yourself" step is the place where things are getting wrong. For example Markdown has no support for the subscript and superscript in CommonMark or GFM and if you have <sup>4</sup><sub>3</sub> becomes 43 and water formula becomes H2O so in the output you get not the ugly but the wrong one. And no converter can fix it, because that's a target format which can't express it. extractors like Defuddle don't save from this also. and non of those extractors or converter throws you just get wrong textby tigrandza
- > Who in their right mind would feed an LLM raw HTML in the first place??
I'd even say, is that even a problem ? If your website isn't a front end blob soup and just a relatively clean HTML file, your LLM is totally capable to handle it anyway.
And I'm still wondering myself which incentive do I have to serve my content to LLMs anyway.
by pjerem - Agree. I think many people forget that not long ago, HTML markup on many sites was a lot richer than it is today. Making it trivial to produce a good trimmed down markdown version.
The reason it may be more difficult today is because we've lost a lot of that. Some of it because of modern JS frameworks, but some also because publishers simply don't want to make it easy for the useful stuff to be scraped and extracted easily.
I'm not convinced that's changing because of AI agents (it's getting worse in many ways with anti-agent rules). Maybe improving for documentation pages intended for agents. But if it is changing, I think it'd be far easier to improve the HTML and let the agent take care of the rest.
by k1m - > in many cases I’d love to skip the CSS/JS and read the markdown directly
I've been enjoying https://defuddle.md since learning about it. Works great. Not affiliated.
by alsetmusic - I'll do that once any of the top 4 AI chatbots says they'll start making requests with this header. Before that it's just a neat idea with no adoption.
I also think it's exceedingly unlikely that any of the top 4 chatbots would choose to load websites this way. Too many risks for a 0.01% adoption even years later.
by lekevicius - I mean, it's exceptionally easy for them to add it. Even with low adoption. Somewhere in the bowels of chatgpt there is code that queries for html, and then parses that to markdown or something similar. All this would be is adding the logic to request markdown if available, then bypass the html->markdown step if you get markdown back.by kennywinker
- Also it's not that hard at crawl time to transform HTML into markdown before passing to the LLM.by russ_j
- You can see current support here:
- Claude Code’s been doing this for at least a year already and I assume Claude Cowork does it too as a result. That’s already millions of users, so it’s not “just a neat idea with no adoption”.by JimDabell
- What about clean, semantic HTML?
It was already optimized for bots and search engines (which are bots) and it has been used for decades. Why we need to serve in markdown now?
There are also many parts of the HTML, like navs, that are useful for bots and AI and may be removed in the markdown version.
by collimarco - Presumably markdown uses far fewer tokens.by slowin
- I agree. I think a lot of people here are assuming that the full HTML retrieved has to go into the LLM eating up tokens. But why wouldn't the agent try to clean up first and remove bloat and convert to markdown itself, before feeding into LLM. Semantic HTML would make that easier.by k1m
- >What about clean, semantic HTML?
Which React package is this?
by meindnoch - Please don’t tell me we are willing to make it easier for AI to read our content than we are to make it easier for humans to read our content.
Better idea: make your UI always prioritize the content.
by lo_fye - Don't point it out, please. I, for one, look forward to accessing content without all the bloat.