Launch HN: Parsewise (YC P25) – Reason Across Documents with an API

Join the discussion

• Hacker News
• Ah probably should add a link to our website: https://www.parsewise.ai/api
  by gergelycsegzi
• "retaining lineage"
  by stevesimmons
• Added above :)
  by dang
• Just use claude. Not another wrapper
  by hnuser
• If Claude is good enough for your use case then for sure. If you need scale, persistent structure and verifiability we can help:)
  by gergelycsegzi
• llamaparse also do it, what is different here?
  by mauryaudayan
• Mostly cross-doc reasoning at scale (e.g., 90k-page corpora) as opposed to doc-to-markdown conversions.
  by maxhofer
• Similar to my other comment, we assume that llamaparse and others can provide the individual page OCR. But once you have that the way that you can integrate it into your workflows often requires additional complexity around combining results from different sources. Here is a deeper dive I wrote on the complexities of building extraction pipelines: https://www.parsewise.ai/doc-processing-pipelines
  by gergelycsegzi
• I say this with a lot of love: The vibecoded applications in your demo reek of AI slop design.

  This isn't a critique of your product. It's just that the a beige-orange theme, the pill components, and the left-border highlight give me that visceral reaction as reading a paragraph littered with em dashes and "not X but Y." It makes me take you less seriously.

  Cool demo otherwise.

  by red_hare
• Haha no appreciate it! That's on me for not calling it out explicitly (was trying to make the video as short as possible), but the demo UIs were literally vibe coded to show the ease of integration https://youtu.be/F1cSuZal03s?si=1H4zTcO-8cosLbVr&t=70
  by gergelycsegzi
• Hey ! Is this kind of like structured output over a large scale document corpora ?
  by rdksu
• Hey, that's exactly it!
  by gergelycsegzi
• I am seeing my client using things like this heavily (not exactly this). Also, what I would call "business awareness" is declining.
  by rogerthis
• I can see why, it's tempting to go for full automation. The reason we go for fine grained sourcing is so that people can build their awareness quickly. Plus many of our customers work in regulated industries where full automation is prohibited.
  by gergelycsegzi
• I worked recently on an internal tool to achieve this kind of things, mostly plugging mistral OCR to gemini to extract structured data from documents. We then perform automated diffs too.

  There seems to be an insane amount of competition in the "Intelligent Document Processing" market, like for instance parseur, whose founder is often on HN himself.

  What do you think sets you apart from competition like : 1) Mistral document AI : depending on the model, it looks way cheaper than yours, OCR model pricing ranges from 0.001 to 0.004 EUR / page and they have structured output wired in the OCR API if needed (things then get fed to one of their LLMs) + EU-based and GDPR ready 2) parseur / rossum / docsumo / nanonets (which is YC 2017) ?

  by gorgmah
• Great question!

  1. We are working with the assumption that OCR is (or soon will be) solved at super low prices.

  So if we have the extracted data, what can we do with it? Where we see Parsewise making a difference is for use cases that span across documents. I.e. if you are extracting the same 5 fields from every invoice, there are lots of solutions as you listed (+ reducto etc). However, once you have a set of documents (e.g. an entire mortgage application package) and you are trying to get a structured response out, then your option is either an LLM API (if things fit into context and you are okay with limited citations), or building a pipeline with LLMs. I posted it in another comment but an example of trawling through 90k pages is here: https://www.parsewise.ai/officeqa-sota

  2. While we rely on LLMs, the outcomes will be non-deterministic, so the bottleneck is and will remain the human verification (that is for somewhat complex use cases). The architecture that we have built is optimizing for the human reviewer to provide as granular values and citations as possible. This is either through our platform, or API clients.

  by gergelycsegzi
• Hi, Parseur founder here :D

  I understand what they are trying to do, but to me it feels like the moment when MongoDB entered the database space, with semi-structured, "flexible" storage format. It has its uses, for prototyping mostly.

  But in high-volume, production workloads, giving a structure to the data you extract (what Parseur does through defining the Fields in your Mailbox, basically giving your output data a schema) adds a ton of value, and the larger the dataset, the truer it is.

  Usually, you start by defining where you want your data to go, and which structure it should have, before working backwards from here and starting to extract the data. This is the key to automating your document workflow.

  by joss82
• Interesting product! Do you think it would work for e-discovery? I have around 120GB of emails, contracts, and the like, and I need to search for data and where certain expressions are referenced.
  by vmandrade
• clickhouse?
  by dennis16384
• Potentially, but at that scale cost and latency may actually become an issue, so probably better to consider some sort of indexing or keyword searching.
  by gergelycsegzi
• Might be interested in orthogonal reading - "The Textual Warehouse" (ISBN-10:‎ 163462954X) by data warehouse pioneer Bill Inmon. He is and always has been ahead of his time with his thinking!
  by sixdimensional
• This does indeed look really interesting. We have deterministic validations (and some deterministic excel transformations) but using more deterministic transformations for text based on traditional NLP would be a nice complement.
  by gergelycsegzi
• "With experience and support from" is a nice landing trick!

  How do you extract and relate to each other the facts from the documents that require comprehension and not simple similarity matching using common embeddings models?

  by dennis16384
• Haha thanks, the reader can try and guess which is which;)

  We actually don't use embeddings or vector similarity, since those tend not to work well in specialist domains (e.g. for the OfficeQA benchmark where we have 90k pages talking about US treasury numbers, they would be mostly mapped to a very small embedding space because it's all the same topic, with small variations across years, expense categories etc.).

  We use LLMs for the extraction and comparison as well, and we route between different models depending on the complexity of the comprehension of the given step required (and by this I mean routing between our pipeline steps; we currently do not dynamically try to judge individual cases for complexity like OpenRouter Fusion).

  by gergelycsegzi
• This looks great for digital humanities, specifically archival work. Would love to try it.
  by vinaigrette
• Fully agree, that's why we quite like the Databricks OfficeQA benchmark.. it made us experts on historical US treasuries haha Some screenshots in here: https://www.parsewise.ai/officeqa-sota
  by gergelycsegzi
• How portable are your agent definitions? If I build one for insurance documents, how much work is needed to adapt it to a completely different domain like legal contracts or healthcare?
  by chaitralikakde
• In practice we find that each domain (and even each organisation) ends up having highly customized definitions.

  At first, fairly generic templated definitions sort of work, but what we've seen is that over time data comes up that is out of distribution, and there was no explicit instruction on how to deal with it. In such cases we tend to flag this and offer suggestions to the users on how they can improve the specificity of agents.

  Another structure we have seen play out is having a manager review ratings and feedback comments from their team and updating the definitions accordingly over time (where we offer them the capability to see results of before and after side -by-side for all existing data as well, so they are more confident in the change before committing).

  The amount of work is dependent on how good the initial definitions are and how complex the use case is (and how much it evolves - new data sources etc). A bit of an unsatisfying answer but it can be anywhere between a few hours one off or a couple of minutes per day on an ongoing basis.

  by gergelycsegzi
• Does this also extract semantic relationships and data dependencies between fields?

  In the past I'd built an internal tool that transforms insurance PDFs to structured data. I wanted to extract explicit data dependencies between fields to perform validation.

  Insurance forms can sometimes have 30-40 pages and they can have fields on page 40 that depend on fields on page 4 with a few nested if conditions. Would Parsewise be able to extract those relationships?

  If yes, how do you do it for large documents?

  by nilirl
• Yes, we do it by having multiple stages to the pipeline. First we would extract the independent data points (from say both page 4 and 40) and a second pass step establishes relationship (we call this resolution).

  On the scale aspect, because we go in multiple passes, we break the scope into small enough pieces and then build it back up in a later step. Iirc the largest document I've seen a customer use was over 1k pages.

  There are more complex data dependency scenarios where we find that the data that's extracted and combined (e.g. from page 4 and 40), needs to then be further transformed in different ways (e.g. having an evaluation and a clarification outcome at the end). To make these be aligned in value we are soon releasing a feature for what we call derived agents.

  by gergelycsegzi