Discussion summary
Parsewise, a YC-backed API, offers cross-document reasoning at scale, such as processing 90k-page corpora. Users discuss its features compared to Claude and llamaParse, emphasizing scalability and integration.
What the discussion says
- Some users prefer Claude for simplicity.
- Parsewise is suitable for large-scale reasoning.
- Others mention alternatives like llamaParse.
- UI design in demos was criticized for aesthetics.
“If Claude is good enough for your use case then for sure.”
“Mostly cross-doc reasoning at scale, e.g., 90k-page corpora.”
Comments
Hacker News
Interesting product! Do you think it would work for e-discovery? I have around 120GB of emails, contracts, and the like, and I need to search for data and where certain expressions are referenced.
by vmandrade
Might be interested in orthogonal reading - "The Textual Warehouse" (ISBN-10: 163462954X) by data warehouse pioneer Bill Inmon. He is and always has been ahead of his time with his thinking!
"With experience and support from" is a nice landing trick!
How do you extract and relate to each other the facts from the documents that require comprehension and not simple similarity matching using common embeddings models?
by dennis16384
This looks great for digital humanities, specifically archival work. Would love to try it.
by vinaigrette
Does this also extract semantic relationships and data dependencies between fields?
In the past I'd built an internal tool that transforms insurance PDFs to structured data. I wanted to extract explicit data dependencies between fields to perform validation.
Insurance forms can sometimes have 30-40 pages and they can have fields on page 40 that depend on fields on page 4 with a few nested if conditions. Would Parsewise be able to extract those relationships?
If yes, how do you do it for large documents?
by nilirl
How portable are your agent definitions? If I build one for insurance documents, how much work is needed to adapt it to a completely different domain like legal contracts or healthcare?
I built a similar tool some time ago called Struktur (https://struktur.sh).
It’s much more limited in scope but fully open source and highly customisable. In fact it’s made for people to build their own pipelines on top of, providing the scaffolding needed to do so in a reliable way.
During development I’ve found it to be hard to truly generalise agent/llm-based data extraction, especially around the unlimited number of input types without task specific instructions (many files of the same kind, single large files, mixed kinds, bad quality files, docx/pdf/png/… the list goes on). Users sadly wanna upload all of these, and developers want a „one size fits all“ solution.
I am interested in how your solution deals with this. I came up with a strategy based approach so every task can be customised if needed, but I’d be delighted to see a technical writeup of how you deal with this endless variety of input + extraction task combos! :)
by capevace
Document parsing is top of my mind lately because in some of the areas we work on the bottleneck is starting to become being able to query documents the same way one queries an api.
I keep thinking the most obvious analogue is we need some way to represent documents the same way we can represent structured data in parquet. Parquet allows easy range bases queries and there is so much tooling built around Arrow.
But for documents I keep hitting a wall to figure out what the right abstractions are. Parquet allows filterable metadata. But what such metadata is there for documents. Then there is the arbitrrariness of chunking, vectorization.
If we could just do this in a 2 step process where every document to process can be represented in a parquet like data format then I think we will atleast have the semblance of a solution.
by whinvik
Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Interesting product! Do you think it would work for e-discovery? I have around 120GB of emails, contracts, and the like, and I need to search for data and where certain expressions are referenced.by vmandrade
- Might be interested in orthogonal reading - "The Textual Warehouse" (ISBN-10: 163462954X) by data warehouse pioneer Bill Inmon. He is and always has been ahead of his time with his thinking!
- "With experience and support from" is a nice landing trick!
How do you extract and relate to each other the facts from the documents that require comprehension and not simple similarity matching using common embeddings models?
by dennis16384 - This looks great for digital humanities, specifically archival work. Would love to try it.by vinaigrette
- Does this also extract semantic relationships and data dependencies between fields?
In the past I'd built an internal tool that transforms insurance PDFs to structured data. I wanted to extract explicit data dependencies between fields to perform validation.
Insurance forms can sometimes have 30-40 pages and they can have fields on page 40 that depend on fields on page 4 with a few nested if conditions. Would Parsewise be able to extract those relationships?
If yes, how do you do it for large documents?
by nilirl - How portable are your agent definitions? If I build one for insurance documents, how much work is needed to adapt it to a completely different domain like legal contracts or healthcare?
- I built a similar tool some time ago called Struktur (https://struktur.sh).
It’s much more limited in scope but fully open source and highly customisable. In fact it’s made for people to build their own pipelines on top of, providing the scaffolding needed to do so in a reliable way.
During development I’ve found it to be hard to truly generalise agent/llm-based data extraction, especially around the unlimited number of input types without task specific instructions (many files of the same kind, single large files, mixed kinds, bad quality files, docx/pdf/png/… the list goes on). Users sadly wanna upload all of these, and developers want a „one size fits all“ solution.
I am interested in how your solution deals with this. I came up with a strategy based approach so every task can be customised if needed, but I’d be delighted to see a technical writeup of how you deal with this endless variety of input + extraction task combos! :)
by capevace - Document parsing is top of my mind lately because in some of the areas we work on the bottleneck is starting to become being able to query documents the same way one queries an api.
I keep thinking the most obvious analogue is we need some way to represent documents the same way we can represent structured data in parquet. Parquet allows easy range bases queries and there is so much tooling built around Arrow.
But for documents I keep hitting a wall to figure out what the right abstractions are. Parquet allows filterable metadata. But what such metadata is there for documents. Then there is the arbitrrariness of chunking, vectorization.
If we could just do this in a 2 step process where every document to process can be represented in a parquet like data format then I think we will atleast have the semblance of a solution.
by whinvik