Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I struggle with FTS inside SQL (SQLite and MSSQL). There is often a fairly significant impedance mismatch between the relational concerns and how the documents need to be stored.
I've always preferred to use SQL as the system of record and then build/maintain an external Lucene index. Do we think these integral FTS capabilities are at the point where a hybrid architecture doesn't make sense anymore? How much customization exists in this provider?
by bob1029 - There's another aspect which none of the FTS search solutions for Postgres do well in my opinion: multi-language support.
For example this one: it doesn't mention support for CJK languages (meaning tokenization for e.g. Chinese will resolve to one token per character, which will technically work and give results, but is inefficient). Also word stemming (databases -> database) is also missing as far as I can see, so the kind of queries where you'd expect related words to show up will be missing. Just doing case-folding and accent-folding is a bit of a functional but bruteforce solution.
Ideally I'd want something that supports:
- language aware tokenization, with ability to define the language per record. Including stemming, etc. And have useful predefined configuration for common languages (e.g. the Postgres built in one is missing many languages).
- CJK support, tokenizing at word boundaries.
- Optional accent- and case-folding.
Most solutions just seem to assume English content, I have not found anything that does all of this yet.
by Doohickey-d - Postgres does have pg_fts (tsvector/tsquery/tsrank) which is a quite sophisticated full text search package integrated with functional indexing and query optimization. Why would I use something vibecoded that isn't part of core Postgres instead?by tannhaeuser
- This was already posted and ignored at https://news.ycombinator.com/item?id=49751888 so i’ll ask the same question: again:
I don’t see any github link, is this 21st century embrace, extend, extinguish ?
by znpy - Interestingly enough SQLites FTS supports Lucene queries out of the box with great performance characteristics. IIRC only writes become pretty slow after a while. I’ve always wondered what exactly would prevent PostgreSQL from strapping that implementation into its own database. My experience with ts_query hasn’t been particularly rosy. It can be better than LIKE but only marginally so and at the cost of insane index sizes… If this extension becomes open source and we can test it out in the real world I’m sure there’s a sweet spot
- Please read the Postgres manual. It has incredible built-in search capability.
- If anyone's curious - https://planetscale.com/docs/postgres/search/get-started#loc...:
They're not providing a local extension with the same performance at the time - it's only offered on their cloud services.
The local version https://github.com/planetscale/lead is mainly just for testing the syntax, it doesn't have the same perf characteristics.
by Tiberium - I think what we're seeing with every database company providing new full-text search capabilities is an example of AI coding productivity showing up in the real world.
It started with paradeDB and pg_search https://www.paradedb.com/blog/introducing-search
Timescale has pg_textsearch https://github.com/timescale/pg_textsearch
Neon and Databricks have Lakebase Search https://docs.databricks.com/aws/en/oltp/projects/lakebase-se...
Now PlanetScale.
AFAIK all of these are implementations of the BM25 algorithm. You can just tell an agent to read about BM25 and implement it in your system of choice. Cool to see. Seems like there's still a lot of juice to be squeezed out of how it's architected and integrated into each system, but you can't help but wonder if this will lead to aggressive commodification