Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- First result for "Jesus":
>Jacob kissed Rachel, and lifted up his voice, and wept. Jacob told Rachel that he was her father’s relative, and that he was Rebekah’s son. She ran and told her father. When Laban heard the news of Jacob, his sister’s son, he ran to meet Jacob, and embraced him, and kissed him, and brought him to his house. Jacob told Laban all these things. Laban said to him, “Surely you are my bone and my flesh.” Jacob stayed with him for a month. Laban said to Jacob, “Because you are my relative, should you therefore serve me for nothing? Tell me, what will your wages be?”
by gaiagraphia - Nice project. The 4GB index / ~15s search part made me think zvec might be a good fit here: https://github.com/alibaba/zvec
It’s an in-process vector DB, so the “local corpus, no separate server” shape is pretty much what it’s designed for. Its benchmark numbers are quite strong, and recent versions also support full-text + hybrid retrieval and DiskANN.
This would be an interesting case to try with zvec: same corpus, same embedding model, then compare indexing time, index size, memory usage, and query latency on normal hardware.
by bigggbob - > The king profits from the field
For the solution, read Henry George!
- Slow, but interesting. I used the query "government" and got back passages in Romans 13 (as I expected), but also passages in Daniel and Ezra describing decrees by government officials, which made sense.by mcswell
- For completeness, this should include all possible books, including Ethiopian, and then it should include a drop-down with pre-defined sets one could choose from (Protestant, Catholic, Orthodox...).by atmanactive
- This is really cool...great job! It's a favorite pastime of mine to index various large corpora.
As for speed, this might help for code referencing: https://github.com/deepbluedynamics/lume
Blog post: https://deepbluedynamics.com/blog/lume-retrieval-primitives
I use a small local model to extract entities for the graph, but it's not necessary.
You can optionally use GTR-T5 which is a few years old now, but still good for generating fast and free embeddings. That step is only run once if you run it in hybrid mode.
Feel free to take and remix or use!
- That's cool! I did the same for the Quran to see how RAG works. I also indexed related works called "Hadith" and the names of Allah. It initially required indexing everything using OpenAI embeddings and then powered by it.
It's also open source
by asim