Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • RAG stands for Retrieval Augmented Generation. The purpose is to search a corpus of text by meaning rather than exact match.

    I had to look it up.

    by klm127
  • That sounds more like semantic search and vector db.

    RAG is simply fetching external data (retrieval) and adding it to LLM context (augmenting) prior to generating a final response.

    Any time LLMs do a grep or a web search to answer the query, it’s RAG. Many people use vector db for their own RAG implementation bc of the semantic search benefits.

    by spunker540
  • Althought I agree with the first point of the author that FTS is underrated in this new RAG-first framework, the whole article really hides all the problems with RAG-pipeline and kind of hand wave everything.

    If you are building a RAG pipeline for your company and are struggling like me, I would recommend this author that has whole series on entreprise documents (start with the one from May 22nd): https://towardsdatascience.com/author/angela.shi/page/4/

    Note: I am not the author, just got her article in my newsletter and found it useful.

    by Otterly99
  • Agentic query rewrite on top of good old fashioned Lucene is the end game. This is effectively providing a lot of the same magic you get with the semantic approach. Allowing the agent to query the document store iteratively is where the capabilities become unbounded.

    Embeddings and semantic search add non determinism on top of non determinism. This seems fundamentally cursed. Lexical is much easier to control, iterate and debug. The tools are incredibly mature. Your users will probably prefer it as well.

    by bob1029
  • There have been many blogs like this over the last years.

    Yes, embeddings are computationally heavy, but they are not at all complicated and they provide a lot of benefit.

    90% of "document" based RAG projects should view semantic search with embeddings as their primary method.

    It's very powerful and so easy to implement that you could try it out and discover whether performance would be an issue rather than trying to anticipate it.

    by 7734128
  • How is pgvector with Sentence Transformers, a CPU-only embedding model, compared to a model hosted by OpenAI?
    by aitchnyu
  • I think it's VERY project specific. If you are looking for anything technical at all, then keyword search almost always does better (in my experience). I'd actually recommend starting with keyword search, and then expanding with embeddings after you have a better idea of what your users are trying to determine.
    by pantsforbirds
  • Embeddings are reasonably simple, but it’s a journey to get there, and I am very proud of the dog-heavy explainer I wrote on them: https://sgnt.ai/p/embeddings-explainer/
    by petesergeant
  • If someone has a Postgres db and want very simple RAG:

    https://github.com/jankovicsandras/plpgsql_bm25 BM25 search implemented in PL/pgSQL ( Unlicense / Public domain )

    The repo includes also plpgsql_bm25rrf.sql : PL/pgSQL function for hybrid search ( plpgsql_bm25 + pgvector ) with Reciprocal Rank Fusion; and Jupyter notebook examples.

    by jankovicsandras
  • Here’s an even simpler take: just embed everything the first time, then track what was changed. Use a cheap model to summarize and clean up the documents/chats with summary and keywords. Unless you have entire libraries of books to embed it’s going to be a few hundred dollars of API calls.

    Then, throw it all in BigQuery. Handles all the vector stuff natively.

    Sprinkle an agentic bot UI thing on top to make it appear all-knowing and magical.

    I assume other vendors than Google have a similar batteries-included approach you can just plug in.

    by refactor_master
  • Yep, lock into some vendor from day 1. Great idea!
    by cpursley
  • I'm sorry is this ironic or not? doesn't sounds simple at all
    by j0selit0
  • > embed everything the first time

    This assumes your text is small. Try embedding pdf reports - though luck. It surely won’t fit into most embeddings. I can think of many more examples: books, news articles, medical reports, insurance claims etc. they’re all too big to “index it all at once”

    by usernametaken29
  • I notice a lot of these AI written articles share this pattern where they'll present idea 1, then idea 2, and finally idea 3 which is some amalgamation of idea 1 and 2. Claude especially will present hybrid options and compromises to avoid having to make a choice then framing the hybrid option as the "best of both worlds" when they're borderline nonsensical.

    "on the fly embedding" and "Sparse + dense reranking" don't really make sense how they're presented and smell like they came from a long claude-driven conversation after multiple cycles of these hybrid compromises across many turns.

    by seamossfet
  • This has Claude written all over it.

    "Recipe 4: On-The-Fly Embedding (The Fresh Data Play)

    The insight If your data changes frequently, why pay to re-embed everything?"

    This reads like every Claude generated presentation I've seen.

    by ChipopLeMoral
  • I have built systems using all of these approaches (all in tandem). For the most part, the juice is not worth the squeeze (in building a highly optimised corpus-specific information retrieval strategy) outside of a very few fringe cases. The amount of technical discussion far outstrips the use case for RAG.
    by alansaber
  • I have a particular antipathy for articles too lazy to spell out acronyms on first use.

    So: https://en.wikipedia.org/wiki/Retrieval-augmented_generation

    by Angostura
  • For those times you need to Red Amber Green your BM25
    by _joel
  • The audience for this piece is already very familiar with RAG. I don't want articles discussing e.g. OLED screens telling me what the acronym is - that would be a sign that the article is far below the level that I need.
    by dotancohen
  • More LLM-generated text about LLMs.

    Is anyone else actually finding it harder and harder to read LLM generated text? I find it quite tiring, my brain just does not want to get through it.

    by jrochkind1
  • I'm Becoming AI-Blind: https://news.ycombinator.com/item?id=49386699
    by inigyou
  • The enshittification of the web, now powered by AI.
    by polynomial
  • In the same boat here.
    by allexander
  • Everything that is generate from a LLM is shit, I don't know why people continue using it. I'm waiting for this bubble to explode once for all so we can return doing things in the sane way.
    by alerighi
  • "This is where the real trade-off lives – not cost, but speed."

    That is where I stopped

    by LukasWeber
  • the biggest giveway is actually not the writing style, but the content

    "using GPT-4o-mini for query rewriting" -> model from 2024, when RAG was trendy, and all the langchain, llama-index, etc, docs mentioned this specific model

    by cpdomina
  • It’s largely because LLMs are reaching for many different types of adjectives or verbs in the same sentence, in a jarring way. While embedding it in a confidently declarative sentence. Everything sounds like some profound insight, dialed to an 11, but written as poetry. Especially those headings. With the short sentences.
    by EGreg
  • Your brain is incredibly adept at pattern recognition; it doesn't focus on LLM-generated text for the same reason it doesn't stare at wallpaper.

    We've all learnt that it's not really communication, and so can be dispensed with.

    by Planktonne
  • RAG is basically good old information retrieval with LLMs doing the querying. This can include vector search but it works without that as well. Treating vector search as magic pixie dust that makes search great without effort is not necessarily going to work that well. Also, it can add a lot of cost and complexity to the equation. And if not tuned properly, you don't necessarily get good results.

    The key thing with RAG is to get the right information in the context with as few queries as possible. That requires good recall (ensuring that if it is there it can be found with a reasonable query) and precision (ensuring the best stuff is on top and minimizing false positives).

    With search, and by extension RAG, the principle of shit in, shit out applies. Most of what search teams did before AI and RAG is still the best way to optimize the experience with RAG. And if you mess that up, search is not going to be working that well and no amount of AI can compensate for that or only at great cost in tokens and time. So, having an ETL pipeline to pre-process what you index, testing & benchmarking search quality, etc. are all helpful.

    The good news is that you don't need that much skills with agentic coding to build something half decent for this. This code almost writes itself. And even a little bit of effort on extracting structure before indexing can make a big difference.

    by jillesvangurp
  • > RAG is basically good old information retrieval with LLMs doing the querying.

    No - rag is doing search before you call the llm to give it context from some corpus like your helpdesk articles.

    by IanCal