Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • > Events, queues and persistent logs are getting more and more important in today’s software systems. Systems like Kafka, RabbitMQ, SQS and others provide that functionality. But maintaining them is annoying, custom and you need the skillset.

    In tech stack choices, I prefer staying simple as long as feasible. That said, you also need to know and understand concepts at a thorough level.

    The above comment from the article suggests PG as a central server that simplifies event architecture. As if the "annoying, custom and needed skillset" around those specific alternatives are unnecessary baggage.

    If you know anything about queues, scaling, availability, access semantics, message formats and concepts such as delivery guarantees, you find out very quickly that the server which stores a queued message is not the high-order bit in that equation.

    by jroseattle
  • To me, your concluding sentence cuts the opposite direction. The server is not the most important thing, and each new kind of server that exists in the system is an appreciable increase in maintenance burden. To me, taken together, this is an argument for waiting until you have a very concrete forcing function to introduce that new kind of server for this purpose.

    I think a good way to think of this is: What empirical metric will the introduction of kafka (or whatever you choose) move in a positive direction? Latency? Oncall burden? The amount of code you have to maintain? Do you have correctness or data integrity metrics that this change would register on? etc.

    This isn't at all intended as an unanswerable question. Many or most organizations will easily say "yes" that they expect some improvement on some metrics by adopting the "right" system for the job. But lots of other organizations are cargo culting "well we know this is the right way, so we should do it this way" long before any metrics they care about would demonstrate the improvement.

    by sanderjd
  • This article seems like a reaction to DuckDB's surge in popularity. Having multiple DB engines to choose from is good and it often doesn't matter which one you use. One could make the same argument about DuckDB. Many database engines are multi-purpose. Though of course there are specific use cases where a different DB may be more appropriate...

    Anyway databases nowadays are a commodity. A sticky commodity but nonetheless they are replaceable; increasingly so in the age of AI where data migrations are easier than ever.

    by socketcluster
  • > PostgreSQL Replacing Your Microservice

    I've done that before and the code was a mess. It works at the beginning but APIs do much more than piping data from the database. When you start dealing with ACL, external calls, code reuse, etc. It's just nice to have all the tools available to you from something like Python or Go.

    by florianherrengt
  • No mention of https://postgis.net/ - shameful
    by _joel
  • Intrigued by this

    > After some performance checks it became clear that PostgreSQL was even faster than reading from the file system for our use-case. PostgreSQL uses the file system very efficiently for its data - and it adds a lot of caching and efficient reading and writing strategies that can outperform writing and reading raw data on a file system.

    This goes against conventional knowledge. I've always heard (and followed best practice) to avoid storing binary data in BYTEA columns that should otherwise be put on a filesystem or an object storage like S3.

    I'd like to find out more about this, because in many cases it would be very convenient indeed to store it in the database itself.

    by sgt
  • The primary reason to avoid doing so is avoiding thrashing your buffers, along with increased size of backups, WAL bloat, etc.

    Can you? Yes. Should you? Not at anything beyond a toy scale, unless you want to pay for more RAM to ensure that your normal OLTP queries don’t take a performance hit.

    by sgarland
  • In our multi-tenant e-commerce application we handled it like this:

    * Metadata for each file is stored in the database

    * The application accesses files through an abstraction based on that metadata, and doesn't care if the actual file is stored in S3 or the database

    * File types which are small and few, are stored in the database. For example letterheads, logos, terms-and-conditions. (a few gigabytes total)

    * File types which are big (e.g. CSV-reports) or many (e.g. invoice-PDFs) are stored in S3 (several terabytes total)

    * Development and test systems often use the database for everything and don't have an associated S3 bucket.

    * Most of the application remains usable without S3 (access to invoice PDFs and CSV-reports is not critical)

    In our case the DB was Mongo, but I expect Postgres to work the same.

    We actually stored all files in the database originally, and only migrated after it reached several terabytes. It worked perfectly fine, but was rather expensive. So next time I'd go for S3 for the start.

    by CodesInChaos
  • These types of articles needed to be written because we have gone way too much in the other direction. The issue is that people use too many tools prematurely when they are not needed at their stage. So yea, in most cases, you are probably better off just with Postgres. I m a culprit of this myself so I wouldn't say that I know better. It is just too tempting to setup too many tools to feel cooler or feeling that "we must use elasticsearch as no one does search in db".
    by codegeek
  • I tend to agree with quite a few points in the article, but some topics warrant some careful scrutiny.

    * As a message queue: Only if your required features are very basic, like if you need cluster communication and run your own coordination protocol on top.

    * High Volume Time Series: TimeScale works, but composes badly with other workloads on the same DB server ( from an operational perspective at scale )

    * Vector Database: The same issues as with TimeScale.. PgVector for example lives in its own seperate "world" and the query planner sees it as a very opaque thing. Forget about adding vector storage to an existing high volume db, that must server other complex queries.. PGVector will either trash your caches, or take over your cpu so that workloads that used to work fine stall. This is IMO not a pgvector problem itself ( Kudos to those guys ) but rather that postgresql extension apis are not very good at exposing custom costs and tradeoffs to the system as a whole.

    * Raw Data: Works for small files... why anyone would want to store large amounts of data in it would be a mystery, where it shines is accessing LOTS of small files where internal caching etc help a lot compared to raw filesystem access ( also a bit dependent on the filesystem and its tuning though )

    * Microservice: If your service is ONLY exposing json data from some database model, then it should not exist at all IMO. Create a view and be done with it.

    by Gluber
  • I like to consider Postgres the starting point for all of these things, that can be outgrown and replaced when appropriate. I do love just shoving everything in Postgres and seeing that I only end up needing a few additional dedicated services as the product groups. Redis is usually the next pickup for me.
    by jjice
  • > Microservice: If your service is ONLY exposing json data from some database model, then it should not exist at all IMO. Create a view and be done with it.

    Yeah, this has nothing to do with Postgres. If the service is accessing a database that isn't internal to the service, then that database is already a standalone service in itself.

    by Kinrany
  • Also to note: (Not a fault of PGVector again just a limit of our algorithmic knowledge) PGVector does HSNW or IVFlat indices ... (there is nothing better persistent) however it breaks down with high latency at LARGE amounts of vectors ( 100MIO+ ) that seems like a high ceiling, but when designing production RAG systems, you tend to do per chunk embeddings, or even visual patch embeddings... e.g one page of a document becomes 1024 vectors in itself (for visual patch embeddings ) ... so you hit those limits at 100000 pages already.. something larger organizations definitly have.
    by Gluber
  • PostGIS is also another very useful addition for storing, indexing, and querying geospatial data.

    http://www.postgis.net

    by silvestrov
  • I use SQLite for everything, and I'm perfectly happy with it. I'm aware of the concurrent writer issues, but at my scale it doesn't even matter.
    by replwoacause
  • Yes, especially for a web app where there’s realistically only a need for one VM/server. By the time you outgrow that approach, a very straightforward migration to Postgres is probably the least complex problem you face.
    by bensyverson
  • Same, I posted some corrections to Dr. Bauer's article: https://joecode.com/2026-08-19-sqlite3/
    by joewils
  • The main issue with SQLite is the very poor type system, after testing it for an app I was shocked.
    by Thaxll
  • Perfectly reasonable:

    I'm a huge PG fan, so I start everything with it, but SQLite is sane, and it generally has a happy upgrade path to PG If you need it.

    by zulux
  • This kind of post (Postgres! It's all you need!) is getting pretty tiresome. Postgres does not even come close to a full replacement for Elastic, and that's just the first bullet.

    Looking down the list it is pretty easy to go: Yes, postgres can be used instead of that for extremely basic use cases, but it all goes out the window you actually need any of the power of these other tools.

    by devin
  • You're right that vanilla Postgres doesn't come close to replacing Elastic. There are efforts to resolve this, though, like ParadeDB: https://github.com/paradedb/paradedb (disclaimer: I work for ParadeDB)
    by philippemnoel
  • I'm partial to Typesense, especially for smaller data sets, since it runs primarily in memory, is easy to use and is hella fast. For bigger data sets, I hear good things about Meilisearch.
    by jjordan
  • Postgres its all you need means to me(IMHO) postgres for all Olap (DB + message) , not all analytical databases.
    by airocker
  • That's just it - most use-cases are pretty basic, and if you don’t know what you need then Postgres is probably a great place to start.

    If you’re just starting out, keep things simple. Otherwise, you probably already know exactly why you need something more than Postgres.

    by wolttam
  • The power of the other tools mostly shines in large scales. For most applications, though, performance of postgres more than suffices.

    I tried to use rabbitmq for a small app, installed it, configured it and then it didn't work. Spent a day jumping through hoops getting it right.

    Dumped it and used postgres, in half an hour. Worked like a charm.

    by kumarvvr
  • The point is in general for people to just consider it, often people start out on their side projects or internal company projects and commission Elastic, Redis, Postgres, Kafka before even getting started. In reality they could fit it all into Postgres for a very long time.

    Nobody is saying that a huge ecommerce store with complicated filtered search logic should throw away their Elasticsearch cluster and switch to Postgres.

    by dewey
  • > Postgres does not even come close to a full replacement for Elastic

    Size matters!

    For most of the application out there elastic (or kafka or any other specialized tool) is just too much(and too costly). They can do fine with postgres or mysql. Actually, I'd argue that in a lot of cases even postgres is too much, probably sqlite is enough.

    by vb-8448
  • I think it would be helpful if some of these posts included scale. There are almost always two groups talking past each other

      - I run my B2B application, Postgres only, and it is perfect for my 50k MAU. No complaints, sleeping soundly with the low complexity and a two man team. 
  - I work at FAANG, where we have 1 billion DAU, and this is a joke. Would fall over immediately. The dedicated ops teams for Kubernetes, Elastic, and Redis have never complained about scaling issues.
    by 0cf8612b2e1e
  • My general rule of thumb is "Use Postgres until you've discovered why you can't use Postgres."

    Anything you introduce is another moving part you have to operate and maintain, and in the beginning, Postgres can probably handle it. Wait for load, see where its failing, and then you'll have a better idea if adding another tool is worth the cost.

    by psadauskas
  • Another rule of thumb. Use what you are comfortable with until it stops doing what you want.
    by boznz