Why we built yet another Postgres connection pooler

Join the discussion

• Hacker News
• Is there a pooler handling schema switching in PostgreSQL? like something in front of django-tenant ?
  by babayega2
• we moved our django app behind pgbouncer transaction pooling a few days ago and the surprise wasn't SET so much as queryset.iterator(). it relies on server side cursors, which don't survive being pooled, so we had to disable it everywhere and let it fall back to client side. also had to move statement_timeout out of the app's connection options into the pooler's own connect query, since libpq startup params just get silently ignored behind it.
  by mmakeev
• Using server-side cursors is a sign of bad design. You use the PostgreSQL server's resources as a cache when you're fetching and processing stuff row by row, which is very bad. Just get the data you requested in the first place in one go and do your stuff in your app. Or process all data on the server and then get the processed data in one batch as well.
  by HackerThemAll
• Handling cursors is tough - they are very much session-level objects, so even if we, say, pinned your client while it uses that cursor, which would work, that would decrease the performance of connection pooling overall.

  So, what's better, breaking your app initially so you know to remove that feature, or letting it work silently while the connection pool isn't 100% in transaction mode? Tough call.

  by levkk
• Clickhouse also just put out a fun article on scaling pgbouncer too, talking about scaling out so_reuseport while not having to shard so harshly (a major limitation pgdog here is addressing via rewrite), https://clickhouse.com/blog/pgbouncer-clickhouse-managed-pos... https://news.ycombinator.com/item?id=48814152
  by jauntywundrkind
• Well tbf pgdog looks extremely amazing on paper and goes way beyond multi threading.

  The notify/listen fix and automatic query routing to read replicas and auto sharding might bringt Postgres finally closer to vitess

  by merb
• Doesn't this NOTIFY performance fix mean that it isn't transactional any more?
  by inigyou
• From the strictest CAP theorem definition, that's correct, it is not. But, it's pretty close. I know that in the database world, that's not a good answer, but in practice, it will deliver the vast majority of messages, so maybe that's good enough? We'll see.

  We show that it's possible to come close without breaking the DB or the app, but I suspect, it's not quite yet at the level you'd expect from a _durable_ work queue, e.g., Kafka. Not going to replace that one anytime soon.

  by levkk
• Any plans to add Query Caching for Selects?

  as per:

  https://www.pgpool.net/docs/latest/en/html/runtime-in-memory...

  by khurs
• I hope not. It introduces a lot more problems than it solves.
  by HackerThemAll
• I don't think so. Too much trouble. Caching is a really hard problem without context, and the context, typically, is app-specific.
  by levkk
• Can someone explain it to me like I am 5. Why did postgres win vs mysql? I don't know many companies at scale that use postgres. Slack, youtube, etc all use a mysql based sharding system https://vitess.io/. I thought the war was lost for postgres, but it seems to keep going. License issues? (Disclosure, I manage 'a few' mysql vms).
  by ransom1538
• > Why did postgres win vs mysql?

  They did? By social media? According to a lot of social media devs Java is also “dead”.

  Having said that the issue is MySql / Mariadb is moving more and more behind commercial products e.g. Galera and Heatwave. Postgres continues to be the open community effort.

  But hence the divide you see. Large companies. Real traffic use pragmatic solutions to make money. The tutorial developers and hype does whatever.

  by re-thc
• MySQL is owned by the lawn mover* and barely kept alive.

  * https://www.youtube.com/watch?v=-zRN7XLCRhc

  by Maledictus
• I don't understand the idea of "winning" in this context.

  If your situation doesn't require specialized features of a particular database, then it doesn't really matter. Just pick one. There's too much premature optimization in the world.

  If your situation does require something that one database or another excels at, then be grateful that there isn't really such thing as a winner and you can pick the one that works best for your context.

  by anticorporate
• Maybe it's because Postgres is easier to write extensions for? I do think MySQL is still used more widely, but since people on HN and other hacker/tech circles write extensions, you hear about Postgres more. Those extensions also increase Postgres's actual usage, like I know people who only know about Postgres because of PostGIS.

  Regardless of popularity, idk, Postgres feels nicer to use for me.

  n.b. PgDog isn't an extension

  by frollogaston
• Postgres features still "just work" behind the pooler.
  by icevl
• > Since connection poolers reuse connections between clients, the connection state of one client “leaks” into the connection state of another.

  Wow this is very bad. This actually happens in typical Postgres setups?

  by petters
• You'll see this kind of fun in other databases that support "persistent connections." When you start up, you have absolutely no idea what the state of the database is. If a previous process errored out, you might find yourself in the middle of a broken transaction for example. Did the last session do some weird SET magic to make things work? Did it create temporary tables? Well guess what, it's all still there!
  by McGlockenshire
• Yes, as a consequence of how aggressively transparent to the postgres wire protocol pgbouncer wants to be. This article does a good job explaining it: https://www.augusteo.com/blog/how-pgbouncer-works
  by llimllib
• pgbouncer has different pool modes you can pick from that have some impact over what's possible to leak https://www.pgbouncer.org/features.html
  by nijave
• It's not unique to postgres, as others have said; the same thing can happen with e.g. MySQL poolers/proxies/etc., since the behavior of the connection can be changed dynamically and it persists for the lifetime of the connection.

  Example: legacy client A connects to MySQL via the bouncer and says 'I want all of our conversations to use latin-1, not utf-8'. This changes the character set that MySQL parses queries with and returns responses in. The legacy client does some queries and then disconnects.

  Now a new client connects to MySQL, and the bouncer just assigns it to the still-open connection from before. The new client is fully UTF-8 compatible and since this is the default for our database it doesn't explicitly say so; it just assumes that UTF-8 is the way to go. Unfortunately, the database server is still thinking in latin-1, meaning that if this new client sends UTF-8 data it will be parsed as latin-1; latin-1 is a subset of UTF-8, meaning that queries will actually work fine unless they need to use a character outside of latin-1, in which case they will get an error, or corrupted data, from the server.

  The only solutions around this are:

  1. Ensure that every client is using the same settings; if your database is for a single app that uses the same ORM, then this is automatic.

  2. Ensure that every client is always explicit about everything it might need to change e4very time, so that every UTF-8 client explicitly sets UTF-8 connections even when that's the default; clients that need utf8mb4 ask for it explicitly and clients that can't handle it ask for something else. One way of ensuring this happens is to configure the server (or the bouncer) to use defaults which are not valid for anyone, or which are going to cause errors frequently and not rarely (e.g. setting the default character set to 7-bit swedish, which would cause frequent errors).

  3. Use a bouncer which can either disallow these changes or detect and revert them after the original client has disconnected. I'm not sure if this exists for MySQL at least.

  4. Use separate bouncers for each application that might be different (extension of #1); in other words, instead of having a bouncer or set of bouncers for each pool of database servers, you have them for each application; your web app gets one, your legacy reporting tool gets one, your ODBC connector gets one, and so on.

  It's kind of a huge mess in theory; in practice, a lot of installations fall into the #1 case so it never matters, but that makes the occasional instance where it does matter extremely difficult to debug.

  by danudey
• by definition connection poolers re-use connections so it it can happen with any connection pooling setup, PG or no.

  in pgbouncer the connection is reset via a customisable command [0] which should reset the connection to a clean state.

  [0] https://www.pgbouncer.org/config.html#server_reset_query

  by vizzier
• And why it should be done upstream with a thread pool model and an internal scheduler as MS SQL server does.
  by rastignack
• What really interests me most is the sharding and the possibility of using this for multitenancy - is the hooks / plugin architecture sufficient so you can run a small sidecar to add shards or tenants to the TOML file dynamically? Would be a game changer.
  by rubenvanwyk
• Yes. The plugin architecture supports this currently. That being said, you can use any number of sharding functions we support to do this too.
  by levkk
• We found it pretty easy to build a little k8s controller for our own purposes to do this -- see https://news.ycombinator.com/item?id=48478994 . You probably don't need to implement this as a plugin or hook, pgdog supports dynamic reload of its configuration without dropping existing connections.

  Although I'm the type to shy away from adding extra layers in my architectures when I can help it, pgdog has been an absolute breeze to use :)

  by apt-get
• It's awesome to see AGPL instead of the horrible BSL variants that have been going around.
  by 27183
• It is quite telling that most cloud companies stay away from AGPL. Their business model is antisocial.
  by alecco
• Thanks! We try to be very open and explicit about why we chose AGPL. Personally, I like it because it's an extension of GPL, which is a huge reason why I was able to self-teach programming. Just trying to give back.
  by levkk