Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- What I want from a query language is for it to not be a language, but some kind of specification of components that can then be transmitted to the engine in a machine-readable way. The bizarre thing about SQL is that it has to be written in SQL. What if instead a JSON representation of the query (something akin to the AST, with a bit of processing done to boil it down) is what the engine took in? Then each language could specify the query in whatever syntax is most idiomatic for that language. Right now we do this with ORMs, but it's a huge pain because the ORMs are like a transpiler with a really free-form and messy language as the target. All the ORMs are just pasting strings together. If the "native" query format were more structured, we could have a lot more cool approaches for wrapping that on the programmer-facing side.by BrenBarn
- The problem with alternative query languages is that the people who have the most knowledge about creating queries and of the relational domains underlying their businesses are all experts in SQL. Introducing something else, then, means your most natural user base must migrate away from something they understand how to use well, and that's a hard sell.
So, until the ultimate query language is developed, I'll take SQL with pipes. It's an easy sell and good enough to eliminate 90% of my gripes about SQL.
by tmoertel - Exactly, we all know the merits of Esperanto, but few have switched away from English.by likium
- For example, Elastic. Much extra learning curve for little obvious gain.
I like to say, with zero research basis, that the New Shiny has to be an order of magnitude better than the Old Thing for people to say "Oh yeah, I gotta have that."
by smitty1e - This is exactly the same problem facing people trying to develop new music notations. In order to grasp the domain enough, they have to be experts in the existing music notation, and once you're an expert in it, the motivation to create something new goes away. From what I've seen, the people who want a new music notation are mostly people uncomfortable with sight reading.by taybin
- SQL is what it is today because it is battle tested and has to handles a very hard problem of handling arbitrary concurrent reads/writes, so the likely scenario is that trying to replace general SQL wholesale will just end up making a worse, less tested version of SQL that developers are less familiar with. So, I think the best query language is probably whatever query feature that's already in your backend language, LINQ for C# for example. The only room for an SQL replacement in my opinion is if you are willing to trade flexibility for speed a la TigerBeetle.
The good thing about having built your own programming language via LLM nowadays is that you don't really have to speculate about a theoretical language when you can just have Codex/Claude implement it and try it out for yourself. I did it yesterday when I wanted to try out this theoretical high-performance database architecture that I had in mind and just added query functionalities to the language I already have.
If anyone is interested about the results, the default naive mode for this new database is ~0.2x the speed of concurrent durable mutation workloads, but if you specialize it to the particular application, you can get ridiculous 50-100x performance increases on filters and maps at the cost of flexibility and more upfront design. Experimental results are promising, definitely not production ready though.
by YuechenLi - doesnt meant the syntax isnt a pile of dog farts
- No, SQL is what it is today because it was crappy in the beginning and no one managed to replace it.
It was only a partial implementation of the relational model, we could have been so much better had it not become the standard
by g-b-r - by nigwil_
- PRQL is one of the best attempts at a new query language IMO
I've been working on a Lean4-based query lang that compiles to substrait, I think the power it has wrt to types and functional programming could improve on SQL ergonomics a good deal
by nylonstrung - Can someone explain to me why SQL error messages are so bad? I routinely have some monster query where the message is effectively, "Illegal syntax somewhere, dufus".by 3eb7988a1663
- This sounds like a proper of the parser, rather than of the languageby klysm
- I'd guess people just haven't put much effort into it. Lots of programming language compilers have absolutely terrible error messages. In SQL its usually just one line, so "somewhere" isn't that big a place.by bawolff
- My ask is 15 years old[1], a live SQL extension. Allow a query to be a subscription to a database, so any updates get streamed as deltas to a listening client. There were a ton of times in my time using SQL where the same query is run over and over, just to get/handle that delta.
Wouldn't it be a lot more efficient to just work that way in the first place?
[1] http://livesql.org/ <--- just a few paragraphs of text from 2011
by mikewarot - Sounds like you want to read the log file to be honest.by grebc
- pg logical replication is close to this. but ideally you want incremental query updates, which I believe Materialize provides.
but yes, I agree this is quite often what one wants, and would remove a lot of grot from the client
by convolvatron - Snowflake has STREAM which can be crated on a view. It also has Dynamic Tables, from which you read a delta using STREAM or using row timestamp.
SQL server has Query Notification.
You can also read from debezium or other cdc, but thats more like table change than query result change.
by DenisM - The issue is that incremental computation is a non-trivial problem that cannot be solved on a language design level.
There are some new interesting players in the field though, eg. https://github.com/feldera/feldera
by mkleczek - BTW, Oracle supports this under the name "continuous query notification".
https://docs.oracle.com/en/database/oracle/oracle-database/2...
You can get callbacks from the driver as query results change, or have notifications be sent to stored procedures, or posted to a message queue (and from there turned into web hooks etc). The notification comes with info about the deltas.
The main issue with it is that the queries it can monitor live are a subset of all queries. It's really more like using SQL to select database cells to watch, than propagating changes through arbitrary query plans. For example, it can't handle a SELECT COUNT(*) FROM statement. Obviously you can use it as a trigger for re-running more advanced queries though.
by mike_hearn - Sounds a lot like Spark before it became so enterprise-focused. Back in my day we wrote scala to run our queries, and once we figured out how to get our compiler and runtime set up, we liked it!
I’ve been getting into Postgres recently and I was very surprised how easy it is to introduce new types/operators/etc through C code. I’m not talking about domains. Just write some C and you can have whatever type you want. It really demystified “extensions” for me, I actually think that is an actively harmful name (it sounds clunky, gross, based on my experience dealing with “extension” and “plugins” elsewhere) for what is essentially just custom types/functions. More people should try writing their own postgres extensions. It’s not very difficult at all!
I’ve been cooking in this space for quite a while (HDFS/spark, Apache Pinot, proprietary stuff, an experimental functional ORM over SQLite). The biggest problem, I think, is the interface between the management/admin, application, and “query” layers. I think something like grpc/protoc (or indeed the way Spark used the JVM) is needed to provide non-leaky abstractions and more programmatic/structured interfaces from the DB to its clients. Happy to share more, but basically, the database needs to become capable of general (meta-)parsing with a reflective type system, I think.
by weitendorf - https://www.geldata.com/blog/we-can-do-better-than-sql
(https://news.ycombinator.com/item?id=24106608, https://news.ycombinator.com/item?id=19871051)
by mcc1ane - As a meta comment, I can handle code blocks without syntax highlighting, and I can handle code blocks that wrap. But both together with long comments just turn into line noise. There's no longer any useful visual signal for how to read them. On my phone the code blocks are simply impossible to meaningfully parse.by bastawhiz
- landscape it is not good but okay-ish on my phone display. hth.by froh
- I would be genuinely curious what the OP other and folks here think of Mangle Datalog in this context. The go implementation is here: https://codeberg.org/TauCeti/mangle-go and the Rust implementation is here: https://codeberg.org/TauCeti/mangle-rs
The implementations are not high performance, but if you can fit everything in memory or you can organize your data and integrate it through external queries, you should get something workable for a lot of use cases.
I did not set out to replace SQL, and while I don't mind adoption, that is not why I am sharing it here. The open sourcing was motivated by making datalog more widely known. I did some research and found out that I needed a datalog implementation with particular characteristics, I for sure knew I didn't want to use SQL for what I needed.
There are structured types and recursion and being able to name predicates and compose queries... Mangle has some users and there is a few application that take advantage of the queries-as-logic-programming approach.
I think an insight one can draw in this discussion that a query language and the system (DBMS implementation) that it is part of can hardly be separated when it comes to the inevitable performance requirements one has.
by burakemir - That's a pretty cool pair of repos. I've been working on a C++ Datalog myself [0] for embedded queries over in-memory datasets. It's been surprisingly useful for a bunch of different usecases (e.g. build graphs). My implementation is faster than Souffle on all the benchmarks I've tried, but apparently I still have some work to do optimizing startup time on small graphs.
- Some overlap here with some of my favorite essays on why SQL is lacking:
https://www.scattered-thoughts.net/writing/against-sql
That particular post ends with a wish-list of items so it's the most similar to the OP. But there are others on the site that I quite enjoy (click on the home icon and search "SQL" on the page).
My personal take is that SQL will continue to reign for a long time because of the how monumental the task of replacing it is due to the inherent complexity of databases. LLMs make this worse because they're really good at translating prose to SQL. Now that it matters less how annoying SQL is to programmers, SQL will become more like assembly over time: something mostly computers write because it's complicated for humans to deal with directly. This is deeply ironic given that SQL was ostensibly designed to read like prose, i.e. to be easy for humans.
- LLMs are also very good at writing code for newly invented languages, especially if they can execute it and iterate. I strongly believe the barrier to switch languages is lowered in a post LLM world.
Ten years ago I was at a startup where we used Datomic, and it was okay, but six months in the sales team was like “ok how do I run SQL queries so I can triage leads”. We had no answer of course.
Today it would simply be: type what you want in natural language and we’ll generate the query with Claude.
I just tried one representative query from that startup against a hypothetical datalog query tool in Rust and it did just fine.
- The end of that article ends with a pertinent quote [1] by Michael Stonebraker [2]. I've included more of the original quote here:
> My biggest complaint about System R is that the team never stopped to clean up SQL. [...] All the annoying features of the language have endured to this day. SQL will be the COBOL of 2020, a language we are stuck with that everybody will complain about.
> My second biggest complaint is that System R used a subroutine call interface (now ODBC) to couple a client application to the DBMS. I consider ODBC among the worst interfaces on the planet. To issue a single query, one has to open a data base, open a cursor, bind it to a query and then issue individual fetches for data records. It takes a page of fairly inscrutable code just to run one query. [...] Only recently with the advent of Linq and Ruby on Rails are we seeing a resurgence of cleaner language-specific enbeddings (sic).
by Rendello - Rawdogging SQL when you're not a seasoned DB administrator basically makes an arcane art look occult.
Most people reach out towards an ORM or query building engine and otherwise don't really go far beyond the basic CRUD, joins, and some simple aggregations with groups. Since they try to be DB agnostic you'll rarely get an adaptor over CTEs or window functions or partitioning.
An LLM is great at exposing what a database is capable of doing with SQL and might even manage to navigate the most poorly designed of schemas. And it might even manage to design one to an acceptable standard if it has enough domain knowledge in its context.
by ljm