Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- The end of the AI's first sub-phrase in the video is wrong.
You started by playing a simple I - vii - I as the first sub-phrase. Then the AI started its response with a dominant 7th-chord. In the vast majority of cases I can think of from the Classical era, the AI's sub-phrase would end with a half-cadence. So, including what you played, the first two sub-phrases would look like this:
I - vii - I
V(7) - I - V
But instead, the AI took the unusual step of having a full cadence for that 2nd sub-phrase. This in music is what someone might call a "double beat" in screenwriting.
Moreover, there are no half cadences in the phrase after that. It's all full cadences in the tonic key, plus two plagal cadences over a tonic pedal there in the 2nd phrase. Each sub-phrase just extends the tonic. There are idiomatic ways to just stay in the tonic for awhile, like "Horn fifths." But this excerpt isn't idiomatic and sounds like the composer accidentally forgot to develop the harmony.
I'm going to claim that this excerpt would sound like the musical equivalent of a run-on sentence to any composer who lived from the time of J.S. Bach to Robert Schumann. And from Mahler on they'd assume you're thumbing your nose at convention.
Put another way: the kind of student who'd use this to finish a harmony exercise would end up getting the same grade they would have gotten doing it on their own.
Edit: clarifications
by jancsika - The results strike me as comparable or worse than you could get with a Markov model. I think it reveals the gap in understanding between LLMs and music. I think you need to either: - Set up a pipeline to decompose music into, say, harmonic sequences and melodic sequences, and then have the LLM work on some more fundamental or more high-level layer of musical composition and then re-translate it back into actual sounds. - Develop a better dataset and train the LLM more natively on musical examples.
Does anybody know of a project that has produced more convincing results?
- There is no LLM in the ML pipeline provided by the OP.by Computer0
- I'm not at all there yet, but this sort of thing is on my roadmap for my AI music project at tunesage.com ... That is, generating musical ideas while being conscious of phrase / melodic / thematic structure that a user can then play around with.
While LLMs (and ANNs in general) are definitely great for musical analysis (though still limited), I think they're likely overkill for the generation part, plus they tend to suffer from "creative collapse" as they prefer averages rather than exploration (which can be good or bad depending on what you want).
by shannifin - Ah, MIDI files. The only type of music you could realistically download from the internet back in the day, and you had to wake up at ungodly hours so that your dialup modem would not rack up a massive phone bill.by hliyan
- I remember downloading MOD files. It was just a bit larger than midi, but sounded better. I had a PC but I think it was an Amiga thingby xtracto
- MIDI is still widely used for professional music production. It sounded goofy back in the day because synthesizer it was played on was not very good.by killerstorm
- I would love something like that, except that I play the melody, and it produces proper 3-4 part accompaniment, preferably in good baroque style. Extra bonus if it could also write it into a file in a format suitable for music editing programs.
- This feels like a natural next step. Starting with a simple melody and having the system fill in the rest while still following your playing style could make it much more useful for experimentation than generating a complete piece from scratch.by Naitik88
- Did you ever try Ludwig? It was rule-based by the Fritz chess engine maker. To my understanding Ludwig was grounded in traditional harmony, counterpoint, voice-leading, and orchestration principles taught in formal music education. It never took off so they eventually stopped it.
Some details: https://shop.chessbase.com/en/products/ludwig_3_engl
You can still download it for free: https://www.heise.de/download/product/ludwig-58854 (Heise is a renowned German publisher, nothing shady)
I am still waiting for something like this based on generaitive AI.
- That is an incredibly hard challenge though. Creating the whole backing track (in any meaningful way other than just basic chords) from just melody will require an amazingly high number of highly subjective choices and random gen will not lead to good outcomes since our ears like intentional and artistical creativity in general.by altmanaltman
- That's a fun idea. You could start playing the piano and it kicks in with a base and drums for a jazz band.by morkalork
- An early attempt at this was Microsoft Songsmith [1] all the way back in 2009, which would take a melody (usually recorded by mic) and try to scaffold an accompaniment around it though obviously not realtime in any sense of the word.
The closest we've had to realtime orchestration around a melody in the "real world" is probably arranger keyboards though your left hand is still responsible for the chord progression itself.
[1] - https://en.wikipedia.org/wiki/Microsoft_Research_Songsmith
by vunderba - Interesting. I do feel letting a machine generating notes is taking the joy out of improvisation, is it not?
I think one of the great, early, joys of learning a piano is gaining the following intuitions: The seemingly harder path of learning sheet is actually faster. Your mind _should_ learn to think in two dimensions Spatial — where fingers go — and Time — pitch and tempo – ** when learning. The _internalization_ of Space and Time queues guide the fingers in a dance that is vastly satisfying. This skill leads you to the final part of the journey that is improvisation and the one more exciting than what i am on now.
---
**
Space: Your finger placement on keys right, e.g. knowing how to go from landmark/anchor notes(mid-c, G, F etc) and then go to the others above and below it. Crudely this is some what like typing from your landmark f and j qwerty keyboard
Time: The out singing/verbalizing of the notes/beats on a time measure as you play them(per the time measure). e.g. you can say out loud 1-2-3-4 for 4/4 measure, if the measure has quarter notes say out loud. And `1-e-and-a-2-e-and-a-3-e-and-a-4-e-and-a` for a 4/4 with 1/16th note granularity. Do this as you play the notes and you get a sense of tempo.
by ghm2199 - as with every discipline that AI and therefore computers can reason through suddenly:
you could always do the discipline you liked before for self fulfillment purposes, and you can still do it for self fulfillment purposes now
by yieldcrv - Reminds me of a story a coworker told me about his band rehearsal when the new drummer counted one-two-three-four-five-six-se-ven-eight.
- OP found a different source of joy: "I wanted the fun of working through the problem myself, rather than just implementing someone else’s research."by elwell
- Reminds me of Francois Pachet’s Continuator (all the way back in 2003, using hierarchical markov models)by timmb
- Really cool! It reminds me of a demo I saw live at NeurIPS: https://www.youtube.com/watch?v=8s3V922h3CU (paper at https://openreview.net/pdf?id=3yeBer3J5z).
If you like such creative AI work, I would recommend looking at the other "NeurIPS Creative AI Track 2025" submissions as well.
by mare5x - Hearing the start of Für Elise, and then it being taken in an incredibly different direction, is surprisingly disconcerting.by karmelapple
- Can we hear it?by h2aichat
- I found it refreshing and interesting, not really disconcerting. Show's how AI can extrapolate in novel ways. An interesting feature.by ABNW
- I found it just... bad and out of whack.by archon810
- There's an incredibly funny bit (if you're into absurdist comedy) by Hans Teeuwen about the disconcerting direction a well known song can take:by gregsadetsky
- Reminds me of this project to generate every melody possible algorithmically in order to fight music copyright lawsuits. https://allthemusic.info/by goda90
- Thanks. I enjoyed watching the TEDx talk on that page.by ronyeh
- Classical pianist and software product designer here.
I see so much in common with this project and the numerous AI-based UX design tools out there. Whether it's music or UI, now that the "generation" portion of the work costs zero, all that remains is taste.
And so much of taste comes from exploring and killing off possibilities that turn out to be dead-ends. I love the idea that models like these will help us find the dead ends faster, or even produce a gem here and there.
P.S. if you want another uncanny version of Fur Elise, listen to Beethoven's own 1822 revision: https://www.youtube.com/watch?v=s24TtiGgb6k. His 1810 version that we all know was simpler and more balanced. But for what it's worth, Beethoven didn't publish either of them.
- Interesting! It reminds me of something I once heard another banjo picker say... something like "I spent most of my youth figuring out how to cram more notes into a piece, and most of my maturity trying to remove them".
I am not fond of the extra jazzy notes.
by stickfigure - Thanks. I found the 1822 version actually quite agreable
- I think this is a great project and very HN. Not sure why the comments are so focused on the deliverable- you learned way more and had a much more interesting experience.
One think I didn't see mentioned in the post- maybe I missed it- how large was the data? How many samples did you use to pretrain and post-train
by jasonjmcghee