Birbla

Discussion summary

30papers.com offers a beginner-friendly list of essential ML papers, but users criticize its UX and readability issues. The site features animated scrolling and formatting problems, leading to mixed feedback.

What the discussion says

  • Users find the website's UX and animations distracting and hard to read.
  • Formatting issues with LaTeX formulas are common.
  • Some appreciate the curated list of ML papers despite design flaws.
“Main page UX is terrible. If you go for quirky, fine, but I would not want to use it.”
— lostmsu
“After scrolling a bit, I find myself having a mild headache and slight dizziness.”
— solarengineer

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Ilya has never made the list public.

    The list is simply made up, as confirmed by John Carmack.

    """ John Carmack @ID_AA_Carmack I rather expected @ilyasut to have made a public post by now after all the discussion of the AI reading list he gave me. A canonical list of references from a leading figure would be appreciated by many. I would be curious myself about what he would add from the last three years. """

    https://x.com/ID_AA_Carmack/status/1622673143469858816

    by _giorgio_
  • No need stupid moving texts.

    CS231n: Convolutional Neural Networks for Visual Recognition - https://cs231n.github.io/

    The Unreasonable Effectiveness of Recurrent Neural Networks - https://karpathy.github.io/2015/05/21/rnn-effectiveness/

    Understanding LSTM Networks - https://colah.github.io/posts/2015-08-Understanding-LSTMs/

    ImageNet Classification with Deep Convolutional Neural Networks - https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436...

    Deep Residual Learning for Image Recognition - https://arxiv.org/abs/1512.03385

    Multi-Scale Context Aggregation by Dilated Convolutions - https://arxiv.org/abs/1511.07122

    Identity Mappings in Deep Residual Networks - https://arxiv.org/abs/1603.05027

    Recurrent Neural Network Regularization - https://arxiv.org/abs/1409.2329

    Deep Speech 2: End-to-End Speech Recognition in English and Mandarin - https://arxiv.org/abs/1512.02595

    Order Matters: Sequence to Sequence for Sets - https://arxiv.org/abs/1511.06391

    Neural Machine Translation by Jointly Learning to Align and Translate - https://arxiv.org/abs/1409.0473

    Pointer Networks - https://arxiv.org/abs/1506.03134

    Attention Is All You Need - https://arxiv.org/abs/1706.03762

    The Annotated Transformer - https://nlp.seas.harvard.edu/annotated-transformer/

    Neural Turing Machines - https://arxiv.org/abs/1410.5401

    A Simple Neural Network Module for Relational Reasoning - https://arxiv.org/abs/1706.01427

    Relational Recurrent Neural Networks - https://arxiv.org/abs/1806.01822

    Neural Message Passing for Quantum Chemistry - https://arxiv.org/abs/1704.01212

    Scaling Laws for Neural Language Models - https://arxiv.org/abs/2001.08361

    GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism - https://arxiv.org/abs/1811.06965

    Keeping Neural Networks Simple by Minimizing the Description Length of the Weights - https://www.cs.toronto.edu/~hinton/absps/colt93.pdf

    A Tutorial Introduction to the Minimum Description Length Principle - https://arxiv.org/abs/math/0406077

    The First Law of Complexodynamics - https://scottaaronson.blog/?p=762

    Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton - https://arxiv.org/abs/1405.6903

    Kolmogorov Complexity - https://onlinelibrary.wiley.com/doi/book/10.1002/047174882X

    Variational Lossy Autoencoder - https://arxiv.org/abs/1611.02731

    Machine Super Intelligence - https://www.vetta.org/documents/Machine_Super_Intelligence.p...

    by cute_boi
  • this is a goldmine, worth bookmarking.
    by glerk
  • Notebooklm would be a perfect home for these
    by algoth1
  • Upvoted. Did you compile that list just now, pulled it from bookmarks, or other source?
    by RetroTechie
  • Possibly the original X tweet that popularized this list? 2024, 876k views

    https://x.com/keshavchan/status/1787861946173186062

    In my opinion, whether it was actually by Ilya or not is not worthy of debate. Many of them are widely recognized for being good pedagogical resources (e.g. annotated transformer, unreasonable effectiveness of RNNs, understanding LSTM networks), and others are landmark papers which anyone interested in the field would benefit from reading:

    - Krizhevsky et al. (2012) introduced AlexNet

    - Bahdanau et al. (2014) introduced attention

    - He et al. (2015) introduced ResNet

    - Vaswani et al. (2017) introduced the Transformer

    Other papers are more specialized. Of them, I think Kaplan et al. (2020) by OpenAI is probably most important.

    by amemi
  • Even if Ilya didn't really create this list I have a very good opinion about every paper on this page that I've read (most of them) so I think it's a great resource. Lately during my off time I want to do something related to AI research (which I am already doing full time atm so I need something light) and I am for sure going to read through this.
    by gekoxyz
  • Nice presentation of the list!

    I'd recommend watching a few of his talks/podcasts before during reading these to get the overview and how all the bits in these works tie together.

    https://www.dwarkesh.com/p/ilya-sutskever

    https://simons.berkeley.edu/talks/ilya-sutskever-openai-2023...

    https://www.dwarkesh.com/p/ilya-sutskever-2

    by imenani
  • I was confused for a minute, I thought this was "top 30 papers by Ilya" and was then wondering why "Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton" is on the list.

    > In additition, even though I have read the vast majority of the papers featured on the website, I have not read through each of the website's versions end to end.

    Website's versions, as in - the actual text or the "explanations"? Either way this is a big red flag.

    by quibono
  • always a fun day when someone rediscovers these. Lucky 10000

    in case folks are interested, i wrote up a ~layman's review of each paper over the course of two years a while back. Several of those reviews ended up doing reasonably well on hn. Full analysis of the ~23 docs that were papers and not massive books

    https://12gramsofcarbon.com/p/ilyas-30-papers-to-carmack-tab...

    by theahura
  • Noting the theory papers on Kolmorogov complexity. For those not familiar, Ilya argues that the reason why neural networks generalize -- why they work at all -- is because they are effectively finding a simple description of their training data, converging down onto the limit of the Kolmorogov complexity. [1]

    [1] https://www.youtube.com/watch?v=AKMuA_TVz3A

    by jawarner
  • That's true of all statistical models, it's not some magic property of neural networks.
    by niksmather
  • After seeing this for the first time, I've build PdfToMp3 to listen to these papers. It has now evolved into ListenDock. Fun fact: PdfToMp3 existed before NotebookLM and I already had "overviews", but I called them teacher explanations.

    Here is an example of a "Teacher Explanation" of the paper "Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton"

    https://listendock.com/e/quantifying_the_rise_and_fall_of_co...

    by janpmz
  • Why do I get downvoted whenever I post something here? Do you think its too spammy? Because its AI? Do I have a downvote bot following me?
    by janpmz
  • For beginners I'd recommend the Welch Labs Illustrated Guide To AI if your not well versed in reading papers. Its a beautiful book that I've enjoyed going through. I'd recommend going through these papers after reading that to get a deep understanding.
    by lwarfield
  • Bought because of this comment. Thanks!
    by mauz
  • I wish this were organized according to suggested/logical reading order. For example, the paper introducing the attention mechanism probably ought to precede "attention is all you need".
    by clintonc
  • Second this! And if the papers are in "logical reading order", it would be very useful if this is stated on top!
    by eirikbakke
  • Author here. First year CS student at Trinity College Dublin. I Built this because when I was getting into reading research papers I ended up burning a ton of my Claude usage asking questions other people have probably already asked. The website is just a side project and definitely a WIP. Happy to answer questions or take PRs on GitHub.
    by notmcrowley
  • What is the easily accessible part? Just that they are on one website? At first I thought there would be more explanation or other addons.
    by fl0id
  • Thanks for sharing this. It appears your README.md's first paragraph is truncated. It ends with "Carmack which reportedly contains..." What were you intending to say next?
    by mjg2
  • Hi ! The "artistic direction" is a bit original, but that's your thing, and you're free to present it anyway you want, of course !

    I think the biggest problem people / I had when reading the list was the "based on a rumoured list of papers that Ilya Sutskever gave to John Carmack."

    Where did you get the "rumoured list" from ? Why should a reader trust the rumours ? That seems to be a pretty big appeal to authority, and it's okay if it's only "word of mouth" (as most papers seems legit), but it's really weird not to give a source, or a backstory, or references, etc...

    Especially since you claim to only have 27 ;)

    by phtrivier
  • I think it'd be interesting to hear what you think the goal of the site is.

    Is it just rehosting the list, plus a reformatted copy of the papers? I was hoping you'd have at least annotated them with what you'd learned?

    by groby_b
  • An option to disable animation and show the paper links in a simple list would be helpful.
    by gowld
  • Hey guys, I really appreciate all of the attention this post has received. I honestly thought it was going to be just a small project to help some of my friends get into reading research papers.

    A large number of people complained about how intense some of the backgrounds/animations were (I might have been a bit too focused on making something that looked cool over usability). In response I have added toggles for both the movement on the page and the backgrounds for the papers.

    Other people mentioned that they would have liked some more personalised reflections on each paper. I currently have already done some of these for the more popular papers on my X @notmcrowley . I would have no problem adding these to the site if people think it will help. I feel the need to warn that I have not been formally educated on ML or AI so any interpretation will just be mine and may not necessarily be the correct one. (If anyone with more experience would like to contribute to this feel free to reach out).

    by notmcrowley
  • Even with the motion/background button toggles you are still left with tiny fonts that make it hard to read.

    It actually made me check my browser wasn't set to zoom out. But then using zoom changes nothing, which breaks accessibility.

    Also why does the header need to take up 3/4 of the screen?

    [edit] Clicking on a paper doesn't even bring the paper up. I have to hit another click to get to it.

    by EagnaIonat
  • Please add them on the site for those of us who have never had Twitter and don’t plan to open one ever. Thanks for this compilation, I am — like your friends — trying to get into reading research papers and this is right up my alley right now.
    by SirHackalot