Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Surely the roots, if we skip over the early preceptron work', are in backpropagation and Hinton, and the work going on at Edinburgh and elsewhere in the 80s.

    Indeed I remember buying a set of three conference-papers-as-books around that time, titled Artificial Neural Networks .. proceedings of the whatever the conference was.

    No doubt Schmidhuber made important contributions, but I see him pop up claiming to be the 'root' of it all every couple of years.

    by jacknews
  • https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber
    by emmelaich
  • This article, too, was originally discovered by Jürgen Schmidhuber in 1991!
    by trashburger
  • worth separating: LSTM (Hochreiter & Schmidhuber 1997) is ironclad and widely cited. the transformer attention priority claims are far shakier. conflating them is how Schmidhuber undermines himself
    by cold_harbor
  • There's this crowd on HN which is very vocal against academia. From what I've seen, the main points are that academia isn't efficient, most of the science coming out of academia is useless and that the whole system is just a waste of taxpayers money. Instead, what is often argued, all good research is done in private labs. Then pointing to SpaceX, Moderna, OpenAI, Google, etc.

    And while it is very true that often the research coming out of Academia is useless, what is always neglected are the roots of the research done in private labs.

    When Jürgen Schmidhuber and team published their work on Neural Nets back in 1991 it was also useless. Unless you had a supercomputer and very, very deep pockets you were not going to do anything with what came out of their lab.

    But still, 30 years later here we are, standing on top of the shoulders of this useless research.

    by jcattle
  • TU Munich and Nipkow, Makarius et.al. are also at the center of the influential Isabelle theorem prover. TU Munich is cool :-)
    by practal
  • Also see Schmidhuber's take on the Hinton + Hopfield Nobel prize: https://people.idsia.ch/~juergen/physics-nobel-2024-plagiari...
    by MeteorMarc
  • The current AI boom has more to do with NVIDIA, and the popularity of computer gaming giving us GPU compute, than who was using neural networks back in 1990's.

    More specifically, it was really AlexNet, the 2012 ImageNet entry, running on two NVIDIA GTX 580's, that highlighted the practicality and utility of running large scale neural nets on affordable hardware. CUDA had been released in 2006, but cuDNN (the CUDA library for neural nets) didn't come out until 2014 - after AlexNet had already kickstarted the demand.

    What followed from AlexNet was a few years of intense competition on the ImageNet benchmark, and larger and larger/deeper neural nets (CNNs), which gave rise to a lot of the algorithms and concepts still used today such as residual connections (originally from ResNet), ADAM (training algorithm), ReLU/etc, normalization, dropout, etc... all the fundamentals that made building large neural nets possible.

    Schmidhuber's continual reminding everyone that he was working on neural nets back in the 1990s is beyond tiresome. Yes, he should have been recognized alongside Hinton/Bengio/LeCun as one of the pioneers, but time for him to get over it.

    by HarHarVeryFunny

Explore Birbla archives