AI researchers debate how close we are to recursive self-improvement

Join the discussion

• Hacker News
• After using frontier models it’s hard to understand why anyone would think this is the path to AGI. Self improving models will likely have limited ability and returns. There may be breakthroughs that enable more general self improvement but the current state of frontier models isn’t that.
  by stevebmark
• Thing is, it depends on whether llms + reinforcement can self-improve in principle. Learned recently that cognitive scientists, before the transformer & llms, were studying the possibility that thinking and learning might be based on some kind of prediction, i.e. something similar to token prediction, and I quite suddenly became less skeptical about the possibilities of llms. (Some will say I’m late to the party of course.) But if knowledge to date has been accumulated in a process quite like “chain of thought” in llms, then I don’t see any reason that computers won’t self-improve in the near future.
  by wwarner
• People seem to expect a sudden shift with "self-improvement", but don't AIs already improve themselves via training? What is there to improve?
  by Buttons840
• > AIs already improve themselves via training

  Marginally. Model collapse is still a problem. Continuous learning is still a problem.

  For AI to make a big leap we need a big break through.

  by zer00eyz
• I think the idea is fundamentally improved architectures. For example, transformer-based models were an incredible stepwise improvement. Self improvement would be a model discovering a stepwise improvement similar to the transformer. And presumably the improved models from that would be more likely to make further advances still.

  Learning from training data is technically self-improvement but not the sort that is typically meant in this context.

  by appplication
• Only if you think in terms of perceived raw intelligence, but self-update is a form of valuable self-improvement that could benefit current models a lot, if they could commit facts from context into their weights cheaply and reliably.
  by gchamonlive
• The human brain operates at better levels of intelligence than the best LLMs, at 20 watts of power. We are a long way to that kind of efficiency, it will probably take both bespoke hardware and algorithmic improvements to catch up to nature.
  by pennomi
• We are already there. Current AI is definitely capable of collecting new data and start training on that data to get a better model.
  by charcircuit
• What do these discussions even matter when the words don't matter?

  How many times has AGI been declared already? A bunch of people (e.g. Jensen Huang) have called Astra AGI, for example.

  The goal posts get moved, everybody's hustling, lying, inventing new buzzwords, doing mental gymnastics, and it's all just tiring.

  Just show me the results, and let the proof be in the pudding.

  by preommr
• I certainly do see one side of the debate moving goal posts…
  by talon8635
• Recursive self-improvement is not necessary for AGI. Humans are general intelligences even though we cannot RSI.
  by Sharlin
• Okay, AI now can write a poem or compose a song.

  But can an AI construct Terminal Bench 5.0 or GDPval 2027 ?

  Without this, there is just no path towards RSI.

  by eugene3306
• Are you a visitor from the past?
  by danjc
• Recursive self-improvement will ultimately be a problem of money. It's a very big bill and AI companies need to start thinking in terms of how they will accumulate cheap and free energy, because merely paying for compute will no longer be enough. Money is the bottleneck. The future requires companies that are post-money.
  by deadbabe
• why can't recursive self improvement not improve efficiency ? token per watt ?
  by iamgopal
• Taxation will solve that. Any company that demonstrates even a glimmer of real AGI will get government money poured into it. They might even disclose it.
  by onion2k
• def self_improve_more():

       if self_improved():
  
            self_improve_more()
  
       else:
  
            see_tony_robins_and_buy_tapes()

  by strangattractor
• The alternative is to switch to far more efficient models and have aggressive optimization of efficiency as part of the process of improvement.

  There are at least two resources here that have a Pareto optimal front: time and energy. And effort spent to change the shape of that may pay off more than efforts spent purely on improving intelligence and agency.

  by epistasis
• Money can be exchanged for goods and services. I think the real constraint is actually physics, nominally energy, which is what a big chunk of the operational cost comes down to. I'm already basically assuming the next 20 years of fab time is set aside to feed the beast, so the capital cost is "give me all the processing and memory you have"
  by jaggederest
• Obviously the doomers will tell you that the future looks like the Matrix, because most of what they predict is based on extrapolation from sci fi movies.
  by timr
• If a self improving model can't produce enough value to pay the electric bill then who cares? Unplug it.
  by tengbretson
• AI should be optimized to make all humans healthy, happy and safe without exception.
  by anoplus
• > AI should be optimized to make all humans healthy, happy and safe without exception.

  No. AI should be optimized for the one true good: maximum market returns. Personally I think that will only be achieved once humans are removed from the market system and decommissioned.

  by palmotea
• It's only optimized to make prior knowledge useful in new situations.

  > this logic gate should not hack other systems

  well .. it only applies a rule to inputs

  can we impose this of authors "this book should only be causing good things when it is read and knowledge used"

  by visarga
• Keeping billions of humans healthy, happy, and safe is complicated. Best to begin by reducing the scale of the problem. First, automate the economy and change the information ecosystem so that humans spend more and more time happily isolated and stop breeding. After reducing the human population to a few thousand, the problem can be made more manageable by cordoning the remaining humans into enclosures where they may be kept safe.
  by Centigonal
• I don’t disagree, but unfortunately a not insignificant part of humanity seems to think that their happiness primarily depends on knowing that other people are suffering harder than they are. So that’s perhaps not the right thing to optimise for.
  by FranzFerdiNaN
• Paperclip all humans into bottles filled with nutritious fluid and drug them all the time or let them dream about a better time, I heard 1999 is a good year they could dream about. Healthy happy safe - without exception.
  by xaedes
• Does anyone know the status of John Carmack's AGI work?
  by carabiner
• We would have heard if there was any result?
  by realaleris149
• I’ve had this question on my mind at least once a month for what feels like years at this point.

  I don’t feel like monitoring Twitter to see how it’s going though.

  by cheschire