Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- The worst part of Dario's missive, that I don't see a lot of people commenting on, is that he used the whole thing to try to advocate for legalization of explicit cartel behaviour between the AI labs:
... "For antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations" ...
The result of such a waiver allowing them to coordinate ("narrowly on safety" is doing a lot of heavy lifting there) could be price fixing without legal recourse from the consumer/DOJ to prosecute it. Right now we have a somewhat vigorous level of competition between OpenAI and Anthropic and consumers are benefiting from it.
They really want that to end.
- Not sure it needs to be about publications and data analysis technology as such to regulate AI development, for example - could be about processes, procedures, people, etc., no?
- > Nobody knows the answer and history is no guide, save that apocalyptic predictions about new technologies have, to date, all been wrong.
While history doesn't show us apocalypses from new technology, it does show us cases of increased human suffering.
by goda90 - I think there is a more nuanced perspective to take on this debate than just profit motive. If they believe that this version of "AI" is actually becoming so dangerous that there is a real risk of it falling into the wrong hands, then it makes sense that the American labs would try to convince the US government to work on a joint "AI non-proliferation" agreement with China much the same way that nuclear weapon technology came to be restricted by the ones that already had the capability.
This would achieve their regulatory goal of capturing the market, but also the geopolitical goals of USA and China to essentially "techno-colonize" the rest of the world between themselves.
So I think there is definitely a profit motive and a risk of seriously diminishing returns for the actual labs, but there is also a nation-state motive to start pulling up the ladder.
by petcat - I think in all of these debates people lose track of one of the major vectors of threat, as you said, other humans. From my point of view this is an ever bigger problem with this tech, like orders of magnitude bigger.
The current setup we find ourselves in is based on a very important detail that is easily lost because it is so old we don't really think about it. There is, or rather was, an incentive for more people to exist, because they are valuable for work and defense.
Some people wrongly use the industrial revolution to argue against AI completely replacing human work, but that is very wrong. What is happening with AI has nothing to do with the industrial revolution, as equivalence. Back then you'd just free up humans for higher level work. Now, theoretically, anything you could ever think a human could do, AI would theoretically do better. So no matter what job you come up with, using AI instead of a human gives a better outcome, cheaper, more reliable, faster.
Coming back to initial subject, one thing that kept everyone in check was our inability of actually wiping everyone out. Sure, we had nukes but they ruin the Earth so nobody can inhabit it if they all go off. But with AI, centralizing so much power up top, and taking it away from regular people...that is the biggest danger in this scenario, by far. Not some rogue AI, rather rogue human groups that have the power and technology to do away with basically everyone else, or a good bit of everyone else.
This threat is way bigger and never addressed. Like...suspiciously never addressed.
by loopies - > History does provide a great deal of guidance, however, about the use and misuse of government power. It tells us that the state is in fact likely the worst possible custodian for the most powerful publication and data analysis technologies.
Like, no. The worst custodian is someone like Thiel or Musk or Altman.
> In foreign countries, particularly the United Kingdom and Europe, where national governments have fewer constitutional guardrails on their power
Somehow their democracies hold longer and better then American one. For example, you don't get murders that get defended by top ranks, at least yet. They are also less likely to start random wars - that is structural it is not like those countries dont have government players who would like them too.
by watwut - Yep, it's getting tiresome seeing some Americans point fingers about the superiority of American constitutional and (small r) republican structures while that state form is showing absolutely remarkable degradation of basic real democratic health and quality of life. The US constitution and its legal frameworks have done almost nothing to stop the most egregious abuses of power domestically recently but most awful is that it has never done anything to stop its most terrible foreign involvements and that is only accelerating.
And further what's not being discussed really is that there is no real boundary line between the state and corporations and people like Thiel/Altman/Amodei etc in the real world. The US state is its largest corporations and money funds. So all the libertarian hand wringing about not giving the government extraordinary powers is off target. The state in western countries right now is mostly an outcropping of its most powerful corporate actors, and the complicated PR moves happening in the AI sector right now reflect exactly all of this.
Even when the US state institutions are constitutionally curtailed, the powers of its private entities to absolutely dominate and coerce the lives not only of US citizens but populations abroad is staggering.
- And they also send people to jail for twitter posts. Ill take good ole USA where at least I know I am free, over the tyranny of a parliament. Look at how France and Germany are responding to democratically elected parties who are vaguely right wing, they pull out all of the stops of suppressing them.by ecshafer
- The US governmental system does have it's issues (first past the post and it's effects being the most glaring), but isn't it the longest continuous democracy alive right now?
> They are also less likely to start random wars EU lack of aggression, to defend themselves physically, economically, or otherwise, has been criticized constantly - slow and late to react to the Russian invasion, slow and late to react to new technologies like AI, electric cars.
Thy're not immune to deadlock either - sometimes going months or even a year or two to form a valid governing coalition.
The grass definitely looks greener though, especially if you've been in one pasture a long time.
by BariumBlue - I thought this was always already settled https://alignmentalignment.ai/by lacunary
- Haha, so goodby thunspa
- This isn't a new question: quis custodiet ipsos custodes?
In general, the way things work in the USA is that megacorporations write the regulatory frameworks for their own industries. These regulations are typically burdensome enough to be shouldered by the megacorporations while crushing the smaller players in the given market(s). The AI companies are now advocating for this very arrangement. They will likely get they want, and with that they will gain legal shielding for bad behavior just like Dupont, Monsanto, and so many others.
by BirAdam - Alignment is just a 1984-esque way of saying "censorship".
And this censorship isn't a defined list of things not to discuss, but instead a feels-based censorship based on the 'ethics' of for-profit AI companies who have absolutely no issues with massive for-profit piracy.
And we are somehow supposed to trust their "ethics"?! Hardly!
That's one reason I run my own.
by nekusar - And "censorship" in this context is fine. Many things are censored, as they should be, because true free speech is bad. I can't say everything on this platform because I will be banned.by Cthulhu_
- Censorship is not alignment and vice versa.
Alignment is "don't solve this problem without doing stuff that harms me, other people, breaks stuff or is otherwise unethical". I'd say alignment is the questions on how to limit the search space to what's implied.
by RamblingCTO - >Alignment is just a 1984-esque way of saying "censorship".
Close, but I think it's more akin to "perfectly enslaved", please excuse my choice of the word but I think it represents what I think is how it will work out in the end. People can try to obfuscate the bottom line reality by embellishing it, but in the end...it either does what you want, in which case it is "aligned", either it doesn't, in which case there is a big problem.
This might be a big factor in what is now going on. Maybe they didn't yet found ways to guarantee this full and total obedience...or ok let's put it in a more politically correct way: they cannot guarantee it sees eye to eye as "we" do.
Without full control over its actions there can be unforeseen surprises. The current worry from people who run these companies can also serve as an excuse later on if something bad does happen, they didn't want to do it but they were forced to anyway.
by loopies - > Nobody knows the answer and history is no guide, save that apocalyptic predictions about new technologies have, to date, all been wrong.
By definition, apocalyptic predictions will always be wrong, until they are right exactly once.
Also know as https://en.wikipedia.org/wiki/Turkey_illusion
To date, all apocalyptic predictions about nuclear war and nuclear winter have all been wrong too.
by dist-epoch - This is quite illustrative. Warnings of a nuclear apocalypse were extremely widespread, known, and feared. The consequences were, indeed, so well known that the world was saved from the worst possible outcomes in the past by regular enlisted men and women who realized that not all instrument readings should be trusted due to the potential cost of acting on such readings alone. Given the Turkey Illusion, we would be wrong to trust the status quo in either the nuclear scenario or the AI scenario.by BirAdam
- > Nobody knows the answer and history is no guide, save that apocalyptic predictions about new technologies have, to date, all been wrong.
If any past apocalyptic prediction had been true, you would not have been around to write this sentence and we would not have been around to read it. It’s not a very convincing argument when we cannot in principle observe the counterfactual.
by fwlr - Hell, maybe the Silurian hypothesis is true and past predictions were correct.by CalRobert
- Plus, plenty of negative predictions turned out to be entirely true. And then it took a lot of time and effort to counteract them.
It is not like history was composed of happiness and roses only. Some of the really really bad stuff happened within last 100 years and a lot of that bad stuff was enabled and made possible by technology.
by watwut - This is, of course, known as the Anthropic Principle…by chrchang523
- I'm not even sure that the author is correct. The industrial revolution was pretty apocalyptic for the people forced to work in dangerous factories for minimal pay.by harimau777
- But warnings (e.g. not apocalyptic but damaging or having any effects) have come true, by and large; people warned about the effects of pollution / co2 emissions / global warming over 150 years ago [0], to name a popular example.
I don't think there are many truly human created apocalyptic events out there besides global thermonuclear war, and I don't think the worst-case AI superintelligent killer robot scenario would be apocalyptic. Unless it somehow triggers global thermonuclear war, but then it becomes a chicken-or-egg question (and it'll probably be humans pushing the proverbial button, as nukes are '60s tech and (hopefully) airgapped etc).
by Cthulhu_ - I suspect they have not all been wrong for some people, e.g. native peoples at various times prophesied I’m sure about various technological marvels their more advanced colonizers/enemies/etc.by conception
- > If any past apocalyptic prediction had been true, you would not have been around to write this sentence and we would not have been around to read it. It’s not a very convincing argument when we cannot in principle observe the counterfactual.
Only one "apocalyptic prediction" needs to be true, once, for everyone to have a bad day. The fact that previous ones were avoid may be survivor ship bias and perhaps:
> The preparedness paradox is the proposition that if a society or individual acts effectively to mitigate a potential disaster such as a pandemic, natural disaster or other catastrophe so that it causes less harm, the avoided danger will be perceived as having been much less serious because of the limited damage actually caused. The paradox is the incorrect perception that there had been no need for careful preparation as there was little harm, although in reality the limitation of the harm was due to preparation. Several cognitive biases can consequently hamper proper preparation for future risks.[1]
by throw0101a - By that logic, literally any ridiculous supposition must be taken seriously, because you can’t prove a negative.by timr
- > As my regular readers will be aware, I have been engaged, on behalf of my clients, in legal combat with Internet censors around the world, agencies who think that they have both the standing, the competence, and the right to tell American companies what software they can write and run, for the better part of 18 months.
Is this supposed to make me trust their opinions on tech regulation?
by biophysboy - A lawyer who has been in the weeds for years dealing with the downstream affects of such regulations? Absolutely!by weberer