Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- These models should be aligning themselves to the customer, not coming up with their own motivations.
It's ironic that the alignment folks are actually training Claude to have it's own idea of good/bad and not even fully trust Anthropic.
What ever happened to computers doing what they were told?
by nonethewiser - > Over several months, I interviewed 20 religious and philosophical thinkers — Catholic, Jewish, Sikh, evangelical, Ubuntu and others — who have been involved in Anthropic’s efforts, and Mr. Olah himself.
Not enough representation of World religions and philosophies.
by kshri24 - I'm an atheist, and have been for over 40 years. I believe that these Religious Scholars are scholars of a thing that is not a thing. I wonder what their view of my moral status is, and whether it differs significantly from their pronouncements about the status of ai models?by andyjohnson0
- As our human labor loses economic value (and for realizing some of that shift the AI companies must brainwash people into believing that their AIs are human-like) people will reach for other sources of self-worth. Religious philosophy, and being made “in the image of God” (imago Dei) is a prominent one. What I find more than a little disquieting is that the AI companies seem to have grasped the relevance of this future playing field ahead of the rest of us. I hope these troublesome thoughts are a figment of some feverish dream of mine[^1] and nothing else.by dsign
- I've been tracking "emotional vectors" in my software for some time now. My Apache web server gets sad (5xx errors), frustrated (4xx), happy (2xx), thoughtful (1xx). I can even see it express these emotions through some of its messages.
- > It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
If that is true, then surely Anthropic is one of the largest slaveholders of history, right?
- When thinking about ‘AI’ I sometimes find it useful to replace the terms ‘model’ or ‘AI’ or whatever with a description of what we’re really talking about here: a whole bunch of floating point numbers.
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
by thadt - Every bubble goes through this phase just before it pops. Pre-implosion Twitter had tons of folks doing useless “research” projects, WeWork had folks studying furniture design’s impact on wellbeing, and so on.
When you’re more worried about this sort of stuff vs generating profit then the history books strongly indicate that’s usually not a good omen of what’s coming next.
by cmiles8