Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Surprised she left as probably had stock options.
But she was Meta before so can’t be a question of her principles, so indicates a personal reason
by fsuts - Every OpenAI news cycle the board members who were unceremoniously dumped in the wake of trying to fire Sam prior to the transition to a “for profit” company seem to be proven right.by mikeryan
- COO just resigned too. Coincidence?by bashtoni
- So many responses saying variations of nice "run". Is that common corporate speak nowadays? It reminds me of "well deserved".by dieselgate
- I don't know. Every big tech company I worked at has a lot of turnover. Then every Blind post says "its so over" or "everything is going to change" But that doesn't happen.
It's hard to read anything into it.
by softwaredoug - > In recent times, Johannes Heidecke, Head of OpenAI's Safety Systems team, also left the company – as did OpenAI’s Chief Futurist Joshua Achiam.
It's OK though, they don't need an ethics team because everyone at OpenAI is ethical:
> “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."
by fwipsy - In this article: no explanation of why she left.by jondwillis
- > The kinds of questions that AI ethics brings up, are the same kinds of questions that people have been asking for centuries - Chloé Bakalar, former AI Ethics Lead at OpenAI
This might actually be the reason they pushed her out. OpenAI and Anthropic base their whole business plan and philosophy on the idea that LLMs are a unique technology to the point they can cause infinite harm or benefit to humanity depending on who controls them, so the only rational choice is to invest all your resources in getting to ASI first so you can tell it to stop any other attempts. Linking AI to old questions defeats that idea because it exposes AI as not so unique.
The other more likely option is she was asking uncomfortable questions, either about the social impact of building AI controlled by a for-profit entity or about the possibility that AI systems are conscious.
by tancop - I'm a broken record on this, but I'd like to reiterate:
1. Academic ethics is the theoretical study of what you can get away with. (Law is the applied branch). Professors in ethics are not experts at being good, or knowing what is good, they're experts in how to get away with things. If you want an expert in being good, you need a saint, not an academic studying ethics. Saints are in short supply.
2. That said, ethics academics can still be decent. They don't have to be sanctimonious corporate shills . However, an "ethics expert" who hasn't threatened to quit and gone through with it at least once, should be assumed to be one.
I respect those who quit - or, at least, I respect them more. I wonder if it's more a reflection of the people, or of the state of the industry, that the list of (AI) ethics experts who have quit in protest is getting long.
by vintermann - People confuse morals with ethics. It can be perfectly acceptable, ethically, to do something that is morally reprehensible.by iamflimflam1
- Thank you for this, there is a a lot of speculation in this thread [Not surprisingly] but very little on the definitions, objective goals. Maybe I'm the one that's uneducated in this specific field, but I don't see anyone talking about what this actually means other than "be good".by mlrtime
- IME companies hire an ethics team to say they have an ethics team. The ethics team has no sway, no influence, and will never be able to move the business. They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.by forestrywat
- > They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.
> “It is difficult to get a man to understand something, when his salary depends upon his not understanding it!” - Upton Sinclair
Think I first saw that on a usenet post, as true now as then and just as true as it was in the 1930's when he wrote it (in at least that form).
by noir_lord - Maybe in this case all they needed an ethics team for was to teach the LLM about ethics?by amelius
- Scapegoat team (when working for an obviously unethical company).by temp0826
- If you ever feel like your job is useless, remember that raytheon has an ethics team.by RobotToaster
- OpenAI was probably able to at least pretend that her role mattered... until they let their AI autonomously hack a rival company, then used the incident to score marketing points. I'm not surprised she left.by wavemode
- The article has no detail that might serve to explain her reasoning for leaving, but does note that she left after the HuggingFace hacking incident. The implication could be that model alignment is not being taken seriously, sure. But it could just as likely be that there was collusion between HuggingFace and OpenAI and that the incident was orchestrated as a publicity stunt.
I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.
by bogzz - Cynic in me says basically part of her implicit job description was taking fake falls when shit happens . Kind of like a CEO is paid for more than the work they do, but also the responsibility they must appear to take publicly (for big money).by chermi
- It's unlikely HugginFace colluded given they specifically cited they used open models to save them from the hacking.by WiSaGaN
- The rats are fleeing because the captain doesn't care if the ethical ship sinks... Is what my take away would be if I didn't know, that the ship sank a long time ago and we are just pretending anyone really cares.
Let's be honest unless AI gains literal limbs, and the records a video of itself presenting a manifesto about world domination and then kills actual humans with its cold mechanical hands.
We won't care about AI safety & ethics enough, atleast not enough to do anything about it.
People thought how could skynet happen, it had to be something no one saw coming and bam... But honestly if I started a new AI lab named it Skynet Corp and hired some AI researcher friends ex-Google, I think I could raise a few hundred million dollars.
by minraws - We Reddit & Facebook nowby Razengan
- What makes you think this is rats fleeing a sinking ship?
She probably just got a better offer somewhere else
- > But honestly if I started a new AI lab named it Skynet Corp
you're too late
by walrus01 - The ethics of the people controlling the AI is always going to matter more than the ethics of the AI itself.by slg
- In five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire.
My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is going to be aligned to the ethical stances the company chooses. If they can't, the waste to time and money is huge if the model requires retraining for ethics reasons. That's a different job than pondering roko's basilisk or whether AI is alive.
The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.
by madrox - > The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.
Your disdain is showing.
That you jump immediately to criticizing AI ethicists as counting angels on pins and implying that's why they left OpenAI rather than OpenAI not being a place where AI ethics is a priority is, well, let's say generous to these AI labs.
- This reminds me of the marketers who hated programmers and coding in general in the early 2000s era, and fundamentally because the developers were seen as more important in a company. So, you know what they did? They didn't learn programming, they still hated it, instead, they renamed their titles to "hackers". Growth "hacking" would go on to become a term and had nothing to do with hacking - ethical or otherwise. It was a glorified marketing term for marketers and by marketers to justify their exorbitant salaries but also wanted the aura of a real hacker without working for it. I would imagine someone with that title or mindset joining an ethics role in an AI organization is going to find out quickly it isn't all about fluff pieces and grandiose statements at all (we are here to change the world!)by neya
- > I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is going to be aligned to the ethical stances the company chooses.
Which like any company will be entirely driven by legal constraints and money. Or just money if it's cheaper to break the law for profit and pay fines. There will be no "this is what's good for humanity, economics be damned".
Much like HR isn't to help employees but just the company.
by afro88 - Read “Careless People” or whatever that book by that Facebook employee that pretended not to see how awful all her colleagues were until her priorities shifted is called.by apercu