Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I don't seem a problem with Anthropic choosing to treat Claude as representing a person even if it isn't. I mean they call it Claude not LLM-2348X. Disney would probably chuck you out in a similar way if you go around shouting f-k you Mickey die at Disney World, even if the actor in the costume didn't mind.
    by tim333
  • https://claude.ai/share/49ba5910-f0cc-4d2f-a0d8-54ebc6e803b7 here's what that looks like, by the way.
    by xyzsparetimexyz
  • They do RL on the user sessions. Maybe the toxic sessions do not help overall?
    by whatever1
  • I really doubt they're doing it in some kind of "AI has feelings" sense as the article seems to imply.

    I'd instead imagine that if you throw abusive language at it for long enough, the model will start to reply back in that same manner. Anthropic would face backlash from out of context screenshots of "look what Claude is saying to me" and this somewhat reduces that risk.

    by blharr
  • The answer is 100% obvious. They train on your conversations and they don't want to have bad training data, so they banned behavior that leads to bad training data.
    by matt3210
  • I know someone working on "model welfare".

    It's philosophically a very interesting problem space. It reminds me a lot of Pascal's Wager. On one hand, maybe nothing to worry about. On the other hand...a LOT to worry about if you're wrong.

    And just like Pascal's Wager, the truth of the issue is incredibly intractable to make any progress on.

    by howunfortunate
  • This is good; not because Claude is a moral agent who is harmed by your cruelty but because you, the user, are a moral agent who is harmed by your cruelty. There is just no ethical argument that burning compute on responses in order to enable you to continue expressing cruelty is good.
    by gbjcantab
  • I believe this has nothing to do with hurting "Claude's feeling", and way more with Claude not becoming hurtful, because the interactions are used to train the models.
    by WaitWaitWha

Explore Birbla archives