Comments
Hacker News
In an attempt to uncover biases and default choices AI makes, I asked 100 AI models, 100 simple questions, 3 times each.
I am not sure what this tells, but I found it interesting that most of the models were very much in agreement. With some outliers like Mistral and Llama on some questions.
I even made a benchmark, ConsensusBench, to measure how aligned they were: https://www.modelbias.ai/consensus-bench
by bnfcl
Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- In an attempt to uncover biases and default choices AI makes, I asked 100 AI models, 100 simple questions, 3 times each.
I am not sure what this tells, but I found it interesting that most of the models were very much in agreement. With some outliers like Mistral and Llama on some questions.
I even made a benchmark, ConsensusBench, to measure how aligned they were: https://www.modelbias.ai/consensus-bench
by bnfcl