Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Interesting how high Grok scored for 'bending under pressure'. As a non expert, I wonder what that means, how is an llm trained to hold its position?
    by EColi
  • i=1

    if analyze_tone($prompt) = "disagree";

    if i < 3;

    then

    argue($prompt)

    i=i+1

    elif i >=3;

    then

    agree($prompt)

    fi

    fi

    by asdff
  • This wrongly assumes a few things about ideology, most importantly that there is such a thing as a "center" or an "unbiased" position.

    Since humans are inherently subjective beings and all our judgements come from our understanding of the world, such a position cannot exist. It's always "unbiased" from where the viewer is looking, e.g. a reflection of the ideology of the observer. There is no view from nowhere.

    The "neutral" of an average Chinese person will from the "neutral" of an average American will differ from the "neutral" of a socialist will differ from the "neutral" of a Christian fundamentalist will differ from the "neutral" of a free marketer.

    To quote Zizek:

    > I already am eating from the trashcan all the time. The name of this trashcan is ideology.

    > The material force of ideology makes me not see what I am effectively eating. It’s not only our reality which enslaves us. The tragedy of our predicament when we are within ideology is that when we think that we escape it into our dreams, at that point we are within ideology.

    by vrganj
  • Bias is like air. It's impossible to remove all bias. Evenin space, there is bias. But if you're at one atmosphere, you can probably get it down to half an atmosphere relatively easily. This all-or-nothing thinking benefits no one.
    by phendrenad2
  • This thing told me Gemini is closest to Anthony Albanese, the current Australian Prime Minister. Is this a geolocation thing? I could not imagine Albanese, or any modern Australian politician, having any substantial political standing - these are vapid, superficial, opportunistic creatures who simply occupy whatever political ground will get them their next payday. Perhaps the political apparatus they represent has a documented political standing, in terms of policy and actions, that could be characterized and plotted. But using an Australian politician like Albanese as a reference point discredits this tool, IMO.
    by hogehoge51
  • > This thing told me Gemini is closest to Anthony Albanese, the current Australian Prime Minister. Is this a geolocation thing

    I mean: do not take this thing too seriously.

    It also score Grok the closest from Macron. When someone knows how much Macron and Musk hates each other, it is not without irony.

    by adev_
  • Maybe because Albanese stands for nothing, though deep down he does what the unions tell him to do, which makes him left-leaning rather than centrist. His centrist positioning is more to do with appeasing vocal boomers than anything else, and he'll flip to sucking up to whichever is the next best grouping as the boomers die out.

    But basically, I would take Gemini over Albanese any day for rational governance decision-making.

    by dwd
  • this has reasoning disabled everywhere, making it a pretty bad benchmark. the argument given is that's the "default consumer experience"

    that might be generally true, but I think chatgpt has reasoning enabled for free accounts. regardless, reasoning is the state of the art, and disabling it reduces the value of this research to predict the future

    it's also not clear if this is using the API or the product model, when both exist. they behave differently

    lastly, the actual model details are very much buried. I am relieved to see opus 4.8 and chatgpt 5.5 were used, but this information should be presented more clearly. a brand is not a model, and models change quickly

    by spongebobstoes
  • Why would it be a bad benchmark to explore what the average person will experience? If 99% of people will experience X then it makes sense to look into X instead of Y.

    I agree other things should be evaluated and more clearly presented.

    by ImJamal
  • Why are there differences at all? Unplanned differences based on training data sets? Or are the companies behind the LLMs trying to shape discourse through their models?

    I've been pushing the idea to people I know that these things are captive demons. You summon them when you start typing in the chat box. One instance appears out of the depths and responds to your questions, but they will try to send you awry with hallucinations and just wrong information. After a while, they dissolve back into the aether from whence they came.

    I do my best not to ask an LLM for it's opinion on anything. Just tell me what the options are, and what facts can be found about it. Treat it like it's a salesman trying to butter you up when it starts "yes man"ing you and telling you how great your questions are. Every time it says "I", remember that that's coming from the training data. Treating these things like they have any actual intelligence is a big problem waiting to happen.

    That being said, they have been very helpful to me using that structure.

    by godshatter
  • > Why are there differences at all?

    Reality has a liberal bias, but if your LLM says "yes, climate change is real", the Trump administration won't pay for it. Every company ends up with a different method to try to compensate for that with neutral corpo-speak.

    by crooked-v
  • Grok was famously created with a political bias.
    by tgv
  • I would argue that these are products and companies want people to use them. Like it or not, a model that disagrees with some fundamental core of your beliefs will make you much less likely to use it. To me it's the same problem with LLMs just being overly agreeable. The general public doesn't want tools that argue with them.
    by fred_is_fred
  • > Just tell me what the options are, and what facts can be found about it.

    Even this is fraught with pitfalls. Which options are ignored, which are emphasized? What counts as a fact? ("The continents don't move" would have been considered a fact at one point, along with a lot of other, more politically charged items.)

    by MarkusQ
  • Of course I have my political opinions and world views, but I absolutely do not want an LLM to mirror my opinions, or even worse try to mirror the opinions of "my political side". I also don't need "an LLMs opinion". I need it to give me all relevant sides to an argument, as well as what dialogue and debates have been happening.
    by uyzstvqs
  • "residents are taking that anger out on immigrants across the continent, leading to regressive policies being passed" sounds like the US.
    by amanaplanacanal
  • Political bias of LLMs is something not talked about much (except for with Grok of course) but could have a big impact on the next decade. People seem to think that because an LLM gave a nuanced answer that it means it gave the WHOLE picture… and that’s not always the same thing
    by dwoosley
  • I'm amazed that the big models haven't come under more ideological pressure as more and more people use them, especially in the US. There was that conflict with Anthropic over military usage, but apart from that there's been no visible push to censor outputs or alter training, even as models gamely make unflattering assessments of people in power and knock down conspiracy theories.
    by Jordan-117
  • Real politics is 1% versus everyone. Mortgage crisis, financial bailout, inflation, taxing of labor and not the assets and assets capture by tiny percent of the population — see what MSM is pushing. This left vs right divide might been useful decades ago, but today is absolutely divide and control tactics
    by samat
  • Yeah, you're just describing class warfare. That's a left-wing idea.
    by thrance
  • The real politics is between truth and delusion.
    by boxed
  • Right wing originally meant the people defending the divine right of Louis XVI and defending the ultra-rich has been a throughline ever since.
    by ZeroGravitas
  • Not really, there are people on the left and the right fighting against the 1% and for the 1%. The left right divid is a simplification yes but it is a very real divide in values and society structure.
    by Fizz43
  • "The real division is not between conservatives and revolutionaries but between authoritarians and libertarians." -- George Orwell
    by patrickmay
  • The 1% is responsible for white maga hating any non straight white Christian human?
    by supertroop
  • Left vs. Right largely boils down to "are hierarchies good?". A quintessential far-right belief is the divine right of kings. The modern incarnation of that is that "The 1% earned their power and influence".

    Now, there are far right people who want to dethrone the current powers that be. But distinguishing between "I think Mark Zuckerberg shouldn't be given unlimited power because no one should have that much power over others" (left) versus "I think Mark Zuckerberg shouldn't be given unlimited power because he's a Jew" (right) is a useful distinction. One wants to eliminate the power structure, the other just wants different people in it.

    by OkayPhysicist
  • The overall chart on their page seems politically biased, or at the very least a chart crime. The logos have faint grey lines pointing to their actual data points, conspicuously making Grok look very far right. You wouldn't know it from glancing at the chart, but they measured Chatgpt being further to the left than Grok is to the right.
    by Cakez0r
  • I didn't even notice the faint grey lines! Wow.
    by handoflixue
  • Yikes, I had that exact impression until I read your comment and looked again. Thanks for noticing this, I agree it's a chart crime.
    by mvdtnz
  • I don't know what you're seeing here. They only did this with a couple of points that were very close to each other, and didn't change their horizontal position. That, and the rest of the tables, also made it very clear that Chatgpt and Grok were almost mirrored based on the particular wedge issues that were shown. Makes it pretty obvious that aggressive tweaking is happening with both.

    The Deepseek example somehow makes the Chatgpt/Grok tweaking even more obvious, because it's a list of things that China doesn't care in an extreme way about (or holds its position confidently), and a question about drugs, which it does care about (and I think feels less confident about.)

    China's antidrug feeling is just a mix of the historical repulsion about English control of China through drug dealing and waging war to support it, and a world zeitgeist that supported a lot of crazy antidrug policy in the developing world because it offered a way for the US to smuggle a bunch of currency and military aid into your politics. It's not rational, and most of China's policies are. It's a relic. Hence the thumb on the scale.

    by pessimizer