Birbla

Discussion summary

Users discuss potential performance issues with GPT-5.5 Codex, especially in reasoning tasks, leading some to consider refunds. The problem may also affect earlier versions like 5.4.

What the discussion says

  • Some users report degraded reasoning performance with GPT-5.5 Codex.
  • Concerns about nondeterminism and overall reliability.
  • At least one user considers requesting a refund.
“I almost never use it for reasoning anymore.”
— ProofHouse
“Now I want a refund after upgrading to Pro.”
— linzhangrun

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I swear some days ago someone here claimed Openai succeeded cutting down their compute cost by half with a breakthrough optimization. So this is it?
    by siva7
  • this explains so much why gpt 5.5 has been so bad lately it was really puzzling why it struggled so much where when it first came out it was one shotting stuff totally amazing, i tried the prompt that will tell you if your plan is degraded:

        codex exec --json --skip-git-repo-check --ephemeral -s read-only --disable memories -m gpt-5.5 -c model_reasoning_effort=high "Do not use external tools. A black bag contains candies with counts: round apple 7, round peach 9, round watermelon 8; star apple 7, star peach 6, star watermelon 4. Shape is distinguishable by touch before drawing; flavor is not. What is the minimum number of candies to draw to guarantee having apple and peach candies of different shapes, i.e. round apple + star peach or round peach + star apple? Give reasoning and final number. The local project dir is irrelevant for this task, do not consult it. "
    1. 516, 24

    2. 516, 27

    3. 516, 12

    4. 516, 21

    5. 516, 21

    This means that the whole time we've been paying for a product that was silently routing to something completely different and inferior from gpt 5.5

    Also I read through the github issues and it seems like they closed a previous issue without addressing it ???!!

    whooo boy somebody from OpenAI is getting fired over this if not a class action lawsuit is almost guaranteed at this point.

    by zuzululu
  • I love that Codex is open source and issues like these can surface/be addressed publicly.
    by laurels-marts
  • For me, the encrypted reasoning contents, when looking at the base64 string lengtht, show this effect. However, the server-reported reasoning tokens don't. So I assumed it was part of the encryption and/or obfuscation purely. So I don't think there is a real issue.

    This is the biggest downside of GPT; thinking is encrypted, so it's more of a black box than kimi/glm/deepseek. You still get thinking summaries though. It's awkward, but workable.

    by edg5000
  • Deja Vu... This looks just like the Claude Code performance regression back in April. I just quit my Claude subscription when that happened and went to Codex.

    Now I'm kinda thinking of trying per token for both, using GLM 5.2 on Fireworks for most tasks, shelling out to the big boys only when needed. Not totally confident I'll break even though.

    by resonious
  • You can use this small Python script to display an histogram of `reasoning_output_tokens` in your past Codex sessions. I do see a spike at 516 indeed.

      import os, glob, re
  import matplotlib.pyplot as plt
  vals = []
  for f in glob.glob(os.path.expanduser(r"~\.codex") + r"\**\*", recursive=True):
      if os.path.isfile(f):
          try:
              s = open(f, "r", encoding="utf-8", errors="ignore").read()
              vals += [int(x) for x in re.findall(r'"reasoning_output_tokens"\s*:\s*(\d+)', s)]
          except Exception:
              pass
  plt.hist(vals, bins=200, range=(0, 5000), weights=[100 / len(vals)] * len(vals))
  plt.xlabel("reasoning_output_tokens")
  plt.ylabel("%")
  plt.show()
    by josephernest
  • I’ve definitely experienced step jumps down in quality on an almost daily basis. I usually used xhigh. The experience of relying on codex’s outstandingly thorough coding earlier in the year has evaporated for me. I’m seeing incredibly stupid implementations intermittently, and have simply switched to Claude until openai takes the issue seriously. As far as i could tell they haven’t taken it seriously for the several months I’ve been personally seeing it.
    by zenapollo
  • Oh this seems bad, and is fairly easy to reproduce using codex cli. You give it a puzzle prompt that it has to reason about and solve, occasionally it will seemingly short circuit and think for exactly 516 tokens, and return the wrong result. When it ends up using 6000-8000 thinking tokens it returns the correct result.

    Maybe some issue with adaptive thinking? Another point for local models I guess, don't have to worry about silent server side changes.

    Edit: To follow up, it seems to happen quite often. Out of 10 runs of the exact same prompt, 4/10 had this 516 thinking token issue, and every one of these had the wrong solution. So nearly half the time, 5.5 xhigh could be short circuiting and degrading performance. Granted the sample size is small.

    by nsingh2

Explore Birbla archives