What I learned by putting GitHub Copilot behind a MitM proxy

Join the discussion

• Hacker News
• We stopped trying to keep the key away from the model and made it cheap instead: per-run, spend-capped, deleted at teardown. Assume it reads the file.
  by tomveber
• When I did this I just used a custom token in visual studio and then looked at the logs in the provider's web UI.

  I don't think they care very much about this. The encrypted blobs for reasoning models is a different matter.

  by bob1029
• Disagree with the conclusion, even without carefully curated context every high end LLM perform just as well, maybe with an extra detour. In contrast if even one of the learnings is not up to date or doesn't apply to the current situation you find yourself with a long detour or even a failure.
  by _davide_
• Interesting to see how big companies make compromises with security for innovation and i feel that it's comprehensible and better that doing nothing. But i guess it also show how we can see governance problems as real opportunity for involved peoples to build good systems with an agent native perspective.
  by mathieu_aithos
• while this approach gives much more insights on the various requests made, if you are mainly interested in the underlying harness and its moving parts, there is already an in-built feature.

  to use it, open the meatballs menu (...) of your current copilot conversation and click "show agent debug logs". it opens a tab showing all the various tool calls and prompts being sent out behind the scenes and how model selection is happening (if auto). it also gives insights on token consumption as well. moreover, vscode has been quietly shipping updates and recently you can connect your own otel service to get this information in a way you can put to use yourself.

  while copilot has turned me off post their pricing changes, they have been doing tons at their own pace. highly recommend going through this feature if you find the op interesting.

  by rldjbpin
• I wish copilot was better at coding Java. It's like using ChatGpt 5.1, even with Fable 5 or Opus 5 as models.

  The other issue with copilot is how episodic memory works. Copilot writes memories after a task is completed, which means a lot of context is lost from the intermediate exploration, success/failure steps (turns), for what? Codex's multithreaded model adds the turn outputs to episodic memory (both agents submit their episodic data to ... themselves for summary) which gives better insight when working on multi-step problems.

  by Supermancho
• I have found beads works pretty well for this
  by YawningAngel
• Nice one! Really shows why we should run those in sandboxes without env access. I like the proxy swap approach
  by bartek_gdn
• Nice deep dive, I always wondered how copilot worked compared to similar tools. I'm shocked at the lack of of a rule for env files, I at least thought with a tool more integrated with github as a whole that would be a default but alas.
  by tolugenius
• Minor factual correction: The Codex client is open source. https://github.com/openai/codex
  by ameliaquining
• thanks! corrected in the article
  by j0selit0
• One thing I found that I thought was a fun addition, is using eBPF made this even easier. No need to fight with anyone that is using certificate pinning, mTLS or anything else, you just get the raw plaintext data straight of the wire (right before encryption and right after decryption) and works nicely for most of the agents and IDE's.

  That will in practice give you everything from telemetry to prompts, and its funny to see just how much some of them collect/run that is not at all related to your own ask..

  A handy alternative when certain applications tend to make it harder to apply a MiTM proxy and you can dump it straight into your own scripts/programs to filter out and store it in whichever format you want for more analysis.

  by p1llus
• Surprised that works, I thought TLS was done entirely in process space.

  I think I found it, My first thought was some sort of builtin ssl library backdoor, but it looks like you do some ld.preload shenanigans to inject a eBPF monitor. I am not sure exactly what the BPF brings to the table here. A convenient interface to intercept the accept() syscall?

  by somat
• Out of curiosity: How?

  They don't offload TLS to the kernel, do they? Most apps do it in userspace linked against openssl afaik.

  Do you patch that lib? If ebpf "just" operates at network/packet level, I don't see how it can do more than Mitmproxy in regard to avoid DH-PFS/Pinning

  by kro
• I was curious to understand how Copilot implements its harness, and also how I was exhausting my quota so quickly. End up going down a rabbit hole of intercepting its network traffic with mitmproxy.

  A few interesting things I found along the way:

  - watched model/capability discovery and routing happen in real time - looked at what gets injected into context and sent with ghost completions - found that recent edits can pull in context from files other than the one you're currently editing (including infamous .env) - found the SQLite session store behind Chronicle, including previous prompts/responses - watched the model query that history through tool calls

  I then went through the VS Code source to reconcile some of what I was seeing on the wire with the actual implementation.

  Overall some interesting lessons around how their harness is implemented.

  by j0selit0
• How do you actually cleanly solve that .env issue?

  Anything cross platform and coding agent agnostic?

  I suppose that .env file should be removed, but then things aren’t easy: no native multiplatform secret manager, or the std lib of the language doesn’t offer an API over the native secret store, etc.

  Or a "secret injection proxy" for some cases could work I guess.

  by jiehong