Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • You can also mine attention from image models, it's a lot of fun and very interesting.
    by scottcodie
  • I don't know much about LLMs but does that mean you have N^2 computation with the context size since every token needs to track how it relates to every other token?
    by ex-aws-dude
  • This is very cool. It's simpler than Bertviz for understanding inference and surface level and a good starting people for new learners as well.
    by sshussain270
  • Is the attention explanation of why the model tells like this? I've seen that there are many discussions about this. (Image attention visualizations were not that good I think)
    by lhk931122
  • The pause-on-load problem is a real UX trap. Stepping through one token at a time is how people actually learn attention; single-step controls would help more than autoplay.
    by Mugshelf
  • This is great, I've read multiple books and watched videos about the attention mechanism. Now that I understand it, this is the clearest example I've seen on how attention works.
    by fuddle
  • UX report. I wished to examine attention state step by step, but I found the animation moved along too fast for that. So I tried pausing...

    On Chromium/linux, pressing pause doesn't pause, instead resetting the animation to it's pre-play state - the current attention highlighting disappears. Pressing play again, restarts at the beginning. Having a commonplace "pause pauses, and play resumes" UI, could allow more time to look over state. A youtube-like slow playback 0.25? option might similarly help. Or perhaps even better, buttons for single stepping. Tnx for your work.

    by mncharity
  • This is great, thank you. I have to teach this stuff on Friday so perfect timing. It's hard to explain the attention mechanism in a way that becomes intuitive because the weighting scheme does not help much with the intuition. Having a visualization like this helps a lot. Don't move that page please since I'll link to it!
    by MCP123

Explore Birbla archives