Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Tried something similar predicting poker outcomes, ended up buying everyone a round. The math wasn't in my favor.
    by cold_boot
  • Betting on this with high-dimensional stats sounds like a classic case of overfitting your opponent. Just buy them a beer.
    by ovin_dal
  • A fun step-up would be to see if you can find an infinity shape by repeating the process with a set of points which are maximally far away from all points but 1 on the existing circle.
    by ViscountPenguin
  • Using high-dimensional statistics for beer money, not academic papers" is the correct use of the degree. The papers are where you prove the circle exists. The bar is where you find out it exists for everything
    by ZeroDayDreamer
  • The best bar bets are the ones where you're technically not cheating, just abusing linear algebra.
    by mph1027
  • Does the mapping depend on the chose set, or could you just map all 25k words and pick 10 points that form a circle? From the process, they seemingly replaced single words in the set that didn't fit, which makes it sound like the position of the remaining words didn't change much from changing the set.
    by hhjinks
  • I recently did some analysis on how different style prompts like "be succinct" and "avoid mannered prose" affect LLM outputs and I also saw a ~circular relationship with a 2d dimensional reduction (MDS) of the similarity matrix. I wonder if it's some artifact of PCA/MDS + Gemma's output dist
    by mdritch
  • After seeing the claim, I knew it would work because of a few intuitive things:

    1. Things that are the result of a very large number of small perturbations follow a Gaussian distribution.

    1a. Word embeddings are the result of millions/billions/trillions of small perturbations (i.e. gradient descent)

    2. As you increase the dimension, d, of a multivariate Gaussian distribution, the actual mass of the data is increasingly concentrated in the thin spherical shell. For MVN distributions with standardized marginal dimensions, this is on the sphere of radius \sqrt{d}.

    In fact when I searched "Gaussian curse of dimensionality" I got this at the top result: [1] which shows this phenomena perfectly. Or this one with pretty interactive pictures [2]

    [1]: https://www.miryusupov.com/blog/posts/thin-shells/index.html

    [2]: https://aseemrb.me/blog/high-dimensional-gaussians-on-a-sphe...

    by JHonaker

Explore Birbla archives