Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Watching an LLM interact with the Honeycomb MCP is interesting. When returning a query result it returns not only the tabular data but also an ascii art chart. Presumably they have tested this and it improves the LLM’s ability to interpret the data.
    by IneffablePigeon
  • My money is on they used agentic AI coding tools to help set up all the metrics/experiments with traditional models.
    by apwheele
  • One step further are those who want to point an llm directly at the data warehouse to get the data needed to run predictions
    by pjot
  • Interesting work.

    I'm surprised that hypothesis 2 (that CSV serialization format mangles table columns) was falsified. Back in the gpt-3.5-turbo and gpt-4o era, I did needle-haystack tests and found that table format mattered a lot (csv, tsv, markdown). Most models "could not read vertically" for csv (they were horrible), but they could for markdown. I concluded that serialization format or tokenization played a major role.

    Nowadays, LLM performance on csvs is much improved (I'm guessing after being explicitly trained on CSV question-answering.) But I still carry the impression that LLMs read columns only by "memorizing" column positions in a format-dependent manner. Maybe this impression is out of date.

    by gw32
  • Unsure if it's LLMs that fail at tabular data or its just that tree boosting are spectacular at that task.
    by ismailmaj
  • Look at the white text on white background in Appendix F. Pretty funny.
    by rustyconover
  • Google's TabFM [1] (and its previous TimesFM) seem the best approach so far on this area

    1. https://research.google/blog/introducing-tabfm-a-zero-shot-f...

    by tough
  • The first thing I'd do if working with an LLM on tabular data is to ask what the best tool would be to work with that data and build up a proper harness to work with the data sensibly. Rawdogging LLM isn't the tool for forecasting like this, as they found.
    by _joel

Explore Birbla archives