Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Project page: https://microsoft.github.io/flint-chart/by chenglong-hn
- >Instead of requiring verbose low-level parameters such as scales, axes, spacing, and layout.
Ok, Microsoft is conflating two different things here: LLMs don't really care about code being low level and verbose, they can read things like Assembly and SPIR-V just fine: visualization is the real issue in that LLMs have no natural understanding of spatial composition through visual comparison because they literally "see" things differently than humans, so the way to get around that is provide them with "visualization" in code form that they can easily reason about and understand, so basically anything that's not deeply nested and has hidden states that they have to reason about.
Also, Flint being stringly typed in JSON is a decision that I don't think I agree with. Looking at the actual spec, this could have just been a normal, human usable TypeScript library, and it would have been 100x better. Using their own example (excuse the formatting):
type SemanticType = "Category" | "YearMonth" | "Profit";
type ChartType = "Heatmap" | "BarChart" | "LineChart" | "ScatterPlot"; // extend as needed
interface ChartEncodings { x: string; y: string; color?: string; size?: string; tooltip?: string; }
interface ChartProperties { colorScheme: string; [key: string]: unknown; // allow other optional properties }
interface ChartSpec { chartType: ChartType; encodings: ChartEncodings; chartProperties: ChartProperties; }
type SemanticTypes = Record<string, SemanticType>;
interface ChartConfig<TData = Record<string, unknown>> { data: TData; semantic_types: SemanticTypes; chart_spec: ChartSpec; }
// The actual typed object literal: const chartConfig: ChartConfig = { data: {}, // replace with your actual data shape/type semantic_types: { game: "Category", period: "YearMonth", newUsers: "Profit", }, chart_spec: { chartType: "Heatmap", encodings: { x: "period", y: "game", color: "newUsers", }, chartProperties: { colorScheme: "redblue", }, }, };
EDIT:
Went and actually looked at the source instead of just eyeballing it from the docs, and it was a lot more complete and sophisticated than my assumed mockup already.
Core complaint (string-keyed JSON vs. a real generic authoring surface) still stands, but the specific types I posted aren't what Flint has. My bad.
by YuechenLi - > simple chart specs can be reliable, but generated charts are often of low quality due to reliance on system defaults; - complex chart specs with explicit details can produce good-looking charts, but they are verbose and agents can struggle with reliability
N of only a few of us working on an analytics agent, I don't think we've been finding this to be the case. We've been impressed with just how good LLMs (even smaller open weight models) are at using Python and R for visualization. Often any shortcomings go away if we iterate a bit to about ambiguity. Are there any threads of research that could better support this claim or highlight where issues might be?
by nrub - > requiring them to explicitly make visual decisions that are supposed to be handled by a good compiler
Isnt graphviz there for the same reason?
Edit: I see it is using JSON as the declaration language, I am OK with llms being "good at json" but a syntax also consumable by humans it is not!
by theK - Since it isn’t mentioned on the page, I’ll mention that accessibility is a really important thing to design in when creating data visualizations.
This podcast (I only have a spotify link) has a really good short interview on this:
https://open.spotify.com/episode/18dHTAxCCeIaLOTch6tRld
The interview is with Frank Elavsky who seems to be a rock star in the field (and no, I don’t know him and am not him) and made a project called Chartability relating to this which has heuristics, principles, and guidelines for a11y audits:
by natch - Is there a specific explanation about how this is better or different than vega itself? https://vega.github.io/vega/docs/specification/
My understanding is that Vega was already an expressive DSL for visualizations and its probably already well spread through LLM training data.
by kveykva - There’s an emerging pattern in agentic systems and this project is a great example.
A deterministic layer like a compiler or generator of code with some kind of IR that the LLM generates and feeds it with.
I feel we will be seeing this more and more in the near future.
by cpard - "For AI agents". I understand why everything needs to be marketed in this way, but it's just ... an easy-to-generate language for expressing charts. That's impressive! That's useful.by rbalicki