JSON Query

Join the discussion

• Hacker News
• In the k8s world there's a random collection of json path, json query, some random expression language.

  Just use jq. None of the other ones are as flexible or widespread and you just end up with frustrated users.

  by arccy
• https://jsonpath.com/ or https://jsonata.org/
  by HatchedLake721
• I'd like to know how it compares to https://jsonata.org
  by jawns
• I read the man page of `jq` and learned how to use it. It's quite well-written and contains a good introduction.

  I've observed that too many users of jq aren't willing to take a few minutes to understand how stream programming works. That investment pays off in spades.

  by gcr
• DuckDB can read JSON - you can query JSON with normal SQL.[1] I prefer to Malloy Data language for querying as it is 10x simpler than SQL.[2]

  [1] - https://duckdb.org/docs/stable/data/json/overview [2] - https://www.malloydata.dev/

  by mrtimo
• You just have to wrap your mind around jq. It's a) functional, b) has pervasive generators and backtracking. So when you write `.a[].b`, which is a lot like `(.a | .[] | .b)` what you get is three generators strung together in an `and_then` fashion: `.a`, then `.[]`, and then `.b`. And here `.a` generates exactly one value, as does `.b`, but `.[]` generates as many values as are in the value produced by `.a`. And obviously `.b` won't run at all if `.a` has no values, and `.b` will run for _each_ value of `.a[]`. Once you begin to see the generators and the backtracking then everything begins to make sense.
  by cryptonector
• Doesn't the command-line utility `jq` already define a protocol for this? How do the syntaxes compare?

  (LLMs are already very adept at using `jq` so I would think it was preferable to be able to prompt a system that implements querying inside of source code as "this command uses the same format as `jq`")

  by tcdent
• crucial jq insight which unlocked the tool for me: it's jsonl, not json.

  it's a pipeline operating on a stream of independent json terms. The filter is reapplied to every element from the stream. Streams != lists; the latter are just a data type. `.` always points at the current element of the stream. Functions like `select` operate on separate items of the stream, while `map` operates on individual elements of a list. If you want a `map` over all elements of the stream: that's just what jq is, naturally :)

  stream of a single element which is a list:

      echo '[1,2,3,4]' | jq .
      # [1,2,3,4]
  

  unpack the list into a stream of separate elements:

      echo '[1,2,3,4]' | jq '.[]'
      # 1
      # 2
      # 3
      # 4
      echo '[1,2,3,4]' | jq '.[] | .' # same: piping into `.` is a NOP:
  

  only keep elements 2 and 4 from the stream, not from the array--there is no array left after .[] :

      echo '[1,2,3,4]' | jq '.[] | select(. % 2 == 0)'
      # 2
      # 4
  

  keep the array:

      echo '[1,2,3,4]' | jq 'map(. * 2)'
      # [2,4,6,8]
  

  map over individual elements of a stream instead:

      echo '[1,2,3,4]' | jq '.[] | . * 2'
      # 2
      # 4
      # 6
      # 8
      printf '1\n2\n3\n4\n' | jq '. * 2' # same
  

  This is how you can do things like

      printf '{"a":{"b":1}}\n{"a":{"b":2}}\n{"a":{"b":3}}\n' | jq 'select(.a.b % 2 == 0) | .a'
      # {"b": 2}
  

  select creates a nested "scope" for the current element in its parens, but restores the outer scope when it exits.

  Hope this helps someone else!

  by nothrabannosir