Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • If you created a format that is so difficult to parse that it cannot be parsed with simple readable C code then the problem is the format not the parser code.
    by r3d
  • Why care about lang which doesnt really support strings well?
    by tester756
  • Can you feel the irony when typing this? "Simple readable C code" itself not being able to be parsed by "simple readable C code".
    by jrimbault
  • Nice, but

    > trims the stray trailing \r that malformed input likes to leave behind

    how does this distinguisg the non-stray variety?

    by chrisjj
  • If this is "simple", then I don't think the author has tried a good parser combinator library yet.
    by aappleby
  • I now use nom to write my parsers. Once you understand it, it’s simple and parsing complex data becomes a _fun_ puzzle. I recommend it.

    https://github.com/rust-bakery/nom

    by speedgoose
  • I'm going to collect this post after 24 hours, extract the methodologies from everyone's comments, and write them down in my notes. The reason I like HN is that people freely share their tips in the comments
    by jdw64
  • I'll just drop this here for you and anyone else who wants it: https://github.com/bablr-lang/language-en-regex-vm-pattern/b...

    No codegen, just function calling.

    by conartist6
  • > if (!line.accept('[').isEmpty() ) // [section] header.

    Is this really ergonomic?

    by alexjurkiewicz
  • Yeah those all read a bit brain teasy to me.

    "if not accept this character, so this is skipping the match, oh wait, if the result of trying to match is empty, no wait again, it was not empty, i.,e we ARE matching...".

    by roaringrocky
  • It's not, and the article's attempt at putting lipstick on this pig is nonsensical:

    > StringView has no operator bool, so a successful match reads as !scanner.accept('=').isEmpty(). Noisier than returning a bool, but the matched text comes back with the answer instead of requiring a second call to go get it.

    Clearly there is a second call, it's the call to isEmpty. Plus, the actual text is lost in this particual example (though we know what it was).

    An actual ergonomic way of using this would set things up so that the INI parser's inner code could be written something like this:

        bool success =
        (name = trim(acceptUntil('='))) &&
        accept('=') &&
        accept(Space) &&
        (property = acceptAll());
    by derdi
  • A parser/compiler could obviously be improved with an LLM (AI!) to suggest improvements to invalid input. That is actually a super good use case of LLM/AI.

    Having clang/gcc, or any other parser, implement that is of course impossible, they are too conservative and would rather die than to implement modern helpful tools.

    by langbn
  • You're presumably using an LLM to drive the compiler, and your LLM should be just as capable of reading the compiler's error message and fixing the problem. So why double the work?
    by derdi
  • FORTH parsers are ultra simple - get the next space-separated token, if it is a number, push it on the stack, otherwise it's a word - look it up in the dictionary and (if it exists there) execute it.
    by zabzonk
  • Even simpler is you go the colorforth route, part of the source code is "pre parsed" by the editor, a prefix byte is added to each word (which is shown as different colors), then a simple dispatch loop with that prefix as a sort of opcode.
    by roaringrocky
  • > LineReader splits input into lines, handles \n and \r\n, and trims the stray trailing \r that malformed input likes to leave behind

    Is there a common source of extra \r in malformed inputs, beyond those existing as part of \r\n? Or is this just a dig at Windows-style line endings? If there's something weird going on I think I'd rather fail loudly.

    > Bounding the inner scanner to a single line makes “run past the end of a malformed line” unrepresentable rather than merely unlikely.

    I don't really see what makes it "unrepresentable", and this reads more like "if you used the right scanning logic, you can't have used the wrong scanning logic".

    by Retr0id
  • End of line on Classic Mac is \r.
    by thesz
  • Sure, start with \r\n, split on \n, now you have a stray \r at the end of every input.
    by inigyou
  • > Is there a common source of extra \r in malformed inputs, beyond those existing as part of \r\n?

    Old Macs and some other systems use \r as their EOL, I still sometimes see that with string values in CSV files y code has to deal with (though I don't think I've seen it as an EOL marker in the format itself for a _long_ time).

    Sometimes incorrect cleaning steps can leave them behind, such as replacing \r\n with \n but that replacement not being global: it tests fine on strings with zero or one \r\n but subsequent ones will retain their \r. Also code splitting on \n assuming it will always see just that as EOLs will leave trailing \r characters in place. Also, code cleaning EOLs from strings that are supposed to be one-line-only may replace \n (or \n or \r\n, ignoring the possibility of just \r) with a space or a comma and a space, that could be where the \r characters in certain string values I see in files from clients are coming from.

    I suspect that off-by-one errors caused by character counting bugs in UTF8/UTF16 handling may cause splitting on EOLs to be a bit off in some cases, though here you will probably be seeing other data corruption at the same time and an errant \r is one of your smaller problems.

    by dspillett
  • If you draw a line from 'ad-hoc byte-wrangling nonsense' to 'parser combinators', this can't be more than 20% along it.

    Looking at the linked URL parser, why doesn't it look like

      url = do scheme
           authority
           path
           query
           fragment
  where
  scheme = ...
  authority = ...
  etc.
    It looks totally ad-hoc.
    by mrkeen
  • Unfortunately, simple URL parsing breaks on so many things. There is a reason on why every URL parsing library is at least a few thousand LOCs.

    One common way to test it is just to pass ipv6 url: http://[f021:d981:b487:e57d:193e:550e::]/

    by f311a
  • Yeah it's complicated, and that's the thing about parsing anything, the more complicated and unpredictable the input and the harder it is to parse.

    Does it need to be human readable, does it need to work across all platforms. Does the it need to be secure. These things change everything. Speed, reliability, security pick one.

    Your point about being compliant with the real spec is the difference between a 20 line scannf and and a 1000 line function. Yeah. Ha.

    by r3d
  • Is that so?

    RFC 3986 Appendix B [1] "Parsing a URI Reference with a Regular Expression":

    The following line is the regular expression for breaking-down a well-formed URI reference into its components.

      ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?

      scheme    = $2
      authority = $4
      path      = $5
      query     = $7
      fragment  = $9

    Let's test your URI with this regex, shall we? [2]

      $2 (scheme) = http
  $4 (authority) = [f021:d981:b487:e57d:193e:550e::]
  $5 (path) = /
    Seems correct to me.

    [1] https://datatracker.ietf.org/doc/html/rfc3986#appendix-B

    [2] https://regexr.com/8nqop

    by meindnoch
  • The hardest thing about writing a parser is cognitively accepting what is going to be considered valid input. You can make the best parser that is fast and well specified but invariably someone will (ab)use it in an unexpected way.

    Famous examples: despite so many initial good intentions, html tags don’t need to be closed, JSON numbers are too often encoded as strings, YAML can look like what most people expect or it can look progressively more like JSON… and on and on.

    by imoverclocked
  • > html tags don’t need to be closed

    That's a very explicit and much debated feature, even self closing tags. It is also one of the main factors that makes HTML distinct from xml. And a big reason why xhtml was created.

    I agree that it makes for a much more complicated interpretation.

    by trashb
  • And harder than that? Report the error, in a way that make some sense.

    This is compounded by the fact that you need the semantics involved, the environment (ie: everything on scope), the source (that means you need to keep carrying big strings around).

    And what is efficient means to be destructive, but you need instead the opposite for semantics, error messages, optimizations and the like.

    by mamcx
  • > JSON numbers are too often encoded as strings

    There's a good reason for that, since JSON comes from JavaScript, many JSON parsers treat numbers as double-precision floats. By encoding your number as a string, you ensure that the JSON parser has not modified your number.

    https://blog.json-everything.net/posts/numbers-are-numbers-n...

    by craftkiller
  • I think the second-hardest thing is to accept that CS spent decades optimizing parsing algorithms and grammars, and this is still a significant part of CS curricula in many places. But the practical reality is that parsing is almost never a bottleneck.

    If what you're parsing is within the capacity of humans to interact with (so in the range of tens of kilobytes), a grammar that requires an O(N^2) parser is totally fine.

    by simonask