Philosophical Issues with Rakudo’s .parse

For a little bit now I’ve had the ABC module working on both Niecza and Rakudo. I got a new Linux box yesterday; this morning I was playing around with my ABC tools there, trying to see if I could get them to work. I installed them with panda, so I was using ABC with Niecza by default on my MBP and with Rakudo by default on my new machine. I tried it with a file of my own compositions. Apparently I hadn’t tried this in a while, because it had several issues. And the two systems handled them completely differently.

On Niecza, I got this message:

Unhandled exception: Did not match ABC grammar: last tune understood:
 A Hat for the Whales
d|:e~A3 BGEF|G2BG DG (3ABd|eBdB ~A3G|Bdef gfed|
e~A3 BGEF|G2BG DG (3ABd|eBdB AAGA|[1Beed e3d:|

On Rakudo, there were no error messages, but “A Hat for the Whales” was the last tune processed, even though there were a dozen more tunes in the ABC file after it.

Without thinking about it too much I corrected this batch of errors. Now on Niecza the entire file parsed, but I got the error

Unhandled exception: Illegal key signature B minor

Under Rakudo, the same ABC file just appeared to work.

What’s going on here? Well, key signature processing looks something like this (simplified):

    token key-def { <basenote> <chord_accidental>? <mode>? }
    token mode { <minor> | <major> }
    token minor { "m" ["in" ["o" ["r"]?]?]? }
    token major { "maj" ["o" ["r"]?]? }

No space allowed between basenote and mode. So when you invoke parse with that rule on “B minor”, in Niecza it fails, because the entire string was not successfully parsed. In Rakudo it succeeds, because “B” is a valid key signature, and it doesn’t care that the entire string was not parsed. In practice, this means that three of the tunes in the collection got the wrong key signature: B major instead of the correct B minor.

To my mind, Niecza’s behavior is more intuitive. If you’re trying to parse a file, parsing half of it and then silently ignoring the rest is wildly unhelpful behavior. In every use I’ve had for parse so far, I’ve wanted it to parse everything in the string I sent to the function.

But hey, I understand that some people might have different needs. Can we at least add a flag that will give you a parsing failure if the entire string is not parsed? :all, perhaps?

4 Responses to “Philosophical Issues with Rakudo’s .parse”

  1. Ron Says:

    antlr has the same “problem”. There you need to add an explicit token in the rule to be parsed.

    • Ron Says:

      That is “explicit EOF token”. If there is garbage in the input, the EOF token cannot be matched and the parse fails.
      Parsing a partial file can be useful when parsing a file to do syntax highlighting a programming language or extracting information like defined functions on the fly in an editor.

  2. ttjjss Says:

    How about just adding ^ and $ to the TOP rule?

    • skids Says:

      That would be less flexible to the user, if they want to parse such a thing with trailing garbage. Perhaps supplying both TOP and another rule which could be selected by :target. However, lots of grammars are going to need this, and it would be pathetic to have a smattering of differently named rules for this, so there is a good argument for a standardized flag.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: