Comments on: Parsing STEP

By: Jakub Narebski (@jnareb)

Jakub Narebski (@jnareb) — Wed, 17 Oct 2012 18:23:04 +0000

I wonder how hard would be to add proper Perl6 interface to libmarpa C library (the core of Marpa)…

By: skids

skids — Thu, 11 Oct 2012 00:15:27 +0000

When faced with this same problem in defining an IPv4 address parser that understands masks/wildcards, I settled on a very similar solution. Rather than define a “loose” grammar though, I decided it was best just to document the tokens that users could override in the manpage, because there were several ways to loosen the grammar up and a user might want any given combination of them.

Note that one can also do the same thing with the companion “actions” class of production rules. You can also override the grammar’s parse method to link the default production rules with the class:


our grammar Strict {
    method parse($target, :$rule?, :$actions = Actions, *%opt) {

        nextwith($target, :rule($rule), :actions($actions), %opt);

    }
    method emit($target, :$rule = 'TOP', :$emit = Emit, *%opt) {

        # for now just call designated rule.

        $emit."$rule"($target);

    }

}

…the second “emit” method is something I added that mirrors the “actions” and produces the correct text to serialize an AST after modifying it. I think that’s a pattern we will see a lot of so it might be worthy of some spec work…

By: colomon

colomon — Wed, 10 Oct 2012 19:55:02 +0000

That’s actually an interesting question. By spec, the files are ASCII. In practice, you frequently see files in…. err, I don’t know the correct technical term, but 8-bit characters with high bit clear meaning ASCII and high bit set meaning ISO 8859 page 1 I think. Being able to read both would be a good thing. I would not be surprised to someday find UTF-8 files as well. Really, only characters in STEP strings can sensibly be non-ASCII, so it’s not too hard to suss out the meaning intended. On the other hand, I have no idea how to usefully handle those differences in Perl 6 input.

By: Bruce

Bruce — Wed, 10 Oct 2012 18:58:53 +0000

If the files are ASCII and not UTF-8, be sure to flag it as such. Parsing involves many substr-type operations which are much more expensive on UTF-8 strings than strings of fixed-width characters.

By: colomon

colomon — Wed, 10 Oct 2012 12:26:29 +0000

I haven’t, but that will probably be my fallback if I cannot get things going fast enough in Rakudo. But jnthn++ has gotten me another 18 seconds already today, and is working on further improvements…

By: Jakub Narebski (@jnareb)

Jakub Narebski (@jnareb) — Wed, 10 Oct 2012 12:20:01 +0000

Have you tried using Perl 5′s Marpa?