Checking the syntax of Perl code

Discussion in 'Perl Misc' started by Samuel, May 18, 2007.

  1. Samuel

    Samuel Guest

    Hi,

    I have written a code generator (in Perl) that produces Perl code and
    would like to throw an assertion if the result contains invalid
    syntax. Any idea if there is a way to perform a syntax check that does
    not involve firing up a new process with "perl -c" or eval?

    -Samuel
    Samuel, May 18, 2007
    #1
    1. Advertising

  2. Samuel wrote:

    > Hi,
    >
    > I have written a code generator (in Perl) that produces Perl code and
    > would like to throw an assertion if the result contains invalid
    > syntax. Any idea if there is a way to perform a syntax check that does
    > not involve firing up a new process with "perl -c" or eval?
    >
    > -Samuel


    Is there a problem with using eval()?

    Tim
    Tim Southerwood, May 18, 2007
    #2
    1. Advertising

  3. On 18 May 2007 06:49:50 -0700, Samuel <> wrote:

    >Subject: Checking the syntax of Perl code


    perl -c

    >I have written a code generator (in Perl) that produces Perl code and
    >would like to throw an assertion if the result contains invalid
    >syntax. Any idea if there is a way to perform a syntax check that does
    >not involve firing up a new process with "perl -c" or eval?


    D'Oh! However... no, I don't think so, since "nothing but perl can
    parse Perl". However eval() does not fire up a new process, that I
    know.

    wolfgang:~ [16:24:15]$ perl -e 'eval q{system "ps x"}'
    PID TTY STAT TIME COMMAND
    18527 ? S 0:01 sshd: blazar@pts/4
    18528 pts/4 Ss 0:01 -bash
    10213 pts/4 S+ 0:00 perl -e eval q{system "ps x"}
    10214 pts/4 R+ 0:00 ps x


    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
    Michele Dondi, May 18, 2007
    #3
  4. Samuel

    Uri Guttman Guest

    >>>>> "S" == Samuel <> writes:

    S> I have written a code generator (in Perl) that produces Perl code and
    S> would like to throw an assertion if the result contains invalid
    S> syntax. Any idea if there is a way to perform a syntax check that does
    S> not involve firing up a new process with "perl -c" or eval?

    how were you planning on actually compiling this code? if you want it
    inside this process you have to use eval STRING. note that any code
    outside of subs will be executed. externally you can run perl -c and it
    won't execute main level code but use commands will be run and those
    modules can execute code. so there really is no way to just test for
    perl syntax all by itself.

    if you want to see how generated code is evaled and used, check out
    Sort::Maker on cpan. it is a very rare problem that needs code
    generation and it can be tricky doing it right. why do you think you
    need code gen for your problem? i saw one case where someone generated
    code for class accessors when closures would have been simpler and
    better.

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
    Uri Guttman, May 18, 2007
    #4
  5. Samuel

    Samuel Guest

    On May 18, 4:16 pm, Tim Southerwood <> wrote:
    > Is there a problem with using eval()?


    Well, maybe I made an incorrect assumption there - is there a way to
    use eval for checking the syntax *without executing the code* that I
    am not aware of?

    There is no problem with eval per-se (in fact, I do compile the code
    using eval at a later time in the program), but I would like to catch
    syntax errors sooner.

    -Samuel
    Samuel, May 18, 2007
    #5
  6. Samuel

    Samuel Guest

    On May 18, 6:11 pm, Uri Guttman <> wrote:
    > how were you planning on actually compiling this code? if you want it
    > inside this process you have to use eval STRING. note that any code
    > outside of subs will be executed.


    Ahh, you may be onto something here, I guess wrapping this into a
    function should do the trick. I'll try this when I'm back in the
    office on Monday.

    > why do you think you
    > need code gen for your problem?


    We are using a highly specialized template language for controlling
    appliances, which we compile into Perl code. There are probably other
    ways to do this, but none that were similarly easy and straight
    forward to do.

    Thanks a lot for your help, Guys!

    -Samuel
    Samuel, May 18, 2007
    #6
  7. Samuel

    Uri Guttman Guest

    >>>>> "S" == Samuel <> writes:

    >> why do you think you
    >> need code gen for your problem?


    S> We are using a highly specialized template language for controlling
    S> appliances, which we compile into Perl code. There are probably other
    S> ways to do this, but none that were similarly easy and straight
    S> forward to do.

    that is definitely not the best (nor easiest) way to do
    templating. there are (too) many templating modules already out there. i
    am sure one of them does what you want and it will likely be faster and
    better.

    and what you may call a highly specialized template language, someone
    else may call a trivial feature already implemented. so post some of
    this template language and i bet you will be able to save a lot of
    coding when you get answers to your real problem (not the eval
    question).

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
    Uri Guttman, May 18, 2007
    #7
  8. Samuel

    Samuel Guest

    On May 18, 8:24 pm, Uri Guttman <> wrote:
    > that is definitely not the best (nor easiest) way to do
    > templating. there are (too) many templating modules already out there.


    Alright, I'll bite. The template looks like any other template
    language:

    {if device.vendor() is "cisco"}
    show conf {extract /^interface (\S+)\.(\d+)/ as interfaces, units}
    {loop interfaces as interface}
    show interfaces {$interface} controller
    {end}
    {end}

    I'm not in the office, but this is a simplified version of the
    compiled template:

    if ($device->vendor() eq "cisco") {
    my @response = $stream->send("show conf");
    my @interfaces;
    my @units;
    for (@response) {
    next unless /^interface (\S+)\.(\d+)/;
    push(@interfaces, $1);
    push(@units, $2);
    }
    for my $interface (@interfaces) {
    my @response = $stream->send("show interfaces $interface
    controller");
    }
    }

    (I skipped the included error handling, locking for parallelization
    via threads, and some more.) This took less than a day to fully
    implement.

    Yes, some template languages are somewhat flexible and allow you to
    register your own functions, but that's really not good enough. Also,
    I believe that parsers and lexers do not really gain you a lot when
    parsing this. The code isn't open sourced yet (it will be soon), but I
    bet it will be hard to implement this with significantly less LOC
    (approx. 350 currently), even by introducing new dependencies. Either
    way, the time it took to write this is probably even less than the
    time it would take to research any other template language.

    -Samuel
    Samuel, May 18, 2007
    #8
  9. >>>>> "Samuel" == Samuel <> writes:

    Samuel> Well, maybe I made an incorrect assumption there - is there a way to
    Samuel> use eval for checking the syntax *without executing the code* that I
    Samuel> am not aware of?

    Not absolutely.

    There are definitely Perl constructs that cannot be parsed (even by
    /usr/bin/perl) without having executed *some* Perl in the process.

    Parsing Perl as a static document is impossible in the general case.

    --
    Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
    <> <URL:http://www.stonehenge.com/merlyn/>
    Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
    See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

    --
    Posted via a free Usenet account from http://www.teranews.com
    Randal L. Schwartz, May 18, 2007
    #9
  10. Samuel

    Guest

    Samuel <> wrote:
    > On May 18, 4:16 pm, Tim Southerwood <> wrote:
    > > Is there a problem with using eval()?

    >
    > Well, maybe I made an incorrect assumption there - is there a way to
    > use eval for checking the syntax *without executing the code* that I
    > am not aware of?


    If the code doesn't use BEGIN or END blocks of its own, or their moral
    equivalents, then you could do something like this:

    eval $whatever . q{;BEGIN {die "reached end\n"}};
    unless ($@ eq "reached end\n") {
    Something_is_wrong_with($whatever, $@);
    };


    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
    , May 18, 2007
    #10
  11. Samuel

    Peter Scott Guest

    On Fri, 18 May 2007 06:49:50 -0700, Samuel wrote:
    > I have written a code generator (in Perl) that produces Perl code and
    > would like to throw an assertion if the result contains invalid
    > syntax. Any idea if there is a way to perform a syntax check that does
    > not involve firing up a new process with "perl -c" or eval?


    Nothing can do as complete a job as perl itself, but the closest anyone
    has come is http://search.cpan.org/~adamk/PPI-1.118/lib/PPI.pm .

    --
    Peter Scott
    http://www.perlmedic.com/
    http://www.perldebugged.com/
    Peter Scott, May 19, 2007
    #11
  12. Samuel

    Samuel Guest

    On May 18, 7:52 pm, Samuel <> wrote:
    > Ahh, you may be onto something here, I guess wrapping this into a
    > function should do the trick. I'll try this when I'm back in the
    > office on Monday.


    I got to test this sooner, and it works in any case that I needed it
    to.

    Thanks again!

    -Samuel
    Samuel, May 19, 2007
    #12
  13. Samuel

    Uri Guttman Guest

    >>>>> "S" == Samuel <> writes:

    S> On May 18, 8:24 pm, Uri Guttman <> wrote:
    >> that is definitely not the best (nor easiest) way to do
    >> templating. there are (too) many templating modules already out there.


    S> Alright, I'll bite. The template looks like any other template
    S> language:

    S> {if device.vendor() is "cisco"}
    S> show conf {extract /^interface (\S+)\.(\d+)/ as interfaces, units}
    S> {loop interfaces as interface}
    S> show interfaces {$interface} controller
    S> {end}
    S> {end}

    S> I'm not in the office, but this is a simplified version of the
    S> compiled template:

    S> if ($device->vendor() eq "cisco") {
    S> my @response = $stream->send("show conf");
    S> my @interfaces;
    S> my @units;
    S> for (@response) {
    S> next unless /^interface (\S+)\.(\d+)/;
    S> push(@interfaces, $1);
    S> push(@units, $2);
    S> }
    S> for my $interface (@interfaces) {
    S> my @response = $stream->send("show interfaces $interface
    S> controller");
    S> }
    S> }

    i wouldn't call that a typical template system. it is more like a
    different language that you parse and tranlate/expand into perl. very
    little of the original source shows up unaltered in the output. in
    general most template systems have text pages which can be modified by
    data during their rendering.

    this means you could also make an interpreter for your system which
    might be easier to manage than code generation. you parse your lang and
    build up a data tree. then you scan that tree and execute each
    operation. you obviously have a set of canned ops like 'loop' and
    'show'. all you need to do is write an op for each that does the same as
    the code you generate. this would be much easier to manage and improve
    than a codegen style.

    S> Yes, some template languages are somewhat flexible and allow you to
    S> register your own functions, but that's really not good enough. Also,
    S> I believe that parsers and lexers do not really gain you a lot when
    S> parsing this. The code isn't open sourced yet (it will be soon), but I
    S> bet it will be hard to implement this with significantly less LOC
    S> (approx. 350 currently), even by introducing new dependencies. Either
    S> way, the time it took to write this is probably even less than the
    S> time it would take to research any other template language.

    your api is also very different. you are calling some external methods
    with your generated code. registering functions could do that but not
    always cleanly. but by doing an interpreter (and you are well over
    halfway there), you eliminate this last messy step of codegen.

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
    Uri Guttman, May 20, 2007
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Clifford W. Racz

    Checking XML DTD syntax and validating XML

    Clifford W. Racz, Jun 7, 2004, in forum: XML
    Replies:
    8
    Views:
    1,623
    Clifford W. Racz
    Jun 23, 2004
  2. Jeff Duffy

    Python switch for syntax checking

    Jeff Duffy, Nov 19, 2004, in forum: Python
    Replies:
    6
    Views:
    546
    Scott David Daniels
    Nov 19, 2004
  3. George Sakkis

    Komodo syntax checking for python2.4

    George Sakkis, Apr 26, 2005, in forum: Python
    Replies:
    2
    Views:
    350
    George Sakkis
    Apr 26, 2005
  4. Count Dracula

    Checking for valid number syntax

    Count Dracula, Dec 20, 2006, in forum: C Programming
    Replies:
    9
    Views:
    337
    Jorgen Grahn
    Dec 22, 2006
  5. Richard Bos

    A different syntax for error checking

    Richard Bos, Jan 18, 2008, in forum: C Programming
    Replies:
    10
    Views:
    471
    Willem
    Jan 18, 2008
Loading...

Share This Page