Checking the syntax of Perl code

S

Samuel

Hi,

I have written a code generator (in Perl) that produces Perl code and
would like to throw an assertion if the result contains invalid
syntax. Any idea if there is a way to perform a syntax check that does
not involve firing up a new process with "perl -c" or eval?

-Samuel
 
T

Tim Southerwood

Samuel said:
Hi,

I have written a code generator (in Perl) that produces Perl code and
would like to throw an assertion if the result contains invalid
syntax. Any idea if there is a way to perform a syntax check that does
not involve firing up a new process with "perl -c" or eval?

-Samuel

Is there a problem with using eval()?

Tim
 
M

Michele Dondi

Subject: Checking the syntax of Perl code

perl -c
I have written a code generator (in Perl) that produces Perl code and
would like to throw an assertion if the result contains invalid
syntax. Any idea if there is a way to perform a syntax check that does
not involve firing up a new process with "perl -c" or eval?

D'Oh! However... no, I don't think so, since "nothing but perl can
parse Perl". However eval() does not fire up a new process, that I
know.

wolfgang:~ [16:24:15]$ perl -e 'eval q{system "ps x"}'
PID TTY STAT TIME COMMAND
18527 ? S 0:01 sshd: blazar@pts/4
18528 pts/4 Ss 0:01 -bash
10213 pts/4 S+ 0:00 perl -e eval q{system "ps x"}
10214 pts/4 R+ 0:00 ps x


Michele
 
U

Uri Guttman

S> I have written a code generator (in Perl) that produces Perl code and
S> would like to throw an assertion if the result contains invalid
S> syntax. Any idea if there is a way to perform a syntax check that does
S> not involve firing up a new process with "perl -c" or eval?

how were you planning on actually compiling this code? if you want it
inside this process you have to use eval STRING. note that any code
outside of subs will be executed. externally you can run perl -c and it
won't execute main level code but use commands will be run and those
modules can execute code. so there really is no way to just test for
perl syntax all by itself.

if you want to see how generated code is evaled and used, check out
Sort::Maker on cpan. it is a very rare problem that needs code
generation and it can be tricky doing it right. why do you think you
need code gen for your problem? i saw one case where someone generated
code for class accessors when closures would have been simpler and
better.

uri
 
S

Samuel

Is there a problem with using eval()?

Well, maybe I made an incorrect assumption there - is there a way to
use eval for checking the syntax *without executing the code* that I
am not aware of?

There is no problem with eval per-se (in fact, I do compile the code
using eval at a later time in the program), but I would like to catch
syntax errors sooner.

-Samuel
 
S

Samuel

how were you planning on actually compiling this code? if you want it
inside this process you have to use eval STRING. note that any code
outside of subs will be executed.

Ahh, you may be onto something here, I guess wrapping this into a
function should do the trick. I'll try this when I'm back in the
office on Monday.
why do you think you
need code gen for your problem?

We are using a highly specialized template language for controlling
appliances, which we compile into Perl code. There are probably other
ways to do this, but none that were similarly easy and straight
forward to do.

Thanks a lot for your help, Guys!

-Samuel
 
U

Uri Guttman

S> We are using a highly specialized template language for controlling
S> appliances, which we compile into Perl code. There are probably other
S> ways to do this, but none that were similarly easy and straight
S> forward to do.

that is definitely not the best (nor easiest) way to do
templating. there are (too) many templating modules already out there. i
am sure one of them does what you want and it will likely be faster and
better.

and what you may call a highly specialized template language, someone
else may call a trivial feature already implemented. so post some of
this template language and i bet you will be able to save a lot of
coding when you get answers to your real problem (not the eval
question).

uri
 
S

Samuel

that is definitely not the best (nor easiest) way to do
templating. there are (too) many templating modules already out there.

Alright, I'll bite. The template looks like any other template
language:

{if device.vendor() is "cisco"}
show conf {extract /^interface (\S+)\.(\d+)/ as interfaces, units}
{loop interfaces as interface}
show interfaces {$interface} controller
{end}
{end}

I'm not in the office, but this is a simplified version of the
compiled template:

if ($device->vendor() eq "cisco") {
my @response = $stream->send("show conf");
my @interfaces;
my @units;
for (@response) {
next unless /^interface (\S+)\.(\d+)/;
push(@interfaces, $1);
push(@units, $2);
}
for my $interface (@interfaces) {
my @response = $stream->send("show interfaces $interface
controller");
}
}

(I skipped the included error handling, locking for parallelization
via threads, and some more.) This took less than a day to fully
implement.

Yes, some template languages are somewhat flexible and allow you to
register your own functions, but that's really not good enough. Also,
I believe that parsers and lexers do not really gain you a lot when
parsing this. The code isn't open sourced yet (it will be soon), but I
bet it will be hard to implement this with significantly less LOC
(approx. 350 currently), even by introducing new dependencies. Either
way, the time it took to write this is probably even less than the
time it would take to research any other template language.

-Samuel
 
R

Randal L. Schwartz

Samuel> Well, maybe I made an incorrect assumption there - is there a way to
Samuel> use eval for checking the syntax *without executing the code* that I
Samuel> am not aware of?

Not absolutely.

There are definitely Perl constructs that cannot be parsed (even by
/usr/bin/perl) without having executed *some* Perl in the process.

Parsing Perl as a static document is impossible in the general case.
 
X

xhoster

Samuel said:
Well, maybe I made an incorrect assumption there - is there a way to
use eval for checking the syntax *without executing the code* that I
am not aware of?

If the code doesn't use BEGIN or END blocks of its own, or their moral
equivalents, then you could do something like this:

eval $whatever . q{;BEGIN {die "reached end\n"}};
unless ($@ eq "reached end\n") {
Something_is_wrong_with($whatever, $@);
};


Xho
 
S

Samuel

Ahh, you may be onto something here, I guess wrapping this into a
function should do the trick. I'll try this when I'm back in the
office on Monday.

I got to test this sooner, and it works in any case that I needed it
to.

Thanks again!

-Samuel
 
U

Uri Guttman

S> Alright, I'll bite. The template looks like any other template
S> language:

S> {if device.vendor() is "cisco"}
S> show conf {extract /^interface (\S+)\.(\d+)/ as interfaces, units}
S> {loop interfaces as interface}
S> show interfaces {$interface} controller
S> {end}
S> {end}

S> I'm not in the office, but this is a simplified version of the
S> compiled template:

S> if ($device->vendor() eq "cisco") {
S> my @response = $stream->send("show conf");
S> my @interfaces;
S> my @units;
S> for (@response) {
S> next unless /^interface (\S+)\.(\d+)/;
S> push(@interfaces, $1);
S> push(@units, $2);
S> }
S> for my $interface (@interfaces) {
S> my @response = $stream->send("show interfaces $interface
S> controller");
S> }
S> }

i wouldn't call that a typical template system. it is more like a
different language that you parse and tranlate/expand into perl. very
little of the original source shows up unaltered in the output. in
general most template systems have text pages which can be modified by
data during their rendering.

this means you could also make an interpreter for your system which
might be easier to manage than code generation. you parse your lang and
build up a data tree. then you scan that tree and execute each
operation. you obviously have a set of canned ops like 'loop' and
'show'. all you need to do is write an op for each that does the same as
the code you generate. this would be much easier to manage and improve
than a codegen style.

S> Yes, some template languages are somewhat flexible and allow you to
S> register your own functions, but that's really not good enough. Also,
S> I believe that parsers and lexers do not really gain you a lot when
S> parsing this. The code isn't open sourced yet (it will be soon), but I
S> bet it will be hard to implement this with significantly less LOC
S> (approx. 350 currently), even by introducing new dependencies. Either
S> way, the time it took to write this is probably even less than the
S> time it would take to research any other template language.

your api is also very different. you are calling some external methods
with your generated code. registering functions could do that but not
always cleanly. but by doing an interpreter (and you are well over
halfway there), you eliminate this last messy step of codegen.

uri
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top