Syntax checker wtf?

[Firstname Surname]

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker
frequently spits out 'syntax error' with the last line of the code as
the error line (the error is nowhere near the last line). This happens
with missing chars, extra ., all sorts of minor syntax errors, not just
missing 'end's. I found myself copy and pasting functions, checking,
then pasting again to find the error, which is pretty ridiculous.

This is something I'd expect in a pre 1.0 compiler, but ruby is at
1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?


Cheers,
Newb

 

[Phillip Hutchings]

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker
frequently spits out 'syntax error' with the last line of the code as
the error line (the error is nowhere near the last line). This happens
with missing chars, extra ., all sorts of minor syntax errors, not just
missing 'end's. I found myself copy and pasting functions, checking,
then pasting again to find the error, which is pretty ridiculous.

This is something I'd expect in a pre 1.0 compiler, but ruby is at
1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?

No, it's just that Ruby is so flexible that your incorrect syntax
could well be right if you tidied it up at the end.

 

[Robert Klemme]

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker
frequently spits out 'syntax error' with the last line of the code as
the error line (the error is nowhere near the last line). This happens
with missing chars, extra ., all sorts of minor syntax errors, not just
missing 'end's. I found myself copy and pasting functions, checking,
then pasting again to find the error, which is pretty ridiculous.

This is something I'd expect in a pre 1.0 compiler, but ruby is at
1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?

I *think* this is at least partly due to Ruby's convenient and very
flexible syntax. IOW, the parser has no means to detect the error line
as it could be in several places - likely too many to report. Also,
this is how parsers work. From my limited insight into the parser
generation business it would require a) a different parser generator
that is much smarter or b) a lot of effort that it's not worth IMHO.

I'll have to add that this is rarely an issue for me. Maybe you should
use an editor with automatic bracket closing and "end" insertion.

Kind regards

robert

 

[Firstname Surname]

No, it's just that Ruby is so flexible that your incorrect syntax
could well be right if you tidied it up at the end.

It's not a bug, it's a feature! Geeze, django is looking better already.

 

[Just Another Victim of the Ambient Morality]

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker
frequently spits out 'syntax error' with the last line of the code as
the error line (the error is nowhere near the last line). This happens
with missing chars, extra ., all sorts of minor syntax errors, not just
missing 'end's. I found myself copy and pasting functions, checking,
then pasting again to find the error, which is pretty ridiculous.

This is something I'd expect in a pre 1.0 compiler, but ruby is at
1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?

While I'm a newbie myself, I'm willing to guess that there's little you
can do about this problem except to try to relax and be a little more
careful with your code.
To be honest, I really don't think this problem is particularly bad in
Ruby. I don't know exactly what kind of errors you're making (and perhaps
you can elaborate some more on that) but even the latest Microsoft C++
compiler often gives less than helpful clues on the source of the problem,
or even it's location! Brace mismatching and quote mismatching are two
errors that neither parsers can nail down to your satisfaction and I get the
sense that these constitute the majority of your problems (again, please
elaborate on this). The former can be alleviated by proper indentation and
the latter can be totally eliminated by syntax highlighting, although I
don't use this technique myself (since I'm a lazy gvim user).

 

[Gregor Kopp]

As "Firstname Surname" said, the error messages from the ruby
interpreter are not really significant, they could be more verbose.
Maybe Ruby 2.0 will do it better.
However, a good editor may help, but it can not be the solution, or?

Regards, Gregor

 

[Firstname Surname]

As "Firstname Surname" said, the error messages from the ruby
interpreter are not really significant, they could be more verbose.
Maybe Ruby 2.0 will do it better.
However, a good editor may help, but it can not be the solution, or?

Regards, Gregor



Robert Klemme schrieb:

I think the syntax checker could be a little more verbose, and I don't
think it would take much work.

The trouble I had was with a line that had an extra '.' like:
@entry = Entry.find(params[:id].
instead of
@entry = Entry.find(params[:id]

and another error that i think was just a missing bracket maybe like
@entry = Entry.find(params[:id

I've used many different languages with various levels of syntax error
help, but never one this bad. I just wrote a quick script to remove
functions from 'def' to 'end' in order and check the syntax over, thus
quickly finding which function the error is in. This is a lot better
than looking through a couple hundred line controller file. If I can do
this in 10 minutes, I can't help but think it's just laziness on the
part of the syntax checker not to provide at least hints where it thinks
the problem might be.
I understand that ruby is flexible (I'm not not sure this is a merit
when using for a large web app), but the checker could easily check
based on 'standard' expected syntax and be more helpful about error
location 99% of the time.

 

[David Vallner]

the checker could easily check
based on 'standard' expected syntax and be more helpful about error
location 99% of the time.

That's what a lint style program would do, not the compiler. Compilers are
there to check correctness against the formal language definition, not to
make opinions about your code, and quoting James Britt: Ruby assumes the
developer is a grown-up. If you're newbly enough to make trivial syntax
errors, noone's forcing you to use the language.

If you think the syntax checking in the compiler could be made better, go
ahead and hack the parser to do it. But I don't recall this topic being
present on the list and complained about any often or so vocally, so I
doubt it's critical that effort be spent from the core Ruby developer team
in that direction.

Oh, before I forget: Whining gets you nowhere, and trolling belongs to
slashdot. Cut that out.

David Vallner

 

[Just Another Victim of the Ambient Morality]

I think the syntax checker could be a little more verbose, and I don't
think it would take much work.

There's a standard response to comments like this. The code is open
source, so if you really think it's that easy, why don't _you_ do it? It
works better for wikis...

The trouble I had was with a line that had an extra '.' like:
@entry = Entry.find(params[:id].
instead of
@entry = Entry.find(params[:id]

and another error that i think was just a missing bracket maybe like
@entry = Entry.find(params[:id

I've used many different languages with various levels of syntax error
help, but never one this bad. I just wrote a quick script to remove
functions from 'def' to 'end' in order and check the syntax over, thus
quickly finding which function the error is in. This is a lot better
than looking through a couple hundred line controller file. If I can do
this in 10 minutes, I can't help but think it's just laziness on the
part of the syntax checker not to provide at least hints where it thinks
the problem might be.

Okay, judging from your examples, I think the problem is that you're
hitting a rather sore spot in the Ruby interpreter. I've done a few
examples and it looks as if the Ruby parser does a lazy evaluation of
parenthesized (and bracketed) things. It just so happens that these are
exactly the kind of mistakes you make so they really seem to bite you.
The only thing I can suggest is that you close all your brackets and
functions before filling them. So, while editing, you'll write things in
this order:

def some_function
end

..and then...

def some_method
@member[]
end

...and finally...

def some_method
@member[@other + 2]
end

I understand that ruby is flexible (I'm not not sure this is a merit
when using for a large web app), but the checker could easily check
based on 'standard' expected syntax and be more helpful about error
location 99% of the time.

Well, you're welcome to try Django but you should be warned that the
Python syntax is every bit as flexable as Ruby, so if you really don't
think that's a merit for a large web application...
Python software is generally more mature than Ruby (probably because
it's much older) but the syntax is less consistent (perhaps because it's
much older?), so it's a trade off. You'll have to decide what's important
to you and go from there...
Good luck!

 

[Just Another Victim of the Ambient Morality]

That's what a lint style program would do, not the compiler. Compilers
are there to check correctness against the formal language definition,
not to make opinions about your code, and quoting James Britt: Ruby
assumes the developer is a grown-up. If you're newbly enough to make
trivial syntax errors, noone's forcing you to use the language.

That's sufficiently harsh that it's rather unfair.
Okay, so you fed the interpeter bad input but, you know what? Good
software tells you why your input was bad and the original poster isn't
getting the feedback he was hoping to get. Now, perhaps his expectations
aren't realistic but you can simply tell him why they're not realistic
rather than effectively telling him that he just sucks...

Personally, I don't think there's anything wrong with protecting code
from human error considering how it's written for humans. Ruby still has
access permissions. If Ruby really trusted the programmer, it wouldn't
bother with public and private methods. If you're not supposed to call
that method, then don't call it! It's that simple. That's what Python
does...

If you think the syntax checking in the compiler could be made better, go
ahead and hack the parser to do it. But I don't recall this topic being
present on the list and complained about any often or so vocally, so I
doubt it's critical that effort be spent from the core Ruby developer
team in that direction.

Well, it may be true that we have better things to do (I, too, have
never encountered the problems the original poster is having), but he's
simply telling us what he finds important. If enough people complained
about it, I'd imagine work would be done on the problem...

Oh, before I forget: Whining gets you nowhere, and trolling belongs to
slashdot. Cut that out.

You could cut him some slack. It's pretty easy to get frustrated with
these stupid machines. A lot of things really should "just work" and it
can get frustrating when they don't, especially after everyone proselytizes
it as the solution to all their problems...

 

[Paul Battley]

Personally, I don't think there's anything wrong with protecting code
from human error considering how it's written for humans. Ruby still has
access permissions. If Ruby really trusted the programmer, it wouldn't
bother with public and private methods. If you're not supposed to call
that method, then don't call it! It's that simple. That's what Python
does...

Ruby does trust the programmer. The privacy screen is very flimsy:

class Foo
private
def dont_call_me
puts("I said DON'T CALL ME!")
end
end

f = Foo.new
f.dont_call_me
# NoMethodError: private method `dont_call_me' called for #<Foo:0x34293c>

f.__send__dont_call_me)
# I said DON'T CALL ME!

Paul.

 

[David Vallner]

That's sufficiently harsh that it's rather unfair.
Okay, so you fed the interpeter bad input but, you know what? Good
software tells you why your input was bad and the original poster isn't
getting the feedback he was hoping to get. Now, perhaps his expectations
aren't realistic but you can simply tell him why they're not realistic
rather than effectively telling him that he just sucks...

Personally, I don't think there's anything wrong with protecting code
from human error considering how it's written for humans. Ruby still has
access permissions. If Ruby really trusted the programmer, it wouldn't
bother with public and private methods. If you're not supposed to call
that method, then don't call it! It's that simple. That's what Python
does...



Well, it may be true that we have better things to do (I, too, have
never encountered the problems the original poster is having), but he's
simply telling us what he finds important. If enough people complained
about it, I'd imagine work would be done on the problem...



You could cut him some slack. It's pretty easy to get frustrated
with
these stupid machines. A lot of things really should "just work" and it
can get frustrating when they don't, especially after everyone
proselytizes
it as the solution to all their problems...

 

[Austin Ziegler]

I think the syntax checker could be a little more verbose, and I don't
think it would take much work.

It'd take more work than you think. See, you've actually got several
issues that you keep repeating in your example code, and that could be
the problem. The use of a high-quality editor and good practices will
also help.

Here's the real trick that you're forgetting that makes it a bit harder
for Ruby to deal with what you're talking about: it is not just
line-oriented, it is expression oriented. Expressions may be terminated
either by an unambiguous line ending (that is, it doesn't end with a
continuation character \, comma, period, or an operator that says that
the next item will be on the next line) or semicolons.

This is perfectly legal:

abc .
foo

although it's really bad style.

Remember, though, that:

abc
foo

is the same thing as

abc; foo

So if you have something that can expect the result of an expression
waiting open, you've got a mismatch immediately. More on that in a
moment.

The trouble I had was with a line that had an extra '.' like:
@entry = Entry.find(params[:id].

Note that you've got a syntax error on this line *without* the extra
period. You're missing a period. What this means *very* specifically is
that you're now working on an expression that should ultimately return
something and be closed with a parenthesis. Now, if you *had* a closing
parenthesis, Ruby would give you a syntax error on a multiline
expression like that for a method call (or so it seems with 1.8.2 on my
as-yet default Tiger install).

It would give you the error at the *end* of the code:

irb(main):030:0> abc.foo(1
irb(main):031:1> +2
irb(main):032:1> +3
irb(main):033:1> +4)
SyntaxError: compile error
(irb):31: syntax error
(irb):33: syntax error
from (irb):33

So Ruby's smart enough to tell you what's going on as best it can. If
you reach an "end" before your parameter list finishes, it's going to
complain about the "end", unless it's matched, like so:

irb(main):039:0> abc.foo(1
irb(main):040:1> end
SyntaxError: compile error
(irb):40: syntax error
from (irb):40
irb(main):041:0> abc.foo(1
irb(main):042:1> if
irb(main):043:2* end
irb(main):044:1> end
SyntaxError: compile error
(irb):42: syntax error
from (irb):44

As you can see, Ruby is quite a bit smarter than you think -- but it
can't tell that that parenthesis is the start of the problem. I don't
think that a C/C++ compiler could tell that, either.

and another error that i think was just a missing bracket maybe like
@entry = Entry.find(params[:id

Again, [] expects the result of an expression. These two are equivalent:

params[:id]
params.[](id)

I've used many different languages with various levels of syntax error
help, but never one this bad.

I personally doubt that. I think that you're expecting a magic bullet
because you're hearing a lot of hype about Rails, and it's not. Believe
me, Ruby's syntax errors may not be the best, but there are two points
to consider toward making them better:

1. There is no compile step as such. Compiling and execution time are
the same. This means that:
2. Increasing the amount of data kept around during parsing to report
better syntax errors increases the amount of data that must be kept
in the interpreter for your program and probably not disappear for
the life of your program.

Can they be better? Yes. But for Ruby to keep track of where each
parenthesis was opened (etc.) and report back that value instead of
where the specific ending token was found that Ruby *does* report on
would require significantly more memory during parsing.

It has been suggested that we might be able to take advantage of a
lint-like tool, but most people who have suggested such have not found
it to be worth the amount of effort as opposed to using a really good
editor with folding and syntax highlighting.

I just wrote a quick script to remove functions from 'def' to 'end' in
order and check the syntax over, thus quickly finding which function
the error is in.

A good editor would help you with that far more than having to write a
quick script.

This is a lot better than looking through a couple hundred line
controller file.

Your controller may be too large, then.

If I can do this in 10 minutes, I can't help but think it's just
laziness on the part of the syntax checker not to provide at least
hints where it thinks the problem might be.

If it doesn't carry that information around, then it can't. And it's not
laziness on the syntax checker's part. I think you're misunderstanding
what's necessary for parsing this sort of error.

I understand that ruby is flexible (I'm not not sure this is a merit
when using for a large web app), but the checker could easily check
based on 'standard' expected syntax and be more helpful about error
location 99% of the time.

Not really. The "standard" expected syntax has changed. Remember, both
of these are legal and equivalent:

foo = bar :baz
foo = barbaz)

I think you're expecting more than you're going to get from any syntax
checker except a lint-like thing.

-austin

 

[Austin Ziegler]

That raises an interesting question (at least for a noob like me). Is
there any (good) lint available for Ruby (and for RoR as a second question)?

No. Most people who want to write one tend to lose interest quickly as
they start adopting better practices.

-austin

 

[Daniel Martin]

I *think* this is at least partly due to Ruby's convenient and very
flexible syntax. IOW, the parser has no means to detect the error
line as it could be in several places - likely too many to report.
Also, this is how parsers work. From my limited insight into the
parser generation business it would require a) a different parser
generator that is much smarter or b) a lot of effort that it's not
worth IMHO.

I'll note that perl has a similarly flexible syntax, yet I don't hear
people new to perl complaining about this. Perhaps this is because
perl has more to complain about, from a newbie perspective, but I
think it's also because perl gives better syntax error messages. For
example, if you get a runaway unclosed string or regexp (//) operator,
the perl interpreter will, in the syntax error, also say something
like "possible runaway string beginning line NN".

This seems to me like it would be a simple addition to the ruby
parser, to mention the line number of the thing it's trying to close
when expecting tEND, or ']', or some other nested thing that can go
off the end of the file.

 

[M. Edward (Ed) Borasky]

It'd take more work than you think. See, you've actually got several
issues that you keep repeating in your example code, and that could be
the problem. The use of a high-quality editor and good practices will
also help.

Here's the real trick that you're forgetting that makes it a bit harder
for Ruby to deal with what you're talking about: it is not just
line-oriented, it is expression oriented. Expressions may be terminated
either by an unambiguous line ending (that is, it doesn't end with a
continuation character \, comma, period, or an operator that says that
the next item will be on the next line) or semicolons.

I think what's most troubling to the original poster is something that I
have such a problem with that I more or less refuse to use it. That is
the pseudo-flexibility offered by having "open syntactic elements"
designate that the next line continues a statement or expression, rather
than forcing an error as older languages did.

In short, if I type

a = (b +

to an interpreter, I want it to complain that I'm missing an operand and
a right paren, *not* go blindly ahead expecting

c) /
sqrt(7**15
) - (olive**oil)

Yes, that's a contrived example, but as far as I'm concerned, if end of
line terminates a "statement", then continuation of a line should
require an *explicit* continuation designator, such as "\".

 

[John Johnson]

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker ...
1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?

I get messages like this occasionally. I've found that a good editor is
the most helpful thing in finding these errors. I use TextMate on my Mac.
If I get an error like you metioned, I just give it the command to
reindent everything, then look for the "funny indent". You can also do the
same in Emacs with ruby-mode. I'm sure jEdit, Eclipse and dozens of other
editors have similar features. Auto-indenting of lines as you enter them
is helpful too. (Use ^J instead of [return] in Emacs)

Learning a new language and framework can be frustrating at times, but I
encourage you not to be too hasty in giving up and moving on.

Regards,
JJ

 

[Mat Schaffer]

Yes, that's a contrived example, but as far as I'm concerned, if
end of
line terminates a "statement", then continuation of a line should
require an *explicit* continuation designator, such as "\".

But then you open up the reverse problem with an example like

if (very long condition that hangs off the end of the page)
|| (another condition)

Raising a syntax error that you have to go back and add a \ to. I
don't know if you can say for sure which is the lesser of two evils.
Requiring ;'s to terminate statements or requiring \'s to continue
them. So ruby just avoids them both by having really flexible syntax.

As for my opinion/idea on the topic, what about adding some lint
checking when you use ruby -c ? That way a regular app wouldn't
incur the memory, but you could still get the job done without
requiring an external lint program. Obviously someone would have to
write this, and it's not gonna be me. Just trying to get ideas
circulating.

-Mat

 

[James Edward Gray II]

I don't think so. For example, you must terminate statements with
a ";" (or did that change in the latest version?) and you cannot
omit parentheses.

Both Perl and Ruby allow parenthesis to be dropped in most cases:

$ perl -we 'print("Hello world!\n")'
Hello world!
$ perl -we 'print "Hello world!\n"'
Hello world!

James Edward Gray II

 

[Austin Ziegler]

I'll note that perl has a similarly flexible syntax, yet I don't hear
people new to perl complaining about this. Perhaps this is because
perl has more to complain about, from a newbie perspective, but I
think it's also because perl gives better syntax error messages. For
example, if you get a runaway unclosed string or regexp (//) operator,
the perl interpreter will, in the syntax error, also say something
like "possible runaway string beginning line NN".

This seems to me like it would be a simple addition to the ruby
parser, to mention the line number of the thing it's trying to close
when expecting tEND, or ']', or some other nested thing that can go
off the end of the file.

Perl also has a separate compile step. (It may not seem like it, but
it does, internally.)

-austin