Obscure syntax error

[Rolf Timmermans]

Hi all,

Today I came across a very strange scenario that raises a syntax error =
in pretty much all Ruby implementations. I have absolutely no =
explanation for the following behaviour:

x :y
#=3D> bar.rb:1:in `<main>': undefined method `x' for main:Object =
(NoMethodError)

x =3D nil
x "y"
#=3D> bar.rb:2:in `<main>': undefined method `x' for main:Object =
(NoMethodError)

x =3D nil
x :y
#=3D> bar.rb:2: syntax error, unexpected ':', expecting $end

Why does that latter case raise a syntax error during parsing? Why would =
this be invalid Ruby? Why is it only invalid when supplying a symbol =
argument, and why only when the local variable with the same name is =
already defined? I would expect all three examples above to be run and =
abort with a NoMethodError.

The behaviour is consistent in Rubinius, JRuby, Ruby 1.9 and Ruby 1.8. =
What's going on, does anyone know?


Kind regareds,

Rolf Timmermans

 

[Peter Hickman]

The format

x ...followed by something else...

means that x is a method and anything that follows it is a parameter.
Since you don't have a method called x you get "undefined method `x'"
just like it says in the error message.

When x is a reference to a variable by itself it just returns its
value, but when you put a parameter after it it looks for a method
called x, because you are passing it a parameter "y" and the only
things you pass parameters to are methods so you get the "undefined
method `x'" message again.

In the final case again you have no method called x and therefore a
parameter, in this case the :y, is unexpected. In that you don't pass
parameters to a variable only to methods.

It's good that you checked it out in all the various implementations
of Ruby but the error message was quite clear, "undefined method `x'"
means that Ruby thinks x should be a method but you don't have a
method x defined.

 

[Alex Gutteridge]

The format

x ...followed by something else...

means that x is a method and anything that follows it is a parameter.
Since you don't have a method called x you get "undefined method `x'"
just like it says in the error message.

When x is a reference to a variable by itself it just returns its
value, but when you put a parameter after it it looks for a method
called x, because you are passing it a parameter "y" and the only
things you pass parameters to are methods so you get the "undefined
method `x'" message again.

In the final case again you have no method called x and therefore a
parameter, in this case the :y, is unexpected. In that you don't pass
parameters to a variable only to methods.

I think Rolf's point is that in *both* of those last two cases there is
no method called x - the only difference being that in the first case he
tries to pass a string to the non-existent method in the second a
symbol. So why is the first (correctly) throwing a NoMethodError and the
second gives a *syntax* error?

 

[Jesús Gabriel y Galán]

The format

x ...followed by something else...

means that x is a method and anything that follows it is a parameter.
Since you don't have a method called x you get "undefined method `x'"
just like it says in the error message.

When x is a reference to a variable by itself it just returns its
value, but when you put a parameter after it it looks for a method
called x, because you are passing it a parameter "y" and the only
things you pass parameters to are methods so you get the "undefined
method `x'" message again.

In the final case again you have no method called x and therefore a
parameter, in this case the :y, is unexpected. In that you don't pass
parameters to a variable only to methods.

It's good that you checked it out in all the various implementations
of Ruby but the error message was quite clear, "undefined method `x'"
means that Ruby thinks x should be a method but you don't have a
method x defined.

What I don't understand is the difference between x "y" and x :y

irb(main):001:0> x = nil
=> nil
irb(main):002:0> x "y"
NoMethodError: undefined method `x' for main:Object
from (irb):2
from :0
irb(main):003:0> x :y
SyntaxError: compile error
(irb):3: syntax error, unexpected ':', expecting $end
from (irb):3
from :0

Why the second case is not a NoMethodError too?

Jesus.

 

[Peter Hickman]

Well, and I have to admit I am guessing here, the source is parsed in
the context of the code that precedes it. So 'x :y' is parsed when
nothing (no method or variable) called x exists so the parser is
thinking that x should be a method, because it has not been told
otherwise and the :y makes sense as a parameter to a method.

In the case of 'x = nil; x :y' the 'x = nil' means that we now have
something defined for x (a variable) which the parser can then take
into account when it gets to 'x :y'. Whereas before with 'x :y' it
assumes that x is a method because x has not been defined. But with 'x
= nil; x :y' we know that x is a variable and the parser no longer
assumes that x is a method and tries to make sense of 'a variable'
followed by 'a symbol'. Which make no sense, thus the error.

I don't know if this is the case as I do not routinely browse the Ruby
parser code but from what I know this makes sense - even if I may not
have explained it clearly.

 

[Rolf Timmermans]

I think Rolf's point is that in *both* of those last two cases there =

is no method called x - the only difference being that in the first case =
he tries to pass a string to the non-existent method in the second a =
symbol. So why is the first (correctly) throwing a NoMethodError and the =
second gives a *syntax* error?

Exactly. And to be entirely clear, the fact that the method does not =
exist doesn't even matter. If the method does exist, the same syntax =
error is raised if the first argument is a symbol. And when using =
parentheses, no syntax error occurs...

Compare:

def x(*args); end
x :y
# No error

def x(*args); end
x =3D nil
x "y"
# No error

def x(*args); end
x =3D nil
x :y
# foo.rb:3: syntax error, unexpected ':', expecting $end

def x(*args); end
x =3D nil
xy)
# No error

def x(*args); end
x =3D nil
x :y, "z"
# foo.rb:3: syntax error, unexpected ':', expecting $end

def x(*args); end
x =3D nil
x "y", :z
# No error


Kind regards,

Rolf Timmermans

 

[Colin Bartlett]

...
Exactly. And to be entirely clear, the fact that the method does not exist doesn't even matter.
If the method does exist, the same syntax error is raised if the first argument is a symbol.
And when using parentheses, no syntax error occurs
...
def x(*args); end
x :y
# No error

def x(*args); end
x = nil
x :y
# foo.rb:3: syntax error, unexpected ':', expecting $end

def x(*args); end
x = nil
xy)
# No error

I don't have an explanation for why using the symbol gives a
SyntaxError rather than a NoMethodError, but the above three of your
new examples seem to me to be the key examples, together with maybe
the following on Ruby1.9.1. Note that using "eval" with x not being
defined as a variable also raises a SyntaxError.

## two types of error message:
## NoMethodErr === #<NoMethodError: undefined method `x' for main:Object>
## SyntaxErr === #<SyntaxError: syntax error, unexpected ':', expecting ')'
y = nil
puts "** not eval; defined?( x ) #=> #{defined?( x ).inspect}"
(x :y) rescue (p $!) # NoMethodErr
(x 'y') rescue (p $!) # NoMethodErr
(x y) rescue (p $!) # NoMethodErr

puts "** eval; defined?( x ) #=> #{defined?( x ).inspect}"
begin; eval "(x :y)"; rescue SyntaxError; p $!; end # SyntaxErr
eval "(x 'y')" rescue (p $!) # NoMethodErr
eval "(x y)" rescue (p $!) # NoMethodErr

 

[Ryan Davis]

x =3D nil
x "y"
#=3D> bar.rb:2:in `<main>': undefined method `x' for main:Object = (NoMethodError)
=20
x =3D nil
x :y
#=3D> bar.rb:2: syntax error, unexpected ':', expecting $end

I think the explanation is simple: you (most likely) found a bug in the =
parser.

You should report this to ruby-core@

 

[Abinoam Jr.]

I think the explanation is simple: you (most likely) found a bug in the parser.

You should report this to ruby-core@

In my humble opinion I think it's even simpler than that.
Sorry if I'm wrong, but...

If you have an object (expression returning an object reference) what
are the syntactic constructs allowed after that?

When the variable is not defined the parser can think that it is a
method name (an undefined method error) and the "Symbol" is an
argument. But, when it is defined, it returns an object, and the colon
is not allowed syntactically in this context.
I just couldn't figure out why using constants renders different behavior.

Look

nil is an object.

Try this:

nil:y
nil :y
nil: y
nil : y
nil : y

class A; end
a = A.new

a:y
a :y
a: y
a : y

A:y (No Method)
A :y (No Method)
A: y
A : y

Only "A:y" and "A :y" raises a Method error. The others, all of them
raises Syntax error.
(tested against ruby 1.9).

Abinoam Jr.

 

[Ryan Davis]

=20
In my humble opinion I think it's even simpler than that.
...
If you have an object (expression returning an object reference) what
are the syntactic constructs allowed after that?

A symbol should be allowed anywhere any other literal is allowed.

 

[Benoit Daloze]

I think the explanation is simple: you (most likely) found a bug in the parser.

You should report this to ruby-core@

Hi,

Sorry, I already found it: http://redmine.ruby-lang.org/issues/show/3163

However, it seems to have re-appeared

Doing in a file:
p = 3
p :sym

Results in:
ruby 1.9.3dev (2010-12-28 trunk 30412) [x86_64-darwin10.5.0]
/Users/benoitdaloze/Desktop/test.rb:210: warning: `:' after local
variable is interpreted as binary operator
/Users/benoitdaloze/Desktop/test.rb:210: warning: even though it seems
like symbol literal
/Users/benoitdaloze/Desktop/test.rb:210: warning: possibly useless use
of a variable in void context
/Users/benoitdaloze/Desktop/test.rb:210: syntax error, unexpected ':',
expecting $end
p :sym
^

If I remember correctly, it worked with Nobu's patch. It must have
been reverted since.

Please post your ideas/suggestions about this bug, it is not so easy
with the operators.

So, for the record, the Symbol is not accepted because it is believed
as part of the ternary operator ? :

 

[Jimmy Zhang]

VTD-XML 2.10 is now released. It can be downloaded at
https://sourceforge.net/projects/vtd-xml/files/vtd-xml/ximpleware_2.10/.
This release includes a number of new features and enhancement.

* The core API of VTD-XML has been expanded. Users can now perform
cut/paste/insert on an empty element.
* This release also adds the support of deeper location cache support for
parsing and indexing. This feature is useful for application performance
tuning for
processing various XML documents.
* The java version also added support for processing zip and gzip files.
Direct processing of httpURL based XML is enhanced.
* Extended Java version now support Iso-8859-10~16 encoding.
* A full featured C++ port is released.
* C version of VTD-XML now make use of thread local storage to achieve
thread safety for multi-threaded application.
* There are also a number of bugs fixed. Special thanks to Jozef Aerts, John
Sillers, Chris Tornau and a number of other users for input and suggestions

 

[Ryan Davis]

VTD-XML 2.10 is now released. It can be downloaded at = https://sourceforge.net/projects/vtd-xml/files/vtd-xml/ximpleware_2.10/.
This release includes a number of new features and enhancement.
=20
* The core API of VTD-XML has been expanded. Users can now perform =

cut/paste/insert on an empty element.

* This release also adds the support of deeper location cache support =

for parsing and indexing. This feature is useful for application =
performance tuning for

processing various XML documents.
* The java version also added support for processing zip and gzip =

files. Direct processing of httpURL based XML is enhanced.

* Extended Java version now support Iso-8859-10~16 encoding.
* A full featured C++ port is released.
* C version of VTD-XML now make use of thread local storage to achieve =

thread safety for multi-threaded application.

* There are also a number of bugs fixed. Special thanks to Jozef =

Aerts, John Sillers, Chris Tornau and a number of other users for input =
and suggestions=20

It might help if you say what it is in the announcements.

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

cut/paste/insert on an empty element.
parsing and indexing. This feature is useful for application performance
tuning for
Direct processing of httpURL based XML is enhanced.
thread safety for multi-threaded application.
John Sillers, Chris Tornau and a number of other users for input and
suggestions

It might help if you say what it is in the announcements.

It would be even better if it had anything at all to do with Ruby

John

 

[Sean O'Halpin]

It might help if you say what it is in the announcements.

From the home page: "The world's only XML parser that allows you to
use XPath to process 256 GB XML documents."

I'm getting hives just thinking about that.

 

[Ryan Davis]

=20
=46rom the home page: "The world's only XML parser that allows you to
use XPath to process 256 GB XML documents."
=20
I'm getting hives just thinking about that.

I shouldn't have to click on a url to begin with... esp when that url =
has both "sourceforge" and "xml" in it. Two points against just to =
start.