Errors in line numbers reported?

[Hal Fulton]

Has anyone ever seen Ruby report incorrect line numbers
where runtime errors occur?

It's doing it to me, but it's hard to reproduce without
the (large) file I'm using.

Matz: If you're interested, I'll send you a tarball.


Hal

 

[Yukihiro Matsumoto]

Hi,

In message "Re: Errors in line numbers reported?"

|Matz: If you're interested, I'll send you a tarball.

Send me, unless the source file is larger than 8191 lines.
It's implementation restriction.

matz.

 

[Ara.T.Howard]

Hi,

In message "Re: Errors in line numbers reported?"

|Matz: If you're interested, I'll send you a tarball.

Send me, unless the source file is larger than 8191 lines.
It's implementation restriction.

matz.

you mean __LINE__ cannot go > that that?

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
===============================================================================

 

[Hal Fulton]

Hi,

In message "Re: Errors in line numbers reported?"

|Matz: If you're interested, I'll send you a tarball.

Send me, unless the source file is larger than 8191 lines.
It's implementation restriction.

OK, I never knew that.

The file is 11,756 lines (down from 22,000).

I know that's a lot, and in general is a very bad idea.

But this file is generated code. I'm generating a large
number of child classes based on data.

Is there a workaround -- perhaps the line reported is
real_line mod 8192? In that case I could just add 8K
once or twice...


Thanks,
Hal

 

[Yukihiro Matsumoto]

Hi,

In message "Re: Errors in line numbers reported?"
on Tue, 26 Oct 2004 11:34:05 +0900, (e-mail address removed) writes:

|> Send me, unless the source file is larger than 8191 lines.
|> It's implementation restriction.

|you mean __LINE__ cannot go > that that?

Unfortunately yes. It's balance between performance and my
intelligence. Maybe someone wiser than me can enlighten me.

matz.

 

[Yukihiro Matsumoto]

Hi,

In message "Re: Errors in line numbers reported?"

|> Send me, unless the source file is larger than 8191 lines.
|> It's implementation restriction.
|
|Is there a workaround -- perhaps the line reported is
|real_line mod 8192? In that case I could just add 8K
|once or twice...

It should be. It is a bad restriction. I should have fix that. But
I couldn't think of the way to fix the problem without hindering
performance.

matz.

 

[Ara.T.Howard]

Hi,

In message "Re: Errors in line numbers reported?"
on Tue, 26 Oct 2004 11:34:05 +0900, (e-mail address removed) writes:

|> Send me, unless the source file is larger than 8191 lines.
|> It's implementation restriction.

|you mean __LINE__ cannot go > that that?

Unfortunately yes. It's balance between performance and my
intelligence. Maybe someone wiser than me can enlighten me.

matz.

with a lead in like that you can't expect many patches can you? ;-)

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
===============================================================================

 

[Markus]

with a lead in like that you can't expect many patches can you? ;-)

I agree with Ara; so instead I will sign up under the always
implicit "more foolhardy" clause in the hopes of learning a little by
claiming less.

I have a patch (attached). It removes the 8k line limit on error
tracking but increases the size of the RNODE to 36 bytes (if I calculate
correctly) from the easier to align 32 bytes. I suspect that this is
where the performance hit would come from, but I have been unable to
measure any consistent speed difference between the patched and
unpatched versions, so I may not be testing the right cases. It may
also be that the savings of not having to shift the bits around makes up
for the alignment hit somewhat, but I have not tested this idea.

-- Markus

diff -u ruby-1.8.2/eval.c ruby-1.8.2-8klines/eval.c
--- ruby-1.8.2/eval.c 2004-07-27 23:32:37.000000000 -0700
+++ ruby-1.8.2-8klines/eval.c 2004-10-25 22:13:13.000000000 -0700
@@ -8296,7 +8296,7 @@

Data_Get_Struct(self, struct BLOCK, data);
if ((node = data->frame.node) || (node = data->body)) {
- len += strlen(node->nd_file) + 2 +
(SIZEOF_LONG*CHAR_BIT-NODE_LSHIFT)/3;
+ len += strlen(node->nd_file) + 2 + (SIZEOF_LONG*CHAR_BIT)/3;
str = rb_str_new(0, len);
sprintf(RSTRING(str)->ptr, "#<%s:0x%.*lx@%s:%d>", cname, w,
(VALUE)data->body,
node->nd_file, nd_line(node));
diff -u ruby-1.8.2/node.h ruby-1.8.2-8klines/node.h
--- ruby-1.8.2/node.h 2004-10-25 22:05:16.000000000 -0700
+++ ruby-1.8.2-8klines/node.h 2004-10-25 22:20:04.000000000 -0700
@@ -130,6 +130,7 @@
typedef struct RNode {
unsigned long flags;
char *nd_file;
+ unsigned long nd_line;
union {
struct RNode *node;
ID id;
@@ -159,11 +160,8 @@
#define nd_set_type(n,t) \

RNODE(n)->flags=((RNODE(n)->flags&~FL_UMASK)|(((t)<<FL_USHIFT)&FL_UMASK))

-#define NODE_LSHIFT (FL_USHIFT+8)
-#define NODE_LMASK (((long)1<<(sizeof(NODE*)*CHAR_BIT-NODE_LSHIFT))-1)
-#define nd_line(n) ((unsigned
int)(((RNODE(n))->flags>>NODE_LSHIFT)&NODE_LMASK))
-#define nd_set_line(n,l) \
-
RNODE(n)->flags=((RNODE(n)->flags&~(-1<<NODE_LSHIFT))|(((l)&NODE_LMASK)<<NODE_LSHIFT))
+#define nd_line(n) (((RNODE(n))->nd_line))
+#define nd_set_line(n,l) (RNODE(n)->nd_line=(l))

#define nd_head u1.node
#define nd_alen u2.argc

 

[ts]

M> I have a patch (attached). It removes the 8k line limit on error
M> tracking but increases the size of the RNODE to 36 bytes (if I calculate
M> correctly) from the easier to align 32 bytes.

The game is to not change the size of R struct


Guy Decoux

 

[Brian Candler]

The game is to not change the size of R struct

As long as the size of the file is recorded somewhere, then you could use
this to have a varying granularity of line numbers. e.g.

file size recorded accuracy
--------- -----------------
0..8191 exact
8192..16383 to within 2 lines
16384..24575 to within 3 lines
24576..32767 to within 4 lines
... etc

def encode_line(n, filesize)
raise "oops" if n > filesize
x = (filesize >> 13) + 1
n / x
end

def decode_line(n, filesize)
x = (filesize >> 13) + 1
(n*x)..(n*x + x - 1)
end

You get the same lack of resolution in the current system. However, if the
file size is 20000 lines (say), then an error in line 17 could be in line
17, line 8209 or line 16401 - rather than line 15, 16 or 17.

Just a thought.

Regards,

Brian.

 

[ts]

B> You get the same lack of resolution in the current system. However, if the
B> file size is 20000 lines (say), then an error in line 17 could be in line
B> 17, line 8209 or line 16401 - rather than line 15, 16 or 17.

Sincerely do you want to look at a file which has more than 8192 lines


Guy Decoux

 

[linus sellberg]

Sincerely do you want to look at a file which has more than 8192 lines

If you have generated it you might want to look at it to be able to
debug it?

 

[Yukihiro Matsumoto]

Hi,

In message "Re: Errors in line numbers reported?"

|> Sincerely do you want to look at a file which has more than 8192 lines
|
|If you have generated it you might want to look at it to be able to
|debug it?

Of course it's good thing to have. The point is whether it is worthy
for the cost of 12.5% more memory consumption per object.

matz.

 

[trans. (T. Onoma)]

On Tuesday 26 October 2004 07:05 am, Yukihiro Matsumoto wrote:
| Hi,
|
| In message "Re: Errors in line numbers reported?"
|
| on Tue, 26 Oct 2004 19:54:05 +0900, linus sellberg <[email protected]>
writes:
| |> Sincerely do you want to look at a file which has more than 8192 lines
| |>
| |
| |If you have generated it you might want to look at it to be able to
| |debug it?
|
| Of course it's good thing to have. The point is whether it is worthy
| for the cost of 12.5% more memory consumption per object.

This has come up else where and I am wondering about the effects of this.
12.5% is relative. Were talking just a couple of bytes per object, yes? But
also that means a 12.5% increase in total memory footprint? How large is that
footprint now, in general?

Also are there any speed issues with this increase?

Thanks,
T.

 

[Ara.T.Howard]

M> I have a patch (attached). It removes the 8k line limit on error
M> tracking but increases the size of the RNODE to 36 bytes (if I calculate
M> correctly) from the easier to align 32 bytes.

The game is to not change the size of R struct

how about storing the line in some word aligned bytes __behind__ nd_file?

nd_file = "a.rb\0xxxx";

the nd_line(n) would be something like

(unsigned long)*((unsinged long *)(nd_file + ((strlen(nd_file) + 4) % 4) - 1));

you get the idea...

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
===============================================================================

 

[Florian Gross]

M> I have a patch (attached). It removes the 8k line limit on error
M> tracking but increases the size of the RNODE to 36 bytes (if I calculate
M> correctly) from the easier to align 32 bytes.

The game is to not change the size of R struct

What about using the first 4 bytes of nd_file for storing the line
number? Would this indirection be too slow?

 

[Markus]

I agree with Ara; so instead I will sign up under the always
implicit "more foolhardy" clause in the hopes of learning a little by
claiming less.

I have a patch (attached). It removes the 8k line limit on error
tracking but increases the size of the RNODE to 36 bytes (if I calculate
correctly) from the easier to align 32 bytes. I suspect that this is
where the performance hit would come from, but I have been unable to
measure any consistent speed difference between the patched and
unpatched versions, so I may not be testing the right cases. It may
also be that the savings of not having to shift the bits around makes up
for the alignment hit somewhat, but I have not tested this idea.

The game is to not change the size of R struct

Yes, I see that. That is why I made a point of calling out the
fact that is _does_ change the size. But what I am trying to learn is
why we care. I assumed (on the bases of comments about speed and my
knowledge of how processors work) that it was an alignment problem--that
changing the size would break the alignment and make it slower. But I
was not able to measure any difference.


On Tue, 2004-10-26 at 04:21, trans. (T. Onoma) wrote:

This has come up else where and I am wondering about the effects of
this. 12.5% is relative. Were talking just a couple of bytes per
object, yes? But also that means a 12.5% increase in total memory
footprint? How large is that footprint now, in general?

I think the 12.5% increase only applies to the generated code size,
which I suspect is only a fraction of the total memory footprint; I
think it comes to 4 bytes per node, which (estimating ~5 nodes per line,
average) comes to around 160K for a 8k line program.

Also are there any speed issues with this increase?

Probably, but I have not learned how to detect them. In my
(admittedly simple) tests, there is no consistent difference between the
patched and unpatched versions,

how about storing the line in some word aligned bytes __behind__ nd_file?

nd_file = "a.rb\0xxxx";

the nd_line(n) would be something like

(unsigned long)*((unsinged long *)(nd_file + ((strlen(nd_file) + 4) % 4) - 1));

I thought of this. The problem is that it makes the memory use
increase much faster, since you now have to duplicate the whole file
name for each node (before, it is shared).

What would, I think, work better is to store a line number OFFSET
with the file name (but as an unsigned integer preceding the file name
rather than as text following it); this would initially be zero, but a
new name-and-offset structure would have to be allocated every 8k lines
(adding ~100*2*(file_name_length+4)/(8k*32+file_name_length) ~= 80k/256k
~= 0.3% to the memory footprint).

I will try to work up a patch that uses this method later today,
unless someone else wants to try their hand at it.

-- Markus


P.S. I am still interested if anyone can show a measurable speed impact
from 36 byte RNODEs ve the 32 bytes. I'm not so interested in the
arguments that it should, in theory, make things slower--which I can
already clearly see for myself--but rather want to discover under what
circumstances this could actually be demonstrated.

 

[Markus]

What about using the first 4 bytes of nd_file for storing the line
number? Would this indirection be too slow?

The problem is that that would require replicating the filename for
each line, approximately doubling the memory footprint. But see my
discussion at the end of my previous post for a hybrid solution that I
think would work.

-- Markus

 

[Yukihiro Matsumoto]

Hi,

In message "Re: Errors in line numbers reported?"

|This has come up else where and I am wondering about the effects of this.
|12.5% is relative. Were talking just a couple of bytes per object, yes? But
|also that means a 12.5% increase in total memory footprint? How large is that
|footprint now, in general?

4 more bytes per object.

|Also are there any speed issues with this increase?

I guess speed would not be an issue here.

matz.

 

[ts]

M> I think the 12.5% increase only applies to the generated code size,
M> which I suspect is only a fraction of the total memory footprint; I
M> think it comes to 4 bytes per node, which (estimating ~5 nodes per line,
M> average) comes to around 160K for a 8k line program.

rb_newobj() in gc.c

Guy Decoux