Ruby 1.9.2-p0: segmenfault in multithreading when using a custom Cextension

[Iñaki Baz Castillo]

Hi, I've coded a SIP protocol parser (very close to the Ragel HTTP
parser) as a Ruby C extension. It works nice (even with very high
traffic)... until now when I've tryed to add multithread logic to
handle the parser SIP message. It seems complex but I've been able to
create a very simple test script that generates the segmentfault.


To summarize, the following code DOES NOT crash:

---------------------------------------------------------------------------=
--
data =3D "INVITE sip:[email protected] SIP/2.0\r\n\r\n"
parser =3D SIP::MessageParser.new

100000.times do
msg =3D SIP::Message.new
puts "msg.object_id =3D #{msg.object_id}"
parser.execute(msg, data.to_str, 0)
parser.reset

t =3D Thread.new { }
end
---------------------------------------------------------------------------=
--


but the following code DOES crash:

---------------------------------------------------------------------------=
--
threads =3D [ ]
data =3D "INVITE sip:[email protected] SIP/2.0\r\n\r\n"
parser =3D SIP::MessageParser.new

100000.times do
msg =3D SIP::Message.new
puts "msg.object_id =3D #{msg.object_id}"
parser.execute(msg, data.to_str, 0)
parser.reset

threads << Thread.new { }
end
---------------------------------------------------------------------------=
--


The segmentfault occurs randomly in any iteration of the loop (usually
after 4900 iterations).
As you can see I do nothing within the thread. The only difference
between both cases is the fact that in the second case (the crashing
one) the thread is stored in an array.

And worse: I get segmentfault for MANY reasons (randomly too):

a) In line: threads << Thread.new { }
-- control frame ----------
c:0008 p:---- s:0032 b:0032 l:000031 d:000031 CFUNC :initialize
c:0007 p:---- s:0030 b:0030 l:000029 d:000029 CFUNC :new

b) In line: parser.execute(msg, data.to_str, 0)

c) When Message#initialize() executes "@timestamp =3D Time.now":
-- control frame ----------
c:0011 p:---- s:0039 b:0039 l:000038 d:000038 CFUNC null)
c:0010 p:---- s:0037 b:0037 l:000036 d:000036 CFUNC :now
c:0009 p:0017 s:0034 b:0034 l:000033 d:000033 METHOD /xxxx/message.rb:66

d) In line: puts "msg.object_id =3D #{msg.object_id}"



In other server with ruby 1.9.1p376 I've got, just once, a different
error (not a segmentfault in fact):

in `execute': method `hash' called on terminated
object (0x00000017c48ac8) (NotImplementedError)



Any help please? I cannot imagine the reason of this issue, neither
I'm sure that it's caused by my C extension.

Thanks a lot.


--=20
I=C3=B1aki Baz Castillo
<[email protected]>

 

[Iñaki Baz Castillo]

Hi, I've coded a SIP protocol parser (very close to the Ragel HTTP
parser) as a Ruby C extension. It works nice (even with very high
traffic)... until now when I've tryed to add multithread logic to
handle the parser SIP message. It seems complex but I've been able to
create a very simple test script that generates the segmentfault.

The segment fault also occurs with Ruby 1.9.2-p136.

--=20
I=C3=B1aki Baz Castillo
<[email protected]>

 

[Iñaki Baz Castillo]

The segment fault also occurs with Ruby 1.9.2-p136.

I've repeated the same test but using the Ragel HTTP parser (in
em-http-request gem) and it doesn't crash, so I expect some bug in my
C code

--=20
I=C3=B1aki Baz Castillo
<[email protected]>

 

[Iñaki Baz Castillo]

I've repeated the same test but using the Ragel HTTP parser (in
em-http-request gem) and it doesn't crash, so I expect some bug in my
C code

I think I've found the problem! and it could be related to a bug in Ruby:

In my SIP parser (Ruby C extension like Ragel HTTP parser) I generate
Ruby string encoded in UTF-8 using:

rb_enc_str_new(s, len, rb_utf8_encoding());

But if I change it to:

rb_str_new(s, len)

then the segment fault does NEVER occur!

--=20
I=C3=B1aki Baz Castillo
<[email protected]>

 

[Iñaki Baz Castillo]

I think I've found the problem! and it could be related to a bug in Ruby:

In my SIP parser (Ruby C extension like Ragel HTTP parser) I generate
Ruby string encoded in UTF-8 using:

=C2=A0rb_enc_str_new(s, len, rb_utf8_encoding());

But if I change it to:

=C2=A0rb_str_new(s, len)

then the segment fault does NEVER occur!

In order to demostrate the problem I've reported the bug in Ruby
tracker by providing a code that generates the segmentfault just when
using rb_enc_str_new:

http://redmine.ruby-lang.org/issues/show/4272

--=20
I=C3=B1aki Baz Castillo
<[email protected]>

 

[Suraj Kurapati]

Hello I=C3=B1aki,

2011/1/12 I=C3=B1aki Baz Castillo <[email protected]>:
In order to demostrate the problem I've reported the bug in Ruby
tracker by providing a code that generates the segmentfault just when
using rb_enc_str_new:

http://redmine.ruby-lang.org/issues/show/4272

Thanks for taking the time to file that issue, along with instructions
on how to reproduce the problem.

In my experience, C extension problems can be disheartening (especially
when nobody seems to care on the ruby-(talk|core) mailing lists) and I
know it takes a lot of energy to continue to solve or workaround those
problems.

I just wanted to say that I appreciate your efforts and I encourage you
to continue following up on the Ruby Issue Tracking System periodically.
It may take months, even a year, before you get any response, but do not
lose heart, for your efforts shows true determination and spirit.

Cheers!

-- =

Posted via http://www.ruby-forum.com/.=

 

[Iñaki Baz Castillo]

Thanks for taking the time to file that issue, along with instructions
on how to reproduce the problem.

In my experience, C extension problems can be disheartening (especially
when nobody seems to care on the ruby-(talk|core) mailing lists) and I
know it takes a lot of energy to continue to solve or workaround those
problems.

I just wanted to say that I appreciate your efforts and I encourage you
to continue following up on the Ruby Issue Tracking System periodically.
It may take months, even a year, before you get any response, but do not
lose heart, for your efforts shows true determination and spirit.

Hi Suraj, thanks a lot for your response

BTW I've tryed the test script with ruby-1.9.1-p376 and it DOES NOT crash!


--=20
I=C3=B1aki Baz Castillo
<[email protected]>