JRuby, ffi, bad file descriptor

[Daniel Berger]

Hi,

JRuby 1.3.1
OS X 10.4.9

The following code is a port of a tempfile library I wrote. Ruby 1.8.6
and 1.9.x work alright, but with JRuby I'm getting a bad file descriptor
error.

Any ideas? Here's the code:

require 'ffi'

class FileTemp < File
extend FFI::Library

attach_function 'fileno', [ointer], :int
attach_function 'fclose', [ointer], :int
attach_function 'mkstemp', [:string], :int
attach_function 'tmpfile', [], ointer
attach_function 'tmpnam', [:string], :string
attach_function 'umask', [:int], :int

TMPDIR = ENV['TEMP'] || ENV['TMP'] || '/tmp'

def initialize(delete = true, template = 'rb_file_temp_XXXXXX')
@fptr = nil

if delete
@fptr = tmpfile()
fd = fileno(@fptr)
else
begin
omask = umask(077)
fd = mkstemp(template)
raise SystemCallError, 'mkstemp()' if fd < 0
ensure
umask(omask)
end
end

super(fd, 'wb+')
end

def close
super
fclose(@fptr) if @fptr
end

def self.temp_name
TMPDIR + tmpnam(nil) << '.tmp'
end
end

if $0 == __FILE__
fh = FileTemp.new
fh.print 'hello'
fh.close
end

Result:

temp.rb:42:in `initialize': Bad file descriptor - Bad file descriptor
(Errno::EBADF)
from temp.rb:56:in `new'
from temp.rb:56

Regards,

Dan

 

[Charles Oliver Nutter]

Hmm, did you post this to JRuby dev list or ruby-ffi list yet? Could
be something simple...

You might also try getting a raw Java backtrace by passing this flag to JRu=
by:

-J-Djruby.backtrace.style=3Draw

That will show us if it's happening in JRuby's IO code or in FFI somewhere.

- Charlie

 

[Daniel Berger]

Hmm, did you post this to JRuby dev list or ruby-ffi list yet? Could
be something simple...

You might also try getting a raw Java backtrace by passing this flag to J= Ruby:

-J-Djruby.backtrace.style=3Draw

That will show us if it's happening in JRuby's IO code or in FFI somewher=

e.

I tried to post to jruby-dev, but I guess you can't post directly to
the list via google groups.

Here's the output using the flag you suggested:

daniel-bergers-computer:~/Documents/workspace/file-temp/lib/file
djberge$ jruby -J-Djruby.backtrace.style=3Draw temp.rb
Thread.java:1409:in `getStackTrace': Bad file descriptor - Bad file
descriptor (Errno::EBADF)
from RubyException.java:139:in `setBacktraceFrames'
from RaiseException.java:160:in `setException'
from RaiseException.java:72:in `<init>'
from Ruby.java:3033:in `newRaiseException'
from Ruby.java:2789:in `newErrnoEBADFError'
from RubyIO.java:895:in `initialize'
from RubyFile.java:405:in `initialize'
from org/jruby/RubyFile$i_method_0_2$RUBYFRAMEDINVOKER
$initialize.gen:-1:in `call'
from DynamicMethod.java:176:in `call'
from SuperCallSite.java:344:in `cacheAndCall'
from SuperCallSite.java:188:in `callBlock'
from SuperCallSite.java:193:in `call'
from temp.rb:31:in `method__1$RUBY$initialize'
from temp#initialize:-1:in `call'
from CachingCallSite.java:238:in `cacheAndCall'
from CachingCallSite.java:46:in `callBlock'
from CachingCallSite.java:51:in `call'
from RubyClass.java:593:in `newInstance'
from RubyIO.java:822:in `newInstance'
from org/jruby/RubyIO$s_method_0_0$RUBYFRAMEDINVOKER
$newInstance.gen:-1:in `call'
from DynamicMethod.java:160:in `call'
from DynamicMethod.java:156:in `call'
from CachingCallSite.java:258:in `cacheAndCall'
from CachingCallSite.java:77:in `call'
from temp.rb:45:in `__file__'
from temp.rb:-1:in `load'
from Ruby.java:592:in `runScript'
from Ruby.java:514:in `runNormally'
from Ruby.java:360:in `runFromMain'
from Main.java:268:in `run'
from Main.java:113:in `run'
from Main.java:97:in `main'

Regards,

Dan

 

[Charles Oliver Nutter]

Got the source of that temp.rb handy? It looks like it's passing a
bad/closed descriptor to IO.new or something...

 

[Daniel Berger]

Got the source of that temp.rb handy? It looks like it's passing a
bad/closed descriptor to IO.new or something...

require 'ffi'

class FileTemp < File
extend FFI::Library

attach_function 'fileno', [ointer], :int
attach_function 'fclose', [ointer], :int
attach_function 'mkstemp', [:string], :int
attach_function 'tmpfile', [], ointer
attach_function 'tmpnam', [:string], :string
attach_function 'umask', [:int], :int

TMPDIR =3D ENV['TEMP'] || ENV['TMP'] || '/tmp'

def initialize(delete =3D true, template =3D 'rb_file_temp_XXXXXX')
@fptr =3D nil

if delete
@fptr =3D tmpfile()
fd =3D fileno(@fptr)
else
begin
omask =3D umask(077)
fd =3D mkstemp(template)
raise SystemCallError, 'mkstemp()' if fd < 0
ensure
umask(omask)
end
end

super(fd, 'wb+')
end

def close
super
fclose(@fptr) if @fptr
end

def self.temp_name
TMPDIR + tmpnam(nil) << '.tmp'
end
end

if $0 =3D=3D __FILE__
fh =3D FileTemp.new
fh.print 'hello'
fh.close
end

 

[Charles Oliver Nutter]

Ahhh. Yeah I see the problem.

In JRuby, because we don't normally have access to the "real" file
descriptor for any IO channel, all our logic for fileno is basically
fake. We keep an artificial list of numbers that map to IO channels
and use that as our file descriptor table. In this case, you're
pulling in a real file descriptor from the system, which does not
exist in our table, so we raise an error.

In order for us to support arbitrary descriptors in our IO we'd
probably need an implementation of IO that used all low-level C APIs
rather than Java IO APIs. There's currently no (public) way to get
from a file descriptor to an IO channel in Java APIs.

So I'll divert this issue by asking: what does this get you that our
built-in tempfile support does not? We do not use MRI's tempfile.rb;
we've implemented our own on top of Java's tempfile support that
performs quite a bit better. Perhaps there's a missing feature we can
add.

It would also be worth discussing this with Wayne Meissner. It would
certainly be *nice* if we could transparently support real file
descriptors, and he may know a way to do it.

Got the source of that temp.rb handy? It looks like it's passing a
bad/closed descriptor to IO.new or something...

require 'ffi'

class FileTemp < File
=C2=A0 extend FFI::Library

=C2=A0 attach_function 'fileno', =C2=A0[ointer], :int
=C2=A0 attach_function 'fclose', =C2=A0[ointer], :int
=C2=A0 attach_function 'mkstemp', [:string], =C2=A0:int
=C2=A0 attach_function 'tmpfile', [], =C2=A0 =C2=A0 =C2=A0 =C2=A0 ointe= r
=C2=A0 attach_function 'tmpnam', =C2=A0[:string], =C2=A0:string
=C2=A0 attach_function 'umask', =C2=A0 [:int], =C2=A0 =C2=A0 :int

=C2=A0 TMPDIR =3D ENV['TEMP'] || ENV['TMP'] || '/tmp'

=C2=A0 def initialize(delete =3D true, template =3D 'rb_file_temp_XXXXXX'= )
=C2=A0 =C2=A0 =C2=A0@fptr =3D nil

=C2=A0 =C2=A0 =C2=A0if delete
=C2=A0 =C2=A0 =C2=A0 =C2=A0 @fptr =3D tmpfile()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 fd =3D fileno(@fptr)
=C2=A0 =C2=A0 =C2=A0else
=C2=A0 =C2=A0 =C2=A0 =C2=A0 begin
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0omask =3D umask(077)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fd =3D mkstemp(template)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0raise SystemCallError, 'mkstemp(= )' if fd < 0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ensure
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0umask(omask)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 end
=C2=A0 =C2=A0 =C2=A0end

=C2=A0 =C2=A0 =C2=A0super(fd, 'wb+')
=C2=A0 end

=C2=A0 def close
=C2=A0 =C2=A0 =C2=A0super
=C2=A0 =C2=A0 =C2=A0fclose(@fptr) if @fptr
=C2=A0 end

=C2=A0 def self.temp_name
=C2=A0 =C2=A0 =C2=A0TMPDIR + tmpnam(nil) << '.tmp'
=C2=A0 end
end

if $0 =3D=3D __FILE__
=C2=A0 fh =3D FileTemp.new
=C2=A0 fh.print 'hello'
=C2=A0 fh.close
end

 

[Eleanor McHugh]

Ahhh. Yeah I see the problem.

In JRuby, because we don't normally have access to the "real" file
descriptor for any IO channel, all our logic for fileno is basically
fake. We keep an artificial list of numbers that map to IO channels
and use that as our file descriptor table. In this case, you're
pulling in a real file descriptor from the system, which does not
exist in our table, so we raise an error.

In order for us to support arbitrary descriptors in our IO we'd
probably need an implementation of IO that used all low-level C APIs
rather than Java IO APIs. There's currently no (public) way to get
from a file descriptor to an IO channel in Java APIs.

So I'll divert this issue by asking: what does this get you that our
built-in tempfile support does not? We do not use MRI's tempfile.rb;
we've implemented our own on top of Java's tempfile support that
performs quite a bit better. Perhaps there's a missing feature we can
add.

It would also be worth discussing this with Wayne Meissner. It would
certainly be *nice* if we could transparently support real file
descriptors, and he may know a way to do it.

I'd be interested in support for that as well if it's possible: a lot
of fun tricks on Unix platforms rely on misappropriating file
descriptors and injecting them into IO objects.


Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

 

[Daniel Berger]

Ahhh. Yeah I see the problem.

In JRuby, because we don't normally have access to the "real" file
descriptor for any IO channel, all our logic for fileno is basically
fake. We keep an artificial list of numbers that map to IO channels
and use that as our file descriptor table. In this case, you're
pulling in a real file descriptor from the system, which does not
exist in our table, so we raise an error.

In order for us to support arbitrary descriptors in our IO we'd
probably need an implementation of IO that used all low-level C APIs
rather than Java IO APIs. There's currently no (public) way to get
from a file descriptor to an IO channel in Java APIs.

Ok, that's unfortunate.

So I'll divert this issue by asking: what does this get you that our
built-in tempfile support does not? We do not use MRI's tempfile.rb;
we've implemented our own on top of Java's tempfile support that
performs quite a bit better. Perhaps there's a missing feature we can
add.

Where is it? I looked through the source but didn't see it.

It would also be worth discussing this with Wayne Meissner. It would
certainly be *nice* if we could transparently support real file
descriptors, and he may know a way to do it.

That would be good.

Regards,

Dan

 

[Jörg W Mittag]

Where is it? I looked through the source but didn't see it.

The mapping for Ruby Classes in JRuby is Foo -> org.jruby.RubyFoo, so
Tempfile is in src/org/jruby/RubyTempfile.java:

https://GitHub.Com/JRuby/JRuby/blob/master/src/org/jruby/RubyTempfile.java

I find it a bit hard to get around the project structure, too,
especially since I'm not used to navigating big Java projects. (Also,
the fact that the directory structure has "organically" grown over,
what, 9(?) years now doesn't help, although it's still not 10% as bad
as YARV.)

jwm