JRuby, ffi, bad file descriptor

Discussion in 'Ruby' started by Daniel Berger, Aug 1, 2009.

  1. Hi,

    JRuby 1.3.1
    OS X 10.4.9

    The following code is a port of a tempfile library I wrote. Ruby 1.8.6
    and 1.9.x work alright, but with JRuby I'm getting a bad file descriptor
    error.

    Any ideas? Here's the code:

    require 'ffi'

    class FileTemp < File
    extend FFI::Library

    attach_function 'fileno', [:pointer], :int
    attach_function 'fclose', [:pointer], :int
    attach_function 'mkstemp', [:string], :int
    attach_function 'tmpfile', [], :pointer
    attach_function 'tmpnam', [:string], :string
    attach_function 'umask', [:int], :int

    TMPDIR = ENV['TEMP'] || ENV['TMP'] || '/tmp'

    def initialize(delete = true, template = 'rb_file_temp_XXXXXX')
    @fptr = nil

    if delete
    @fptr = tmpfile()
    fd = fileno(@fptr)
    else
    begin
    omask = umask(077)
    fd = mkstemp(template)
    raise SystemCallError, 'mkstemp()' if fd < 0
    ensure
    umask(omask)
    end
    end

    super(fd, 'wb+')
    end

    def close
    super
    fclose(@fptr) if @fptr
    end

    def self.temp_name
    TMPDIR + tmpnam(nil) << '.tmp'
    end
    end

    if $0 == __FILE__
    fh = FileTemp.new
    fh.print 'hello'
    fh.close
    end

    Result:

    temp.rb:42:in `initialize': Bad file descriptor - Bad file descriptor
    (Errno::EBADF)
    from temp.rb:56:in `new'
    from temp.rb:56

    Regards,

    Dan
     
    Daniel Berger, Aug 1, 2009
    #1
    1. Advertising

  2. Hmm, did you post this to JRuby dev list or ruby-ffi list yet? Could
    be something simple...

    You might also try getting a raw Java backtrace by passing this flag to JRu=
    by:

    -J-Djruby.backtrace.style=3Draw

    That will show us if it's happening in JRuby's IO code or in FFI somewhere.

    - Charlie

    On Fri, Jul 31, 2009 at 9:54 PM, Daniel Berger<> wrote:
    > Hi,
    >
    > JRuby 1.3.1
    > OS X 10.4.9
    >
    > The following code is a port of a tempfile library I wrote. Ruby 1.8.6 an=

    d
    > 1.9.x work alright, but with JRuby I'm getting a bad file descriptor erro=

    r.
    >
    > Any ideas? Here's the code:
    >
    > require 'ffi'
    >
    > class FileTemp < File
    > =C2=A0 extend FFI::Library
    >
    > =C2=A0 attach_function 'fileno', =C2=A0[:pointer], :int
    > =C2=A0 attach_function 'fclose', =C2=A0[:pointer], :int
    > =C2=A0 attach_function 'mkstemp', [:string], =C2=A0:int
    > =C2=A0 attach_function 'tmpfile', [], =C2=A0 =C2=A0 =C2=A0 =C2=A0 :pointe=

    r
    > =C2=A0 attach_function 'tmpnam', =C2=A0[:string], =C2=A0:string
    > =C2=A0 attach_function 'umask', =C2=A0 [:int], =C2=A0 =C2=A0 :int
    >
    > =C2=A0 TMPDIR =3D ENV['TEMP'] || ENV['TMP'] || '/tmp'
    >
    > =C2=A0 def initialize(delete =3D true, template =3D 'rb_file_temp_XXXXXX'=

    )
    > =C2=A0 =C2=A0 =C2=A0@fptr =3D nil
    >
    > =C2=A0 =C2=A0 =C2=A0if delete
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 @fptr =3D tmpfile()
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 fd =3D fileno(@fptr)
    > =C2=A0 =C2=A0 =C2=A0else
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 begin
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0omask =3D umask(077)
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fd =3D mkstemp(template)
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0raise SystemCallError, 'mkstemp(=

    )' if fd < 0
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 ensure
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0umask(omask)
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 end
    > =C2=A0 =C2=A0 =C2=A0end
    >
    > =C2=A0 =C2=A0 =C2=A0super(fd, 'wb+')
    > =C2=A0 end
    >
    > =C2=A0 def close
    > =C2=A0 =C2=A0 =C2=A0super
    > =C2=A0 =C2=A0 =C2=A0fclose(@fptr) if @fptr
    > =C2=A0 end
    >
    > =C2=A0 def self.temp_name
    > =C2=A0 =C2=A0 =C2=A0TMPDIR + tmpnam(nil) << '.tmp'
    > =C2=A0 end
    > end
    >
    > if $0 =3D=3D __FILE__
    > =C2=A0 fh =3D FileTemp.new
    > =C2=A0 fh.print 'hello'
    > =C2=A0 fh.close
    > end
    >
    > Result:
    >
    > temp.rb:42:in `initialize': Bad file descriptor - Bad file descriptor
    > (Errno::EBADF)
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from temp.rb:56:in `new'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from temp.rb:56
    >
    > Regards,
    >
    > Dan
    >
    >
     
    Charles Oliver Nutter, Aug 2, 2009
    #2
    1. Advertising

  3. On Aug 2, 2:32=A0am, Charles Oliver Nutter <> wrote:
    > Hmm, did you post this to JRuby dev list or ruby-ffi list yet? Could
    > be something simple...
    >
    > You might also try getting a raw Java backtrace by passing this flag to J=

    Ruby:
    >
    > -J-Djruby.backtrace.style=3Draw
    >
    > That will show us if it's happening in JRuby's IO code or in FFI somewher=

    e.

    I tried to post to jruby-dev, but I guess you can't post directly to
    the list via google groups.

    Here's the output using the flag you suggested:

    daniel-bergers-computer:~/Documents/workspace/file-temp/lib/file
    djberge$ jruby -J-Djruby.backtrace.style=3Draw temp.rb
    Thread.java:1409:in `getStackTrace': Bad file descriptor - Bad file
    descriptor (Errno::EBADF)
    from RubyException.java:139:in `setBacktraceFrames'
    from RaiseException.java:160:in `setException'
    from RaiseException.java:72:in `<init>'
    from Ruby.java:3033:in `newRaiseException'
    from Ruby.java:2789:in `newErrnoEBADFError'
    from RubyIO.java:895:in `initialize'
    from RubyFile.java:405:in `initialize'
    from org/jruby/RubyFile$i_method_0_2$RUBYFRAMEDINVOKER
    $initialize.gen:-1:in `call'
    from DynamicMethod.java:176:in `call'
    from SuperCallSite.java:344:in `cacheAndCall'
    from SuperCallSite.java:188:in `callBlock'
    from SuperCallSite.java:193:in `call'
    from temp.rb:31:in `method__1$RUBY$initialize'
    from temp#initialize:-1:in `call'
    from CachingCallSite.java:238:in `cacheAndCall'
    from CachingCallSite.java:46:in `callBlock'
    from CachingCallSite.java:51:in `call'
    from RubyClass.java:593:in `newInstance'
    from RubyIO.java:822:in `newInstance'
    from org/jruby/RubyIO$s_method_0_0$RUBYFRAMEDINVOKER
    $newInstance.gen:-1:in `call'
    from DynamicMethod.java:160:in `call'
    from DynamicMethod.java:156:in `call'
    from CachingCallSite.java:258:in `cacheAndCall'
    from CachingCallSite.java:77:in `call'
    from temp.rb:45:in `__file__'
    from temp.rb:-1:in `load'
    from Ruby.java:592:in `runScript'
    from Ruby.java:514:in `runNormally'
    from Ruby.java:360:in `runFromMain'
    from Main.java:268:in `run'
    from Main.java:113:in `run'
    from Main.java:97:in `main'

    Regards,

    Dan
     
    Daniel Berger, Aug 3, 2009
    #3
  4. Got the source of that temp.rb handy? It looks like it's passing a
    bad/closed descriptor to IO.new or something...

    On Sun, Aug 2, 2009 at 11:17 PM, Daniel Berger<> wrote:
    >
    >
    > On Aug 2, 2:32=C2=A0am, Charles Oliver Nutter <> wrote=

    :
    >> Hmm, did you post this to JRuby dev list or ruby-ffi list yet? Could
    >> be something simple...
    >>
    >> You might also try getting a raw Java backtrace by passing this flag to =

    JRuby:
    >>
    >> -J-Djruby.backtrace.style=3Draw
    >>
    >> That will show us if it's happening in JRuby's IO code or in FFI somewhe=

    re.
    >
    > I tried to post to jruby-dev, but I guess you can't post directly to
    > the list via google groups.
    >
    > Here's the output using the flag you suggested:
    >
    > daniel-bergers-computer:~/Documents/workspace/file-temp/lib/file
    > djberge$ jruby -J-Djruby.backtrace.style=3Draw temp.rb
    > Thread.java:1409:in `getStackTrace': Bad file descriptor - Bad file
    > descriptor (Errno::EBADF)
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from RubyException.java:139:in `setBacktraceFr=

    ames'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from RaiseException.java:160:in `setException'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from RaiseException.java:72:in `<init>'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from Ruby.java:3033:in `newRaiseException'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from Ruby.java:2789:in `newErrnoEBADFError'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from RubyIO.java:895:in `initialize'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from RubyFile.java:405:in `initialize'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from org/jruby/RubyFile$i_method_0_2$RUBYFRAME=

    DINVOKER
    > $initialize.gen:-1:in `call'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from DynamicMethod.java:176:in `call'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from SuperCallSite.java:344:in `cacheAndCall'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from SuperCallSite.java:188:in `callBlock'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from SuperCallSite.java:193:in `call'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from temp.rb:31:in `method__1$RUBY$initialize'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from temp#initialize:-1:in `call'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from CachingCallSite.java:238:in `cacheAndCall=

    '
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from CachingCallSite.java:46:in `callBlock'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from CachingCallSite.java:51:in `call'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from RubyClass.java:593:in `newInstance'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from RubyIO.java:822:in `newInstance'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from org/jruby/RubyIO$s_method_0_0$RUBYFRAMEDI=

    NVOKER
    > $newInstance.gen:-1:in `call'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from DynamicMethod.java:160:in `call'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from DynamicMethod.java:156:in `call'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from CachingCallSite.java:258:in `cacheAndCall=

    '
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from CachingCallSite.java:77:in `call'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from temp.rb:45:in `__file__'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from temp.rb:-1:in `load'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from Ruby.java:592:in `runScript'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from Ruby.java:514:in `runNormally'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from Ruby.java:360:in `runFromMain'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from Main.java:268:in `run'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from Main.java:113:in `run'
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0from Main.java:97:in `main'
    >
    > Regards,
    >
    > Dan
    >
    >
     
    Charles Oliver Nutter, Aug 3, 2009
    #4
  5. On Aug 2, 10:34=A0pm, Charles Oliver Nutter <> wrote:
    > Got the source of that temp.rb handy? It looks like it's passing a
    > bad/closed descriptor to IO.new or something...


    require 'ffi'

    class FileTemp < File
    extend FFI::Library

    attach_function 'fileno', [:pointer], :int
    attach_function 'fclose', [:pointer], :int
    attach_function 'mkstemp', [:string], :int
    attach_function 'tmpfile', [], :pointer
    attach_function 'tmpnam', [:string], :string
    attach_function 'umask', [:int], :int

    TMPDIR =3D ENV['TEMP'] || ENV['TMP'] || '/tmp'

    def initialize(delete =3D true, template =3D 'rb_file_temp_XXXXXX')
    @fptr =3D nil

    if delete
    @fptr =3D tmpfile()
    fd =3D fileno(@fptr)
    else
    begin
    omask =3D umask(077)
    fd =3D mkstemp(template)
    raise SystemCallError, 'mkstemp()' if fd < 0
    ensure
    umask(omask)
    end
    end

    super(fd, 'wb+')
    end

    def close
    super
    fclose(@fptr) if @fptr
    end

    def self.temp_name
    TMPDIR + tmpnam(nil) << '.tmp'
    end
    end

    if $0 =3D=3D __FILE__
    fh =3D FileTemp.new
    fh.print 'hello'
    fh.close
    end
     
    Daniel Berger, Aug 3, 2009
    #5
  6. Ahhh. Yeah I see the problem.

    In JRuby, because we don't normally have access to the "real" file
    descriptor for any IO channel, all our logic for fileno is basically
    fake. We keep an artificial list of numbers that map to IO channels
    and use that as our file descriptor table. In this case, you're
    pulling in a real file descriptor from the system, which does not
    exist in our table, so we raise an error.

    In order for us to support arbitrary descriptors in our IO we'd
    probably need an implementation of IO that used all low-level C APIs
    rather than Java IO APIs. There's currently no (public) way to get
    from a file descriptor to an IO channel in Java APIs.

    So I'll divert this issue by asking: what does this get you that our
    built-in tempfile support does not? We do not use MRI's tempfile.rb;
    we've implemented our own on top of Java's tempfile support that
    performs quite a bit better. Perhaps there's a missing feature we can
    add.

    It would also be worth discussing this with Wayne Meissner. It would
    certainly be *nice* if we could transparently support real file
    descriptors, and he may know a way to do it.

    On Mon, Aug 3, 2009 at 7:21 AM, Daniel Berger<> wrote:
    > On Aug 2, 10:34=C2=A0pm, Charles Oliver Nutter <> wrot=

    e:
    >> Got the source of that temp.rb handy? It looks like it's passing a
    >> bad/closed descriptor to IO.new or something...

    >
    > require 'ffi'
    >
    > class FileTemp < File
    > =C2=A0 extend FFI::Library
    >
    > =C2=A0 attach_function 'fileno', =C2=A0[:pointer], :int
    > =C2=A0 attach_function 'fclose', =C2=A0[:pointer], :int
    > =C2=A0 attach_function 'mkstemp', [:string], =C2=A0:int
    > =C2=A0 attach_function 'tmpfile', [], =C2=A0 =C2=A0 =C2=A0 =C2=A0 :pointe=

    r
    > =C2=A0 attach_function 'tmpnam', =C2=A0[:string], =C2=A0:string
    > =C2=A0 attach_function 'umask', =C2=A0 [:int], =C2=A0 =C2=A0 :int
    >
    > =C2=A0 TMPDIR =3D ENV['TEMP'] || ENV['TMP'] || '/tmp'
    >
    > =C2=A0 def initialize(delete =3D true, template =3D 'rb_file_temp_XXXXXX'=

    )
    > =C2=A0 =C2=A0 =C2=A0@fptr =3D nil
    >
    > =C2=A0 =C2=A0 =C2=A0if delete
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 @fptr =3D tmpfile()
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 fd =3D fileno(@fptr)
    > =C2=A0 =C2=A0 =C2=A0else
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 begin
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0omask =3D umask(077)
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fd =3D mkstemp(template)
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0raise SystemCallError, 'mkstemp(=

    )' if fd < 0
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 ensure
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0umask(omask)
    > =C2=A0 =C2=A0 =C2=A0 =C2=A0 end
    > =C2=A0 =C2=A0 =C2=A0end
    >
    > =C2=A0 =C2=A0 =C2=A0super(fd, 'wb+')
    > =C2=A0 end
    >
    > =C2=A0 def close
    > =C2=A0 =C2=A0 =C2=A0super
    > =C2=A0 =C2=A0 =C2=A0fclose(@fptr) if @fptr
    > =C2=A0 end
    >
    > =C2=A0 def self.temp_name
    > =C2=A0 =C2=A0 =C2=A0TMPDIR + tmpnam(nil) << '.tmp'
    > =C2=A0 end
    > end
    >
    > if $0 =3D=3D __FILE__
    > =C2=A0 fh =3D FileTemp.new
    > =C2=A0 fh.print 'hello'
    > =C2=A0 fh.close
    > end
    >
    >
     
    Charles Oliver Nutter, Aug 3, 2009
    #6
  7. On 3 Aug 2009, at 14:06, Charles Oliver Nutter wrote:
    > Ahhh. Yeah I see the problem.
    >
    > In JRuby, because we don't normally have access to the "real" file
    > descriptor for any IO channel, all our logic for fileno is basically
    > fake. We keep an artificial list of numbers that map to IO channels
    > and use that as our file descriptor table. In this case, you're
    > pulling in a real file descriptor from the system, which does not
    > exist in our table, so we raise an error.
    >
    > In order for us to support arbitrary descriptors in our IO we'd
    > probably need an implementation of IO that used all low-level C APIs
    > rather than Java IO APIs. There's currently no (public) way to get
    > from a file descriptor to an IO channel in Java APIs.
    >
    > So I'll divert this issue by asking: what does this get you that our
    > built-in tempfile support does not? We do not use MRI's tempfile.rb;
    > we've implemented our own on top of Java's tempfile support that
    > performs quite a bit better. Perhaps there's a missing feature we can
    > add.
    >
    > It would also be worth discussing this with Wayne Meissner. It would
    > certainly be *nice* if we could transparently support real file
    > descriptors, and he may know a way to do it.


    I'd be interested in support for that as well if it's possible: a lot
    of fun tricks on Unix platforms rely on misappropriating file
    descriptors and injecting them into IO objects.


    Ellie

    Eleanor McHugh
    Games With Brains
    http://slides.games-with-brains.net
    ----
    raise ArgumentError unless @reality.responds_to? :reason
     
    Eleanor McHugh, Aug 3, 2009
    #7
  8. On Aug 3, 7:06=A0am, Charles Oliver Nutter <> wrote:
    > Ahhh. Yeah I see the problem.
    >
    > In JRuby, because we don't normally have access to the "real" file
    > descriptor for any IO channel, all our logic for fileno is basically
    > fake. We keep an artificial list of numbers that map to IO channels
    > and use that as our file descriptor table. In this case, you're
    > pulling in a real file descriptor from the system, which does not
    > exist in our table, so we raise an error.
    >
    > In order for us to support arbitrary descriptors in our IO we'd
    > probably need an implementation of IO that used all low-level C APIs
    > rather than Java IO APIs. There's currently no (public) way to get
    > from a file descriptor to an IO channel in Java APIs.


    Ok, that's unfortunate.

    > So I'll divert this issue by asking: what does this get you that our
    > built-in tempfile support does not? We do not use MRI's tempfile.rb;
    > we've implemented our own on top of Java's tempfile support that
    > performs quite a bit better. Perhaps there's a missing feature we can
    > add.


    Where is it? I looked through the source but didn't see it.

    > It would also be worth discussing this with Wayne Meissner. It would
    > certainly be *nice* if we could transparently support real file
    > descriptors, and he may know a way to do it.


    That would be good. :)

    Regards,

    Dan
     
    Daniel Berger, Aug 3, 2009
    #8
  9. Daniel Berger wrote:
    > On Aug 3, 7:06 am, Charles Oliver Nutter <> wrote:
    >> So I'll divert this issue by asking: what does this get you that our
    >> built-in tempfile support does not? We do not use MRI's tempfile.rb;
    >> we've implemented our own on top of Java's tempfile support that
    >> performs quite a bit better. Perhaps there's a missing feature we can
    >> add.

    > Where is it? I looked through the source but didn't see it.


    The mapping for Ruby Classes in JRuby is Foo -> org.jruby.RubyFoo, so
    Tempfile is in src/org/jruby/RubyTempfile.java:

    https://GitHub.Com/JRuby/JRuby/blob/master/src/org/jruby/RubyTempfile.java

    I find it a bit hard to get around the project structure, too,
    especially since I'm not used to navigating big Java projects. (Also,
    the fact that the directory structure has "organically" grown over,
    what, 9(?) years now doesn't help, although it's still not 10% as bad
    as YARV.)

    jwm
     
    Jörg W Mittag, Aug 3, 2009
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    9
    Views:
    744
    Abhijit Soman
    Jan 6, 2005
  2. Vishnu
    Replies:
    0
    Views:
    523
    Vishnu
    Jan 6, 2005
  3. rantingrick
    Replies:
    44
    Views:
    1,268
    Peter Pearson
    Jul 13, 2010
  4. John Croisant
    Replies:
    0
    Views:
    108
    John Croisant
    Oct 25, 2009
  5. Yary H
    Replies:
    0
    Views:
    132
    Yary H
    Oct 3, 2004
Loading...

Share This Page