[ANN] ruby bounties--list of bounties

[Roger Pack]

Fifth time's a charm (durn spam filters)

========Announcing the creation of a "ruby bounty" list=============

Purpose: connect those who want to work on side projects with those who want
to support them.

http://wiki.github.com/rdp/ruby_bounties/ruby-bounties

Is meant to be a place where you can list projects you'd like to see
completed and would be willing to help pay for.

And also a place for people looking for something to do in their spare
time to go.

(Ok, ok so it's mostly a list of projects I wish I could work on, but
don't have time for, but thought I'd try to make a community resource,
if others want to add to the list or what not.)

Enjoy.

-rp

 

[Mark T]

Sure would like to be able to afford a bounty on a Ruby (framework
agnostic) OWASP ESAPI.

http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API

MarkT
--=20
=E3=83=81=E3=82=A7=E3=83=83=E3=82=AF=E3=82=A2=E3=82=A6=E3=83=88=E3=81=8C=E3=
=80=81Jingle =E3=81=A0!
It's a Jingle Out There!

 

[Roger Pack]

Sure would like to be able to afford a bounty on a Ruby (framework

agnostic) OWASP ESAPI.

http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API

MarkT

You could add just like a $10 one to the list--I'll chip in a few bucks.

-r

 

[Charles Oliver Nutter]

Fifth time's a charm (durn spam filters)

========Announcing the creation of a "ruby bounty" list=============

I added a couple more for JRuby:

JRuby C Extension support - we have an early start at such a library
(http://github.com/wmeissner/jruby-cext) but we need more C hackers
willing to help us build it out. We will only support the "safe" C
functions that don't let you manipulate objects internals directly or
get direct access to pointers.

Pure-Java Nokogiri - Just what it sounds like...for places where
libxml isn't available or native libraries are forbidden.

Dive in

- Charlie

 

[Roger Pack]

Pure-Java Nokogiri - Just what it sounds like...for places where
libxml isn't available or native libraries are forbidden.

Dive in

Looks like pure java Nokogiri is something popular--the bounty on it has
already risen to $225

-r

 

[Charles Oliver Nutter]

Looks like pure java Nokogiri is something popular--the bounty on it has
already risen to $225

It's probably the most oft-encountered stumbling block for folks using
JRuby (these days), since Nokogiri itself has become very popular and
is now depended on by many other libraries. A pure-Java version would
never need special handling on any platform, would work on any
platform where JRuby works, and would not require native library
support at all.

I implore gem authors: think about who you might hurt with hard gem
dependencies on native extensions. At least provide an alternative
path.

- Charlie

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

Looks like pure java Nokogiri is something popular--the bounty on it has
already risen to $225

$250 now

 

[Ammar Ali]

$250 now

Does a pure java anything qualify as a ruby bounty? Or is it a java
bounty now? Maybe a ruby-envy bounty?

ammar

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

Does a pure java anything qualify as a ruby bounty? Or is it a java bounty
now? Maybe a ruby-envy bounty?

The goal is a version of Nokogiri without any native code dependencies which
runs entirely within the JVM. That doesn't mean it's written in Java or
even necessarily includes any Java code at all: it could be pure Ruby
interfacing with the native Java XML libraries.

 

[Charles Oliver Nutter]

Does a pure java anything qualify as a ruby bounty? Or is it a java bounty
now? Maybe a ruby-envy bounty?

Considering most of the bounties are about C extensions to MRI, you're
about as far off base as you could possibly be.

- Charlie

 

[Aaron Patterson]

It's probably the most oft-encountered stumbling block for folks using
JRuby (these days), since Nokogiri itself has become very popular and
is now depended on by many other libraries. A pure-Java version would
never need special handling on any platform, would work on any
platform where JRuby works, and would not require native library
support at all.

I thought FFI was supposed to solve this problem. Is it not?

I implore gem authors: think about who you might hurt with hard gem
dependencies on native extensions. At least provide an alternative
path.

I implore Ruby implementors to support the MRI C api, as it too is part
of Ruby's api. Think about who you hurt by not letting people reuse
valuable libraries written in C.

 

[Mike Dalessio]

[Note: parts of this message were removed to make it a legal post.]

It's probably the most oft-encountered stumbling block for folks using
JRuby (these days), since Nokogiri itself has become very popular and
is now depended on by many other libraries. A pure-Java version would
never need special handling on any platform, would work on any
platform where JRuby works, and would not require native library
support at all.

I implore gem authors: think about who you might hurt with hard gem
dependencies on native extensions. At least provide an alternative
path.

I'm in for $200 for a Java Nokogiri implementation. So is my partner in
crime, Aaron.

Consider it "conscience money" for all the puppies who died as a result of
us writing and maintaining the JRuby FFI port of Nokogiri.

 

[Phillip Gawlowski]

I implore Ruby implementors to support the MRI C api, as it too is part
of Ruby's api.

Either provide mswin32 (compiled with VC6), or mingw32 binaries (yes,
binaries, not the source code), or use Java.

 

[Phillip Gawlowski]

Either provide mswin32 (compiled with VC6), or mingw32 binaries (yes,
binaries, not the source code), or use Java.

"mswin32 *and* mingw32" is what I actually wanted to write. My apologies.

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

On Sun, Jan 24, 2010 at 10:29 PM, Aaron Patterson <

I thought FFI was supposed to solve this problem. Is it not?

It "solves" it to a certain degree, but when deploying in enterprise
environments it can be quite a hassle. Think: ancient versions of RHEL with
ancient versions of libxml/libxslt. That's not to mention problems with
32-bit vs 64-bit environments: for some reason the 64-bit version of RedHat
didn't package a 32-bit libxslt, but they did provide a 64-bit one.
However, we originally installed a 32-bit JRE. Eep! 64-bit JRE is an
"unofficially supported" configuration for our environment (they typically
run 32-bit JRE on 64-bit RedHat for some reason).

Nokogiri's great and we have everything working now (we had to do some
symlink hackery though). However, it would definitely streamline the
deployment process a lot if we didn't have native code dependencies, and
right now Nokogiri is the only one we have.

 

[Eleanor McHugh]

=20
I implore Ruby implementors to support the MRI C api, as it too is = part
of Ruby's api. Think about who you hurt by not letting people reuse
valuable libraries written in C.

I implore Ruby developers to write in Pure Ruby and demand all these =
Ruby Implementors solve their "performance" problems ;p


Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

 

[Charles Oliver Nutter]

I thought FFI was supposed to solve this problem. =C2=A0Is it not?

As Tony mentioned, it solves the problem just fine, except that you
still have to have the right libraries available and permission to use
them. Lots of production deployment environments for Java disallow
loading native libraries because they have a much higher likelihood of
crashing the JVM or circumventing its security policies. Google App
Engine, for example, doesn't allow loading native libraries at all,
and of course any small profile Java environments will often be unable
to use native libs as well (like Android).

I implore Ruby implementors to support the MRI C api, as it too is part
of Ruby's api. =C2=A0Think about who you hurt by not letting people reuse
valuable libraries written in C. =C2=A0

The MRI C API is terrible if you want to implement it for anything
other than MRI. It allows direct pointer access to object internals,
allows you to *write* to objects directly in memory, has absolutely no
consideration for concurrent execution, no abstraction around object
locations in a relocating GC, and no methods or standards for handling
reference lifecycle (that I am aware of). As I've stated before (and
told you) we'd be happy to support a C API that did not include all
the unsafe bits (like accessing array, string, or hash contents
*directly*) and only allows you access to objects via a handle-based
indirection mechanism. It's just a matter of time and effort to
implement it...and there's a bounty to do so

Even then, however, it still wouldn't solve the problems of loading
native libraries on secure environments.

I firmly believe the MRI's C API is the #1 thing holding it back. Why
can't we get a better GC in place? Native extensions. Why can't we
have real concurrent threading? Native extensions. Why is it a pain in
the ass to use MRI on environments without a compiler easily
available, like Windows? Native extensions. When used with plain Java
libraries, JRuby works almost identically across all platforms without
modification, without build tools present, and without segfaults. That
beats supporting the current unsafe C API any day.

- Charlie

 

[Mike Dalessio]

[Note: parts of this message were removed to make it a legal post.]

As Tony mentioned, it solves the problem just fine, except that you
still have to have the right libraries available and permission to use
them. Lots of production deployment environments for Java disallow
loading native libraries because they have a much higher likelihood of
crashing the JVM or circumventing its security policies. Google App
Engine, for example, doesn't allow loading native libraries at all,
and of course any small profile Java environments will often be unable
to use native libs as well (like Android).

Charlie, you're making a great case against using FFI.

It sounds like you're recommending that gems containing non-ruby code, like
Nokogiri, need to be written twice: once in C, and once in Java. Is that an
accurate interpretation?

(Unless, of course, you take Eleanor's advice and do everything in pure
Ruby. Hello, REXML! :-D )

 

[Charles Oliver Nutter]

Charlie, you're making a great case against using FFI.

FFI is much better than writing any C code at all, due to the
security, stability, and portability problems of writing your own C
bindings. If you are permitted to load a given library and that
library is available and you *must* use that library, FFI is the only
logical choice. But it doesn't get around the fact that you need the
library you're binding to be available and loadable on your target
platform. FFI > C bindings, but [platform-independent binary] > FFI.
And that usually means Java-based.

I should also point out that you don't necessarily have to write JVM
libraries in Java; you could also use Scala or Fan or similar
languages, and it would be just as portable (albeit a bit larger due
to the runtime dependency on those languages' runtime libraries).

But yes, at the end of the day, I believe writing stuff in a portable
binary format like JVM bytecode (or CLR bytecode) is a better choice
than writing in a language that has to be recompiled for every target
system. You ought to know that already...would I be working on JRuby
if I believed any differently?

And yes...I'd love to be able to recommend that everyone just use Ruby
for everything. But I don't think it's simply a performance issue;
there's some pretty amazing things you can get for free with a rich
static type system.

- Charlie

 

[Aaron Patterson]

FFI is much better than writing any C code at all, due to the
security, stability, and portability problems of writing your own C
bindings.

References please.

Last I checked, it was just as easy to segv from an FFI library as a C
library. Plus with FFI you don't get any benefits of compile time
checks. You can't, for example, check for #define constants.

With FFI you must:

1. Duplicate header files (see below for more problems)
2. Understand struct layouts and the sizeof() for each member
3. Do runtime checking of library features
4. Worry about weak ref maps when using void pointers (see the id2ref
problem in nokogiri)
5. Pay a runtime conversion price from ruby data types to FFI types
6. Educate users on LD_LIBRARY_PATH
7. Worry about 32bit and 64bit issues (like Tony mentioned)

The duplication of header files becomes an even larger problem if the
library you're wrapping changes it's struct layout. Where a simple
recompile would have solved the problem, now (without warning) you're getting
surprising values in your FFI program. Plus typical debugging tools
like gdb get you nowhere.

Example:

Library "foo" ships with a struct like this:

struct awesome {
float hello;
char * world;
};

Then later changes to:

struct awesome {
char * world;
float hello;
};

You wrapped the first one, upgrade the library, then boom. It doesn't work.
With a compiled program, you wouldn't care.

Unfortunately, none of the problems I've just listed off are
theoretical. I have personally run in to every one of them and can
provide you with real world examples. FFI is awesome for certain,
confined, small, stable use cases. I use FFI, and I enjoy it. But
saying that it's "the only logical choice" seems wrong.

I am curious what your experience has been, and why you haven't run in to the
same problems? How do other people overcome these issues?