Native gem roundup!

[James Gray]

Do people generally use the native json gem for performance reasons?
Does a pure-ruby version not cut it? Are there benchmarks?

The documentation for the gems says:

# == Speed Comparisons
#
# I have created some benchmark results (see the benchmarks subdir of
the
# package) for the JSON-Parser to estimate the speed up in the C
extension:
#
# JSON:ure:arser:: 28.90 calls/second
# JSON::Ext:arser:: 505.50 calls/second

The reason I like the extension is that it gets you pretty close to
Marshal's speed:

#!/usr/bin/env ruby -wKU

require "benchmark"
require "yaml"

require "rubygems"
require "json"

TESTS = 10_000
DATA = {"fields" => [1, 2.0, true, false, nil]}

Benchmark.bmbm do |results|
results.report("JSON:") { TESTS.times
{ JSON.parse(DATA.to_json) } }
results.report("Marshal:") { TESTS.times
{ Marshal.load(Marshal.dump(DATA)) } }
results.report("YAML:") { TESTS.times
{ YAML.load(DATA.to_yaml) } }
end
# >> Rehearsal --------------------------------------------
# >> JSON: 0.280000 0.030000 0.310000 ( 0.306849)
# >> Marshal: 0.230000 0.040000 0.270000 ( 0.267113)
# >> YAML: 5.090000 0.860000 5.950000 ( 5.975268)
# >> ----------------------------------- total: 6.530000sec
# >>
# >> user system total real
# >> JSON: 0.270000 0.030000 0.300000 ( 0.294798)
# >> Marshal: 0.230000 0.030000 0.260000 ( 0.264460)
# >> YAML: 5.090000 0.870000 5.960000 ( 5.965469)

With the pure version though, you are more like halfway between
Marshal and YAML:

#!/usr/bin/env ruby -wKU

require "benchmark"
require "yaml"

require "rubygems"
require "json/pure"

TESTS = 10_000
DATA = {"fields" => [1, 2.0, true, false, nil]}

Benchmark.bmbm do |results|
results.report("JSON (pure):") { TESTS.times
{ JSON.parse(DATA.to_json) } }
results.report("Marshal:") { TESTS.times
{ Marshal.load(Marshal.dump(DATA)) } }
results.report("YAML:") { TESTS.times
{ YAML.load(DATA.to_yaml) } }
end
# >> Rehearsal ------------------------------------------------
# >> JSON (pure): 2.780000 0.620000 3.400000 ( 3.409480)
# >> Marshal: 0.230000 0.040000 0.270000 ( 0.264554)
# >> YAML: 5.140000 0.880000 6.020000 ( 6.040840)
# >> --------------------------------------- total: 9.690000sec
# >>
# >> user system total real
# >> JSON (pure): 2.760000 0.630000 3.390000 ( 3.407148)
# >> Marshal: 0.230000 0.040000 0.270000 ( 0.264310)
# >> YAML: 5.130000 0.870000 6.000000 ( 6.004295)

James Edward Gray II

 

[Coey Minear]

Cool, thanks. Are you using ruby-postgres through ActiveRecord itself or
standalone in some way

- Charlie

Just through ActiveRecord.

 

[Ben Bleything]

Cool, thanks. Are you using ruby-postgres through ActiveRecord itself or
standalone in some way

I know you weren't asking me, but I'm using both ruby-postgres and the
pg library with AR, Sequel, and standalone.

Ben

 

[Tom Cloyd]

I disagree. I think the people most likely to respond to a
mailing-list request are the exact people I'm looking to reach. Did it
occur to you that perhaps I know there are download stats for all gems
that I could mine? I'm looking to gauge the importance of gems to
vocal mailing-list participants directly, by posting a general message
here and seeing who responds. And I've gotten a lot of great
responses, both on-list and off. If I wanted an accurate count of gem
downloads (no doubt inflated by older gems including dev-time
libraries as runtime dependencies) I certainly could get that. But
people coming out and saying "I really need X" is far more interesting.

So please, others, feel free to reply with what X you find important.

And Tom, thanks for your input. What native extensions do you find
useful?

- Charlie

Ah, Charlie, most interesting. Your population of interest isn't those
who USE gems but those who talk/yell/gossip/whatever about them. Unusual
notion, I'd say, but hey it's YOUR study, so to speak. I care not.
Thanks for the clarification however. And no, I didn't assume you knew
about about any download stats - I certainly didnt', but I supposed they
might exist.

My favorite gems:

date ruby-debug readline strscan logger fileutils webby

by extension, if you'll excuse my putting it that way, I'd have to
include all their dependencies, too...(e.g, rake, etc.)

Final thought: If you're interested in what people "really need", I
cannot image a better measure of that than what they really USE. And
that is NOT the same universe as what those who respond to your query
happen to talk about. I still think you have a problem - if not a
sampling problem, then a study design problem. Sorry, I'm a bit perverse
about such things. I like to get it right, even with it isn't even my
own question. Some people turn to God in times of trouble. I turn to
statistics. Never could tell the difference, really. (heh heh)

t.

--

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tom Cloyd, MS MA, LMHC - Private practice Psychotherapist
Bellingham, Washington, U.S.A: (360) 920-1226
<< (e-mail address removed) >> (email)
<< TomCloyd.com >> (website)
<< sleightmind.wordpress.com >> (mental health weblog)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

[Rados³aw Bu³at]

Gem::Specification.list.select {|s| s.extensions.size > 0 }.map {|s| s.n=
ame }.sort
=3D> ["RedCloth", "amalgalite", "bcrypt-ruby", "concurrent", "curb",
"do_sqlite3", "ebb", "em_httpserver", "erlectricity", "eventmachine",
"eventmachine_httpserver", "fastthread", "fcgi", "ferret", "ffi",
"fxruby", "god", "hpricot", "json", "linecache", "memcached",
"mongrel", "mysql", "narray", "ncurses", "neuro", "nokogiri",
"oniguruma", "passenger", "raspell", "rbtree", "rcov", "rev", "rice",
"rmagick", "ruby-debug-base", "ruby-opengl", "ruby-postgres",
"ruby-prof", "rubygame", "rubynode", "sqlite3-ruby", "swiftiply",
"termios", "thin", "thin-turbo", "unicode", "zipruby"]

At this moment I have 307 installed gems (I like playing a lot with new thi=
ngs).

--=20
Pozdrawiam

Rados=B3aw Bu=B3at
http://radarek.jogger.pl - m=F3j blog

 

[James Gray]

My favorite gems:

date ruby-debug readline strscan logger fileutils webby

Of those, I believe only ruby-debug and webby are gems. The rest are
standard libraries that ship with Ruby. Also, date, logger,
fileutils, and webby are not the native extensions Charlie is looking
for, since they don't involve C code.

James Edward Gray II

 

[Joel VanderWerf]

=> ["RedCloth", "amalgalite", "bcrypt-ruby", "concurrent", "curb",
"do_sqlite3", "ebb", "em_httpserver", "erlectricity", "eventmachine",
"eventmachine_httpserver", "fastthread", "fcgi", "ferret", "ffi",
"fxruby", "god", "hpricot", "json", "linecache", "memcached",
"mongrel", "mysql", "narray", "ncurses", "neuro", "nokogiri",
"oniguruma", "passenger", "raspell", "rbtree", "rcov", "rev", "rice",
"rmagick", "ruby-debug-base", "ruby-opengl", "ruby-postgres",
"ruby-prof", "rubygame", "rubynode", "sqlite3-ruby", "swiftiply",
"termios", "thin", "thin-turbo", "unicode", "zipruby"]

At this moment I have 307 installed gems (I like playing a lot with new things).

Not sure why this doesn't work for me:

$ ruby -e 'p Gem::Specification.list.select {|s| s.extensions.size > 0
}.map {|s| s.name }.sort'
[]

Still, looking at the output of `find -name '*.so'` on a couple of
systems, and picking out the ones that are really used:

RubyRRDtool
ruby-opengl
json
rbtree
sqlite3-ruby
amalgalite
narray

 

[Rados³aw Bu³at]

Not sure why this doesn't work for me:

$ ruby -e 'p Gem::Specification.list.select {|s| s.extensions.size > 0 }.= map
{|s| s.name }.sort'
[]

Don't ask me why but it will work if you load at least one gem by
rubygems. Try with:

ruby -rubygems -e "require 'rake'" -e 'p
Gem::Specification.list.select {|s| s.extensions.size > 0 }.map {|s|
s.name }.sort'

My previous try was from irb session so it was already loaded many gems. We=
ird.
Rubygems sucks at many things, specially in loading *all* gems specs
so it end up with eating more memory if you have insalled many gems
(as I do).

--=20
Pozdrawiam

Rados=B3aw Bu=B3at
http://radarek.jogger.pl - m=F3j blog

 

[Joel VanderWerf]

Not sure why this doesn't work for me:

$ ruby -e 'p Gem::Specification.list.select {|s| s.extensions.size > 0 }.map
{|s| s.name }.sort'
[]

Don't ask me why but it will work if you load at least one gem by
rubygems. Try with:

ruby -rubygems -e "require 'rake'" -e 'p
Gem::Specification.list.select {|s| s.extensions.size > 0 }.map {|s|
s.name }.sort'

My previous try was from irb session so it was already loaded many gems. Weird.
Rubygems sucks at many things, specially in loading *all* gems specs
so it end up with eating more memory if you have insalled many gems
(as I do).

Thanks, that works.

 

[Shawn Anderson]

Gem::Specification.list.select {|s| s.extensions.size > 0 }.map {|s|
s.name}.uniq.sort
=3D> ["RedCloth", "eventmachine", "fastthread", "ferret", "hpricot",
"mongrel", "mysql", "narray", "ncurses", "neuro", "rcov", "rfuzz",
"rmagick", "ruby-debug-base", "ruby-fann", "ruby-opengl", "ruby-prof",
"rubygame", "rubysdl", "sandbox", "sqlite3-ruby", "termios", "thin"]s.name }.uniq.size
=3D> 23

Not sure why this doesn't work for me:

$ ruby -e 'p Gem::Specification.list.select {|s| s.extensions.size > 0
}.map
{|s| s.name }.sort'
[]

Don't ask me why but it will work if you load at least one gem by
rubygems. Try with:

ruby -rubygems -e "require 'rake'" -e 'p
Gem::Specification.list.select {|s| s.extensions.size > 0 }.map {|s|
s.name }.sort'

My previous try was from irb session so it was already loaded many gems.
Weird.
Rubygems sucks at many things, specially in loading *all* gems specs
so it end up with eating more memory if you have insalled many gems
(as I do).

Thanks, that works.

 

[Tom Cloyd]

Of those, I believe only ruby-debug and webby are gems. The rest are
standard libraries that ship with Ruby. Also, date, logger,
fileutils, and webby are not the native extensions Charlie is looking
for, since they don't involve C code.

James Edward Gray II

Ah, the perils of being a part time programmer. Never quite know what
you're doing. I wondered about that 'readline' entry was doing there.

gems, packages, module, libraries, standard, non-standard,
not-so-standard, definitely NOT standard (oops, that's one of mine). I
feel lucky to muddle through, most days, and I DO do that. I actually
get quite a lot of work done with Ruby, without ever quite knowing
how.... ha!

t.

--

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tom Cloyd, MS MA, LMHC - Private practice Psychotherapist
Bellingham, Washington, U.S.A: (360) 920-1226
<< (e-mail address removed) >> (email)
<< TomCloyd.com >> (website)
<< sleightmind.wordpress.com >> (mental health weblog)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

[Joel VanderWerf]

Ah, the perils of being a part time programmer. Never quite know what
you're doing. I wondered about that 'readline' entry was doing there.

gems, packages, module, libraries, standard, non-standard,
not-so-standard, definitely NOT standard (oops, that's one of mine). I
feel lucky to muddle through, most days, and I DO do that. I actually
get quite a lot of work done with Ruby, without ever quite knowing
how.... ha!

yes, ruby is quite a "gem" (oops!)

 

[Adam Gardner]

I use Gosu for game development, which I suppose is a wrapper around the
Gosu C++ library from the same developer... I think more people use Gosu
from Ruby than from C++, though I haven't asked. Gosu as a whole is a
wrapper around OpenGL, fmod (or SDL), and Cocoa/Win32/GTK (depending on
platform).

I also used RubyCocoa to allow me to construct a native OS X application
in Ruby... I'm not sure how you'd classify that into one of the
categories... it's kind of a wrapper, but around a whole language
(Objective-C) instead of a library. Apple calls it a bridge. I'm not
sure what the utility of having it available in JRuby would be, since
most RubyCocoa apps are distributed as self-contained bundles which
don't rely on an interpreter being already present on the end user's
machine, but it'd be neat to see it.

Other than that, ruby-debug and ruby-prof, same as everyone else.

 

[Julian Leviston]

How would it be technically possible to do jruby cocoa If the machines
weren't macs?

Blog: http://random8.zenunit.com/
Learn rails: http://sensei.zenunit.com/

 

[Adam Gardner]

How would it be technically possible to do jruby cocoa If the machines
weren't macs?

Blog: http://random8.zenunit.com/
Learn rails: http://sensei.zenunit.com/

On 05/02/2009, at 10:46 AM, Adam Gardner <[email protected]>

It wouldn't be, but, so? Who says you can't use JRuby on a Mac?

Perhaps you're following the MVC pattern, and the Model and perhaps
Controller are written using normal old ruby code (perhaps with some
java stuff thrown in), and then you create a distinct View for different
platforms. Of course, I'm sure you could use Swing or AWT or something
(I'm not too familiar with Java GUI frameworks)... it'd be a lot less
effort, but with OS X GUI applications, if the interface is not done in
Cocoa or Carbon, it usually sticks out like a sore thumb.

Again, I'm not particularly sure it'd be worth the effort involved, but
I'd certainly expect it to be possible. Most of the heavy lifting in
RubyCocoa is done with libffi anyway; it'd probably be possible to
switch it to ruby-ffi.

More likely, though, the RubyCocoa project will (eventually) disappear
entirely, replaced by MacRuby, which is pretty much to Objective-C what
JRuby is to Java.

 

[Logan Barnett]

It wouldn't be, but, so? Who says you can't use JRuby on a Mac?

I certainly use JRuby on Mac!
However, I think Julian is asking if his JRuby Cocoa app will run on
things that aren't Macs. My thought is no. Cocoa would need to be
running on the other platforms for that to work. My understanding is
that Apple provides some Java libs that hook into Cocoa, which makes
JRuby development possible.

Perhaps you're following the MVC pattern, and the Model and perhaps
Controller are written using normal old ruby code (perhaps with some
java stuff thrown in), and then you create a distinct View for
different
platforms. Of course, I'm sure you could use Swing or AWT or something
(I'm not too familiar with Java GUI frameworks)... it'd be a lot less
effort, but with OS X GUI applications, if the interface is not done
in
Cocoa or Carbon, it usually sticks out like a sore thumb.

Java Swing apps (including JRuby ones, like when you write a
Monkeybars app - http://monkeybars.rubyforge.org/) by default on Macs
will render components using the System Look and Feel, which under the
hood will use the OS's native code to render buttons and other
widgets. Check out an app like FrostWire (http://www.frostwire.com/ -
Java) or JotBot (http://getjotbot.com/ - JRuby with Monkeybars)

SWT (not to be confused with AWT) is much more close to native look
and feels, but I think it marries you a lot more to a particular
platform. Check out Azureus for an SWT app in Java (http://www.azureus.com/
). Glimmer is the only JRuby SWT framework I know of - https://rubyforge.org/projects/glimmer/

Either way you go, it's possible to make apps that look good and look
native, but how many apps look like iTunes that you'd say also look
native? Pages doesn't look like iTunes, yet it looks native. What
about Colloquy? I think a native look is a fuzzy definition. The most
important thing is that you app does something useful, and does it
easily. That's not to say there isn't value in making a nice interface
that looks like it belongs on the OS with all of the other apps.

 

[Julian Leviston]

The cocoa objective-c java bridge is deprecated from my understanding,
and cocoa is copyright, so there'd be little point, tho I dunno why
jruby couldn't theoretically run the cooaruby code

Blog: http://random8.zenunit.com/
Learn rails: http://sensei.zenunit.com/

 

[Logan Barnett]

The cocoa objective-c java bridge is deprecated from my
understanding, and cocoa is copyright, so there'd be little point,
tho I dunno why jruby couldn't theoretically run the cooaruby code

I'm not sure about the deprecation, although that typically doesn't
mean much in the Java world
I don't understand what you mean by copyright (unless you mean that
you can't port it to another platform).
If the CocoaRuby stuff calls out to native C bindings, JRuby won't be
able to use it unless the bindings were made against FFI (I'm not sure
what the exact process is, but FFI extensions work great in JRuby,
whereas gems and other libs that depend on native C code don't work at
all in JRuby). I strongly believe that CocoaRuby uses native
extensions, but I could be wrong

 

[Adam Gardner]

I'm not sure about the deprecation, although that typically doesn't
mean much in the Java world
I don't understand what you mean by copyright (unless you mean that
you can't port it to another platform).
If the CocoaRuby stuff calls out to native C bindings, JRuby won't be
able to use it unless the bindings were made against FFI (I'm not sure
what the exact process is, but FFI extensions work great in JRuby,
whereas gems and other libs that depend on native C code don't work at
all in JRuby). I strongly believe that CocoaRuby uses native
extensions, but I could be wrong

I'm not 100% sure of the low-level mechanisms involved, but I do know
that RubyCocoa makes heavy use of FFI (that's libffi the C library, not
ruby-ffi), along with a Framework/Library metadata system called
BridgeSupport and a automatic generator of said metadata called
gen_bridge_metadata.rb . That being said, I believe there are also some
native extensions used in areas where metadata-based support was
particularly thorny.

 

[Adam Gardner]

Java Swing apps (including JRuby ones, like when you write a
Monkeybars app - http://monkeybars.rubyforge.org/) by default on Macs
will render components using the System Look and Feel, which under the
hood will use the OS's native code to render buttons and other
widgets. Check out an app like FrostWire (http://www.frostwire.com/ -
Java) or JotBot (http://getjotbot.com/ - JRuby with Monkeybars)

SWT (not to be confused with AWT) is much more close to native look
and feels, but I think it marries you a lot more to a particular
platform. Check out Azureus for an SWT app in Java
(http://www.azureus.com/
). Glimmer is the only JRuby SWT framework I know of -
https://rubyforge.org/projects/glimmer/

Either way you go, it's possible to make apps that look good and look
native, but how many apps look like iTunes that you'd say also look
native? Pages doesn't look like iTunes, yet it looks native. What
about Colloquy? I think a native look is a fuzzy definition. The most
important thing is that you app does something useful, and does it
easily. That's not to say there isn't value in making a nice interface
that looks like it belongs on the OS with all of the other apps.

Point conceded that the most important thing is that your app does
something useful. And you'll never hear me argue, ever, that Apple
follows their own HIG with high consistency. What I'm referring to has a
lot to do with small details in layout and behavior that do not make an
app any less useful; Azureus is the only bittorrent client I use, and
the interface, though sometimes imperfect in its rendering, does not get
in my way in any significant manner. But it remains a fact that the
differences *are* noticeable, at least in the Java applications I've
used.

Ultimately, I brought up RubyCocoa not because I thought it would be
useful in JRuby, but because it was useful for me and it hadn't been
mentioned yet so far. JRuby and RubyCocoa/MacRuby really fill entirely
different needs. RubyCocoa was very useful for me as someone who needed
to make an application which called native OS X APIs*, but who only knew
Ruby, not Objective-C. I wouldn't ever use it to make a cross-platform
application, any more than I would use the ruby-win32 api for a
cross-platform application. That's not the point of having it.

*For reference, the things RubyCocoa allowed me to do were: registering
for NSWorkspace notifications to nicely handle not bugging the user if
the reason the application is being quit is because the user is logging
out (among other things); save user preferences using the NSUserDefaults
API and thus be able to manage those prefs via WorkGroup Manager (not
that I'm currently doing so, but I now could); add auto-updating
functionality via the Sparkle framework; use Interface Builder to create
the UI (say what you will, I really like Interface Builder as a tool);
define a dynamic dock menu. Some of these things undoubtably would have
been possible without directly calling Objective-C APIs. With RubyCocoa,
none of these were not only possible, they were fairly *easy*, and I
don't even know Objective-C to begin with. Ultimately, it's a matter of
choosing the right tool for the job.