RubyGems and RPA

[James Edward Gray II]

Having just read the RubyGems chapter in the Pickaxe II, I'm curious
about how it compares to RPA. I'm not trying to start a war here, but
I would like some information about their differences, if someone would
be so helpful.

Do they solve the same problem, or am I confused? Are there
differences in their design philosophy or fundamental differences in
their operation? Do they offer a similar range of packages? Can both
be installed and peacefully coexist? Any thing else I should know
about either?

Again, I'm not trying to start a fight, I'm just trying to learn more
about my options. Thanks.

James Edward Gray II

 

[Bill Atkins]

RPA and RubyGems both solve the same problem (automatic installation
of Ruby packages and their dependencies), but they have some
philosophical differences on how the software should work and where
the packages should come from. It basically comes down to your choice
of which to use.

With RubyGems, a package's author creates a .gem file and uploads it
to RubyForge. RubyGems installations can then access that .gem file
and use it to install the associated software.

With RPA, installation scripts are managed by a team at RPA (currently
just Mauricio Fernandez, AFAIK) who create the installation scripts
themselves and then offer them from their server.

There are also several differences in the software associated with
each (gem and rpa-base, respetively). RubyGems installs a gem's files
into their own directory; RPA installs them into Ruby's library
directories. RPA builds are atomic, meaning that if something goes
wrong, the installation is completely reversed. RubyGems builds are
not atomic, since the files are installed into a separate directory.

Hope that helps.

Bill Atkins

P.S. I prefer RPA

 

[Gavin Sinclair]

Having just read the RubyGems chapter in the Pickaxe II, I'm curious
about how it compares to RPA. I'm not trying to start a war here, but
I would like some information about their differences, if someone would
be so helpful.

Do they solve the same problem, or am I confused?

Same problem: management of Ruby libraries and applications. Install,
uninstall, and query.

gem install rake

rpa install rake

Pretty similar.

Are there differences in their design philosophy or fundamental
differences in their operation?

Yes. RubyGems puts each gem (package) in its own directory, rather
than installing everything into site_ruby/1.8/. This is specifically
designed so that you can have multiple versions installed, and take
advantage of that to ensure dependencies are met. For example, one
person on this list was using ActiveRecord, but could only use an old
one because of threading issues with the current release. So...

require_gem 'activerecord', '= 0.7.4'

Meant that his code would _only_ work if ActiveRecord v0.7.4 was
installed as a gem on the user's computer.

RPA is more transparent -- installs go into site_ruby and you access
them like any directly-installed package. It's got some other magic
going on, which escapes me at the moment.

With RubyGems, we try to make it as easy as possible, but the ability
to install and use multiple versions of packages puts a limit on the
transparency.

Do they offer a similar range of packages?

Yes. It's RPA's 120 to RubyGems's 100 at the moment, with significant
overlap.

Can both be installed and peacefully coexist?

Yes. That should always be the case.

Any thing else I should know about either?

The main point of RPA is quality control. It's a "Production
Archive". Eventually there will be "RPA versions" so you can access a
suite of libraries all tested to work together. It's centrally
managed.

RubyGems is a decentralised package manager that treats each gem (and
each version of each gem) as independent. If you have a RubyForge
project, you just "release" your .gem file there, and it will go into
the repository.

Again, I'm not trying to start a fight, I'm just trying to learn more
about my options. Thanks.

No worries. Hope you enjoy them. We're lucky to have two package
managers when a year ago we had none. Both were borne of the
frustration of going years without one, I guess.

Gavin

 

[why the lucky stiff]

Yes. That should always be the case.

It'd be great if there was an rpa-base gem, though. And criss-cross.

_why

 

[Chad Fowler]

It'd be great if there was an rpa-base gem, though. And criss-cross.

There was at one point, but it was screwed up (my fault) so we removed
it. Then you could use rubygems to install rpa-base, which you could
use to install rubygems, which you could use to install rpa-base....

 

[Eivind Eklund]

Association disclaimer: I'm the original architect of the RPA vision
(with Mauricio being the one having done most of the work and that
should have most of the credit). I'm also trying to help RubyGems get
the best possible system, and believe that if both systems are well
architected they will serve different roles (RPA for production,
RubyGems for zero-delay packages) and form a symbiosis.

RPA and RubyGems both solve the same problem (automatic
installation of Ruby packages and their dependencies),

Yes and no. RPA attempts to solve a broader problem than this: Making
packages available in a form where they are useable in production.
Automatic installation is one part of this, but another part of it is
to keep the packages available over time, and provide a point of
contact for patches so the packages are maintained over time. You can
see the RPA manifesto here:
http://rpa-base.rubyforge.org/wiki/wiki.cgi?RpaManifesto

In accordance with this, RPA also tries to provide various
documentation that helps its goal. Examples are
http://rpa-base.rubyforge.org/wiki/wiki.cgi?PackageAdvisor (for
finding the packages you need for production),
http://rpa-base.rubyforge.org/wiki/wiki.cgi?GoodAPIDesign (helping the
libraries we package be the best they can), and
http://rpa-base.rubyforge.org/wiki/wiki.cgi?GoodPractices (documenting
what practices makes it easy for us and other packagers).

As a courtesy to other packagers, RPA also has as a goal that it
should be possible to repackagage RPA packages in e.g. Debian or
FreeBSD.


RubyGems attempts to solve the problem of letting authors create
packages. Information about how to package etc is available at
http://rubygems.rubyforge.org/wiki/wiki.pl

For future directions, RubyGems has
http://onestepback.org/articles/rubygemsfacets/possiblerubygemsdirections.html

RubyGems are presently somewhat troublesome to repackage (due to
containing too little policy to be able to fit with requirements from
other systems - the problems are directory structure and require_gem),
but this will hopefully be fixed in the future, as a part of RubyGems
goal of being the "standard" packaging system for authors (and the
only release they need to do.) For the time being, repackagers
request that developers also release as .tar.gz or similar.

Eivind.

 

[Ara.T.Howard]

There was at one point, but it was screwed up (my fault) so we removed it.
Then you could use rubygems to install rpa-base, which you could use to
install rubygems, which you could use to install rpa-base....

you could create the first installer denial of service attack!

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it;
| and a weed grows, even though we do not love it.
| --Dogen
===============================================================================

 

[Alexander Kellett]

With RPA, installation scripts are managed by a team at RPA (currently
just Mauricio Fernandez, AFAIK) who create the installation scripts
themselves and then offer them from their server.

package authors can also publish rpa packages
on their own site via rpa's http install. e.g:

rpa install http://www.lypanov.net/ruvi-prerelease_0.4.13-1.rps



Alex

p.s: if you've got a stable version, remember "rpa remove ruvi" first
i'm too lazy to make a pkg which has a differently named script

 

[Chad Fowler]

RubyGems are presently somewhat troublesome to repackage (due to
containing too little policy to be able to fit with requirements from
other systems - the problems are directory structure and require_gem),
but this will hopefully be fixed in the future, as a part of RubyGems
goal of being the "standard" packaging system for authors (and the
only release they need to do.) For the time being, repackagers
request that developers also release as .tar.gz or similar.

I'm surprised to hear you say this. .tar.gz files contain _less_
policy than RubyGems. And less metadata. I'm sure there are _some_
scenarios where gem authors might do something that makes RPA's job a
little harder, but I would guess that in most cases, thinking about a
package management system will cause them to solve some of the
problems that would have to be solved for RPA. An example is Austin's
work on gem/rpa-ifying Ruwiki, which has required him to change things
about how Ruwiki works.

To accomodate both systems in a package-management-system-neutral way
will probably require additional "standard" abstractions that
packagers and authors can rely on just being there (included with the
ruby distribution, probably). The only way to really find out is to
create more packages.

I'd love to collect specific examples (on a Wiki--not ruby-talk) of
gem packages that do gem-specific things in such a way that it makes
things difficult for RPA. We can polish these rough edges away over
time as we move toward 1.0 of both systems. The ideal scenario is
that any RubyGem can be RPA-ified (pre QA) with a single script. And
vice versa.

Chad

 

[Eivind Eklund]

I'm surprised to hear you say this. .tar.gz files contain _less_
policy than RubyGems. And less metadata.

I was imprecise here; what I meant was a .tar.gz + standard setup.rb.

And I'm surprised that you are surprised; I've been hearing grumbling
from various packagers, not just RPA. This is one of the things that
has been having me frustrated.

To accomodate both systems in a package-management-system-neutral way
will probably require additional "standard" abstractions that
packagers and authors can rely on just being there (included with the
ruby distribution, probably). The only way to really find out is to
create more packages.

I see operating system specific packaging as extremely important, too.
There are a lot of people that just won't use Ruby programs if they
cannot be installed through the native operating system packages - and
there are a lot of packaging gruops that will not accept a
single-directory solution a la RubyGems. This may actually be most
groups - it is at least most groups that I know about.

I'm not sure if we need changes to the Ruby base or not - but
attempting to do the package conversion tools should help us find out.
RPA are working on exports to different packaging formats (we've got
RPM, and are working on.deb and FreeBSD ports at the moment - or
bitserf and Mauricio is, with feedback, help and adoptions from the
rest of us).

I'd love to collect specific examples (on a Wiki--not ruby-talk) of
gem packages that do gem-specific things in such a way that it makes
things difficult for RPA. We can polish these rough edges away over
time as we move toward 1.0 of both systems. The ideal scenario is
that any RubyGem can be RPA-ified (pre QA) with a single script. And
vice versa.

And converted to other packaging systems, including respecting their
policies. I believe this is a very important goal - something that
may be significant enough to make or break Ruby as a platform (though
I'm not sure if it is or isn't.)

If we can make this possible, it would be great - I think it could
help raise adoption of Ruby a lot. Conversely, if we make this
harder, we lower the adoption.

This is sort of orthogonal to the use of the "native Ruby" packaging
systems themselves.

Eivind.

 

[Chad Fowler]

I was imprecise here; what I meant was a .tar.gz + standard setup.rb.

And I'm surprised that you are surprised; I've been hearing grumbling
from various packagers, not just RPA. This is one of the things that
has been having me frustrated.

I'd like to hear the grumbling. Specifically, it's important to hear
of specific scenarios where something is created as a RubyGem and it
_hurt_ the ability to repackage it. As I said below, specifics are
important. "grumbling" doesn't help if we don't hear what people are
grumbling about. Even pointing out areas that might be problems isn't
nearly as valuable as pointing out a specific scenario where something
was made difficult by RubyGems.

That being said, the RubyGems developers aren't encouraging people to
only create gems. But, we _are_ encouraging everyone to make gems for
everything. With tools like Rake, it's dead easy (for most projects)
to generate .tar.gz and .gem files at the same time.

We also want to continue to hear about issues people have in making
gems--where our current system doesn't do everything it needs to for
apps and libs with specific needs. We've gotten a lot of great
feedback from the many users who have been using the system so far,
and I expect that as we see an increasing number of packages, we'll
learn that much more and make RubyGems better and better.

I see operating system specific packaging as extremely important, too.
There are a lot of people that just won't use Ruby programs if they
cannot be installed through the native operating system packages - and
there are a lot of packaging gruops that will not accept a
single-directory solution a la RubyGems. This may actually be most
groups - it is at least most groups that I know about.

In this scenario, as I imagine for RPA eventually, it would probably
work best to auto-generate (at least the skeleton for) native packages
out of gems. People in this category won't want to use "gem install
blah", but I still believe that installation and package creation can
be made easier by starting with a gem file. RubyGems has install-time
and run-time behavior. After you've extracted a gem, you can
literally just point to it directly by changing your $: somehow and
completely forego "require_gem", etc. "gem install" could be an early
step in a semi-atomated .deb or .rpm generator, for example.

I'm not sure if we need changes to the Ruby base or not - but
attempting to do the package conversion tools should help us find out.
RPA are working on exports to different packaging formats (we've got
RPM, and are working on.deb and FreeBSD ports at the moment - or
bitserf and Mauricio is, with feedback, help and adoptions from the
rest of us).


And converted to other packaging systems, including respecting their
policies. I believe this is a very important goal - something that
may be significant enough to make or break Ruby as a platform (though
I'm not sure if it is or isn't.)

If we can make this possible, it would be great - I think it could
help raise adoption of Ruby a lot. Conversely, if we make this
harder, we lower the adoption.

This is sort of orthogonal to the use of the "native Ruby" packaging
systems themselves.

Somewhat orthogonal in terms of the goal, but not very much so technically.

To summarize, I have a request: If anyone is trying to repackage
rubygems for any other system--native or not (RPA)--please keep the
feedback coming. We want to know if it was hard or even if it was
easy. Without hearing your specific stories, it's not going to get
better much faster. The rubygems-developers list on RubyForge is the
best place to send this kind of thing.

Thanks,
Chad

 

[vruz]

To summarize, I have a request: If anyone is trying to repackage

rubygems for any other system--native or not (RPA)--please keep the
feedback coming. We want to know if it was hard or even if it was
easy. Without hearing your specific stories, it's not going to get
better much faster. The rubygems-developers list on RubyForge is the
best place to send this kind of thing.

Extracted Lafcadio from the latest tarball available from Rubyforge,
and installed.
then tried to require the lib:

irb(main):001:0> require 'lafcadio'
LoadError: No such file to load -- rubygems
from /usr/local/lib/ruby/site_ruby/1.8/lafcadio/objectStore.rb:1:in
`require'
from /usr/local/lib/ruby/site_ruby/1.8/lafcadio/objectStore.rb:1
from /usr/local/lib/ruby/site_ruby/1.8/lafcadio/mock.rb:1:in `require'
from /usr/local/lib/ruby/site_ruby/1.8/lafcadio/mock.rb:1
from /usr/local/lib/ruby/site_ruby/1.8/lafcadio.rb:23:in `require'
from /usr/local/lib/ruby/site_ruby/1.8/lafcadio.rb:23
from (irb):1:in `require'
from (irb):1

This is clearly not a bug in Rubygems, but a developer that built the
tarball believing gems is part of the Ruby standard.

The developer wasn't aware, or didn't take the time to look at the
complete set of dependencies. (and didn't disclaim the rubygems
dependency)
Of course if one installs Lafcadio from gems, it won't fail !

This is also clearly one area that should be polished in order to be
repackager-friendly.
(or non-rubygems-user friendly, for that matter)

Mauricio took care to fix the unnecesary dependencies for RPA usage, so
rpa install lafcadio works after some patching.

best,
vruz

PS: I'm subscribing to the rubygems-developers maillist, just wanted
to post this now so that I don't forget mentioning it later.

 

[Richard Kilmer]

If we can make this possible, it would be great - I think it could
help raise adoption of Ruby a lot. Conversely, if we make this
harder, we lower the adoption.

I think you are ignoring a very important platform...Win32. Realize that
Ruby is a very effective language for folks that merely want to automate
their platform, and most computer users are running Win32 (unfortunate, but
true). For them, they want binary versions of libraries and a graphical
installer. RubyGems does support win32, and will include a GUI for gems
management.

-rich

 

[Austin Ziegler]

I think you are ignoring a very important platform...Win32. Realize that
Ruby is a very effective language for folks that merely want to automate
their platform, and most computer users are running Win32 (unfortunate, but
true). For them, they want binary versions of libraries and a graphical
installer. RubyGems does support win32, and will include a GUI for gems
management.

...AND, we Win32 users don't want RubyGems (or any other packaging
system) cluttering our normal installer.

-austin

 

[Chad Fowler]

...AND, we Win32 users don't want RubyGems (or any other packaging
system) cluttering our normal installer.

We had some good conversations with people (including you, I think) at
RubyConf about how an upcoming/proposed post-install hook might help
integrate with the Windows install registry.

So, it remains on the list with everything else that needs to get
done. Will probably have to wait until we finalize how post-install
processing will work.

My sense is that most Windows users really _don't_ care too much about
this, but I believe it's important enough to put in for sake of
"rightness" and for the users who actually do care.

Chad

 

[Mauricio Fernández]

I think you are ignoring a very important platform...Win32. Realize that

How so? I don't see why making RubyGems play nice with other repackagers
could hinder its ability to provide support for win32.

rpa-base does support win32 (and binary packages are supported --- in fact
*used* --- internally, but that functionality hasn't been exposed yet),
while operating in such a way that it can export to other packaging
systems, respecting the associated policies.

Ruby is a very effective language for folks that merely want to automate
their platform, and most computer users are running Win32 (unfortunate, but
true). For them, they want binary versions of libraries and a graphical
installer. RubyGems does support win32, and will include a GUI for gems
management.

Other systems support win32 too, but IMHO that's only a (rather minor)
part of the problem; it will be (much) harder to get an autobuilder in
place to create the binary packages for that platform instead of having
the authors build them. I don't know if the RubyGems project has set
such a thing as one of its goals.

I am not sure a GUI is an absolute must, even on win32. Surely nice to
have, but not something you can't do without; at least for the current
user base (_why's vision of newborns speaking Ruby will have to wait a
bit).

 

[Richard Kilmer]

...AND, we Win32 users don't want RubyGems (or any other packaging
system) cluttering our normal installer.

Yes, well, _our normal installer_ is 11MB and growing, and it does not
include every library in the Ruby community. The idea is to provide a
downloadable ruby installer with basics and a GUI to download any other
library (as a gem) you want so you can work with them. I support about 20
different folks on my project here at DARPA who use Ruby on Win32...they are
all different (from programmers to domain experts). They have all suffered
through trying to install various libraries at different times to help them
use this language I taught them and they now really enjoy. A gems-based GUI
would help tremendously for Win32 users.

-rich

 

[Curt Hibbs]

Yes, well, _our normal installer_ is 11MB and growing, and it does not
include every library in the Ruby community. The idea is to provide a
downloadable ruby installer with basics and a GUI to download any other
library (as a gem) you want so you can work with them. I support about 20
different folks on my project here at DARPA who use Ruby on
Win32...they are
all different (from programmers to domain experts). They have
all suffered
through trying to install various libraries at different times to
help them
use this language I taught them and they now really enjoy. A
gems-based GUI
would help tremendously for Win32 users.

I'm the primary persono who has been maintaining the installer lately, and
this has been my plan. It's impossible to meet everyone's needs with a
single set of packages.

The idea is that the installer would include only an essential core and
would include the RubyGems browser to allow the user to pick additional
packages to install (either right then or any time later on). I would also
like to have a simplified set of categories or groups from which the user
can choose (like "Web Application Development") so that people who are not
familiar with the packages are not overwhelmed with meaningless (to them)
choices.

Curt

 

[Gavin Sinclair]

We had some good conversations with people (including you, I think) at
RubyConf about how an upcoming/proposed post-install hook might help
integrate with the Windows install registry.

What are we actually talking about here? RubyGems is designed for the
packaging of Ruby libraries and applications. Those have no need to
use the registry. Oh.... applications, start menu, all that s..t. I
get it...

So, it remains on the list with everything else that needs to get
done. Will probably have to wait until we finalize how post-install
processing will work.

My sense is that most Windows users really _don't_ care too much about
this, but I believe it's important enough to put in for sake of
"rightness" and for the users who actually do care.

I'm getting lost again. What is it that Windows users care or not
care about?

Gavin

 

[Patrick May]

Hello,

I'm getting lost again. What is it that Windows users care or not
care about?

I use ruby on windows. I

* am frightened of integration with the registry
* am moderately excited about rubygems package management capabilities
* would like to be able to install rubygems w/out package management,
w/out rubygems

To me, the important concern is that packages and metadata be saved in
a form that is usable from many different angles. I've joined the
rubygems devel list, and I will report on some spikes I will try over
there.

This is a coding standards problem. I'll suggest a standard when I
have a better idea than the current incarnation of rubygems.

Cheers,

Patrick