[ANN] rpa-base 0.2.0

[Mauricio Fernández]

rpa-base 0.2.0 is now available at http://rpa-base.rubyforge.org .
Many of the most popular libraries/applications as per Rubyforge
statistics (rails, rake, redcloth, activerecord, sqlite, log4r, copland,
ruvi, to name a few) have been packaged for use with rpa-base 0.2.0.

Foreword
--------

The Ruby Production Archive (RPA) will provide packages of Ruby
libraries and programs in a form that allows production use, engineered
through a stringent process resembling FreeBSD's or Debian's.

rpa-base is a port/package manager designed to support RPA. Its scope and
purposes are different to those of other systems like RubyGems.

Status
------

Please keep in mind that RPA is at an embryonic stage; this means that
it is impossible to commit to all the long term goals stated in the
manifesto (http://rpa-base.rubyforge.org/wiki/wiki.cgi?RpaManifesto) at
the moment. This doesn't make rpa-base any less usable however.

rpa-base requires Ruby 1.8.[12] (certainly 1.8 at least, it might work
on 1.8.0); it has been tested on several Linux distributions (Debian,
Fedora, older RH, Gentoo, etc), FreeBSD, DragonFly BSD, Mac OSX, win32
(cygwin, 'pragmatic installer', WinXP and 2K).

rpa-base is fairly stable at this stage; it has been tested on several
platforms during the last 2 months. Since rpa-base can self-update,
eventual bugs discovered through the intensive testing associated with
a public release could be solved easily by upgrading rpa-base using
rpa-base itself.

We would appreciate any feedback on rpa-base.
A mailing list has been set up for that purpose:
http://rubyforge.org/mailman/listinfo/rpa-base-testers


Features
========

rpa-base is a port/package manager designed to support RPA's client-side
package management. You can think of it as RPA's apt-get + dpkg. It
features the following as of 0.2.0:

* strong dependency management: rpa-base installs dependencies as needed,
keeps track of reverse dependencies on uninstall, and will remove no
longer needed dependencies
* atomic (de)installs: operations on the local RPA installation are atomic
transactions; the system has been designed to survive ruby crashes (OS
crashes too on POSIX systems)
* parallel installs: you can install several ports in parallel; building
will be parallelized and the final phase will be serialized properly
* self-hosting: rpa-base installs and updates itself
* modular, extensible design: the 2-phase install is similar to FreeBSD and
Debian's package creation; rpa-base packages need not be restricted
to installing everything under a single directory ("1 package, 1 dir"
paradigm)
* rdoc integration: RDoc documentation for libraries is generated at install
time (currently disabled on win32)
* ri integration: ri data files are generated for all the libraries managed
by RPA; you can access this information with ri-rpa
* handling C extensions: if you have the required C toolchain, rpa-base can
compile extensions as needed
* unit testing: when a library is installed, its unit tests are run; the
installation is canceled if they don't pass



Several of the above features are illustrated in the screenshots and
animations available at
http://rpa-base.rubyforge.org/wiki/wiki.cgi?Rpa_Base_In_Action

Limitations:
============
A number of features have been pushed back to 0.3.0:
* full support for binary platform-specific packages
* signed packages/ports
* system-wide configuration system
* better user interface

In practice, the first one is the most limiting at the moment since it means
that win32 users in particular need a working C toolchain to install
extensions. This will soon be addressed.


RPA needs your help
===================
RPA is an ambitious project in need for developers. These are some of the
areas that need to be worked on:
* packaging (new software and package maintenance)
* setting up a permanent repository infrastructure
* cross-compilation and build automation
* website development (should provide package indexes, QA section,
bugtracking, etc)

Please contact me at <[email protected]> (adding RPA to the subject
will help get it past the spam filtering or via IRC, batsman @
#ruby-lang on freenode.net if you have any interest or for any additional
information on RPA/rpa-base and their goals.

 

[Joel VanderWerf]

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

Versions:

# rpa -v
rpa (rpa-base 0.2.0-21) RPA 0.0
# ruby -v
ruby 1.9.0 (2004-07-29) [i686-linux]

 

[Mauricio Fernández]

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

Versions:

# rpa -v
rpa (rpa-base 0.2.0-21) RPA 0.0
# ruby -v
ruby 1.9.0 (2004-07-29) [i686-linux]

Can't reproduce with
ruby 1.9.0 (2004-05-19) [i686-linux]

I'll upgrade to a later CVS and try again...

 

[Mauricio Fernández]

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

This is very strange, cause

batsman@tux-chan:/tmp/ruby1.9/lib$ grep -r "unrecognized" *
getoptlong.rb: set_error(InvalidOption, "unrecognized option `#{argument}'")
open-uri.rb: raise ArgumentError, "unrecognized option: #{k}"

batsman@tux-chan:~/src/rpa/rpa-base$ grep -r "unrecognized" *
lib/rpa/.svn/text-base/open-uri.rb.svn-base: raise ArgumentError, "unrecognized option: #{k}"
lib/rpa/open-uri.rb: raise ArgumentError, "unrecognized option: #{k}"

I don't know where that "unrecognized arguments" comes from.

Versions:

# rpa -v
rpa (rpa-base 0.2.0-21) RPA 0.0
# ruby -v
ruby 1.9.0 (2004-07-29) [i686-linux]

Can't reproduce it

batsman@tux-chan:/tmp/ruby1.9$ ~/ruby1.9/bin/ruby -v
ruby 1.9.0 (2004-08-09) [i686-linux]
batsman@tux-chan:/tmp/ruby1.9$ ~/ruby1.9/bin/rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Building ri-rpa (0.1-4).
Building dependencies rdoc.
Getting port rdoc from
http://rpa-base.rubyforge.org/ports/rdoc_1.8.1-3.rps.
100% [========================================] 94208 bytes
Building rdoc (1.8.1-3).
Calculating MD5 digests.
Building package in rdoc_1.8.1-3_i686-pc-linux-gnu.rpa.
Fixed shebang in rpa/tmp/bin/ri-rpa.
Calculating MD5 digests.
Building package in ri-rpa_0.1-4_i686-pc-linux-gnu.rpa.
Installing ri-rpa
Reusing cached package
/home/batsman/ruby1.9/lib/ruby/rpa0.0/packages/ri-rpa_0.1-4_i686-pc-linux-gnu.rpa.
Installing dependencies rdoc.
Starting lightweight (metadata only) transaction for ri-rpa
Reusing cached package
/home/batsman/ruby1.9/lib/ruby/rpa0.0/packages/rdoc_1.8.1-3_i686-pc-linux-gnu.rpa.
Starting lightweight (metadata only) transaction for rdoc
Checking for file conflicts in rdoc.
Starting transaction for rdoc
Package
/home/batsman/ruby1.9/lib/ruby/rpa0.0/packages/rdoc_1.8.1-3_i686-pc-linux-gnu.rpa unpacked.
Finished transaction for rdoc
Finished lightweight (metadata only) transaction for rdoc
Checking for file conflicts in ri-rpa.
Starting transaction for ri-rpa
Package
/home/batsman/ruby1.9/lib/ruby/rpa0.0/packages/ri-rpa_0.1-4_i686-pc-linux-gnu.rpa unpacked.
Finished transaction for ri-rpa
Starting lightweight (metadata only) transaction for ri-rpa
Finished lightweight (metadata only) transaction for ri-rpa
Finished lightweight (metadata only) transaction for ri-rpa
Committed changes

 

[Joel VanderWerf]

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

This is very strange, cause

batsman@tux-chan:/tmp/ruby1.9/lib$ grep -r "unrecognized" *
getoptlong.rb: set_error(InvalidOption, "unrecognized option `#{argument}'")
open-uri.rb: raise ArgumentError, "unrecognized option: #{k}"

batsman@tux-chan:~/src/rpa/rpa-base$ grep -r "unrecognized" *
lib/rpa/.svn/text-base/open-uri.rb.svn-base: raise ArgumentError, "unrecognized option: #{k}"
lib/rpa/open-uri.rb: raise ArgumentError, "unrecognized option: #{k}"

I don't know where that "unrecognized arguments" comes from.

Tried to run it in gdb to see where it was when exit() was called, but
running it in gdb "fixed" the problem:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting
# gdb ruby
GNU gdb 5.3-22mdk (Mandrake Linux)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i586-mandrake-linux-gnu"...
(gdb) r /usr/local/bin/rpa install ri-rpa
Starting program: /home/local/bin/ruby /usr/local/bin/rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Building ri-rpa (0.1-4).
Building dependencies rdoc.
Getting port rdoc from http://rpa-base.rubyforge.org/ports/rdoc_1.8.1-3.rps.
100% [========================================] 94208 bytes
Building rdoc (1.8.1-3).



This experience is repeated with other packages (I tried net-ssh):

# ruby /usr/local/bin/rpa install net-ssh
Installing ports
Getting port net-ssh from
http://rpa-base.rubyforge.org/ports/net-ssh_0.0.5-1.rp
100% [========================================] 117760 bytes
Error: unrecognized arguments net-ssh aborting
# gdb ruby
GNU gdb 5.3-22mdk (Mandrake Linux)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i586-mandrake-linux-gnu"...
(gdb) r /usr/local/bin/rpa install net-ssh
Starting program: /home/local/bin/ruby /usr/local/bin/rpa install net-ssh
Installing ports
Getting port net-ssh from
http://rpa-base.rubyforge.org/ports/net-ssh_0.0.5-1.rp
100% [========================================] 117760 bytes
Building net-ssh (0.0.5-1).
Generating RI data files.
Generating RDoc HTML documentation.
...

What could be different about running in gdb? My normal shell is zsh,
but bash has the same effect.

Once a package is installed, doing "rpa install" on it again succeeds
(detecting no changes).

RPA seems to work fine for me on windows, anyway.

 

[Mauricio Fernández]

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

Could you try to run it with --debug, to get a backtrace?
Alternatively, you could try to put a debug statement in rpafrontend.rb around

307 rescue => e
308 $stderr.puts "Error: #{e} aborting"
309 raise if @options.debug
310 exit 6

(I think this is where the error is detected).

Tried to run it in gdb to see where it was when exit() was called, but
running it in gdb "fixed" the problem:

This experience is repeated with other packages (I tried net-ssh):

What could be different about running in gdb? My normal shell is zsh,
but bash has the same effect.

Once a package is installed, doing "rpa install" on it again succeeds
(detecting no changes).

This makes me think it is somehow related to the ri/rdoc integration
and some shell quoting issue.

RPA seems to work fine for me on windows, anyway.

That's good news at least, win32 was such a PITA to support from the
beginning ;-)

 

[Joel VanderWerf]

This seems to happen for anything I try to install with rpa:

# rpa install ri-rpa
Installing ports
Getting port ri-rpa from
http://rpa-base.rubyforge.org/ports/ri-rpa_0.1-4.rps.
100% [========================================] 264704 bytes
Error: unrecognized arguments ri-rpa aborting

Found it... there was an install.rb on my lib path that was being loaded
instead of the real install.rb. The problem may be in rpa/base.rb, line 256:

begin
Dir.chdir(destdir){ load "install.rb" }
rescue Exception
RPA::Install::AtExitHandler.cancel
raise
end

If install.rb is not found in destdir, $LOAD_PATH is searched, and who
knows what it'll find there. It might be better to use
File.expand_path("install.rb") in this case.

 

[GGarramuno]

This makes me think it is somehow related to the ri/rdoc integration
and some shell quoting issue.

Mauricio, I tried your rpa-base and I admit I loved it, as it really
reminded me of the simplicity of perl's cpan installs.

I do have a minor bug and 3 questions, thou:

a) First the minor bug. It seems documentation creation requires the
downloading of rpa-ri first, which is fine. However, if you download
other modules first and only rpa-ri later, documentation of those
previous modules is not created and requires uninstalling and
re-installing them again.

b) Also, how exactly do I get ri/rpa-ri to show documentation for the
installed packages? I installed my getopt-declare module and it said
it created the ri docs for it, but I don't seem to be able to access
them.

c) As an indirect contributor of a module to rpa-base, how do I make
sure rpa-base is indeed using the latest version of my module? That
is, how do the maintainer(s) of rpa-base get notified of the existance
of a new version of my library?

d) As someone who might contribute other libraries in the future, how
does rpa-base and rubygems interact (if at all). I really would like
to stick to a single format for installs which would hopefully remain
compatible.

 

[Mauricio Fernández]

Found it... there was an install.rb on my lib path that was being loaded
instead of the real install.rb. The problem may be in rpa/base.rb, line 256:

begin
Dir.chdir(destdir){ load "install.rb" }
rescue Exception
RPA::Install::AtExitHandler.cancel
raise
end

If install.rb is not found in destdir, $LOAD_PATH is searched, and who
knows what it'll find there. It might be better to use
File.expand_path("install.rb") in this case.

Thanks a lot for squashing this bug
I wonder why install.rb couldn't be found in the tempdir, though...

 

[Mauricio Fernández]

Mauricio, I tried your rpa-base and I admit I loved it, as it really
reminded me of the simplicity of perl's cpan installs.

I have to say that cpan.pm is NOT the model rpa-base is based on. The
major sources of inspiration are FreeBSD and Debian: I found that
on a technical level, there's very little to be stolen from cpan.pm
(I believe that both RubyGems and rpa-base are more powerful than
cpan.pm); I'm more interested in the CPAN process as a whole, but I still
believe there's more to be learned from FreeBSD/Debian than from CPAN.

I do have a minor bug and 3 questions, thou:

a) First the minor bug. It seems documentation creation requires the
downloading of rpa-ri first, which is fine. However, if you download
other modules first and only rpa-ri later, documentation of those
previous modules is not created and requires uninstalling and
re-installing them again.

This is not accurate Documentation is always generated, it is not
required to have ri-rpa. You need the latter to access the generated ri
information with

$ ri-rpa -p portname

from the command line. (Note that I don't like the -p portname
mechanism, and will try to solve the issue in a more general way.)
So no need to uninstall/reinstall in general.

Not all libraries/apps have rdoc/ri documentation, but for those that
do, you should be able to find it under
$prefix/share/doc/rpa0.0/portname/rdoc

Now, if you're often getting messages like
WARNING: RI datafile generation failed
when installing a port (regardless of whether you have ri-rpa installed
or not, since as I said it is not needed to generate the docs actually),
this means there's a problem with your rdoc installation: make sure you
have it installed (apt-get install rdoc1.8 under Debian) and that it
does indeed work. In the future, I might ship rdoc with rpa-base and/or
ri-rpa, to prevent such problems.

b) Also, how exactly do I get ri/rpa-ri to show documentation for the
installed packages? I installed my getopt-declare module and it said
it created the ri docs for it, but I don't seem to be able to access
them.

The rdoc information should be in
$prefix/share/doc/rpa0.0/getopt-declare/rdoc

You can access the ri information as follows:

$ ri-rpa -p getopt-declare -T DelimScanner
---------------------------------------------------- Class: DelimScanner
A derivative of StringScanner that can scan for delimited
constructs in addition to regular expressions.

------------------------------------------------------------------------

[...]

Note that there's a number of usability issues with the modified ri I
distributed as 'ri-rpa'; I am planning to work on them and eventually
push a new release through the usual channels (rpa update && rpa install
ri-rpa).

c) As an indirect contributor of a module to rpa-base, how do I make
sure rpa-base is indeed using the latest version of my module? That
is, how do the maintainer(s) of rpa-base get notified of the existance
of a new version of my library?

I was hoping that a combination of the following would work:
* regular inspection of release announcements on ruby-talk, rubyforge
and RAA
* direct notification from the author
* requests from the users

I have yet to create an 'official channel' for the second; I think I'll
set up a rpa-sw-authors mailing list in Rubyforge, but for the time
being you can just drop me a note (RPA in the subject to get to the
appropriate mailbox quickly

d) As someone who might contribute other libraries in the future, how
does rpa-base and rubygems interact (if at all). I really would like
to stick to a single format for installs which would hopefully remain
compatible.

There are a number of differences (the most important one being the
approach to the versioning problem), as explained in
http://rpa-base.rubyforge.org/wiki/wiki.cgi?Rpa_FAQ

I am not sure total integration is feasible, but just in case I
contributed the code for the tar/gz package format rpa-base uses to
RubyGems, so at least now the packages are externally similar (there
are still important differences in what is put inside, though .
rpa-base tries to play nice with other package managers:
* it won't overwrite files it didn't "own" before unless you specify
--force while installing
* on uninstall, it will not remove the files that have been modified
(it assumes they are now "owned" by another pkg manager, or needed by
some other program)

In practice, if you install a Gem without stubs, the Gem & RPA versions
can coexist easily; require_gem "foo" would use the Gem version and the
normal require 'foo' would take the RPA one.

If you install the Gem with stubs, the 2 following cases can happen:
* installing the Gem before: rpa-base won't let you install the RPA
version unless you --force, in which case the stubs are overwritten
and require 'foo' would use the RPAfied version. On uninstall,
the files which are now owned by rpa-base would be removed; in the
process, you've lost the stubs that RubyGems used.
* installing the RPAfied version before: when RubyGems is about to
create the stubs, it will complain saying that the files exist already
and (IIRC) choose not to overwrite them. Then both packages would
coexist as usual. If gem overwrote them, on uninstall rpa-base would
detect that and leave them there.

As for having a unified format, I think that
* to the upstream developer (you), the existence of RPA doesn't make
much of a difference: you can keep maintaining your gemspecs, cause we
will maintain the RPAfied version of your software without putting
more strains on you (of course, you can choose to become a part of
'we' and help maintain the RPA version, but you're not required to ;-)
* for the end user, having 2 separate commands to install is not that
inconvenient, as long as the packages can coexist peacefully. As shown
above, this is mostly the case already

Thanks a lot for the feedback.

 

[Jani Monoses]

I have to say that cpan.pm is NOT the model rpa-base is based on. The
major sources of inspiration are FreeBSD and Debian: I found that
on a technical level, there's very little to be stolen from cpan.pm
(I believe that both RubyGems and rpa-base are more powerful than
cpan.pm); I'm more interested in the CPAN process as a whole, but I still
believe there's more to be learned from FreeBSD/Debian than from CPAN.

could it preserve the downloaded files then in something like distfiles ?
if the install aborts for some reason it's quicker the second time after
correcting the situation if you have the files locally

Jani

 

[Mauricio Fernández]

could it preserve the downloaded files then in something like distfiles ?
if the install aborts for some reason it's quicker the second time after
correcting the situation if you have the files locally

Yes, it will soon cache the .rps (port) files. Currently, only the .rpa
(binary package) files generated locally are cached, but this will change.