Accessing SVN through Ruby

[Bob Aman]

Nice, an answer that might get me somewhere. Thank you!

DL is a generic dynamically-linked library wrapper that requires none
of this C compiler business. You load a so/dll and tell Ruby what the
parameters are for the C calls, and it takes care of all the argument
passing for you.

DL ships with ruby 1.8, but there doesn't seem to be an exact home page
for it.

http://www.google.com/search?q=ruby+dl

Will give you several tutorial/documents on it.


Why?

The requirement of having to be able to modify the metadata. I assume
there's no way to modify the metadata in place through the client. If
there is, then that very nearly settles it, I think. (Only thing left
to deal with is my irrational apprehention every time I shell out for
something. Plus my reservations about needing to have Subversion
installed.)

If you build it to be reliable, it will be reliable. Unit tests come
in really handy for this.

Agreed, though you always have to consider the weakest link. It's bad
if the weakest link is something you can't control. That's kinda what
I meant.

There's nothing that prevents an svn wrapper from presenting a
ruby-like API. DL and SWIG only give different tradeoffs in how this
is implemented.

What potential trade-offs are there between DL and SWIG?

While many of the svn commands can work directly with the repository
without local storage, I don't know of any way to avoid writing to disk
for those commands that require a local checkout.

Oh yeah, that reminds me... Let's say you try to run the svn client
from a random directory that isn't a svn working directory. Since
we're trying to avoid writing to disk, this will probably effectively
be the case. Are there any parameters to the command line client to
tell it which repository it should be looking at? It looks like you
can specify the target but not the url?

So how does TortoiseSVN do it with the repo-browser? What
capabilities do the client libraries expose that the command line
client does not?

This is easiest if you just shell out, since it will be easier to
maintain a svn wrapper than maintaining libraries for the platforms
that don't typically ship with a compiler (windows).

Agreed, and it's really the only real attraction to me for wrapping
the command line client.

Sounds like you'd rather to an svn wrapper or DL (in that order).

Yeah, I would agree, assuming that all of the functionality I need can
be had through the command line client.

DL or an svn wrapper will probably get you farther faster than SWIG,
since both have a lower barrier to entry. For an svn wrapper you just
need svn installed. For DL you need to be able to find an svn .dll or
.so, which may be as simple as installing svn.

But judging from the contents of my svn directory, maybe not:

Volume in drive D is PRIMARY
Volume Serial Number is 1CEC-3A16

Directory of D:\Program Files\Subversion\bin

01/26/2005 11:07 PM <DIR> .
01/26/2005 11:07 PM <DIR> ..
01/20/2005 06:25 AM 794,722 svn.exe
01/20/2005 06:25 AM 311,392 svnadmin.exe
01/20/2005 06:25 AM 286,814 svnlook.exe
01/20/2005 06:25 AM 315,488 svnserve.exe
01/20/2005 06:25 AM 663,652 svnversion.exe
01/20/2005 06:25 AM 254,058 svndumpfilter.exe
01/20/2005 06:25 AM 647,168 libdb42.dll
01/20/2005 06:25 AM 127,057 libapr.dll
01/20/2005 06:25 AM 168,026 libaprutil.dll
01/20/2005 06:25 AM 36,956 libapriconv.dll
05/04/2004 08:52 AM 61,519 intl.dll
01/20/2005 06:25 AM 843,776 libeay32.dll
01/20/2005 06:25 AM 159,744 ssleay32.dll
03/31/2004 06:13 PM 61,440 SVNService.exe
14 File(s) 4,731,812 bytes

Maybe I'm missing something, but I'm not really seeing that supposed
svn library that I can stick into a DL here.

For SWIG you need
headers and a compiler, which can be no fun if there's no default
compiler and you have 2 or 3 different compilers to choose from.

If you're having these difficulties, do you expect other people to find
it easier? Do you feel you can maintain what you've done when you've
completed your wrapper? These are important things to think about
before you put your hands on the keyboard. I have failed before when
trying to do something Too Big™ because I didn't think about how hard
it would be to complete.

True, though if I can do it once, I can do it again. The question is
whether it's worth the effort if it's going to take 4 hours every time
there's a new release of Subversion. Beyond that, once I get the
thing started, if it seems to be going reasonably well, I won't be the
only person working on it.

But yeah, the possibility of Too Big™ thing is why I haven't started
writing any code quite yet. I like to know what I'm getting into.

I think that you'll be better off making this a requirement, I would
hate to be stuck duplicating the three different svn access protocols
(http/DAV, file, svn) or even one of them.

The subversion people seem to have done a good job of getting
no-brainer svn packages for all the major platforms.

This is true. Ok, so maybe having svn as a prereq isn't too much of a problem.

I believe that Lyle has a good deal of C mojo in his back pocket, which
would make SWIG much easier for him.

Where's the C mojo when you need it? I think long ago I used to have
it... but thus far I've been happier not even thinking about C.

Ruby SWIG wrappers for svn once existed, but bitrotted due to lack of
maintenance...

Well, if you take a look in the subversion subversion repository,
you'll see that they do exist and there's some nice unit tests and
everything for them. Kou has been working on them fairly recently I
think. But building for Windows isn't any easier, which, for the
moment, seems to be my main problem.

It sounds like you need to step back and re-evaluate all of the options
available.

SWIG seems to be more like a belt-fed, automatic, pneumatic nail gun,
and less like a box of nails and a hammer. Just about anybody can use
a hammer, but a belt-fed, automatic, pneumatic nail gun is going to
take lots of practice to use correctly. Once you have it down though,
you'll be able to pound out houses with ease. If you can invest the
time and energy to build SWIG bindings for subversion, then you'll get
a great tool.

If you can't invest that time, I would instead go for DL, or a wrapper
for svn.

Really, I think you should check out DL, you'll quickly cut your
problems down to those of actually wrapping the svn dll/so rather than
trying to figure out why SWIG can't find your apr or svn headers.

I like the sound of DL, but... since there doesn't seem to be any .so
or .dll for the client, maybe it's not even possible? I don't mind
learning SWIG _too_ much, I still have some C knowledge buried in the
attic, and I can dig it out if I don't have any choice.

So I guess my question now is, can the svn client modify the metadata
in place without writing to disk, or is a checkout necessary? Same
question for the stuff I'm able to do via the SWIG bindings or the
mythical DL? This is really the most important part to me I think.

Since it appears that there's actually no svn client libraries
packaged with svn, (presumably because the functionality is entirely
within svn.exe?) then perhaps to use DL magic, I have to compile the
SWIG stuff and then stick _that_ in the DL? Can I get all of the
resultant DL's (since SWIG seems to generate multiple libraries?) to
package up nicely into one gem without much hassle?

And is there any way to use the svn client to work directly with the
repository in the way I want? If not, how would you manage a
reasonably large multi-user system with lots of different things being
worked on, probably in some kind of temp directory? Because right now
I'm envisioning that kind of thing getting out of control really fast.

And if it's not currently possible to edit metadata in place, would it
be unreasonable to make that a feature request for subversion?

 

[Garrett Rooney]

Ruby SWIG wrappers for svn once existed, but bitrotted due to lack of
maintenance...

That's incorrect. Hand rolled ruby bindings for SVN once existed, but
bitrotted. The SWIG bindings are new, and work just fine, it's just
that nobody has made them work on windows yet because, well, there are
fewer hackers working on windows and building SVN on windows isn't
overly easy. Patches welcome.

-garrett

 

[Bob Aman]

Hey, it is not that bad. See the snippet at the end that
I wrote for a 'native' extension. The code is not all that
great (I was mainly trying to get it to just work) but
that Apache Portable Runtime is some truly horrible stuff.

Yes, it is, and this is one of the main reasons I want something more
"ruby-like" than the straight up Ruby SWIG bindings. "Er, pools? Why
should I have to think about pools? This is Ruby, for goodness
sakes!" The thing is ugly, even by C standards.

Someone is already working on a SVN extension to go with the RSCM
project... I would suggest taking a look if you can help there
instead?

Yes, but it seems that they're creating a wrapper for the svn client,
not using the client libraries directly. So they can get no more
functionality than the client itself offers. Which is quite possibly
(or probably, even) not going to be enough to meet my needs.

 

[Steven Jenkins]

Yes, it is, and this is one of the main reasons I want something more
"ruby-like" than the straight up Ruby SWIG bindings. "Er, pools? Why
should I have to think about pools? This is Ruby, for goodness
sakes!" The thing is ugly, even by C standards.

I've wrapped two libraries with SWIG. In both cases, I ended up writing
a layer of syntax sugar in Ruby, purely to make the API more Ruby-like.
One of those efforts is described here:

https://cvs.comedi.org/cgi-bin/view...?rev=1.4&content-type=text/vnd.viewcvs-markup

It's possible to do some of this in SWIG itself, but Ruby is easier, at
least for me.

Steve

 

[Steven Jenkins]

[snip]

It's possible to do some of this in SWIG itself, but Ruby is easier, at
least for me.

Which is a good time for me to ask Lyle Johnson in public: Are you
coming to RubyConf? Can I solicit a tutorial on intermediate to advanced
SWIG and Ruby? (No pressure ).

I've successfully wrestled SWIG (C only, no C++) to the ground twice,
but I have the nagging feeling that it would be so much easier if I
understood certain things: language-indpendent rules, %rename,
exceptions, etc.

Steve

 

[Eric Hodel]

--Apple-Mail-26--272273634
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII; format=flowed

Nice, an answer that might get me somewhere. Thank you!


The requirement of having to be able to modify the metadata. I assume
there's no way to modify the metadata in place through the client. If
there is, then that very nearly settles it, I think. (Only thing left
to deal with is my irrational apprehention every time I shell out for
something. Plus my reservations about needing to have Subversion
installed.)


What potential trade-offs are there between DL and SWIG?

I think I've illustrated the biggest ones already, with SWIG you can
get exactly what you want, but it will take you more time and effort.
With DL, you may run into some gotcha in the corner that DL doesn't
handle well. DL will require less effort to keep up-to-date than SWIG
will, the difference depends on how quickly the svn internals change.
(It looks like the subversion people do the right thing here, 1.1.3
still seem to have the same library numbers as 1.0.0, so this probably
won't be as big a deal.)

Oh yeah, that reminds me... Let's say you try to run the svn client
from a random directory that isn't a svn working directory. Since
we're trying to avoid writing to disk, this will probably effectively
be the case. Are there any parameters to the command line client to
tell it which repository it should be looking at? It looks like you
can specify the target but not the url?

So how does TortoiseSVN do it with the repo-browser? What
capabilities do the client libraries expose that the command line
client does not?

You'll have to consult with the subversion people more on this, but:

$ svn help propset
propset (pset, ps): Set PROPNAME to PROPVAL on files, dirs, or
revisions.
usage: 1. propset PROPNAME [PROPVAL | -F VALFILE] PATH...
2. propset PROPNAME --revprop -r REV [PROPVAL | -F VALFILE] 1. Creates a versioned, local pro... SIGNATURE----- --Apple-Mail-26--272273634--

 

[Bob Aman]

http://www.swig.org/Doc1.3/Windows.html

I should also note here that as I have said before, the output files from
SWIG are portable to almost any architecture and only need to be generated
once. Thus, if you have SWIG installed say on linux, you can run it to
generate the _wrap files and those exact wrap files can be used on Sun,
Windows, Linux, MacOS, whatever. So you could even get someone to email
you the files that SWIG creates, and you wouldn't even need SWIG installed
to compile the bindings on windows.

The right way to take care of this problem is to have the SWIG generated
files present directly inside the released subversion tarballs, then SWIG
does not need to be installed to build a tarball. Sadly, this feature will
not even be in the 1.2 release, but hopefully future releases will have it.

Note that all I've said is only true for SWIG since version 1.3.23... but
subversion is using 1.3.24 so shouldn't matter.

John

Well, I managed to get a little bit more progress by moving around a
bunch of the .h files. But apr.h seems to be missing. I saw apr.h.in
and that seems to be what I'm looking for. It says apr.h is a
generated file? How do I generate it? On a whim, I tried just
renaming apr.h.in to apr.h and got back an error from SWIG:
E:\svn-source\subversion-1.1.4\subversion\bindings\swig>swig -ruby apr.i
apr.h(257): Error: Syntax error in input.
*** No module name specified using %module or -module.

So I assume that was the wrong thing to do

 

[Joshua J. Kugler]

The requirement of having to be able to modify the metadata. I
assume there's no way to modify the metadata in place through the
client. If there is, then that very nearly settles it, I think.
(Only thing left to deal with is my irrational apprehention every
time I shell out for something. Plus my reservations about needing
to have Subversion installed.)

So I guess my question now is, can the svn client modify the metadata
in place without writing to disk, or is a checkout necessary? Same
question for the stuff I'm able to do via the SWIG bindings or the
mythical DL? This is really the most important part to me I think.

Since it appears that there's actually no svn client libraries
packaged with svn, (presumably because the functionality is entirely
within svn.exe?) then perhaps to use DL magic, I have to compile the
SWIG stuff and then stick _that_ in the DL? Can I get all of the
resultant DL's (since SWIG seems to generate multiple libraries?) to
package up nicely into one gem without much hassle?

I don't know how much reading of the svn manual you have done
<http://svnbook.red-bean.com/> because it explains very clearly how to
edit metadeta with the svn client and the svnadmin tools. It also
explains (IIRC) about the client libraries (but I could be wrong on

And is there any way to use the svn client to work directly with the
repository in the way I want? If not, how would you manage a
reasonably large multi-user system with lots of different things
being worked on, probably in some kind of temp directory? Because
right now I'm envisioning that kind of thing getting out of control
really fast.

You can specify where you check out, but checking out to memory might
require some interesting proramming. If it is via a web client,
though, checking out to memory wouldn't work as the memory would go
away when the script ended.

j----- k-----

 

[Bob Aman]

I don't know how much reading of the svn manual you have done

<http://svnbook.red-bean.com/> because it explains very clearly how to
edit metadeta with the svn client and the svnadmin tools. It also
explains (IIRC) about the client libraries (but I could be wrong on
that). See <http://svnbook.red-bean.com/en/1.1/ch08.html>, especially
the section about "Using the API."

I did read that section on using the API. It's not overwhelmingly
helpful unless you were writing everything in Python. I also know how
to edit metadata under normal circumstances. IE, when you have a
working copy checked out. That's a simple propset command. I need to
edit that metadata without having a working copy. I need to be
editing the metadata from within a multi-user web app, and writing
anything to disk would be a bad idea, especially if the sole purpose
of writing to disk was to make svn know which repository we're
interested in. Compounded on top of all that, a very likely
requirement will be having the same web app connect to more than one
repository.

You can specify where you check out, but checking out to memory might
require some interesting proramming. If it is via a web client,
though, checking out to memory wouldn't work as the memory would go
away when the script ended.

And why is that a problem? Any "checkouts to memory" would only be
occurring for the purposes of enabling in-place metadata changes. If
that's even required. See, the problem is that I'm writing a wrapper
of sorts around Subversion. For example, things like workflow states
will be saved in the svn metadata. I don't particularly want users
editing metadata manually, though I don't see any really, really big
problems with them being able to do it since the metadata is
versioned. Workflow states will probably have some sort of encrypted
authorization token that will be next to impossible for an end user to
fake, and if they change those, someone walks over to their cube and
tells them to cut it out and then restores the workflow state to what
it should be. Not a big deal. (Certainly better than the system that
currently exists. The app will just have to make allowance for
possibly erroneous data. No biggy, I would be remise in my coding
anyhow if I made the assumption that everything was clean.) But for
that to work, the program needs to be able to "promote" or "demote"
files from within the web interface. That's not something you want to
make people do via TortoiseSVN. Especially if you have complex
group/role logic going on to determine who can promote or demote what.