Creating a subversion "tag change summary".

Discussion in 'Java' started by Daniel Pitts, Apr 15, 2012.

  1. Daniel Pitts

    Daniel Pitts Guest

    I know this isn't strictly Java, but I'm using SVNKIT, which is Java :)

    Our deployment process includes building a tag in SVN, which is "mostly"
    the contents of an old tag, but with a couple of files copied from the
    trunk instead.

    I'm trying to build a tool which will do two things: One, it will list
    the files that differ between trunk and the "source" tag. This is easy
    with SVNKIT, just use the SVNDiffClient.doDiffStatus().

    The second part seems to be more difficult. I want to produce a list of
    commits that are different. Basically, I want to warn someone if are two
    or more authors committed to the file that's being deployed, and also
    give them a quick change-log.

    I've written code that does this, which basically does a svn log of the
    file both in the source tag and trunk, and then which trunk revisions
    aren't already present in the tag. The problem is that this is a really
    slow process, taking up to 30 seconds per file.

    I'm using SVNKIT, I was using version 1.3.2, and recently upgraded to
    1.7.4, which is somewhat faster, but still to slow.

    Our current setup only allows http(s) connections to the repository (not
    svn: or direct file access), so I'm not sure if that accounts for some
    of the speed, but I doubt thats the entire problem. I don't have shell
    access to the SVN server, so I can't really see whats going on there
    without involving our admins.

    The current revision number is r788228, so we have a lot of revisions.

    Short of moving the revision history into my own DB (to cache the data),
    is there any other approach you might consider taking to solve this
    problem? Even questions/ideas I can take to our admins are fine, but
    this is kind of a side-project for me, so I wouldn't expect a lot from them.

    Thanks,
    Daniel.
     
    Daniel Pitts, Apr 15, 2012
    #1
    1. Advertising

  2. Daniel Pitts

    Daniel Pitts Guest

    On 4/15/12 2:44 PM, Daniel Pitts wrote:
    > I know this isn't strictly Java, but I'm using SVNKIT, which is Java :)
    >
    > Our deployment process includes building a tag in SVN, which is "mostly"
    > the contents of an old tag, but with a couple of files copied from the
    > trunk instead.
    >
    > I'm trying to build a tool which will do two things: One, it will list
    > the files that differ between trunk and the "source" tag. This is easy
    > with SVNKIT, just use the SVNDiffClient.doDiffStatus().
    >
    > The second part seems to be more difficult. I want to produce a list of
    > commits that are different. Basically, I want to warn someone if are two
    > or more authors committed to the file that's being deployed, and also
    > give them a quick change-log.
    >
    > I've written code that does this, which basically does a svn log of the
    > file both in the source tag and trunk, and then which trunk revisions
    > aren't already present in the tag. The problem is that this is a really
    > slow process, taking up to 30 seconds per file.
    >
    > I'm using SVNKIT, I was using version 1.3.2, and recently upgraded to
    > 1.7.4, which is somewhat faster, but still to slow.
    >
    > Our current setup only allows http(s) connections to the repository (not
    > svn: or direct file access), so I'm not sure if that accounts for some
    > of the speed, but I doubt thats the entire problem. I don't have shell
    > access to the SVN server, so I can't really see whats going on there
    > without involving our admins.
    >
    > The current revision number is r788228, so we have a lot of revisions.
    >
    > Short of moving the revision history into my own DB (to cache the data),
    > is there any other approach you might consider taking to solve this
    > problem? Even questions/ideas I can take to our admins are fine, but
    > this is kind of a side-project for me, so I wouldn't expect a lot from
    > them.


    Well, I did find one way to speed things up. An "info" request of the
    file on the tag is relatively fast, and includes the "last change
    revision" of that file. I can then do a log of that file on the trunk
    (Which is faster by magnitudes), and anything after that revision can be
    considered a change after that file was originally tagged.

    Anyway, I'm still open to any other advice on the matter.
     
    Daniel Pitts, Apr 15, 2012
    #2
    1. Advertising

  3. In article <HIHir.9015$>,
    Daniel Pitts <> wrote:

    > Well, I did find one way to speed things up. An "info" request of
    > the file on the tag is relatively fast, and includes the "last
    > change revision" of that file. I can then do a log of that file
    > on the trunk (Which is faster by magnitudes), and anything after
    > that revision can be considered a change after that file was
    > originally tagged.
    >
    > Anyway, I'm still open to any other advice on the matter.


    This reminds me of a project that had a particularly tangled
    per-directory access control configuration. You might look at the
    speed versus security tradeoff in "Disabling path-based checks."

    --
    John B. Matthews
    trashgod at gmail dot com
    <http://sites.google.com/site/drjohnbmatthews>
     
    John B. Matthews, Apr 16, 2012
    #3
  4. On 15/04/12 22:44, Daniel Pitts wrote:
    > Basically, I want to warn someone if are two or more authors committed
    > to the file that's being deployed,


    I expect I haven't fully understood your requirements, but for that
    specific one, could you use the equivalent of 'svn blame'?

    --
    ss at comp dot lancs dot ac dot uk
     
    Steven Simpson, Apr 16, 2012
    #4
  5. Daniel Pitts

    Daniel Pitts Guest

    On 4/15/12 10:51 PM, John B. Matthews wrote:
    > In article<HIHir.9015$>,
    > Daniel Pitts<> wrote:
    >
    >> Well, I did find one way to speed things up. An "info" request of
    >> the file on the tag is relatively fast, and includes the "last
    >> change revision" of that file. I can then do a log of that file
    >> on the trunk (Which is faster by magnitudes), and anything after
    >> that revision can be considered a change after that file was
    >> originally tagged.
    >>
    >> Anyway, I'm still open to any other advice on the matter.

    >
    > This reminds me of a project that had a particularly tangled
    > per-directory access control configuration. You might look at the
    > speed versus security tradeoff in "Disabling path-based checks."
    >

    I'm not sure I understand what you mean by that, but that does remind me
    that our admins have prevented checkouts that don't have "/branches/*/",
    "/tags/*/" or "/trunk/" in them. Not sure if that is contributing to the
    problem or not.
     
    Daniel Pitts, Apr 16, 2012
    #5
  6. Daniel Pitts

    Daniel Pitts Guest

    On 4/16/12 12:10 AM, Steven Simpson wrote:
    > On 15/04/12 22:44, Daniel Pitts wrote:
    >> Basically, I want to warn someone if are two or more authors committed
    >> to the file that's being deployed,

    >
    > I expect I haven't fully understood your requirements, but for that
    > specific one, could you use the equivalent of 'svn blame'?
    >


    I thought about using that at first, it doesn't really help here. I have
    two versions of the same file, and I need to know who's made the
    difference. svn blame takes exactly one file.

    In another post, I mentioned that svn info runs much faster for my
    tagged file, and it returns basically the last revision that changed
    that version of the file. I'm satisfied with this solution (unless I can
    find "the perfect" solution). This is more of a "double check", so it
    doesn't have to be perfect. The edge cases this misses are going to be
    few and far between. It is already more information than people get
    with our existing tool-set.
     
    Daniel Pitts, Apr 16, 2012
    #6
  7. In article <LpYir.88442$>,
    Daniel Pitts <> wrote:

    > On 4/15/12 10:51 PM, John B. Matthews wrote:
    > > In article<HIHir.9015$>,
    > > Daniel Pitts<> wrote:
    > >
    > >> Well, I did find one way to speed things up. An "info" request
    > >> of the file on the tag is relatively fast, and includes the
    > >> "last change revision" of that file. I can then do a log of
    > >> that file on the trunk (Which is faster by magnitudes), and
    > >> anything after that revision can be considered a change after
    > >> that file was originally tagged.
    > >>
    > >> Anyway, I'm still open to any other advice on the matter.

    > >
    > > This reminds me of a project that had a particularly tangled
    > > per-directory access control configuration. You might look at
    > > the speed versus security tradeoff in "Disabling path-based
    > > checks."
    > >

    > I'm not sure I understand what you mean by that, but that does
    > remind me that our admins have prevented checkouts that don't
    > have "/branches/*/", "/tags/*/" or "/trunk/" in them. Not sure if
    > that is contributing to the problem or not.


    I'm not sure how they're enforcing it, but "Per-directory access
    control" is one way, and "Disabling path-based checks" is one
    alternative, both discussed here:

    <http://svnbook.red-bean.com/en/1.7/svn.serverconfig.httpd.html#svn.serverconfig.httpd.authz.perdir>

    A symptom is that the time required to obtain older `info`
    increases (roughly) geometrically with age.

    --
    John B. Matthews
    trashgod at gmail dot com
    <http://sites.google.com/site/drjohnbmatthews>
     
    John B. Matthews, Apr 17, 2012
    #7
  8. Daniel Pitts

    Daniel Pitts Guest

    On 4/16/12 5:54 PM, John B. Matthews wrote:
    > In article<LpYir.88442$>,
    > Daniel Pitts<> wrote:
    >
    >> On 4/15/12 10:51 PM, John B. Matthews wrote:
    >>> In article<HIHir.9015$>,
    >>> Daniel Pitts<> wrote:
    >>>
    >>>> Well, I did find one way to speed things up. An "info" request
    >>>> of the file on the tag is relatively fast, and includes the
    >>>> "last change revision" of that file. I can then do a log of
    >>>> that file on the trunk (Which is faster by magnitudes), and
    >>>> anything after that revision can be considered a change after
    >>>> that file was originally tagged.
    >>>>
    >>>> Anyway, I'm still open to any other advice on the matter.
    >>>
    >>> This reminds me of a project that had a particularly tangled
    >>> per-directory access control configuration. You might look at
    >>> the speed versus security tradeoff in "Disabling path-based
    >>> checks."
    >>>

    >> I'm not sure I understand what you mean by that, but that does
    >> remind me that our admins have prevented checkouts that don't
    >> have "/branches/*/", "/tags/*/" or "/trunk/" in them. Not sure if
    >> that is contributing to the problem or not.

    >
    > I'm not sure how they're enforcing it, but "Per-directory access
    > control" is one way, and "Disabling path-based checks" is one
    > alternative, both discussed here:
    >
    > <http://svnbook.red-bean.com/en/1.7/svn.serverconfig.httpd.html#svn.serverconfig.httpd.authz.perdir>
    >
    > A symptom is that the time required to obtain older `info`
    > increases (roughly) geometrically with age.

    That actually seems to be a likely cause. I'm going to ask our admins to
    consider disabling path-based checks.

    Thanks for the hint!
     
    Daniel Pitts, Apr 17, 2012
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?iso-8859-1?B?bW9vcJk=?=

    subversion hosting

    =?iso-8859-1?B?bW9vcJk=?=, Oct 17, 2005, in forum: Java
    Replies:
    16
    Views:
    748
    Karl Heinz Marbaise
    Oct 19, 2005
  2. Elaine
    Replies:
    0
    Views:
    352
    Elaine
    May 10, 2006
  3. subversion and ant

    , May 11, 2006, in forum: Java
    Replies:
    0
    Views:
    373
  4. Replies:
    5
    Views:
    796
    John B. Matthews
    Apr 30, 2010
  5. Replies:
    0
    Views:
    383
Loading...

Share This Page