Version control for VHDL projects.

[Symon]

Guys,

I am interested in what people use for version control of VHDL projects
that are run across geographically diverse teams. Currently I use
Subversion over a VPN, with windows PC clients connecting using
TortoiseSVN, which works pretty well. I wonder if anyone is using, or
considering using, distributed version control like GIT or Mercurial? If
so, would you care to share your experiences?

Thanks, Symon.

 

[Philippe]

I am interested in what people use for version control of VHDL projects
that are run across geographically diverse teams.

Hi Symon,

We conducted a survey among VHDL designers over the last few months.
Here are some figures.

SVN: 62%
GIT or Mercurial: 25%
CVS 18%
Clearcase: < 3%

These add up to more than 100% because we allowed people to select
multiple alternatives. There are no guarantees that our population is
statistically representative, so add a pinch of salt. More recently I
heard of a person that works with MS Visual CodeSafe, but I guess that
would hardly show up in the statistics.

Still, I think it is safe to conclude that SVN is the number one
version control system for digital hardware design.

I hope this helps

 

[M. Norton]

I am interested in what people use for version control of VHDL projects
that are run across geographically diverse teams. Currently I use

We have development taking place at our facility and then at a sister
facility three hours away. We've been using CVS though there seems to
be strong support to switching to GIT and there have been some steps
taken to migrate new projects to that version control platform. My
prior workplace was all SVN and the company prior to THAT used
SourceSafe.

I think as long as everyone agrees on the system, you can pretty much
get anything to work. GIT seems to be the new hot thing.

Mark Norton

 

[rickman]

Guys,

I am interested in what people use for version control of VHDL projects
that are run across geographically diverse teams. Currently I use
Subversion over a VPN, with windows PC clients connecting using
TortoiseSVN, which works pretty well. I wonder if anyone is using, or
considering using, distributed version control like GIT or Mercurial? If
so, would you care to share your experiences?

Thanks, Symon.

I wouldn't say I am involved in development across geographically
diverse areas, but I am using SVN and I'm not sure what to make of
it. Reading the manual it seems to be saying that if I want to set a
baseline with a tag or if I want to create a branch, I can copy
everything to a separate folder. Do I need a version control system
for that??? It seems to work in ways that are very different from
what I am used to (very old experience with CVS and clearcase).

Am I just out of the loop or do I just need to work with it more to
get the "feel" of SVN/Tortoise?

Rick

 

[Andrew FPGA]

Hi Symon,
The big win with Git/Mecurial is their branching and merging model.

Yes they are distributed, and that helps me when I'm working from home
or on the road and I don't have an internet connection or the VPN goes
down. I can still keep checking in, branching, merging etc. I just
cant push or pull with the remote(server). We use git, we have 10 or
so RTL developers distributed across 3 sites in both hemispheres.
VHDL, Verilog and SV on a windows(sadly) environment. Git works pretty
well under cygwin (some line ending problems from time to time...).

There is a learning curve with git, and until you embrace the command
line flow and branching/merging model, life is tough. Especially for
any hardware guys who love their GUIs. But once over the learning
curve, SVN via toroise would be a painful step backwards for me.

Compared to Git/Mercurial, the branching and merging model in CVS/SVN
is broken. Branching is always easy of course - just take a copy.
Merging is where all the pain is. Unlike SVN/CVS, git automatically
searches for and finds the common ancestor for a merge. That is a
manual, and error prone process in SVN/CVS. This makes merging in git
a breeze relative to SVN/CVS.

Branching is powerful because it allows you to manage multiple streams
of development concurrently. It is a key enabler for concurrent
development in my view. Multiple RTL developers on the same project
can each work concurrently on separate feature branches without
interfering with each other, and keep a clean/known good main line.
Merge features and bug fixes onto the main line, as they are ready.

Even thought git is distributed, we use a pesudo client/server model.
We have git servers(remotes) at each geographical site, one of which
is a master the others sync to. The local client is actually a git
repository in itself; it pushes to the master server, and pulls from
local server. Commits, diffs, merges, branches, etc all happen in the
local client repo, and don't require a connection to the server.

Feel free to ask more specific questions.
Cheers
Andrew

 

[Marcus Harnisch]

Reading the manual it seems to be saying that if I want to set a
baseline with a tag or if I want to create a branch, I can copy
everything to a separate folder.

That would be SVN's own copy, not a file system copy. The duplication
happens inside the database (repository).

SVN doesn't distinguish tags from branches. Both are copies (see
above) of a particular revision. If you check out this copy, modify
it, and check it back in, you have a branch. If you don't touch the
copy, it is a tag.

HTH
Marcus

 

[rickman]

That would be SVN's own copy, not a file system copy. The duplication
happens inside the database (repository).

I guess I don't understand this. How do I make a copy in a new
directory that is a "SVN copy"?

SVN doesn't distinguish tags from branches. Both are copies (see
above) of a particular revision. If you check out this copy, modify
it, and check it back in, you have a branch. If you don't touch the
copy, it is a tag.

Yeah, I understand the fact that a branch is the same as a tag. I
just don't see how to create either and have them accessible in a way
that is at all similar to what I am used to with conventional VC.

Rick

 

[Marcus Harnisch]

I guess I don't understand this. How do I make a copy in a new
directory that is a "SVN copy"?

The repository presents itself to the users as some kind of versioned
filesystem. "svn copy" causes Subversion to create a copy of a
specified state of this file system (default: head revision) or a
subtree of it[1]. Since at this point there are no differences between
original and copy, yet, this is a zero (almost) overhead operation.

It is common practice to create the repository with three (initially
empty) top-level directories "trunk", "tags", "branches". To
Subversion these names have no special meaning!

To create a workspace, check out "trunk", not the repository
itself. Add, modify, check in, repeat. When you reach a certain
milestone and want to tag it. Just copy everything under "trunk" into
"tags"[2]:

$ svn copy <path to repo>/trunk <path to repo>/tags/my-first-tag

This copy exists only in the repository

To create a branch, you'd do similar:

$ svn copy <path to repo>/trunk <path to repo>/branches/my-first-branch
$ svn checkout <path to repo>/branches/my-first-branch ./my-first-branch

Check the Fine Manual for more examples.

Regards
Marcus


Footnotes:
[1] Note that the possibility to work on subtrees of a project is a
significant difference to Mercurial/Git.

[2] Don't blame me for wrong syntax, none of these commands have been
tested.

 

[Symon]

Even thought git is distributed, we use a pesudo client/server model.
We have git servers(remotes) at each geographical site, one of which
is a master the others sync to. The local client is actually a git
repository in itself; it pushes to the master server, and pulls from
local server. Commits, diffs, merges, branches, etc all happen in the
local client repo, and don't require a connection to the server.

Feel free to ask more specific questions.
Cheers
Andrew

Hi Andrew,

Thank you very much for your post, it encouraged me to do more research
on git.

I'm interested in the specifics of setting up the infrastructure
required for this. With Subversion, it's pretty simple, there's a
central server somewhere, and all the clients, where ever they are,
connect to it.

As for git, I understand it's distributed, meaning that each user has a
complete repository on their own workstation. It seems clients push and
pull data from each other.

One question is, how do all the clients pull data from all the others?
Some (all?) clients are behind firewalls? You seem to say in the
implementation you use, pushes happen to a central server, just like
SVN, but pulls happen from a local server. Is this local server pushed
to by the central server or does the local server pull from the central
server? Or something else?

My second question is, is there necessarily a master copy stored
anywhere? I'm not exactly even sure what this means in git-world, so
excuse me if this is dumb. I think I'm confused as to how clients know
whether their personal repositories are up-to-date with respect to other
clients.

Thanks again for your help, Syms.

 

[Andy Peters]

I wouldn't say I am involved in development across geographically
diverse areas, but I am using SVN and I'm not sure what to make of
it.  Reading the manual it seems to be saying that if I want to set a
baseline with a tag or if I want to create a branch, I can copy
everything to a separate folder.  Do I need a version control system
for that???  It seems to work in ways that are very different from
what I am used to (very old experience with CVS and clearcase).

See, when you create a branch or a tag, you do the copy in the
repository, not on your working computer's local filesystem.

Am I just out of the loop or do I just need to work with it more to
get the "feel" of SVN/Tortoise?

Tortoise REALLY helps.

-a

 

[Andy Peters]

I guess I don't understand this.  How do I make a copy in a new
directory that is a "SVN copy"?

You are making the copy in the repository.

 

[KJ]

See, when you create a branch or a tag, you do the copy in the
repository, not on your working computer's local filesystem.

Be careful of 'tagging' things that use externals though...you'll
likely be disappointed when you find out what 'cheap copy' really
means.

Kevin Jennings

 

[Bill Mills]

Guys,

I am interested in what people use for version control of VHDL projects
that are run across geographically diverse teams. Currently I use
Subversion over a VPN, with windows PC clients connecting using
TortoiseSVN, which works pretty well. I wonder if anyone is using, or
considering using, distributed version control like GIT or Mercurial? If
so, would you care to share your experiences?

Thanks, Symon.

No experiences to offer myself. But Joel on Software recently posted
this. The Mercurial tutorial, near the end, describes branch and
merge using this distributed system.

http://www.joelonsoftware.com/items/2010/03/17.html

 

[Andy Peters]

Be careful of 'tagging' things that use externals though...you'll
likely be disappointed when you find out what 'cheap copy' really
means.

Kevin Jennings

I think we've been through this before, and my advice still stands:

If the externals you use are themselves tags, then a project that
includes them will always give you exactly what you expect.

-a

 

[KJ]

I think we've been through this before, and my advice still stands:

We have...and my advice still stands also

If the externals you use are themselves tags, then a project that
includes them will always give you exactly what you expect.

One thing that I expect from a version control system is a non-manual
mechanism to be able to tag something so that when I come back days
(weeks, months, years) later I can retrieve it exactly as it was back
at the time that it was tagged...whether or not I chose to use
externals in that project...whether or not those external things have
also changed during that interval...and the big word in all of this is
'non-manual'.

Using SVN, the only way you can get this to happen is to manually
assign specific revision levels to each and every external in the
entire project (what you refer to as "externals you use are themselves
tags"). Having to manually assign revision levels has the following
consequences:
- Additional work that can be performed incorrectly (i.e. less
productive, possibly error prone)
- With each and every code freeze that is to be archived, one needs to
first check that each and every external does in fact still have the
correct revision level assigned. Externals that aren't rev specified
would have to be first tagged themselves (there's the extra work I
talked about)
- No way to check that a build from the 'head' revision is actually
correct without deleting all those manually entered revision levels
(kind of calls into question the integrity of the designs in the
repository if nobody is actually snapshotting and validating the
'latest/greatest' somewhat routinely)

Bottom line, is that SVN does what it does, maybe you find it does
what you'd like and don't see the above consequences as a burden...but
it doesn't do what I would expect a version control tool to
do...without heavy duty manual intervention ('heavy duty' that is when
you practice code reuse and the reusable code is also under
development). I'm not alone in that view, just like you're not alone
in yours.

Interestingly enough, it seems there might be movement afoot to
address this limitation when version SVN 1.7.0 comes out based on the
following thread...or maybe I'm just hoping...
http://groups.google.com/group/tortoisesvn/browse_frm/thread/0337cb91d0eedef8?hl=en#

Probably shouldn't beat the horse again on this group so I'll leave it
at that and simply refer the interested reader to check out the
discussion from last May where this got discussed quite a bit.
http://groups.google.com/group/comp...bversion+Kevin+Jennings+VHDL#7ea2dc3837f5819d

Kevin Jennings

 

[Symon]

Dear Andrew,
I know this is rude to email directly, but I don't suppose you would
like to answer my usenet query on comp.lang.vhdl about git? I felt free,
and your original reply was one of the most cogent usenet replies I've seen!
Thanks. Symon.

 

[rickman]

You are making the copy in the repository.

Maybe I don't understand that. I am using SVN with Tortoise and I see
the files in the regular directory tree using Windows Explorer. How
do I make a copy just in the repository? Do I use a special command
for that?

There are a number of other things I am unclear about. I have tried
looking at the docs, but they seemed to be explaining things that I
wasn't confused about. Should I go back and read it again? I can't
imagine this is really that hard. I expect I just have a mindset
about some aspect of this that is making it hard for me.

Rick

 

[Andy Peters]

Maybe I don't understand that.  I am using SVN with Tortoise and I see
the files in the regular directory tree using Windows Explorer.  How
do I make a copy just in the repository?  Do I use a special command
for that?

Yes, there's a special command, svn copy.

If you wish to make a branch from your __checked-out working copy__,
right click on its icon, choose "TortoiseSvn" then "Branch/Tag." A
dialog box titled "Copy (Branch/Tag)" will pop up.

You'll see:

From WC at URL:
svn://eng-andrew/fpga/projects/CameraSim/trunk (not editable)

and

To URL:
svn://eng-andrew/fpga/projects/CameraSim/trunk (list box)

(note that they are the same, to start). This is followed by:

Create Copy in the repository from:
( ) HEAD revision in the repository
(*) Specific revision in the repository [2272] ...
( ) Working copy

followed by an entry box for the log message.

Now remember: In this example, you want to use your checked-out
working copy as the starting point for a new branch. (This working
copy may have edits to any of the files -- this is fine!) First,
select the "Working copy" radio button. This means that the new branch
will use the working copy as its base (and all history is retained).
Then, next to the "To URL:" combo box is a button with ellipsis. Click
it, and the repository browser opens to the project's location in the
repo. In the left-hand pane is the repo tree. If you've used the
standard trunk/branches/tags format, click on the project name in the
left-hand pane, and click the "+" to expand the tree. Click "branches"
then click "OK" and you're back to the dialog. Click in the "To URL"
combo box and put your cursor after "branches" in the URL. Finish the
URL by giving it a useful branch name:

svn://eng-andrew/fpga/projects/CameraSim/branches/testbranch

Enter a useful log message ("Creating branch to test X, Y, Z" or
whatever). Make sure you CHECK the box labeled "Switch working copy to
new branch/tag" then click "OK." Voila, you've now created a branch
and your working copy is now of that branch, not the trunk.

There are a number of other things I am unclear about.  I have tried
looking at the docs, but they seemed to be explaining things that I
wasn't confused about.  Should I go back and read it again?  I can't
imagine this is really that hard.  I expect I just have a mindset
about some aspect of this that is making it hard for me.

it's really not that hard but it can be somewhat non-obvious.

=-a

 

[rickman]

Maybe I don't understand that.  I am using SVN with Tortoise and I see
the files in the regular directory tree using Windows Explorer.  How
do I make a copy just in the repository?  Do I use a special command
for that?

Yes, there's a special command, svn copy.

If you wish to make a branch from your __checked-out working copy__,
right click on its icon, choose "TortoiseSvn" then "Branch/Tag." A
dialog box titled "Copy (Branch/Tag)" will pop up.

You'll see:

From WC at URL:
svn://eng-andrew/fpga/projects/CameraSim/trunk (not editable)

and

To URL:
svn://eng-andrew/fpga/projects/CameraSim/trunk (list box)

(note that they are the same, to start). This is followed by:

Create Copy in the repository from:
( ) HEAD revision in the repository
(*) Specific revision in the repository   [2272] ...
( ) Working copy

followed by an entry box for the log message.

Now remember: In this example, you want to use your checked-out
working copy as the starting point for a new branch. (This working
copy may have edits to any of the files -- this is fine!) First,
select the "Working copy" radio button. This means that the new branch
will use the working copy as its base (and all history is retained).
Then, next to the "To URL:" combo box is a button with ellipsis. Click
it, and the repository browser opens to the project's location in the
repo. In the left-hand pane is the repo tree. If you've used the
standard trunk/branches/tags format, click on the project name in the
left-hand pane, and click the "+" to expand the tree. Click "branches"
then click "OK" and  you're back to the dialog. Click in the "To URL"
combo box and put your cursor after "branches" in the URL. Finish the
URL by giving it a useful branch name:

svn://eng-andrew/fpga/projects/CameraSim/branches/testbranch

Enter a useful log message ("Creating branch to test X, Y, Z" or
whatever). Make sure you CHECK the box labeled "Switch working copy to
new branch/tag" then click "OK." Voila, you've now created a branch
and your working copy is now of that branch, not the trunk.

There are a number of other things I am unclear about.  I have tried
looking at the docs, but they seemed to be explaining things that I
wasn't confused about.  Should I go back and read it again?  I can't
imagine this is really that hard.  I expect I just have a mindset
about some aspect of this that is making it hard for me.

it's really not that hard but it can be somewhat non-obvious.

=-a

Hey, thanks a lot. I didn't see the menu item in Tortoise. I guess
that makes it easier... :^s The explanation helps a lot too.

Rick

 

[Andrew FPGA]

Apologies for the slow reply - wish I had a way of automatically being
told when someone replies to a message I've posted on a thread? I use
google groups....

One question is, how do all the clients pull data from all the others?
Some (all?) clients are behind firewalls? You seem to say in the
implementation you use, pushes happen to a central server, just like
SVN, but pulls happen from a local server. Is this local server pushed
to by the central server or does the local server pull from the central
server? Or something else?

Ok, so for our company, all clients must be on the VPN before can pull/
push with the servers. I don't know the details but the VPN goes
through the various firewalls somehow - standard IT/networking thingy.
Plenty of open source projects use git so they must deal with the
firewall thing somehow.

So for our source control policy, clients don't pull/push from/to each
other. Only with the servers. Here is an example transcript when I
type "git update". This is an in-house script that will get my client
repo up-to-date with the master server.

$ git update
Full Fetch:
$ git config remote.origin.url git://<local server name>
$ git fetch origin
$ git config remote.origin.url git://<remote server name>
$ git fetch origin --tags
$ git fetch origin
Local branch [master] is tracking [origin/master].
Merging changes into current branch:
$ git merge origin/master
Already up-to-date.
Remove stale tracking branches:
$ git remote prune origin

So you will see it fetches from the local server first. Thats fast/
cheap. Then it fetches from the master server which is at remote
geographical location for our site.

I don't know how the local/remote servers stay in sync. I presume the
local server pulls from the remote/master periodically.

When I push I push to the remote(master) server.

My second question is, is there necessarily a master copy stored
anywhere? I'm not exactly even sure what this means in git-world, so
excuse me if this is dumb. I think I'm confused as to how clients know
whether their personal repositories are up-to-date with respect to other
clients.

So for our company, I guess the master server, is well, the master
server. And all these servers must be backed up as per normal IT
stuff.

To find out if my client repo is out of date, I go git fetch. Then I
can go git diff origin/master master to see what the differences are
if any. Read up on git branches and tracking branches.

Hope this helps.
Cheers
Andrew