SVN Page
Catherine suggested that we put all the bits and pieces we learn about SVN in one place. A jolly good idea as I do tend to forget about things!
As a CVS user, migrating to SVN was not without it's hiccoughs. I found the biggest one to be that SVN commands like svn diff, svn status (and maybe a few more) work against the pristine revision located in the .svn subdirectory in your working directory and not the latest revision in the SVN repository. In contrast, CVS commands always work against the latest revision in the CVS repository.
In the discussion that follows, call the pristine revision in the working directory BASE and the latest revision in the SVN repository HEAD.
Another hiccough is the use of revision numbers. This topic has been dealt with very well in the Subversion book [1]. See the section on SVN DIFF for a discussion of some of the issues related to version numbers that might make a CVS user weep.
--alston 18:10, 28 November 2006 (GMT)
SVN Links
- SVN webpage: http://subversion.tigris.org/
- SVN Book: http://svnbook.red-bean.com/
- SVN FAQ: http://subversion.tigris.org/faq.html
- SVN revision numbers and keywords: http://svnbook.red-bean.com/en/1.0/ch03s03.html
Setting up your SVN details
(1) To get an account on the SVN server mail ucc-support@ch.cam.ac.uk . Say that you're from the Theory sector, which research group you're in, and if you're setting up access for the first time for a group, who else in the group needs access. You should cc your group leader on the mail. Details on how to set up svn using password authentication and how to obtain/change your password are on the wwmm wiki.
(2) I found it convenient to define an environment variable pointing to the user and group SVN directories on the WWMM server:
export MYSVN=https://wwmm.ch.cam.ac.uk/svn2/users/am592 export SVN=https://wwmm.ch.cam.ac.uk/svn2/group/stone
You could just add those lines to your .bashrc file.
(3) The first time you access svn on a given computer, it will ask for your password. After that, you're good to go:
jss43@keiko:~/NECI> svn ls $SVN Authentication realm: <https://wwmm.ch.cam.ac.uk:443> WWMM SVN repository Password for 'jss43': [svn ls output] jss43@keiko:~/NECI> svn co $SVN/NECI/trunk . [path is checked out]
Creating a Project
There seem to be a couple of ways of doing this. The method reccommended by the SVN book (Version Control with Subversion - O'Reilly) is to use svnadmin create. This won't work as none of us users have access to the svnadmin command. The following works quite well:
$ svn mkdir $SVN/project1
This will create directory project1 in the repository. The svn mkdir command can also be used to make a directory in your working copy, but more on this later.
Now your project has its own directory on the SVN server. Let's get the files into it.
$ ls my_code a.f90 b.f90 $ svn import my_code $SVN/project1/trunk Adding my_code/a.f90 Adding my_code/b.f90 Committed revision 1.
That's done. Notice that the project is called my_code in my directory but project1 on the server. The names could be the same. Notice that I did not place the codes directly in $SVN/project1/ but rather in the trunk/ subdirectory. This has been done as we may well want to make branches/tags of this project, in which case, they will appear as subdirectories in $SVN/project1/ and trunk/ will contain the main branch - or trunk! --alston 17:57, 19 September 2007 (BST)
To create a project from a former CVS project, you may want to first remove the CVS sub directories. This can be done using
cvs checkout project cd project find . -type d -name CVS | xargs rm -Rf
--alston 12:19, 19 September 2007 (BST)
Checking out a project
To check out a project from the repository:
$ svn checkout $SVN/project1 [PATH]
where $SVN/project1 is the URL for the project in the repository. If PATH is not given, subversion will create the same directory structure for you (if PATH is given, it will create any directories which don't exist).
svn co is shorthand for the svn checkout.
--james 13:50, 31 October 2006 (GMT)
Committing changes
If your changes involve adding/deleting files, then you need to let subversion know about this before committing:
svn add file svn rm file
svn rm also deletes the file.
To commit your local changes to the repository:
svn commit svn update
svn commit will take you into your default editor (vi, hopefully) for you to enter your (suitably verbose) log message and show which paths have been changed. svn update brings your working (BASE) copy up-to-date with the HEAD version.
If the log message is short, you can specify it on the command line:
svn ci -m 'log message'
ci is short for commit.
If you wish to only commit some of the changed files, then specify them:
svn ci [PATH]
By default, all local changes are committed to the repository. --james 13:51, 5 December 2006 (GMT)
Updating
To get the latest version of the code, simply do:
svn update
There are many useful options to update that can be seen using the svn help command. --james 16:54, 30 January 2008 (GMT)
Resolving conflicts
If a local copy has a newer version of a file that is in the repository, it can happen that some lines are conflicting (for example if the content of a line is partially changed). In this case running svn update will result in a C flag near the file in question, and will also create two extra files, for example Makefile.orig and Makefile.r10003. The merged file (Makefile in this example) will contain both conflicting lines, in the following format:
<<<<<<< .mine # Test of cronddddd ======= # Test of cronccc >>>>>>> .r10040
After having edited the file to resolve the conflict, trying to commit changes will result in:
svn: Commit failed (details follow): svn: Aborting commit: 'Makefile' remains in conflict
This is because this file is still being flagged as in conflict. To flag the conflict resolved (of course, after the file has been edited properly), type svn resolved <filename>'.
> svn resolved Makefile Resolved conflicted state of 'Makefile'
--szilard 15:00, 9 July 2008 (GMT)
If you are using a newer version of SVN, svn update might stop when a conflict is detected and ask you to choose what to do. If in doubt (or if you are using a log file template), press p to postpone and deal with as above.
--csw34 11:47, 23 September 2008 (BST)
Help
svn help <command> is very useful, as is the O'Reilly book, which available free online (and for download) here: SVN book.
--james 12:34, 31 October 2006 (GMT)
Differences
It's exceptionally useful to check the difference between a local file and the repository version. svn does provide svn diff, but this uses the standard diff engine, which is somewhat unhelpful. You (apparently) can change your diff engine (e.g. to xxdiff) in your ~/.subversion/config file, but I have never got this to work. Instead, I wrote a function using svn cat in my ~/.bashrc which does the job:
function svndiff () {
if [ "$#" -eq 0 -o "$1" == "-help" -o "$1" == "--help" ]; then
echo "Usage: svndiff <file> [-r revision_number]"
echo "Compare local copy to specified revision in the subversion"
echo "repository (default: last commit) using xxdiff."
else
svn cat $@ | xxdiff $1 - --title2 "Repository copy $1"
fi
}
xxdiff is now installed on the workstations and servers (or, e.g., vimdiff can be used).
--james 11:30, 5 December 2006 (GMT)
For vimdiff, we can't use stdin, so we use the original svndiff function:
svndiff () {
if [ "$#" -eq 0 -o "$1" == "-help" -o "$1" == "--help" ]; then
echo "Usage: svndiff <file> [-r revision_number]"
echo "Compare local copy to specified revision in the subversion"
echo "repository (default: last commit) using vimdiff."
else
tempfile='/tmp/'`basename $1`
svn cat $@ > $tempfile
vimdiff $1 $tempfile
fi
}
--alex 19:16, 6 August 2008 (BST)
I just found this version of svndiff ( http://svn.sunbase.org/repos/svnutils/trunk/src/svndiff) that is a bit more powerful than the above one (sorry James, but this one can use a choice of diff programs and allows you to use -r N:M). I'm placing a slightly modified copy here. --alston 14:15, 5 December 2006 (GMT)
meld is svn aware. Run
meld .
in an svn-controlled directory. It can also take specific filenames.
CVS users use cvs diff in just two ways. The default behaviour of cvs diff is to difference a file against the latest version in the CVS repository. If a difference against a particular revision of a file was needed, a CVS user might use cvs diff -r N file.name. Things are different with SVN. First of all, the default behaviour of svn diff is to show only local differences; that is, differences against the pristine copy in the working copy (the BASE). Second, while you could use the command svn diff -r N file.name, since SVN maintains global revision numbers, version N of your file is quite likely the same as your current version. SVN provides a different set of commands to take care of these issues.
There are three kinds of differences you'd probably like to make:
(1) A difference against the latest version in the repository:
svn diff -r HEAD <file.name> or, if you've defined James' command (above), svndiff <file.name> -r HEAD
As described in the preamble, HEAD refers to the latest version in the SVN repository.
(2) A difference against the pristine version of the file in the working copy:
svn diff <file.name> or svn diff -r BASE <file.name> or svndiff <file.name> or svndiff <file.name> -r BASE
So this is the default behaviour of svn diff. The BASE is optional.
(3) The latest revision in the repository could be the same as the BASE version, so SVN provides a way of differencing against the last revision at which the item changed before the BASE:
svn diff -r COMMITTED <file.name> or svndiff <file.name> -r COMMITTED
SVN also proivides a way of accessing the revision just before the last revision at which the item changed:
svn diff -r PREV <file.name> or svndiff <file.name> -r PREV
Though I still haven't found a situation in which I would want to use this.--alston 11:18, 30 November 2006 (GMT)
If you'd like differences against a revision of a file/directory earlier than possible using the above tags, first use svn log <file.name> to figure out the revision numbers needed. svn log will print logs for only those revisions in which your file/directory changed. Then use svndiff <file.name> -r N where N is the revision number.
PREV could be useful if you committed code without testing it and then realised it was wrong. Of course, we'd never be so incompetent... --james 11:45, 15 January 2007 (GMT)
Logs
Logs can quickly become very length, so I find piping svn log through less is a good option (particularly as I tend to be more interested in recent changes). Note that unless you specify a revision number, you will only see the logs of commits from before your last update (so if you commit code, you won't see your own log message until you update your local version).
The verbose flag prints out the changed paths as well (very handy!):
svn log -v | less
--james 13:41, 5 December 2006 (GMT)
I got this from the SVN FAQ (online):
How can I make svn diff show me just the names of the changed files, not their contents?
svn diff doesn't have an option to do this, but
* If you only are interested in the diffs between, say, revision 10 and the revision just before it,
svn log -vq -r10
does exactly what you want;
* otherwise, if you're using Unix, this works for any range of revisions:
svn log -vq -r123:456 | egrep '^ {3}[ADMR] ' | cut -c6- | sort | uniq
It's rather useful. --alston 14:02, 5 December 2006 (GMT)
Like most SVN commands, the default behaviour of svn log file.name is to give you the log messages of file.name as it is in your working copy, i.e., BASE. You may want the logs of the file as it is in the repository, i.e., the HEAD. To do this use:
svn log -r HEAD file.name
If you want the log messages of the file since your copy was checked out, use
svn log -r BASE:COMMITTED file.name or svn log -r BASE:HEAD file.name
I have yet to test to see which one works best. --james 18:25, 17 January 2007 (GMT) <-- actually Alston!
It is often useful to see all the log messages made over a given time period. This is again possible using the -r r1:r2 syntax:
svn log -r {yyyy-mm-dd}:{yyyy-mm-dd}
The date syntax is especially easy to construct in scripts.
Different revision syntax can be combined. To show all log messages since a given date (eg from 21st Jan 2007 to present), with the most recent first:
svn log -r HEAD:{2007-01-21}
A time can also be given in the format {hh:mm}. You need to quote the string to pass spaces in the date stamp, so a date and time can be given in two ways:
svn log -r HEAD:{2007-01-21T03:30}
svn log -r HEAD:{"2007-01-21 03:30"}
To get all the logs for all commits today is just:
svn log -r HEAD:{00:00}
Note that unless you specify a timestamp as well, svn assumes you wish to go from/to midnight. The range {2007-01-18}:{2007-01-20} thus includes all commits on the 18th and 19th, but none on the 20th. To include the 20th you require either {2007-01-18}:{"2007-01-20 23:59"} or {2007-01-18}:{2007-01-21} (the two are essentially but not formally equivalent). --james 19:00, 22 January 2007 (GMT)
A major trouble with svn log is that it becomes slower as the number of commits increases because it gets log messages from all the commits made. Most of the time we want just the most recent messages, which the --limit option gives us. This function (in ~/.bashrc) gives the last 20 log messages and is much faster:
svnlog() {
svn log -v --limit 20 $@ | less
}
--james 14:06, 26 February 2008 (GMT)
Status & Info
svn status returns the status of all the files in the current directory (or the path specified). This is a pain if you have lots of files produced by compiling your code/tex files, Makefiles and the ilk, which are not (and should not be) under source code management. svn status returns these marked by ?, which is not good if you have a few hundred such files. Fortunately, there is a way to change this! Open up ~/.subversion/config in your favourite text editor (which ought to be vi). Lines starting with # are comment lines. Uncomment [miscellany] and global-ignores. In the global ignores line, add any filenames and extensions you wish to ignore (e.g. *.o).
svn info gives you some useful information on a file/path (or current directory if no path is specified).
--james 12:44, 31 October 2006 (GMT)
svn status will return the status of your working copy with respect to your pristine copy located in the .svn sub-directory (the BASE).
If you wish to find out the status of your working copy with respect to the SVN repository (the HEAD), use
svn status --show-updates or svn status -u
This is non-intitutive for CVS users as cvs status always returns the status of the working copy w.r.t. the CVS repository.
--alston 16:57, 28 November 2006 (GMT)
Exporting (removing .svn)
To create a copy of code outside subversion without having to manually delete every .svn directory (!), use the export command:
svn export SVN_code nosvn
This will copy all code in SVN_code into a new directory called nosvn. This is useful if you want to move a directory within a repository. For example, to move the directory from trunk/A/DIR to trunk/B/DIR
svn export A/DIR B svn rm A/DIR svn add B/DIR svn ci
Done!
Reverting
So, you have made some changes to your code and committed them, and now realise that your changes break everything. One of the strengths of using subversion (and other source code management systems) is to allow you to rollback code, however this is not intuitive in subversion.
svn revert <filename> reverts your *local* version back to the repository version. It doesn't accept a -r flag, so you can't use it to go back to older versions in the repository. It is useful if you've made some temporary changes for testing purposes, as you can revert to your clean code with one command).
(As an aside svn revert -R . will revert all files in the current and lower directories.)
To rollback to a previous version, first you need to find out which revision number you want to revert to. Then use svn update in a cunning way:
svn update -r <revision number>
--james 13:45, 31 October 2006 (GMT)
If you just need to rollback to a previous version without any changes to it, after svn update -r, svn ci will not recognise that you wished to keep the old version. To force this to be the case, you need to use svn merge
If I wish to rollback X to revision 2, I would use
svn merge X@HEAD X@2 svn commit -m "Taking X back to revision 2 to stop it being broken." X
--alex 18:07, 18 July 2008 (BST)
Errors
- Seg fault in VIM when attempting to type in LOG message:
$ svn commit
(VIM is opened and you enter edit mode; press a key; and...)
svn: Commit failed (details follow):
--This line, and those below, will bsvn: system('vim -c 'set tw=72 et'
svn-commit.tmp') returned 11
M BIBLIOGRAPHY.bi[am592@radiant Induction]$ bash: 1: command not found
M tables/E3int.tex
The formatting in your terminal is destroyed and you need to type:
$ reset
Of course you could use svn commit -m "your message goes here", but this doesn't allow multiline log messages.
I found the following workaround. In your .bashrc file include the statement:
alias svncommit='svn commit --editor-cmd vim'
Now use the command svncommit instead of svn commit. This seems to work - at least I haven't seen the SEG fault since doing this. --alston 13:03, 13 December 2006 (GMT)
This is very odd: do you see this error every time, or only occasionally? I've never had such a problem. It could be a bizarre local configuration problem?
You could also try uncommenting the [helpers] and editor-cmd in your ~/.subversion/config file; I'd be interested to see if that also worked.
--james 13:47, 13 December 2006 (GMT)
I only encountered this problem before editing the svn config file to introduce a template so I think your solution works too James --csw34 12:39, 14 July 2008 (BST)
Branching & Tagging
When testing out new code and ideas becomes longer than a few hours, it is convenient to branch the source code, especially if you wish to share these possibly unstable changes with others. This avoids the need for committing code to the main trunk which creates reams of debug output, or breaks previously working features.
To create a branch, one can simply copy the trunk to a branch:
svn copy https://wwmm.ch.cam.ac.uk/svn/groups/alavi/CPMD/branches/QMC/trunk
https://wwmm.ch.cam.ac.uk/svn/groups/alavi/CPMD/branches/dev
This creates a new branch on the server, but does not affect the working copy. You can point your working copy to update to the branch with a switch:
svn switch https://wwmm.ch.cam.ac.uk/svn/groups/alavi/CPMD/branches/dev/trunk
This can be done if you have modified files in your local copy. It contains an implicit svn update, so should be used with care. In particular, it is worth ensuring you run svn update before the switch.
If you have modified your local files, you can commit the changes to the repository with svn commit.
--alex 12:02, 13 February 2007 (GMT)
Tags are created in exactly the same way as branches. SVN treats both the same way. But logically, we think of tags as markers that indicate a particular point in the development of the code. So, a released version could be tagged "Release-5". While a branch is conceptually quite different. So, here's how we might create a tag:
svn copy https://wwmm.ch.cam.ac.uk/svn/groups/alavi/CPMD/trunk \
https://wwmm.ch.cam.ac.uk/svn/groups/alavi/CPMD/tags/release-5.0 \
-m "Tagging Release 5.0 of the CPMD code"
--alston 17:33, 3 October 2008 (BST)
Merging branches
First commit and update to the repository of the working directory. You can merge changes from another branch to this directory by, e.g.:
svn merge -r 250:HEAD https://wwmm.ch.cam.ac.uk/svn/users/jss43/project/branches/dev
This merges all changes from revision 250 to the HEAD revision from a dev branch into the current repository. Do something similar in a working version of the trunk to merge a branch to the trunk.
--james 18:53, 16 July 2007 (BST)
Some more words about merging (I've just been doing that):
If you've been working on a branch and want to merge the final copy into the trunk do the following:
- Change directory to the branch you wish to merge into the trunk. Now you need to find out at what revision number this branch was branched off the main development in the trunk. To do this easily use
cd dev/ svn long --verbose --stop-on-copy
The log messages will stop when SVN detects the copying that had to be done to make the branch. Let's say this is revision 250.
- Now change directory to the trunk and update.
cd trunk/ svn update
- Now for a dry-run of the merge. We are going to merge the HEAD of the branch with the trunk. Now you may have made changes to the trunk *while* you were making changes to the branch. So we need to let SVN know which of those changes need keeping. This is why we need the revision number of the point when the branch was made.
svn merge --dry-run -r 250:HEAD https://wwmm.ch.cam.ac.uk/svn/users/jss43/project/branches/dev
- If all looks well, remove the '--dry-run' and repeat the command.
- You will probably need to resolve some conflicts.
NOTES: - Your log messages from the branch will *not* be included in the trunk. If you want to retain log messages use GIT (ask James). - Or do what I do and maintain a very good Changelog file. --alston 16:42, 3 October 2008 (BST)
RSS feeds
It is quite useful to have rss feeds for the repositories. Even without admin access, this is still quite straightforward (with a bit of python). I have written a python script to do this in conjunction with cron and an xlst stylesheet (slightly modified, originally from here: codingmonkeys.de). The files are hosted on the alavi website, as the wiki has rather strict rules on what can be uploaded (and appears to look unfavourably on xml files).
For each commit, it generates a highlighted diff webpage using hdiff and links to it (example). Note that the supplied version of hdiff contains bug fixes by JSS.
- Get an account/website on arwen.
- Set up key-authentication (see, e.g. handy guide or google it).
- Download svnlogfeed.tar and extract.
- Edit the replocs,user,websitepath and workdir variables in svnlogfeed.py.
- Set up the cron to run svnlogfeed.py: I use */10 * * * * /home/jss43/src/svn/svnlogfeed.py 2> /home/jss43/svnlogfeed.err.
- Subscribe to the feed(s).
It is quite low-profile: files are only copied if a new commit has been made and diffs are only generated for new commits. If a commit changes more than a certain number of files, no diff is produced (otherwise the CPU and server take a huge hammering, particularly if the commit in question is the import of an entire source tree!).
Please note that if you link to the feeds on a publicly-accessible website, then search engines can (and will) crawl them and the diffs, allowing parts of your code to become public. You can avoid this by keeping the feed addresses private, by using Raven to control access to the diffs or by using a suitable robots.txt.
--james 16:56, 30 April 2008 (BST)