Docswiki - User contributions [en]

Wales Group Version control

2020-01-06T23:22:05Z

Src68: Added cautionary tale section to highlight lessons to be learnt

As of 16th July 2008, all the group code is under version control. [[GMIN]], [[OPTIM]] and [[PATHSAMPLE]], along with the [[CHARMM]] and [[AMBER]] code used with them have been added to the repository, along with the official documentation. Changes to the documentation in the repository are automatically applied to the version on the website every night (at midnight) and the RSS feeds below are updated every ten minutes. The next step is to set up the nightly compilation and test suite for each code.

There are instructions on how to set up your SVN details and obtain the group code on the [[SVN setup]] page.

==RSS feeds==

RSS feeds (including diffs of files that have been changed) are generated every 10 minutes from the SVN log files any uploaded to the Wales group web server [http://www-wales.ch.cam.ac.uk/rss/ here]. There is currently one feed per top level directory in the repository:

* [http://www-wales.ch.cam.ac.uk/rss/trunk_log.xml trunk_log.xml] - contains the logs/diffs for every change to the code
* [http://www-wales.ch.cam.ac.uk/rss/gmin_log.xml gmin_log.xml] - only contains the logs/diffs when code in the GMIN directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/pathsample_log.xml pathsample_log.xml] - only contains the logs/diffs when code in the PATHSAMPLE directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/optim_log.xml optim_log.xml] - only contains the logs/diffs when code in the OPTIM directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/charmm_log.xml charmm_log.xml] - only contains the logs/diffs when code in the CHARMM31 directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/amber_log.xml amber_log.xml] - only contains the logs/diffs when code in the AMBER directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/nab_log.xml nab_log.xml] - only contains the logs/diffs when code in the NAB directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/disconnect_log.xml disconnect_log.xml] - only contains the logs/diffs when code in the DISCONNECT directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/doc_log.xml doc_log.xml] - only contains the logs/diffs when code in the DOC directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/scripts_log.xml scripts_log.xml] - only contains the logs/diffs when code in the SCRIPTS directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/ambertools_log.xml ambertools_log.xml] - only contains the logs/diffs when code in the AMBERTOOLS directory is changed

Using Firefox to look at the feeds is ok, but I suggest you set up a different RSS reader, such as [http://www.google.com/reader Google Reader] to keep them all in one place. Note that Google Reader has a delay in updating the feeds, so you won't see new logs up to an hour after they are created and are visible with Firefox.

==Access restrictions==
Raven authentication has been re-enabled for the diffs only. You can view the feeds without logging in (as this allows email clients to be used), but if you are outside the department, you will need to log in via Raven to access the diffs linked from the feed. Also, the diffs directory is now excluded from Google and other bot searches. This should hopefully prevent chunks of code ending up on Google.

==Usage tips==
What follows are a few suggestions to make using SVN a joyful experience (see cautionary tale below, to illustrate how it can go wrong). Please add any others you think are important! In general you should:

* please be careful with svn commands. svn is a very powerful tool, and it is possible to revert all changes made over a long period, which is probably not what you want. Please consult before you revert changes made by other people.
* make sure you followed the setup instructions on the [[SVN setup]] page. The procedure includes setting up a template file to ensure that all logs are in the same basic format
* run 'svn update' on a regular basis to ensure you have the latest bug fixes. It is also important to update regularly to detect any bugs that may have appeared due to changes elsewhere in the code. Please report any changes in behaviour that might indicate bugs.
* '''always''' do an 'svn update' before you do 'svn commit'
* don't forget 'svn add <filename>' to schedule a new file for addition at the next commit!
* get into the habit of running 'svn diff' before committing. And looking closely at the output... Optionally, run 'svn diff | grep Index' to see a summary list of the files that you have changed. Try not to commit extra things you didn't mean to.
* commit your changes regularly. If you wait a two months before commit your changes, you're likely to have to resolve more conflicts in parts of the code that has been changed by others in the meantime.
* if you get a conflict (C flag) when running 'svn update', find out who introduced the change and talk to them about it to make sure you can introduce your code without damaging theirs.
* consider making a development branch (more on this on the [[SVN setup]] page) if your hacking is going to involve a lot of potentially disruptive changes over a significant period of time. Talk to DJW and current developers about this first though...

* to obtain a previous version of one file. In the svn directory where that file lives use svn -r with the revision number, the file name, and pipe the output to a target. For example, to obtain revision 36240 of commons.f90 use
svn -r 36240 cat commons.f90 > commons.f90.36240
* to see a log of svn commits use svn log in the appropriate directory. The log contains the changes entered by the user who made the commit. Please fill in relevant fields for the log when you make changes.

==Cautionary Tale==
In August 2019 the author of this section (namely me) made an svn error that could have had potentially very damaging consequences. Being inexperienced with svn I followed the instructions to revert back to a previous version of the code, thinking I was just rolling back my own personal repo, clearly this was not the case and the whole repo was reverted back to a version from February 2019.

If caught quickly this error is quick and easy to fix, however the missing features, which had been added between February and August were only noticed in Jan 2020. This meant that people had committed code to a much older version of the code and everything was out of line. Fortunately in this case some crafty svn work from John was able to put it right without too much pain, but lessons should be learnt. At this stage I just want to say a quick sorry for the trouble caused and feel like I've at least learnt from it.

'''Lessons to be learnt:
* ask for advice from more experienced members when committing if you're unsure.
* try to make regular updates to your personal repo so that these errors can be spotted more quickly in the future.

==Papers==
We have a repository for preparing papers too: see [[Papers_in_preparation]] for information.

SVN setup

2020-01-06T23:06:53Z

Src68: /* Useful svn commands and good practice */

The group code is now managed under Subversion (SVN) version control. What follows is a quick setup guide to get you up and running with the Wales Group codes - [[GMIN]], [[OPTIM]] and [[PATHSAMPLE]]. Once you are set up with the source code and are happy compiling it, you should read the best practice tips on the group [[Wales Group Version control | version control ]] page.

===Getting an account on the SVN server===
You should email or speak to David (dw34) or Rosie (rgm38) to request access to the Wales Group repository. Make sure you let us know your your CRSID, the bit before @cam.ac.uk in your email address.

Once they reply, you should be able to log in using your Active Directory (AD)/Admitto password. If you're not sure what your AD password is, or have lost it - you can get one [http://www-co.ch.cam.ac.uk/facilities/ad/ here] To check, try logging in [https://svn.ch.cam.ac.uk/svn/wales/groups/wales/trunk/ here]. If you can't, you've done something wrong! Get help!

===Setting up your SVN details===
Open a shell on the machine where you plan on using the group code. To make subversion easy to use, you need to specify some locations in your .bashrc file as environment variables. We do this as follows:
<pre>
vi ~/.bashrc
</pre>
This opens the '''vi''' editor. Press '''SHIFT+G''' to go to the bottom of the file, then press '''I''' to enter insert mode. Add the following to the bottom:
<pre>
export SVN=https://svn.ch.cam.ac.uk/svn/wales/groups/wales
export PAPERSSVN=https://svn.ch.cam.ac.uk/svn/wales/groups/djwpapers
export MYSVN=https://svn.ch.cam.ac.uk/svn/CRSID
</pre>
where CRSID is the user name you use to log into Raven with. The PAPERSSVN repository contains papers in preparation, while the SVN repository contains the group codes. When you're done, we need to save he changes, press '''ESC''' to exit insert mode (check 'Insert' isn't still showing at the bottom!) and then type:
<pre>
:wq
</pre>
and press '''ENTER'''. This should exit the editor, and return you to the directory you were in before. These variables will now be automatically set when you open a new shell, but for now, we need to load them manually. We do this using the source command:
<pre>
source ~/.bashrc
</pre>
We're now ready to check if you have access to the repository from your chosen machine. Lets try listing the contents of the trunk directory. Type the following ($SVN is the environmental variable we set to the location of the Wales Group repository above):
<pre>
svn ls $SVN/trunk
</pre>
You may be prompted to validate a security certificate so press '''p''' and then '''ENTER''' to accept it permanently. Now enter the password you set above (you should only need to do this once!). You should see a list of the directories in the repository, something like this:
<pre>
AMBER/
AMBERTOOLS/
BLAS/
CAMSHIFTDATA/
CHARMM31/
CHARMM35/
CMakeModules/
COPYING
DISCONNECT/
DOC/
GMIN/
INCLUDE/
LAPACK/
NAB/
OPTIM/
PATHSAMPLE/
PLUGINS/
SCRIPTS/
THESES/
UTILS/
</pre>
NOTE: The 'trunk' contains the non-developmental version of the group codes.

Finally, before you download the code, you need to set up the log file template. This ensures that everyone submitting changes to the repository writes their log files in a similar fashion. Firstly, we need to edit the SVN config file
<pre>
vi ~/.subversion/config
</pre>
There is a section in the config file that begins as follows:
<pre>
# [helpers]
# editor-cmd = editor (vi, emacs, notepad, etc.)
</pre>
This should be changed to the following:
<pre>
[helpers]
editor-cmd = vim +"r ~/template"
</pre>
This tell subversion to open the file /home/CRSID/template when it asks for a commit log message to be written.

Interlude: while you're editing the config file, there are some other useful changes you may want to make. To have svn ignore various types of unversioned file (e.g. the products of a compilation) in the report from 'svn status', find the section labelled
<pre>
[miscellany]
</pre>
(it may be commented out with a # currently). Make sure it is uncommented (remove the #) and has no preceeding spaces on the line. Then, add a line underneath it like this:
<pre>
global-ignores = *.o *.mod lib*.a
</pre>
You can use wildcards and give all the suffixes you want to ignore, or even give explicit filenames if you want, separated by spaces as above.

Let's create the commit template file now
<pre>
vi ~/template
</pre>
Enter insert mode by pressing '''i''' and paste the following into vi:
<pre>
<one line summary>

PROGRAM
-------

SUMMARY OF CHANGES
------------------

SUBROUTINES AFFECTED
--------------------

KEYWORDS AFFECTED
-----------------

KNOWN ISSUES
------------
</pre>
Save the file by pressing '''ESC''' and then typing ''':wq'''. When committing, the first line of the template should be replaced with a one line summary of your changes. You're now good to download the group code :)

==='Checking out' the code===
The first time you want to access the code, you need to 'check it out'. This will pull the code from the repository into a local directory for your use.

'''WARNING: It has been found that if you put the svn directory too far from your /home/CRSID/, you will not be able to compile the CHARMM source. This is most likely due to the variable CHARMM uses to contain the locations of files is of a limited length! We hope to fix this problem, but for now you need to ensure it is close enough. '''

This is the furthest the CHARMM31 directory within the repository can be from your home, without causing errors. Adding one more 'x' to the svn directory name will cause problems.
<pre>
/home/csw34/svnxxxxxxxxxxx/CHARMM31
</pre>
As a result, it is HIGHLY recommended that you transfer the code to ~/svn as shown below. This also prevents you from having to edit Makefiles more than is absolutely necessary. The code is checked out as follows:
<pre>
svn co $SVN/trunk ~/svn
</pre>
Notice that I'm transferring the code to:
<pre>
/home/csw34/svn
</pre>
This path is clearly shorter than that above, and so we should have no problem compiling CHARMM if we choose to do so. You'll need to wait a while for the code to transfer! Once it has finished, you'll see a message like this:
<pre>
Checked out revision 9853.
</pre>
The number given is the version number you have just obtained. Whenever changes are make to the code and uploaded to the repository, this number is incremented. It might be increased by more than one between versions as the counter is common for all software on the SVN server, not just ours! That's it! You now have a copy of the group codes. Here are a few notes on its structure for new users.

* '''GMIN/source''' contains the [[GMIN]] source code itself.
* '''GMIN/bin''' contains the GMIN binaries when they are compiled.
* '''PATHSAMPLE''' contains the [[PATHSAMPLE]] code. It has the same directory structure as [[GMIN]].
* '''OPTIM''' contains the [[OPTIM]] code. It has the same directory structure as [[GMIN]] also.
* '''AMBER''' contains the [[AMBER]]9 source code.
* '''NAB''' contains the [[AMBER]]9 nucleic acid builder (NAB) source code.
* '''LAPACK''' and '''BLAS''' contain the maths libraries used by all the group code.
* '''CHARMM31''' contains the [[CHARMM]] source code. You need to compile [[CHARMM]] before you compile [[GMIN]] with it, and you must use the same compiler in both cases. If you don't you'll get errors. The [[OPTIM]] compilation automatically recompiles [[CHARMM]] if necessary.
*'''DOC''' contains the documentation for [[GMIN]], [[OPTIM]] and [[PATHSAMPLE]]. Whenever you introduce a new keyword or feature, it should be documented.
*'''SCRIPTS''' contains useful scripts developed by members of the group. For example, perm-pdb.py generates an OPTIM perm.allow file from an input PDB.
*'''AMBERTOOLS''' contains the full version of Amber Tools 1.2 with our updated LeaP files to fix forcefield problems we have encountered using AMBER9.

'''Note: If you want to use OPTIM with the new C++ implementation of the NEB routine, you will need to obtain the source code for that separately. See [https://wikis.ch.cam.ac.uk/wales/wiki/index.php/OPTIM here] for instructions.'''

'''Note: If you want to use AMBER you will have to copy the libraries for the N-terminal (everything with nt) from AMBERTOOLS to $AMBERHOME/dat/leap/lib. Otherwise, you will potentially use unsymmetrised force field parameters.'''

Now that you have the source codes, you should try to compile and test it. You can find a tutorial on using cmake to compile GMIN [[Compiling GMIN using CMAKE | here]]. If you'd rather not use cmake, or are having problems, a brief tutorial on compiling GMIN with CHARMM just using make can be found [[Compiling GMIN with CHARMM | here]].

===Useful svn commands and good practice===
Info on the commands you'll need to use to add your changes to the code, and retrieve the updates from the repository can be found on the Theory Sector [[SVN Page]].

'''WARNING: svn commit is very powerful and it can be easy to misuse it, therefore it is strongly encouraged to ask for advice from more experienced users when using for the first time or doing something different.

'''GOOD PRACTICE: keep your code up to date with the trunk by regularly updating and committing changes as this will make your life easier and breakages and errors will be spotted more quickly.

* revert a file in the repository to revision xx
<pre>
svn merge -c -xx <fname>

svn commit <fname>
</pre>

* revert the entire repository to revision xx with yy being the last revision to be removed

<pre>
svn merge -r yy:xx $SVN/trunk

svn commit
</pre>

So for example to revert to revision 33122 and removing all revision up to 33134:

<pre>
svn merge -r 33134:33122 $SVN/trunk

svn commit
</pre>

===The 'release' branch===
When you follow the instructions above, you are checking out the group development branch. This version of the code is what we are all working on together. As a result, it is possible that at any time, it might be somehow broken, we're only human after all!

If you are not going to be altering the code yourself and would like a stable (if possibly slightly outdated) version of the codes that should at least always compile, you can check out the current release branch code. We use this to produce the code tarballs on the website for collaborators without access to the repository. To do this, you need to follow the instructions above, but when it comes to checking out the code, replace this command:
<pre>
svn co $SVN/trunk ~/svn
</pre>
with this:
<pre>
svn co $SVN/release ~/svn
</pre>

You can then compile the codes as normal. You should <font color=red>'''NEVER''' commit anything to the release branch</font>. The branch is updated by merging changes in from the main trunk at regular intervals once the code has been checked. If you want to modify the code yourself, check out the 'trunk' as normal above. If you do accidentally commit a change - please revert it. If you are unsure how to do this, tell Chris or Jo ASAP!

Another alternative is to create your own personal development branch as detailed below. This can be useful if you'd like to be able to commit intermediate bits of your new code without having to worry about breaking things for everyone else. Please make sure that you merge in your changes when you are sure everything is working :)

===Personal development branches===
At the same level as $SVN/trunk/ in the repository is a directory $SVN/branches/ . This is the place where the development branches live. A dev branch is a copy of a particular revision of trunk/ (or the subdirectory in trunk/ for one of the programs if the work is going to be that localized...). They are made with the 'svn copy ...' command, and mean that potentially disruptive work can be under version control whilst leaving the trunk stable and well-behaved. The basic work cycle is:
* make the branch with the svn copy .... command
* check out a working copy of the new branch
* make changes in your working copy and commit them, in the usual way
* keep an eye on what's being committed to trunk during this time. Pull the changes on trunk between when you make the branch and the current head into your branch by merging. This will make sure you have the latest bug fixes etc. You can merge as many times as you need, bringing in the changes on trunk between different pairs of revision numbers as time passes. Just make sure you don't try to merge the same changeset into your branch twice.
* make more commits on your branch. After testing, you find your shiny new code is ready to go into the trunk for distribution etc.
* do one more merge of the latest trunk changesets (since the previous such merge) into your branch
* merge the changes on your branch since its creation into trunk
* do something else now... :-)

In the above list, for trunk read "trunk or the subdirectory in trunk/ for one of the programs", i.e. whichever directory you specified in the original svn copy to make the branch.

See [[Branching and Merging | here ]] for more info on the svn commands for branching and merging.

===RSS feeds===
As an added feature, we have set up RSS feeds for the SVN logs. This means that when you commit a change, the source you submitted is automatically diff'd against the previous version, and differences uploaded to the feed, along with the corresponding log message. More info on the feed locations can be found on the group [[Wales Group Version control | version control ]] page.

== Using git-svn ==
Git can interface nicely with svn.

=== Why use git? ===
Git is a more modern version control system than svn which has lots of very nice features that svn doesn't.
* You have an entire copy of the repository locally so you can use git log, git diff, and checkout and older version of the software without needing to contact the svn server with every command.
* You can make lots of commits locally before contacting the server. You only need to contact the svn server when you want to get changes from the server or push your changes to the server. This is nice because you can work on a section of the code for days, weeks or months and have all your work under local version control, but only push your changes to the server when you think it's ready for prime time.
* Making new local branches is completely trivial so you can easily work on two logically separate things (e.g. experimenting with a new potential in GMIN and adding a feature to disconnectDPS) at the same time within the same repository.

=== Clone the svn repository ===
To clone the svn repository, just copy your old stuff in ~/svn to somewhere else for safekeeping. Then,
<pre>
git svn clone https://svn.ch.cam.ac.uk/svn/wales/groups/wales/trunk/ svn/
</pre>
will download the repository to a new directory svn/
Sometimes that stalls midway through the download, so just run
<pre>
git svn fetch
</pre>
until it works.

Here is a good git [http://www.git-tower.com/blog/git-for-subversion-users-cheat-sheet-detail/ cheat-sheet] written for svn users

=== Basic Working Cycle ===

The basic cycle assumes you're doing all your work on the master branch.
The first step is to sync your local repository with the server
<pre>
git svn rebase
</pre>
For git experts: it is important to use rebase here instead of merge because svn
can only deal with linear histories.

Next make some changes to the files and commit the changes
<pre>
git commit -a -m "here is my commit message"
</pre>

Ensure that the local repository is up to date and push the commit (or commits) back to the server.
<pre>
git svn rebase
git svn dcommit
</pre>

=== Useful tools ===
The following git commands are very useful tools and pretty self explanatory
<pre>
git log
git diff
git grep
</pre>

Also, the gui gitk is very useful for visually inspecting the history
<pre>
gitk --all
</pre>

=== Advanced working cycle ===

A more advanced working cycle would take advantage of the powerful branching
ability of git. First make a working branch.
<pre>
git branch mybranch
git checkout mybranch
</pre>

Make changes and commit them to mybranch. Multiple commits are fine.

Sync your branch remotes/git-svn with the svn server (this is doing in two steps what `git svn rebase` would do in one)
<pre>
git svn fetch
</pre>
Bring your working branch up to date
<pre>
git rebase remotes/git-svn
</pre>

push your commits to the svn server
<pre>
git svn dcommit
</pre>

Note: It is important to use git rebase rather than git merge because it maintains a purely linear commit history, which is necessary for easy communication with the svn server. Before using dcommit, ensure that the changes you want to commit are arranged in a nicely linear manner on top of remotes/git-svn.

<pre>
gitk --all
</pre>
is a good tool for examining the branch structure of your repository and making sure everything is linear before a dcommit.

Note that
<pre>
git svn rebase
</pre>
is a shorcut for the two commands
<pre>
git svn fetch
git rebase remotes/git-svn
</pre>

==== Squashing commits ====
It is good to make many small commits when working locally, but when pushing to the trunk it you might want to just make a few, very well-commented commits. You can squash multiple git commits into one single commit and change the commit message at the same time using rebase.

To squash your last four commits together, use
<pre>
git rebase -i HEAD~4
</pre>
For step-by-step instructions, [http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html look here].

SVN setup

2020-01-06T23:05:47Z

Src68: /* Useful svn commands */

The group code is now managed under Subversion (SVN) version control. What follows is a quick setup guide to get you up and running with the Wales Group codes - [[GMIN]], [[OPTIM]] and [[PATHSAMPLE]]. Once you are set up with the source code and are happy compiling it, you should read the best practice tips on the group [[Wales Group Version control | version control ]] page.

===Getting an account on the SVN server===
You should email or speak to David (dw34) or Rosie (rgm38) to request access to the Wales Group repository. Make sure you let us know your your CRSID, the bit before @cam.ac.uk in your email address.

Once they reply, you should be able to log in using your Active Directory (AD)/Admitto password. If you're not sure what your AD password is, or have lost it - you can get one [http://www-co.ch.cam.ac.uk/facilities/ad/ here] To check, try logging in [https://svn.ch.cam.ac.uk/svn/wales/groups/wales/trunk/ here]. If you can't, you've done something wrong! Get help!

===Setting up your SVN details===
Open a shell on the machine where you plan on using the group code. To make subversion easy to use, you need to specify some locations in your .bashrc file as environment variables. We do this as follows:
<pre>
vi ~/.bashrc
</pre>
This opens the '''vi''' editor. Press '''SHIFT+G''' to go to the bottom of the file, then press '''I''' to enter insert mode. Add the following to the bottom:
<pre>
export SVN=https://svn.ch.cam.ac.uk/svn/wales/groups/wales
export PAPERSSVN=https://svn.ch.cam.ac.uk/svn/wales/groups/djwpapers
export MYSVN=https://svn.ch.cam.ac.uk/svn/CRSID
</pre>
where CRSID is the user name you use to log into Raven with. The PAPERSSVN repository contains papers in preparation, while the SVN repository contains the group codes. When you're done, we need to save he changes, press '''ESC''' to exit insert mode (check 'Insert' isn't still showing at the bottom!) and then type:
<pre>
:wq
</pre>
and press '''ENTER'''. This should exit the editor, and return you to the directory you were in before. These variables will now be automatically set when you open a new shell, but for now, we need to load them manually. We do this using the source command:
<pre>
source ~/.bashrc
</pre>
We're now ready to check if you have access to the repository from your chosen machine. Lets try listing the contents of the trunk directory. Type the following ($SVN is the environmental variable we set to the location of the Wales Group repository above):
<pre>
svn ls $SVN/trunk
</pre>
You may be prompted to validate a security certificate so press '''p''' and then '''ENTER''' to accept it permanently. Now enter the password you set above (you should only need to do this once!). You should see a list of the directories in the repository, something like this:
<pre>
AMBER/
AMBERTOOLS/
BLAS/
CAMSHIFTDATA/
CHARMM31/
CHARMM35/
CMakeModules/
COPYING
DISCONNECT/
DOC/
GMIN/
INCLUDE/
LAPACK/
NAB/
OPTIM/
PATHSAMPLE/
PLUGINS/
SCRIPTS/
THESES/
UTILS/
</pre>
NOTE: The 'trunk' contains the non-developmental version of the group codes.

Finally, before you download the code, you need to set up the log file template. This ensures that everyone submitting changes to the repository writes their log files in a similar fashion. Firstly, we need to edit the SVN config file
<pre>
vi ~/.subversion/config
</pre>
There is a section in the config file that begins as follows:
<pre>
# [helpers]
# editor-cmd = editor (vi, emacs, notepad, etc.)
</pre>
This should be changed to the following:
<pre>
[helpers]
editor-cmd = vim +"r ~/template"
</pre>
This tell subversion to open the file /home/CRSID/template when it asks for a commit log message to be written.

Interlude: while you're editing the config file, there are some other useful changes you may want to make. To have svn ignore various types of unversioned file (e.g. the products of a compilation) in the report from 'svn status', find the section labelled
<pre>
[miscellany]
</pre>
(it may be commented out with a # currently). Make sure it is uncommented (remove the #) and has no preceeding spaces on the line. Then, add a line underneath it like this:
<pre>
global-ignores = *.o *.mod lib*.a
</pre>
You can use wildcards and give all the suffixes you want to ignore, or even give explicit filenames if you want, separated by spaces as above.

Let's create the commit template file now
<pre>
vi ~/template
</pre>
Enter insert mode by pressing '''i''' and paste the following into vi:
<pre>
<one line summary>

PROGRAM
-------

SUMMARY OF CHANGES
------------------

SUBROUTINES AFFECTED
--------------------

KEYWORDS AFFECTED
-----------------

KNOWN ISSUES
------------
</pre>
Save the file by pressing '''ESC''' and then typing ''':wq'''. When committing, the first line of the template should be replaced with a one line summary of your changes. You're now good to download the group code :)

==='Checking out' the code===
The first time you want to access the code, you need to 'check it out'. This will pull the code from the repository into a local directory for your use.

'''WARNING: It has been found that if you put the svn directory too far from your /home/CRSID/, you will not be able to compile the CHARMM source. This is most likely due to the variable CHARMM uses to contain the locations of files is of a limited length! We hope to fix this problem, but for now you need to ensure it is close enough. '''

This is the furthest the CHARMM31 directory within the repository can be from your home, without causing errors. Adding one more 'x' to the svn directory name will cause problems.
<pre>
/home/csw34/svnxxxxxxxxxxx/CHARMM31
</pre>
As a result, it is HIGHLY recommended that you transfer the code to ~/svn as shown below. This also prevents you from having to edit Makefiles more than is absolutely necessary. The code is checked out as follows:
<pre>
svn co $SVN/trunk ~/svn
</pre>
Notice that I'm transferring the code to:
<pre>
/home/csw34/svn
</pre>
This path is clearly shorter than that above, and so we should have no problem compiling CHARMM if we choose to do so. You'll need to wait a while for the code to transfer! Once it has finished, you'll see a message like this:
<pre>
Checked out revision 9853.
</pre>
The number given is the version number you have just obtained. Whenever changes are make to the code and uploaded to the repository, this number is incremented. It might be increased by more than one between versions as the counter is common for all software on the SVN server, not just ours! That's it! You now have a copy of the group codes. Here are a few notes on its structure for new users.

* '''GMIN/source''' contains the [[GMIN]] source code itself.
* '''GMIN/bin''' contains the GMIN binaries when they are compiled.
* '''PATHSAMPLE''' contains the [[PATHSAMPLE]] code. It has the same directory structure as [[GMIN]].
* '''OPTIM''' contains the [[OPTIM]] code. It has the same directory structure as [[GMIN]] also.
* '''AMBER''' contains the [[AMBER]]9 source code.
* '''NAB''' contains the [[AMBER]]9 nucleic acid builder (NAB) source code.
* '''LAPACK''' and '''BLAS''' contain the maths libraries used by all the group code.
* '''CHARMM31''' contains the [[CHARMM]] source code. You need to compile [[CHARMM]] before you compile [[GMIN]] with it, and you must use the same compiler in both cases. If you don't you'll get errors. The [[OPTIM]] compilation automatically recompiles [[CHARMM]] if necessary.
*'''DOC''' contains the documentation for [[GMIN]], [[OPTIM]] and [[PATHSAMPLE]]. Whenever you introduce a new keyword or feature, it should be documented.
*'''SCRIPTS''' contains useful scripts developed by members of the group. For example, perm-pdb.py generates an OPTIM perm.allow file from an input PDB.
*'''AMBERTOOLS''' contains the full version of Amber Tools 1.2 with our updated LeaP files to fix forcefield problems we have encountered using AMBER9.

'''Note: If you want to use OPTIM with the new C++ implementation of the NEB routine, you will need to obtain the source code for that separately. See [https://wikis.ch.cam.ac.uk/wales/wiki/index.php/OPTIM here] for instructions.'''

'''Note: If you want to use AMBER you will have to copy the libraries for the N-terminal (everything with nt) from AMBERTOOLS to $AMBERHOME/dat/leap/lib. Otherwise, you will potentially use unsymmetrised force field parameters.'''

Now that you have the source codes, you should try to compile and test it. You can find a tutorial on using cmake to compile GMIN [[Compiling GMIN using CMAKE | here]]. If you'd rather not use cmake, or are having problems, a brief tutorial on compiling GMIN with CHARMM just using make can be found [[Compiling GMIN with CHARMM | here]].

===Useful svn commands and good practice===
Info on the commands you'll need to use to add your changes to the code, and retrieve the updates from the repository can be found on the Theory Sector [[SVN Page]].

WARNING: svn commit is very powerful and it can be easy to misuse it, therefore it is strongly encouraged to ask for advice from more experienced users when using for the first time or doing something different.

GOOD PRACTICE: keep your code up to date with the trunk by regularly updating and committing changes as this will make your life easier and breakages and errors will be spotted more quickly.

* revert a file in the repository to revision xx
<pre>
svn merge -c -xx <fname>

svn commit <fname>
</pre>

* revert the entire repository to revision xx with yy being the last revision to be removed

<pre>
svn merge -r yy:xx $SVN/trunk

svn commit
</pre>

So for example to revert to revision 33122 and removing all revision up to 33134:

<pre>
svn merge -r 33134:33122 $SVN/trunk

svn commit
</pre>

===The 'release' branch===
When you follow the instructions above, you are checking out the group development branch. This version of the code is what we are all working on together. As a result, it is possible that at any time, it might be somehow broken, we're only human after all!

If you are not going to be altering the code yourself and would like a stable (if possibly slightly outdated) version of the codes that should at least always compile, you can check out the current release branch code. We use this to produce the code tarballs on the website for collaborators without access to the repository. To do this, you need to follow the instructions above, but when it comes to checking out the code, replace this command:
<pre>
svn co $SVN/trunk ~/svn
</pre>
with this:
<pre>
svn co $SVN/release ~/svn
</pre>

You can then compile the codes as normal. You should <font color=red>'''NEVER''' commit anything to the release branch</font>. The branch is updated by merging changes in from the main trunk at regular intervals once the code has been checked. If you want to modify the code yourself, check out the 'trunk' as normal above. If you do accidentally commit a change - please revert it. If you are unsure how to do this, tell Chris or Jo ASAP!

Another alternative is to create your own personal development branch as detailed below. This can be useful if you'd like to be able to commit intermediate bits of your new code without having to worry about breaking things for everyone else. Please make sure that you merge in your changes when you are sure everything is working :)

===Personal development branches===
At the same level as $SVN/trunk/ in the repository is a directory $SVN/branches/ . This is the place where the development branches live. A dev branch is a copy of a particular revision of trunk/ (or the subdirectory in trunk/ for one of the programs if the work is going to be that localized...). They are made with the 'svn copy ...' command, and mean that potentially disruptive work can be under version control whilst leaving the trunk stable and well-behaved. The basic work cycle is:
* make the branch with the svn copy .... command
* check out a working copy of the new branch
* make changes in your working copy and commit them, in the usual way
* keep an eye on what's being committed to trunk during this time. Pull the changes on trunk between when you make the branch and the current head into your branch by merging. This will make sure you have the latest bug fixes etc. You can merge as many times as you need, bringing in the changes on trunk between different pairs of revision numbers as time passes. Just make sure you don't try to merge the same changeset into your branch twice.
* make more commits on your branch. After testing, you find your shiny new code is ready to go into the trunk for distribution etc.
* do one more merge of the latest trunk changesets (since the previous such merge) into your branch
* merge the changes on your branch since its creation into trunk
* do something else now... :-)

In the above list, for trunk read "trunk or the subdirectory in trunk/ for one of the programs", i.e. whichever directory you specified in the original svn copy to make the branch.

See [[Branching and Merging | here ]] for more info on the svn commands for branching and merging.

===RSS feeds===
As an added feature, we have set up RSS feeds for the SVN logs. This means that when you commit a change, the source you submitted is automatically diff'd against the previous version, and differences uploaded to the feed, along with the corresponding log message. More info on the feed locations can be found on the group [[Wales Group Version control | version control ]] page.

== Using git-svn ==
Git can interface nicely with svn.

=== Why use git? ===
Git is a more modern version control system than svn which has lots of very nice features that svn doesn't.
* You have an entire copy of the repository locally so you can use git log, git diff, and checkout and older version of the software without needing to contact the svn server with every command.
* You can make lots of commits locally before contacting the server. You only need to contact the svn server when you want to get changes from the server or push your changes to the server. This is nice because you can work on a section of the code for days, weeks or months and have all your work under local version control, but only push your changes to the server when you think it's ready for prime time.
* Making new local branches is completely trivial so you can easily work on two logically separate things (e.g. experimenting with a new potential in GMIN and adding a feature to disconnectDPS) at the same time within the same repository.

=== Clone the svn repository ===
To clone the svn repository, just copy your old stuff in ~/svn to somewhere else for safekeeping. Then,
<pre>
git svn clone https://svn.ch.cam.ac.uk/svn/wales/groups/wales/trunk/ svn/
</pre>
will download the repository to a new directory svn/
Sometimes that stalls midway through the download, so just run
<pre>
git svn fetch
</pre>
until it works.

Here is a good git [http://www.git-tower.com/blog/git-for-subversion-users-cheat-sheet-detail/ cheat-sheet] written for svn users

=== Basic Working Cycle ===

The basic cycle assumes you're doing all your work on the master branch.
The first step is to sync your local repository with the server
<pre>
git svn rebase
</pre>
For git experts: it is important to use rebase here instead of merge because svn
can only deal with linear histories.

Next make some changes to the files and commit the changes
<pre>
git commit -a -m "here is my commit message"
</pre>

Ensure that the local repository is up to date and push the commit (or commits) back to the server.
<pre>
git svn rebase
git svn dcommit
</pre>

=== Useful tools ===
The following git commands are very useful tools and pretty self explanatory
<pre>
git log
git diff
git grep
</pre>

Also, the gui gitk is very useful for visually inspecting the history
<pre>
gitk --all
</pre>

=== Advanced working cycle ===

A more advanced working cycle would take advantage of the powerful branching
ability of git. First make a working branch.
<pre>
git branch mybranch
git checkout mybranch
</pre>

Make changes and commit them to mybranch. Multiple commits are fine.

Sync your branch remotes/git-svn with the svn server (this is doing in two steps what `git svn rebase` would do in one)
<pre>
git svn fetch
</pre>
Bring your working branch up to date
<pre>
git rebase remotes/git-svn
</pre>

push your commits to the svn server
<pre>
git svn dcommit
</pre>

Note: It is important to use git rebase rather than git merge because it maintains a purely linear commit history, which is necessary for easy communication with the svn server. Before using dcommit, ensure that the changes you want to commit are arranged in a nicely linear manner on top of remotes/git-svn.

<pre>
gitk --all
</pre>
is a good tool for examining the branch structure of your repository and making sure everything is linear before a dcommit.

Note that
<pre>
git svn rebase
</pre>
is a shorcut for the two commands
<pre>
git svn fetch
git rebase remotes/git-svn
</pre>

==== Squashing commits ====
It is good to make many small commits when working locally, but when pushing to the trunk it you might want to just make a few, very well-commented commits. You can squash multiple git commits into one single commit and change the commit message at the same time using rebase.

To squash your last four commits together, use
<pre>
git rebase -i HEAD~4
</pre>
For step-by-step instructions, [http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html look here].