Git-NECI

Git and NECI

I've been discussing this for a while, and I'm keen to do a trial run to see if people are comfortable with using git. The move-over from svn to git is reasonably painless, but it isn't possible to easily use both svn and git from more than one repository. I'll therefore maintain the connection with the svn (through git-svn) until such time as we might transfer fully. The upshot of this is that the changes may take a day to filter through back and forth between git and svn versions. If you use just one or the other then changes will be immediate.

I'm going to host it on my home on scepter. Access will be via ssh. At the moment, I'd like people to test checking it out, and see how it would fit into their workflow. I've included some rudimentary instructions below, but there are some tutorials online (e.g. http://git.or.cz/course/svn.html ), and man gittutorial also provides some help. I would advise creating your own local sandbox repository to play and test things out on (separate from NECI).

To get it up and running successfully, it's sensible to have a public key setup (see below). After that, try the checkout commands below.

You may also find JSS's Git page useful. --alex 17:04, 1 October 2009 (BST)

Public Key Setup

You can see if you have one with

ls ~/.ssh/id_?sa.pub

If you don't, you can generate one with:

ssh-keygen -t dsa

I'm a security slacker, so I don't use a passphrase, but I'm sure JSS can let you know about exciting combinations of keyrings and passphrases if you want to use them.

You can copy the key to scepter (if you have an rsa, change dsa to rsa)

scp ~/.ssh/id_dsa.pub @scepter:
ssh scepter
cat ~/id_dsa.pub >>~/.ssh/authorized_keys
rm ~/id_dsa.pub
exit

Now test whether you can access scepter without a password

ssh scepter

This is also possible from a machine outside chemistry. Set up the keys above (on both chimaera and scepter), and then setup a tunnel by adding something like the following to the .ssh/config on the machine outside chemistry.

Host chimaera
    hostname chimaera.ch.cam.ac.uk
    user ajwt3

Host scepter
   ProxyCommand ssh chimaera exec nc %h %p
   user ajwt3

The user means that one can now do things like

ssh scepter

even on computers where one has a different username. Cloning is now simply a matter of:

git clone scepter:~ajwt3/NECI.git SOURCE

--alex 10:08, 9 October 2009 (BST)

Setting up git prompts

There's a nice little prompt utility which allows you to add git information to your prompt and provides tab-completion for the git subcommands, options, branch names etc.

wget "http://repo.or.cz/w/git.git?a=blob_plain;f=contrib/completion/git-completion.bash;hb=HEAD" -O ~/.gitprompt.bash

then in your .bashrc add to the end:

source ~/.gitprompt.bash
PS1='\u@\h:\w$(__git_ps1 " [%s]")\$ '

Feel free to customize your prompt otherwise. Mine is

PS1="\[\033]0;${USER}@${HOSTNAME%%.*}:${PWD/#$HOME/~} ${STY}\007\]\u@\h:\w$(__git_ps1 " [%s]")\$"

Cloning the NECI repository

Go to your favourite directory

cd ~/NECI
git clone scepter:~ajwt3/NECI.git SOURCEgit

This will create a directory ~/NECI/SOURCEgit

You'll want to use the following commands for the moment

git status
git add
git commit
git pull
git push
git help 
git help status (etc.)

Eventually there will be branching and merging (next week hopefully). Then we can have some real fun. DO NOT USE git svn on this repository. It will likely break things horribly owing to creating multiple different histories.

--alex 17:00, 1 October 2009 (BST)

For ease of use you can add some extra commands to your .git/config file:

[alias]
       co = checkout
       ci = commit
[branch "master"]
       remote = origin
       merge  = refs/heads/master

The aliases reduce key-strain, and the branch section means you can type `git pull' without specifying a remote and branch.

Git on sword and tardis

git is not currently installed globally on sword and tardis, but I have installed a version local to me which all can use. Just add

export PATH=$PATH:~ajwt3/local/bin

to your .bashrc on those machines.

--alex 11:00, 6 October 2009 (BST) JSS has done a better job than me on this, including man pages on tardis. You can use his version with this in your .bashrc

export PATH=/home/jss43/local/bin:$PATH && export MANPATH=/home/jss43/local/share/man:$MANPATH 

--alex 09:43, 8 October 2009 (BST)

Catherine has very kindly installed git on tardis. To access it just

module load git

--alex 11:11, 9 October 2009 (BST)

If you'd like to move changes to your code around between different repositories without going through the central one this is easy. git is a distributed VCS and it's exactly what it's designed for.

ajwt3@ajax:~/NE-CI/SOURCEpar [master]$ git pull ../SOURCEser/ master
remote: Counting objects: 11, done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 6 (delta 5), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From ../SOURCEser
 * branch            master     -> FETCH_HEAD
Auto-merged FciMCPar.F90
Merge made by recursive.
 CCMC.F90         |   13 +-
 FciMCLogging.F90 |   70 +-
 FciMCPar.F90     | 2721 ++++++++++++++++++++++++++----------------------------
 mcpaths.F        |    2 +
 4 files changed, 1345 insertions(+), 1461 deletions(-)

Because git tracks the content, these changes will automatically be detected if you push both patches to the central repository without causing conflicts (even if you do things in different orders). Magic. Alternatively you could do this on different machines

ajwt3@tardis:~/NECI/SOURCE [master]$ git pull ajax:NE-CI/SOURCEpar/ master

You could even add some extra options in your .git/config file to make this as simple as `git pull ajax`

--alex 11:34, 6 October 2009 (BST)

Thoughts from an svn user

Just thought I'd give some tips to people who have been svn users about the main differences that I have come across, and some things to beware of.

Your repository is arranged as separate commits in separate branches. 'git branch' will show you the branch you are on, and the others in the repository. 'git checkout branchname' will allow you to swap branches, and 'git checkout -b newbranchname' will allow you to create a new branch. You can then modify the branches, make local commits to them, and merge them back in when you want (or not!). 'git status' will show you any changes to the files in the branch so you can decide whether to commit them or not. Note that checkout has a very different meaning to svn. To commit is also different. You have local commits, and commits to the main branch (you can only commit to this branch at the moment), which are done separately (they are one and the same in svn.) To make a local commit (which needs to be done before a main 'push' - gits name for pushing changes out to a non-local branch), you have to first add the code which makes up that commit. You do that with 'git add file.f90', and then to commit that to the branch as a commit, use 'git commit'. Alternatively, 'git commit -a' will add all modified files and commit in one go.

If you are in the middle of something, then want to work on something else, you can either create a new branch, or simply use 'git stash'. This will stash your changes out of sight and revert your code in that branch to the version at the last commit. You can then work on something else, and then reapply your changes you made at the stash point at a later date. To recover what you stashed and apply the changes to your current branch (it doesn't have to be the same one you stashed from), then you can use 'git stash apply'.

Once you have committed, you may want to merge your changes back into the master branch, since this is currently the only hosted branch on scepter. To do this, change branch to master, then do 'git merge branchtomerge' and the changes will be merged into master. These changes however are still only on your local copy, so you will want to use 'git push' to push them out to everyone who wants to update to get your changes. To update your code to the latest version of the master branch, you can use 'git pull' to get the changes up to date. 'git status' should then show no changes.

Finally, gitk (or qgit) will open up a nice gui, which will show all the log messages and commit history for the local repository you are on. To show the history for all the branches, you can use 'gitk --all'. The links are basically pointers to copies of the code which can be used to trance the development. As an aside, you should initially use git OR svn to make changes to neci, since changing at both will muck up the histories. Also, git and its branching facility, which is the main benefit, will only work if people make commits (local at least - they don't necessarily also need to be pushed) regularly and often. Remember, if you are only making a local commit, it won't break everyone elses code, so there is no reason not to make them!

Hopefully this is enough to get started, but feel free to add more things you find interesting/useful.

--ghb24 16:13, 5 October 2009 (BST)

Initial Setup

Some information on how this repository was setup initially

ajwt3@scepter:~> mkdir NECI.git
ajwt3@scepter:~> cd NECI.git
ajwt3@scepter:~/NECI.git> git init --bare --shared=group
Initialized empty shared Git repository in /home/ajwt3/NECI.git/
ajwt3@scepter:~> cd ..
ajwt3@scepter:~> chgrp -R alavi NECI.git/
ajwt3@scepter:~> chmod -R g+w NECI.git/

Then back to ajax

ajwt3@ajax:~/NE-CI/SOURCE [master]$ git remote add origin scepter:~ajwt3/NECI.git
ajwt3@ajax:~/NE-CI/SOURCE [master]$ git push origin master
ajwt3@ajax:~/NE-CI/SOURCEgit2 [master]$ cd ..
ajwt3@ajax:~/NE-CI$ git clone scepter:~ajwt3/NECI.git SOURCEgit2
ajwt3@ajax:~/NE-CI$ cd SOURCEgit2/
ajwt3@ajax:~/NE-CI/SOURCEgit2 [master]$ 

Thanks to JSS for pointing out the bare and shared flags. --alex 17:19, 1 October 2009 (BST)