Proposed changes to backup and archiving

Background

We need more backup space as workstation disks are getting larger and we have acquired several more clusters. We currently have about 3Tb split over three servers. The current system only lets computer officers do restores; it would be an improvement if people could access their own backups. It would also be very useful to be able to keep some backups for longer than the current two weeks.

Our existing archive server is not user accessible. It is difficult to use, unreliable, and has no free space.

New backup system

I want to buy a new backup server with at least 6Tb of space, move the backups onto that, and make them user-accessible. We would have to keep one of the old backup servers running for technical reasons: some of the older machines are configured in such a way that user accessible backups of those machines would be insecure. Eventually this problem will go away as machines are reinstalled.

There is one potential disadvantage to this proposal: having user-restorable backups means that files which are set to be 'world-readable' become readable by anyone in the entire sector, even if they don't have a user account on the machine the original file was on. Anyone who doesn't want this can protect themselves by changing their file permissions to no longer be world-readable. However IME some people don't do this even if warned, and are then unpleasantly surprised when they discover others can read their files.

The cost of such a new server would be XXX

New archive system

The other two old backup servers would immediately be free for reuse. I would make those into an improved and higher capacity archive system. This would let us clear old homespaces off clusters and workstations while keeping a read-only copy available to users. Assuming people set their file permissions appropriately on the original data, the rest of their research group will be able to access it on the archive server. The same issue with world-readable files arises.

The data won't change so we don't need regular backups of it: the two servers would be set up to mirror each other and be sited a long way apart.

There will have to be some sort of time limit on how long we keep the archive data otherwise we will just run out of space again. This might vary from account to account. One way to do it would be to have a 'deletion date' for each account which could be extended if needed. I could then warn the leader of the group who produced the data when the date came round, before actually deleting data. It is impossible for me to reliably warn the owner of the data because once they've left they don't tell me about changes in their contact details.