Revamping the modules system - Revision history

131.111.115.14: /* Libraries */

2008-02-28T15:05:55Z

Libraries

← Older revision		Revision as of 15:05, 28 February 2008
Line 38:		Line 38:

	Not that I have exposed anyone other than myself to this yet of course...		Not that I have exposed anyone other than myself to this yet of course...

			=== some months later ===

			I implemented this (Pathscale and all) in the 10.2 image. No one's made any comment at all, so I guess at least it broke nothing. And personally I find it really useful.

	== Conflicts ==		== Conflicts ==

import>Cen1001: /* MPI confusion */

2006-12-19T16:35:44Z

MPI confusion

← Older revision		Revision as of 16:35, 19 December 2006
Line 17:		Line 17:
	The current attempt at a fix for this is to make a family of modules called 'environment' which load a compatible set of things. For example, we have environment/32-bit/parallel/lam/pgi on mek-quake, plus many others. The idea is that most people will just load one version of the 'environment' module in their .bash_profile and never see the underlying mpi and compiler modules at all.		The current attempt at a fix for this is to make a family of modules called 'environment' which load a compatible set of things. For example, we have environment/32-bit/parallel/lam/pgi on mek-quake, plus many others. The idea is that most people will just load one version of the 'environment' module in their .bash_profile and never see the underlying mpi and compiler modules at all.

			Having now tried this for a few months, it works rather well provided the user sets their shell up with the right package and forgets it. Attempts to change horses midstream lead to grief!

	Having now tried this for a few months, it works rather well.

	== Libraries ==		== Libraries ==

import>Cen1001: /* Default environment */

2006-12-19T16:34:16Z

Default environment

← Older revision		Revision as of 16:34, 19 December 2006
Line 5:		Line 5:
	One possible improvement would be to have all new user accounts load a module called 'basic' that loads a standard set of modules. The 'basic' module could then be updated centrally as needed. For example I currently have a problem in that one of the clusters sets users up with 'score' 'pgi64' and 'ifort_em64t' by default. Later on we got a new compiler, so we now have an 'icc' module too, I want everyone to load that because it makes sense- it will add to the functionality of the 'score' module. But unless I edit all the users' config files by hand there is no way to do this. If I have to do the editing, I might as well edit them to use 'basic'. Good idea or not?		One possible improvement would be to have all new user accounts load a module called 'basic' that loads a standard set of modules. The 'basic' module could then be updated centrally as needed. For example I currently have a problem in that one of the clusters sets users up with 'score' 'pgi64' and 'ifort_em64t' by default. Later on we got a new compiler, so we now have an 'icc' module too, I want everyone to load that because it makes sense- it will add to the functionality of the 'score' module. But unless I edit all the users' config files by hand there is no way to do this. If I have to do the editing, I might as well edit them to use 'basic'. Good idea or not?

	This idea has now been implemented on the reinstalled destiny cluster, mek-quake, and the workstations. It works OK but you have to be quite careful to purge the environment before loading it.		This idea has now been implemented on the reinstalled destiny cluster, mek-quake, and the workstations. It works OK for people who know exactly what they're doing, but you have to be quite careful to purge the environment before loading one of these package modules (ie it's best only to do it in shell startup files), and they seem to cause a certain amount of confusion amongst those who aren't used to the whole switching environment thing. Jury is still out.

	== MPI confusion ==		== MPI confusion ==

import>Cen1001: /* Address space confusion */

2006-12-05T17:10:39Z

Address space confusion

← Older revision		Revision as of 17:10, 5 December 2006
Line 58:		Line 58:
	Possibly the whole problem just needs pointing out clearly in the documentation for the machines where this is an issue.		Possibly the whole problem just needs pointing out clearly in the documentation for the machines where this is an issue.

	Current approach: give every module (except for those progs like Mathematica which genuinely work out which arch you're on and Do The Right Thing) a 32 and 64 subdirectory. Have 32 bit machines only see a module tree with 32 subdirectories populated. 64 bit machines see both but defaults are always 64. That way people have to work quite hard to mix things up.		Current approach on suse 10.1 image: give every module (except for those progs like Mathematica which genuinely work out which arch you're on and Do The Right Thing) a 32 and 64 subdirectory. Have 32 bit machines only see a module tree with 32 subdirectories populated. 64 bit machines see both but defaults are always 64. That way people have to work quite hard to mix things up.

import>Cen1001: /* Address space confusion */

2006-12-05T17:10:21Z

Address space confusion

← Older revision		Revision as of 17:10, 5 December 2006
Line 50:		Line 50:
	Maybe I should just remove the 32-bit Portland from the 64-bit machines and force everyone to use explicit compiler switches to generate a 32-bit binary. But then there's the problem of stopping people from trying to link to a 64-bit library when they're compiling 32-bit code.		Maybe I should just remove the 32-bit Portland from the 64-bit machines and force everyone to use explicit compiler switches to generate a 32-bit binary. But then there's the problem of stopping people from trying to link to a 64-bit library when they're compiling 32-bit code.

	A few months later I tried a variant of this by installing the new 64-bit cluster tardis without 32-bit Portland modules. ~~What~~ was ~~the~~ ~~first~~ ~~module~~ I ~~was~~ ~~asked~~ to ~~add?~~ That answers that question.		A few months later I tried a variant of this by installing the new 64-bit cluster tardis without 32-bit Portland modules. I was fairly soon asked to install the 32-bit module. That answers that question pretty conclusively I think.

	Two module trees, 32/ and 64/, which conflict? I never managed to make conflicts in .modulercs work quite right though.		Two module trees, 32/ and 64/, which conflict? I never managed to make conflicts in .modulercs work quite right though.
Line 57:		Line 57:

	Possibly the whole problem just needs pointing out clearly in the documentation for the machines where this is an issue.		Possibly the whole problem just needs pointing out clearly in the documentation for the machines where this is an issue.

			Current approach: give every module (except for those progs like Mathematica which genuinely work out which arch you're on and Do The Right Thing) a 32 and 64 subdirectory. Have 32 bit machines only see a module tree with 32 subdirectories populated. 64 bit machines see both but defaults are always 64. That way people have to work quite hard to mix things up.

import>Cen1001: /* Address space confusion */

2006-12-05T16:56:23Z

Address space confusion

← Older revision		Revision as of 16:56, 5 December 2006
Line 49:		Line 49:

	Maybe I should just remove the 32-bit Portland from the 64-bit machines and force everyone to use explicit compiler switches to generate a 32-bit binary. But then there's the problem of stopping people from trying to link to a 64-bit library when they're compiling 32-bit code.		Maybe I should just remove the 32-bit Portland from the 64-bit machines and force everyone to use explicit compiler switches to generate a 32-bit binary. But then there's the problem of stopping people from trying to link to a 64-bit library when they're compiling 32-bit code.

			A few months later I tried a variant of this by installing the new 64-bit cluster tardis without 32-bit Portland modules. What was the first module I was asked to add? That answers that question.

	Two module trees, 32/ and 64/, which conflict? I never managed to make conflicts in .modulercs work quite right though.		Two module trees, 32/ and 64/, which conflict? I never managed to make conflicts in .modulercs work quite right though.

import>Cen1001: /* Libraries */

2006-12-05T16:54:47Z

Libraries

← Older revision		Revision as of 16:54, 5 December 2006
Line 34:		Line 34:
	Finally, what do I do about gcc and g77 to make them use the vars? I can't wrap them because every other security patch would stamp on that. I can't afford to uninstall the system ones and maintain my own. Only putting a dummy somewhere in the path like /usr/local deals with it, and that has other problems: it can't be an NFS PATH because it has to be at the start and then you're vulnerable to the network going down. If it's in /usr/local I can't maintain it on the workstations without building more infrastructure. Does anyone here care about gcc/g77 anyway?		Finally, what do I do about gcc and g77 to make them use the vars? I can't wrap them because every other security patch would stamp on that. I can't afford to uninstall the system ones and maintain my own. Only putting a dummy somewhere in the path like /usr/local deals with it, and that has other problems: it can't be an NFS PATH because it has to be at the start and then you're vulnerable to the network going down. If it's in /usr/local I can't maintain it on the workstations without building more infrastructure. Does anyone here care about gcc/g77 anyway?

	After musing on this for a few months I had another crack at it while working on the SuSE 10.1 image, and discovered the variables LIBRARY_PATH and CPATH. These do Exactly What I Want for gcc and friends: act as a list of directories to search for libs and includes respectively and don't require wrapping the compiler in any way at all (why is this stuff not better documented? I eventually found it in a locked filing cabinet stuck in a disused lavatory etc etc). icc and ifort also support these out of the box, and it was a trivial fix to make Portland do so (siterc file and some help from the Portland FAQ). Nag is just a wrapped gcc so it also supports them, and anyone who needs to have full control just unsets them and everything goes away. I haven't tacked Pathscale yet, but since I had to wrap it extensively to make it work at all, I already have the hooks even if it doesn't turn out to support them.		After musing on this for a few months I had another crack at it while working on the SuSE 10.1 image, and discovered the variables LIBRARY_PATH and CPATH. These do Exactly What I Want for gcc and friends: act as a list of directories to search for libs and includes respectively and don't require wrapping the compiler in any way at all (why is this stuff not better documented? I eventually found it in a locked filing cabinet stuck in a disused lavatory etc etc). icc and ifort also support these out of the box, and it was a trivial fix to make Portland do so (siterc file and some help from the Portland FAQ). Nag is just a wrapped gcc so it also supports them, and anyone who needs to have full control just unsets them and everything goes away. I haven't tacked Pathscale yet, but since I had to wrap it extensively to make it work at all, I already have the hooks even if it doesn't turn out to support them out of the box.

	So basically all library modules adjust these two, and the difficult library modules (fftw and acml if I ever install it) will just have to have dependencies on the compiler modules.		So basically all library modules adjust these two, and the difficult library modules (fftw and acml if I ever install it) will just have to have dependencies on the compiler modules.

import>Cen1001: /* Libraries */

2006-12-05T16:53:55Z

Libraries

← Older revision		Revision as of 16:53, 5 December 2006
Line 32:		Line 32:
	I am also concerned about ordering effects- by using LIBDIRS and INCLUDEDIRS I am forcing an ordering of -L and -I options onto people. Is that likely to go wrong? What do I do when someone needs fine control of their linker? Tell them to mangle the variables by hand, tell them to unload all the modules and do it all the old fashioned way, suggest they run ld by hand (total control but really bad for anyone compiling MPI code), tell them to load modules in exactly the right order? Which of those is least bad?		I am also concerned about ordering effects- by using LIBDIRS and INCLUDEDIRS I am forcing an ordering of -L and -I options onto people. Is that likely to go wrong? What do I do when someone needs fine control of their linker? Tell them to mangle the variables by hand, tell them to unload all the modules and do it all the old fashioned way, suggest they run ld by hand (total control but really bad for anyone compiling MPI code), tell them to load modules in exactly the right order? Which of those is least bad?

	Finally, what do I do about gcc and g77 to make them use the vars? I can't wrap them because every other security patch would stamp on that. I can't afford to uninstall the system ones and maintain my own. Only putting a dummy somewhere in the path like /usr/local deals with it, and that has other problems: it can't be an NFS PATH because it has to be at the start and then you're vulnerable to the network going down. If it's in /usr/local I can't maintain it on the workstations without building more infrastructure. Does anyone here care about gcc/g77 anyway?		Finally, what do I do about gcc and g77 to make them use the vars? I can't wrap them because every other security patch would stamp on that. I can't afford to uninstall the system ones and maintain my own. Only putting a dummy somewhere in the path like /usr/local deals with it, and that has other problems: it can't be an NFS PATH because it has to be at the start and then you're vulnerable to the network going down. If it's in /usr/local I can't maintain it on the workstations without building more infrastructure. Does anyone here care about gcc/g77 anyway?

			After musing on this for a few months I had another crack at it while working on the SuSE 10.1 image, and discovered the variables LIBRARY_PATH and CPATH. These do Exactly What I Want for gcc and friends: act as a list of directories to search for libs and includes respectively and don't require wrapping the compiler in any way at all (why is this stuff not better documented? I eventually found it in a locked filing cabinet stuck in a disused lavatory etc etc). icc and ifort also support these out of the box, and it was a trivial fix to make Portland do so (siterc file and some help from the Portland FAQ). Nag is just a wrapped gcc so it also supports them, and anyone who needs to have full control just unsets them and everything goes away. I haven't tacked Pathscale yet, but since I had to wrap it extensively to make it work at all, I already have the hooks even if it doesn't turn out to support them.

			So basically all library modules adjust these two, and the difficult library modules (fftw and acml if I ever install it) will just have to have dependencies on the compiler modules.

			Not that I have exposed anyone other than myself to this yet of course...

	== Conflicts ==		== Conflicts ==

import>Cen1001: /* MPI confusion */

2006-12-05T16:43:54Z

MPI confusion

← Older revision		Revision as of 16:43, 5 December 2006
Line 16:		Line 16:

	The current attempt at a fix for this is to make a family of modules called 'environment' which load a compatible set of things. For example, we have environment/32-bit/parallel/lam/pgi on mek-quake, plus many others. The idea is that most people will just load one version of the 'environment' module in their .bash_profile and never see the underlying mpi and compiler modules at all.		The current attempt at a fix for this is to make a family of modules called 'environment' which load a compatible set of things. For example, we have environment/32-bit/parallel/lam/pgi on mek-quake, plus many others. The idea is that most people will just load one version of the 'environment' module in their .bash_profile and never see the underlying mpi and compiler modules at all.


			Having now tried this for a few months, it works rather well.

	== Libraries ==		== Libraries ==

import>Cen1001: /* MPI confusion */

2006-07-06T10:27:23Z

MPI confusion

← Older revision		Revision as of 10:27, 6 July 2006
Line 15:		Line 15:
	The problem with MPI is particularly acute because there is no one MPI library I can load as standard that will work for most (or even many) users so most people using MPI are going to have to fight with the modules.		The problem with MPI is particularly acute because there is no one MPI library I can load as standard that will work for most (or even many) users so most people using MPI are going to have to fight with the modules.

	The current attempt at a fix for this is to make modules called 'environment' which load a compatible set of things. For example, we have environment/32-bit/parallel/lam/pgi on mek-quake, plus many others. The idea is that most people will just load one version of the 'environment' module in their .bash_profile .		The current attempt at a fix for this is to make a family of modules called 'environment' which load a compatible set of things. For example, we have environment/32-bit/parallel/lam/pgi on mek-quake, plus many others. The idea is that most people will just load one version of the 'environment' module in their .bash_profile and never see the underlying mpi and compiler modules at all.

	== Libraries ==		== Libraries ==