Catherine Pitt - Revision history

import>Cen1001: /* useful stuff */

2009-03-18T18:19:44Z

useful stuff

← Older revision		Revision as of 18:19, 18 March 2009
Line 19:		Line 19:
	export LD_DEBUG=all		export LD_DEBUG=all

			is my friend, as is http://www.linux.org/docs/ldp/howto/Program-Library-HOWTO/shared-libraries.html
	is my friend

	^[a-z][a-z0-9]{1,7}$		^[a-z][a-z0-9]{1,7}$

import>Cen1001: /* Library linking aide memoire */

2008-07-22T08:23:12Z

Library linking aide memoire

← Older revision		Revision as of 08:23, 22 July 2008
Line 73:		Line 73:
	chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.		chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.

	Then we get into libraries that need other libraries (this happens with MKL and also with ACML) and here it gets really nasty. Say executable file 1 needs library 1 which in turn needs library 2. Even if you compile exe1 with an rpath/runpath that will find both lib1 and lib2, this will not work (on Linux) unless lib1 has its own rpath/runpath that will find lib2, because the runtime linker only uses the runpath of the thing that has the dependency (ie the thing with the NEED in the ELF header). Solaris behaves differently in this respect, I am told (although my own experiments on CUS, which I believe runs Solaris 9, do not bear this out). So you're totally stuck with commercial libraries, because the vendor does not know where you're going to install them and so can't set the runpath of lib1 so that it can find lib2, and LD_LIBRARY_PATH is your only hope. This is where chrpath might be useful.		Then we get into libraries that need other libraries (this happens with MKL and also with ACML) and here it gets really nasty. Say executable file 1 needs library 1 which in turn needs library 2. Even if you compile exe1 with an rpath/runpath that will find both lib1 and lib2, this will not work (on Linux) unless lib1 has its own rpath/runpath that will find lib2, because the runtime linker only uses the runpath of the thing that has the dependency (ie the thing with the NEED in the ELF header). Solaris behaves differently in this respect, I am told (although my own experiments on CUS, which I believe runs Solaris 9, do not bear this out. Incidentally the Solaris equivalent of readelf is elfdump. I have certainly created a program calling a library which needs another library and despite a correct RPATH pointing to both libs in the program, the second one is not found without LD_LIBRARY_PATH). So you're totally stuck with commercial libraries, because the vendor does not know where you're going to install them and so can't set the runpath of lib1 so that it can find lib2, and LD_LIBRARY_PATH is your only hope. This is where chrpath might be useful.

	ld has a -rpath-link option for linking a library that needs another library, which tells the linker to search the contents of -rpath-link for things like lib2, but does not add stuff to the RPATH, RUNPATH, or any other thing in the ELF header that might help any exe calling the first library find the second one again at runtime! The point of this option escapes me completely at present. Anything in an -rpath option when doing this sort of link is used after the -rpath-link option contents and inserted into the header in the usual way.		ld has a -rpath-link option for linking a library that needs another library, which tells the linker to search the contents of -rpath-link for things like lib2, but does not add stuff to the RPATH, RUNPATH, or any other thing in the ELF header that might help any exe calling the first library find the second one again at runtime! The point of this option escapes me completely at present. Anything in an -rpath option when doing this sort of link is used after the -rpath-link option contents and inserted into the header in the usual way.

import>Cen1001: /* Library linking aide memoire */

2008-07-22T08:20:17Z

Library linking aide memoire

← Older revision		Revision as of 08:20, 22 July 2008
Line 73:		Line 73:
	chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.		chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.

	Then we get into libraries that need other libraries (this happens with MKL and also with ACML) and here it gets really nasty. Say executable file 1 needs library 1 which in turn needs library 2. Even if you compile exe1 with an rpath/runpath that will find both lib1 and lib2, this will not work (on Linux) unless lib1 has its own rpath/runpath that will find lib2, because the runtime linker only uses the runpath of the thing that has the dependency (ie the thing with the NEED in the ELF header). Solaris behaves differently in this respect, I am told (I ~~should~~ ~~try~~ ~~this~~ on CUS ~~before~~ ~~they~~ ~~take~~ it ~~away~~). So you're totally stuck with commercial libraries, because the vendor does not know where you're going to install them and so can't set the runpath of lib1 so that it can find lib2, and LD_LIBRARY_PATH is your only hope. This is where chrpath might be useful.		Then we get into libraries that need other libraries (this happens with MKL and also with ACML) and here it gets really nasty. Say executable file 1 needs library 1 which in turn needs library 2. Even if you compile exe1 with an rpath/runpath that will find both lib1 and lib2, this will not work (on Linux) unless lib1 has its own rpath/runpath that will find lib2, because the runtime linker only uses the runpath of the thing that has the dependency (ie the thing with the NEED in the ELF header). Solaris behaves differently in this respect, I am told (although my own experiments on CUS, which I believe runs Solaris 9, do not bear this out). So you're totally stuck with commercial libraries, because the vendor does not know where you're going to install them and so can't set the runpath of lib1 so that it can find lib2, and LD_LIBRARY_PATH is your only hope. This is where chrpath might be useful.

	ld has a -rpath-link option for linking a library that needs another library, which tells the linker to search the contents of -rpath-link for things like lib2, but does not add stuff to the RPATH, RUNPATH, or any other thing in the ELF header that might help any exe calling the first library find the second one again at runtime! The point of this option escapes me completely at present. Anything in an -rpath option when doing this sort of link is used after the -rpath-link option contents and inserted into the header in the usual way.		ld has a -rpath-link option for linking a library that needs another library, which tells the linker to search the contents of -rpath-link for things like lib2, but does not add stuff to the RPATH, RUNPATH, or any other thing in the ELF header that might help any exe calling the first library find the second one again at runtime! The point of this option escapes me completely at present. Anything in an -rpath option when doing this sort of link is used after the -rpath-link option contents and inserted into the header in the usual way.

import>Cen1001: /* Library linking aide memoire */

2008-07-22T07:45:10Z

Library linking aide memoire

← Older revision		Revision as of 07:45, 22 July 2008
Line 66:		Line 66:

	As well as RPATH, ELF also has the RUNPATH. RUNPATH is allegedly better than RPATH because it's searched after LD_LIBRARY_PATH, giving the user a way to override it. goes and tests On my 32-bit SuSE 10.2 system this turns out to be misleading; LD_LIBRARY_PATH appears to override RPATH. Maybe RUNPATH behaving differently is a Solaris thing? Wait, if RUNPATH and RPATH are <i>both</i> set (as GNU ld does with -rpath) then RPATH is ignored so RUNPATH (set) is overridden by LD_LIBRARY_PATH if present. So that does actually make sense, and makes use of -rpath a lot less evil than you might think.		As well as RPATH, ELF also has the RUNPATH. RUNPATH is allegedly better than RPATH because it's searched after LD_LIBRARY_PATH, giving the user a way to override it. goes and tests On my 32-bit SuSE 10.2 system this turns out to be misleading; LD_LIBRARY_PATH appears to override RPATH. Maybe RUNPATH behaving differently is a Solaris thing? Wait, if RUNPATH and RPATH are <i>both</i> set (as GNU ld does with -rpath) then RPATH is ignored so RUNPATH (set) is overridden by LD_LIBRARY_PATH if present. So that does actually make sense, and makes use of -rpath a lot less evil than you might think.

			Unfortunately some elderly versions of GNU ld only set RPATH and not RUNPATH when passed -rpath (eg the one on zero). Curiously enough the gcc on zero doesn't honour LIBRARY_PATH either.

	And then there is LD_RUN_PATH. This is an environment variable used by the linker at link time, if and ONLY if you didn't use a -rpath option. Its contents are then used to set the RPATH (and the RUNPATH? Who knows? Well I just tested it and it sets both as one would hope and expect). I suspect use of -rpath is better as it's more finely tuned- setting LD_RUN_PATH generally rather than using rpath per compiler would involve every binary you created having a huge RPATH, containing stuff for all the compilers you had kicking around.		And then there is LD_RUN_PATH. This is an environment variable used by the linker at link time, if and ONLY if you didn't use a -rpath option. Its contents are then used to set the RPATH (and the RUNPATH? Who knows? Well I just tested it and it sets both as one would hope and expect). I suspect use of -rpath is better as it's more finely tuned- setting LD_RUN_PATH generally rather than using rpath per compiler would involve every binary you created having a huge RPATH, containing stuff for all the compilers you had kicking around.

import>Cen1001: /* useful stuff */

2008-07-09T15:11:00Z

useful stuff

← Older revision		Revision as of 15:11, 9 July 2008
Line 35:		Line 35:

	alt-printsreen is good		alt-printsreen is good

			module snippets from http://www.lrz-muenchen.de/services/software/utilities/modules/modules_admin.html

			# make application usage profile entry
			# (do not use for very heavily used stuff like compilers)
			if { [module-info mode] == "load" } {
			exec /lrz/sys/bin/application-control pac $VERSION $env(HOSTNAME)
			}

			# need to be member of group "pac" to use
			set groups [ exec groups $env(USER) ]
			if { [ lsearch -exact $groups pac ] == -1 && [module-info mode] == "load" }
			{
			puts stderr "Your account is not in the UNIX group pac.
			Please contact LRZ support to have it entered. Aborting module load."
			break
			}

	== Library linking aide memoire ==		== Library linking aide memoire ==

import>Cen1001 at 14:44, 25 June 2008

2008-06-25T14:44:17Z

← Older revision		Revision as of 14:44, 25 June 2008
Line 12:		Line 12:

	[[Torque and Maui]]		[[Torque and Maui]]

			[[Intel Trace Analyzer and Collector]]

	== useful stuff ==		== useful stuff ==

import>Cen1001: /* useful stuff */

2008-03-27T16:25:20Z

useful stuff

← Older revision		Revision as of 16:25, 27 March 2008
Line 31:		Line 31:

	fixing up node installation sources on a Streamline cluster		fixing up node installation sources on a Streamline cluster

			alt-printsreen is good

	== Library linking aide memoire ==		== Library linking aide memoire ==

import>Cen1001: /* Library linking aide memoire */

2008-03-26T18:10:59Z

Library linking aide memoire

← Older revision		Revision as of 18:10, 26 March 2008
Line 34:		Line 34:
	== Library linking aide memoire ==		== Library linking aide memoire ==

	LIBRARY_PATH is used by many gccs at the link stage as a list of directories to search for shared libraries at link time. I have also tweaked the local PGI and Intel compilers to honour it (Pathscale and nag are gcc frontends so honour it out of the box). This is equivalent to the -L option that most compilers support.		LIBRARY_PATH is an environment variable used by many gccs at the link stage as a list of directories to search for shared libraries at link time. I have also tweaked the local PGI and Intel compilers to honour it (Pathscale and nag are gcc frontends so honour it out of the box). This is equivalent to the -L option that most compilers support.

	There is a note in one of the CV modulefiles on tardis saying that Pathscale doesn't use bog-standard 'ld' when in -Ofast mode, and so LIBRARY_PATH doesn't work in that case, and indeed this is correct. For Pathscale one must use PSC_GENFLAGS or whatever to make things work with -Ofast. See (for example) the workstation acml modules.		There is a note in one of the CV modulefiles on tardis saying that Pathscale doesn't use bog-standard 'ld' when in -Ofast mode, and so LIBRARY_PATH doesn't work in that case, and indeed this is correct. For Pathscale one must use PSC_GENFLAGS or whatever to make things work with -Ofast. See (for example) the workstation acml modules.
Line 40:		Line 40:
	LD_LIBRARY_PATH is used by the runtime linker as a list of directories to search for shared libraries at RUNTIME. Should be avoided where possible, as setting it means that every binary you call must search its contents which is a) slow and b) means you might pick up an incorrect library version if you don't manage it carefully. Especially dangerous if you set it to include NFS paths.		LD_LIBRARY_PATH is used by the runtime linker as a list of directories to search for shared libraries at RUNTIME. Should be avoided where possible, as setting it means that every binary you call must search its contents which is a) slow and b) means you might pick up an incorrect library version if you don't manage it carefully. Especially dangerous if you set it to include NFS paths.

	-rpath is a linker option that sets the RPATH in the ELF header of the resulting binary when you link the thing. At runtime the RPATH is used by the runtime linker as a list of directories to search for shared libraries. I have tweaked the local commercial compiler drivers to pass appropriate -rpath options that match the setting of LIBRARY_PATH, usually plus some compiler-specific ones (eg the particular compiler's own runtime libs)		-rpath is a linker option that sets the RPATH and RUNPATH in the ELF header of the resulting binary when you link the thing. At runtime the RPATH is used by the runtime linker as the first list of directories to search for shared libraries. I have tweaked the local commercial compiler drivers to pass appropriate -rpath options that match the setting of LIBRARY_PATH, usually plus some compiler-specific ones (eg the particular compiler's own runtime libs)

	[http://people.debian.org/~che/personal/rpath-considered-harmful Some people don't like rpath] but LD_LIBRARY_PATH is even worse, see above. (Also that article applies to using it in a Linux distro that can be upgraded in place, whereas we're using it to try to stitch together a system comprising one Linux distro and a great many multi-versioned third party packages whose authors expect people to put up with LD_LIBRARY_PATH, and I never intend to upgrade in place)		[http://people.debian.org/~che/personal/rpath-considered-harmful Some people don't like rpath] but LD_LIBRARY_PATH is even worse, see above. (Also that article applies to using it in a Linux distro that can be upgraded in place, whereas we're using it to try to stitch together a system comprising one Linux distro and a great many multi-versioned third party packages whose authors expect people to put up with LD_LIBRARY_PATH, and I never intend to upgrade in place)

	As well as RPATH, ELF also has RUNPATH. RUNPATH is allegedly better because it's searched after LD_LIBRARY_PATH, giving the user a way to override it. goes and tests On my 32-bit SuSE 10.2 system this turns out to be misleading; LD_LIBRARY_PATH appears to override RPATH. Maybe RUNPATH behaving differently is a Solaris thing? Wait, if RUNPATH and RPATH are <i>both</i> set (as GNU ld does with -rpath) then RPATH is ignored so RUNPATH (set) is overridden by LD_LIBRARY_PATH if present. So that does actually make sense, and makes use of -rpath a lot less evil.		As well as RPATH, ELF also has the RUNPATH. RUNPATH is allegedly better than RPATH because it's searched after LD_LIBRARY_PATH, giving the user a way to override it. goes and tests On my 32-bit SuSE 10.2 system this turns out to be misleading; LD_LIBRARY_PATH appears to override RPATH. Maybe RUNPATH behaving differently is a Solaris thing? Wait, if RUNPATH and RPATH are <i>both</i> set (as GNU ld does with -rpath) then RPATH is ignored so RUNPATH (set) is overridden by LD_LIBRARY_PATH if present. So that does actually make sense, and makes use of -rpath a lot less evil than you might think.

	And then there is LD_RUN_PATH. This is an environment variable used by the linker at link time, if and ONLY if you didn't use -rpath ~~(gah!)~~. Its contents are then used to set the RPATH (and the RUNPATH? Who knows? Well I just tested it and it sets both as one would hope and expect). I suspect use of -rpath is better as it's more finely tuned- setting LD_RUN_PATH generally rather than using rpath per compiler would involve every binary you created having a huge RPATH, containing stuff for all the compilers you had kicking around.		And then there is LD_RUN_PATH. This is an environment variable used by the linker at link time, if and ONLY if you didn't use a -rpath option. Its contents are then used to set the RPATH (and the RUNPATH? Who knows? Well I just tested it and it sets both as one would hope and expect). I suspect use of -rpath is better as it's more finely tuned- setting LD_RUN_PATH generally rather than using rpath per compiler would involve every binary you created having a huge RPATH, containing stuff for all the compilers you had kicking around.

	chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.		chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.
Line 52:		Line 52:
	Then we get into libraries that need other libraries (this happens with MKL and also with ACML) and here it gets really nasty. Say executable file 1 needs library 1 which in turn needs library 2. Even if you compile exe1 with an rpath/runpath that will find both lib1 and lib2, this will not work (on Linux) unless lib1 has its own rpath/runpath that will find lib2, because the runtime linker only uses the runpath of the thing that has the dependency (ie the thing with the NEED in the ELF header). Solaris behaves differently in this respect, I am told (I should try this on CUS before they take it away). So you're totally stuck with commercial libraries, because the vendor does not know where you're going to install them and so can't set the runpath of lib1 so that it can find lib2, and LD_LIBRARY_PATH is your only hope. This is where chrpath might be useful.		Then we get into libraries that need other libraries (this happens with MKL and also with ACML) and here it gets really nasty. Say executable file 1 needs library 1 which in turn needs library 2. Even if you compile exe1 with an rpath/runpath that will find both lib1 and lib2, this will not work (on Linux) unless lib1 has its own rpath/runpath that will find lib2, because the runtime linker only uses the runpath of the thing that has the dependency (ie the thing with the NEED in the ELF header). Solaris behaves differently in this respect, I am told (I should try this on CUS before they take it away). So you're totally stuck with commercial libraries, because the vendor does not know where you're going to install them and so can't set the runpath of lib1 so that it can find lib2, and LD_LIBRARY_PATH is your only hope. This is where chrpath might be useful.

			ld has a -rpath-link option for linking a library that needs another library, which tells the linker to search the contents of -rpath-link for things like lib2, but does not add stuff to the RPATH, RUNPATH, or any other thing in the ELF header that might help any exe calling the first library find the second one again at runtime! The point of this option escapes me completely at present. Anything in an -rpath option when doing this sort of link is used after the -rpath-link option contents and inserted into the header in the usual way.
⚫	CPATH is used by many C compiler drivers as a list of directories to search for headers. I think FPATH may do something similar. Equivalent to -I .

			I need to investigate $ORIGIN, whatever that is. People on the Internet occasionally sing its praises for fixing this kind of problem, but I have a feeling it's a Solaris thing.

		⚫	And finally and not very relevantly CPATH is an environment variable used by many C compiler drivers as a list of directories to search for headers. I think FPATH may do something similar. Equivalent to -I .

	== very old stuff ==		== very old stuff ==

import>Cen1001: /* Library linking aide memoire */

2008-03-26T17:59:59Z

Library linking aide memoire

← Older revision		Revision as of 17:59, 26 March 2008
Line 49:		Line 49:

	chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.		chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.

			Then we get into libraries that need other libraries (this happens with MKL and also with ACML) and here it gets really nasty. Say executable file 1 needs library 1 which in turn needs library 2. Even if you compile exe1 with an rpath/runpath that will find both lib1 and lib2, this will not work (on Linux) unless lib1 has its own rpath/runpath that will find lib2, because the runtime linker only uses the runpath of the thing that has the dependency (ie the thing with the NEED in the ELF header). Solaris behaves differently in this respect, I am told (I should try this on CUS before they take it away). So you're totally stuck with commercial libraries, because the vendor does not know where you're going to install them and so can't set the runpath of lib1 so that it can find lib2, and LD_LIBRARY_PATH is your only hope. This is where chrpath might be useful.

	CPATH is used by many C compiler drivers as a list of directories to search for headers. I think FPATH may do something similar. Equivalent to -I .		CPATH is used by many C compiler drivers as a list of directories to search for headers. I think FPATH may do something similar. Equivalent to -I .

import>Cen1001: /* Library linking aide memoire */

2008-03-26T17:42:21Z

Library linking aide memoire

← Older revision		Revision as of 17:42, 26 March 2008
Line 46:		Line 46:
	As well as RPATH, ELF also has RUNPATH. RUNPATH is allegedly better because it's searched after LD_LIBRARY_PATH, giving the user a way to override it. goes and tests On my 32-bit SuSE 10.2 system this turns out to be misleading; LD_LIBRARY_PATH appears to override RPATH. Maybe RUNPATH behaving differently is a Solaris thing? Wait, if RUNPATH and RPATH are <i>both</i> set (as GNU ld does with -rpath) then RPATH is ignored so RUNPATH (set) is overridden by LD_LIBRARY_PATH if present. So that does actually make sense, and makes use of -rpath a lot less evil.		As well as RPATH, ELF also has RUNPATH. RUNPATH is allegedly better because it's searched after LD_LIBRARY_PATH, giving the user a way to override it. goes and tests On my 32-bit SuSE 10.2 system this turns out to be misleading; LD_LIBRARY_PATH appears to override RPATH. Maybe RUNPATH behaving differently is a Solaris thing? Wait, if RUNPATH and RPATH are <i>both</i> set (as GNU ld does with -rpath) then RPATH is ignored so RUNPATH (set) is overridden by LD_LIBRARY_PATH if present. So that does actually make sense, and makes use of -rpath a lot less evil.

	And then there is LD_RUN_PATH. This is an environment variable used by the linker at link time, if and ONLY if you didn't use -rpath (gah!). Its contents are then used to set the RPATH (and the RUNPATH? Who knows!). I suspect use of -rpath is better as it's more finely tuned- setting LD_RUN_PATH generally rather than using rpath per compiler would involve every binary you created having a huge RPATH, containing stuff for all the compilers you had kicking around.		And then there is LD_RUN_PATH. This is an environment variable used by the linker at link time, if and ONLY if you didn't use -rpath (gah!). Its contents are then used to set the RPATH (and the RUNPATH? Who knows? Well I just tested it and it sets both as one would hope and expect). I suspect use of -rpath is better as it's more finely tuned- setting LD_RUN_PATH generally rather than using rpath per compiler would involve every binary you created having a huge RPATH, containing stuff for all the compilers you had kicking around.

	chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.		chrpath is a neat little program that hacks at the ELF header of an existing binary and allows you to replace or delete the RPATH or RUNPATH. Source is in ~cen1001/src/utils on liberator or you can pull it out of Debian with apt-get source.