NECI Parallelization - Revision history

import>Jss43 at 13:25, 18 June 2008

2008-06-18T13:25:53Z

← Older revision		Revision as of 13:25, 18 June 2008
Line 64:		Line 64:
	CPMD parallel progress:		CPMD parallel progress:

			Integrals done. There might be some efficiencies to be gained via blocking on-the-fly integral calculation.
	* Can run KS determinant and exchange energy calculations.
	* Cannot do shifted integrals in parallel, but as I've discussed, I think I know how to do these without further communication. I just need to delve into some rather dark code...
	* Need to merge back into trunk.

		⚫	--[[User:jss43\|james]] 14:25, 18 June 2008 (BST)
	I'll try to look at shifted integrals this weekend/early next week. We can still start on parallelising the NECI part, so long as we realise the results are wrong...

⚫	--[[User:jss43\|james]] 18:12, 29 ~~March~~ 2008 (~~GMT~~)

	After discussion with James, it seems that if the integrals are to be equally distributed among processors, the <ij\|ab> integrals must be stored separately from the <ki\|ka> integrals. James will handle this rewrite of UMatCache.		After discussion with James, it seems that if the integrals are to be equally distributed among processors, the <ij\|ab> integrals must be stored separately from the <ki\|ka> integrals. James will handle this rewrite of UMatCache.

	--[[User:ajwt3\|alex]] 11:25, 18 June 2008 (BST)		--[[User:ajwt3\|alex]] 11:25, 18 June 2008 (BST)

			I think the phrase was "I can do some of it"...

			--[[User:jss43\|james]] 14:25, 18 June 2008 (BST)

import>Ajwt3 at 10:25, 18 June 2008

2008-06-18T10:25:01Z

← Older revision		Revision as of 10:25, 18 June 2008
Line 71:		Line 71:

	--[[User:jss43\|james]] 18:12, 29 March 2008 (GMT)		--[[User:jss43\|james]] 18:12, 29 March 2008 (GMT)

			After discussion with James, it seems that if the integrals are to be equally distributed among processors, the <ij\|ab> integrals must be stored separately from the <ki\|ka> integrals. James will handle this rewrite of UMatCache.

			--[[User:ajwt3\|alex]] 11:25, 18 June 2008 (BST)

import>Jss43 at 18:12, 29 March 2008

2008-03-29T18:12:00Z

← Older revision		Revision as of 18:12, 29 March 2008
Line 59:		Line 59:

	--[[User:ajwt3\|alex]] 16:02, 29 March 2008 (GMT)		--[[User:ajwt3\|alex]] 16:02, 29 March 2008 (GMT)

			Arse. It would be convenient if we could parallelise all types of calculation in the same way, so any later work is automatically parallel.

			CPMD parallel progress:

			* Can run KS determinant and exchange energy calculations.
			* Cannot do shifted integrals in parallel, but as I've discussed, I think I know how to do these without further communication. I just need to delve into some rather dark code...
			* Need to merge back into trunk.

			I'll try to look at shifted integrals this weekend/early next week. We can still start on parallelising the NECI part, so long as we realise the results are wrong...

			--[[User:jss43\|james]] 18:12, 29 March 2008 (GMT)

import>Ajwt3 at 16:02, 29 March 2008

2008-03-29T16:02:51Z

← Older revision		Revision as of 16:02, 29 March 2008
Line 49:		Line 49:

	--[[User:jss43\|james]] 11:43, 29 March 2008 (GMT)		--[[User:jss43\|james]] 11:43, 29 March 2008 (GMT)

			It would require different routines for star and sum. Sum is easier first (and we get MP2 out of it).
			Star will have the advantage that the whole star calculation can be parallelized, making it a lot faster.

			The blocking may well have some extra advantages - <ij\|ab> and <ij\|ac> use the same co-density ia, and would be sequentially requested,
			so probably within the same block.

			Is the CPMD parallel code shipshape?

			--[[User:ajwt3\|alex]] 16:02, 29 March 2008 (GMT)

import>Jss43 at 11:43, 29 March 2008

2008-03-29T11:43:54Z

@@ Line 1: / Line 1: @@
 It seems that there are two main aims of NECI Parallelization
 * To Reduce Integral Cache storage needs (and also to therefore compute the stored integrals in parallel)
@@ Line 35: / Line 33: @@
 --[[User:ajwt3|alex]] 04:54, 29 March 2008 (GMT)

import>Ajwt3 at 04:54, 29 March 2008

2008-03-29T04:54:35Z

← Older revision		Revision as of 04:54, 29 March 2008
Line 19:		Line 19:

	In order to parallelize efficiently, the four-index 2-electron (rather than two-index 2-electron) integrals should be used only where they are stored.		In order to parallelize efficiently, the four-index 2-electron (rather than two-index 2-electron) integrals should be used only where they are stored.

⚫	--[[User:ajwt3\|alex]] 04:54, 29 March 2008 (GMT)
	To achieve this it is possible to divide integrals up according to their first (i) index, and store only a subset of (i,j) at each compute node.		To achieve this it is possible to divide integrals up according to their first (i) index, and store only a subset of (i,j) at each compute node.
	It is similarly possible to divide excitations up across compute nodes.		It is similarly possible to divide excitations up across compute nodes.
Line 33:		Line 33:
	that it needs for those excitations (the batch size). These are collated, and then all the integrals calculated and distributed to each node, where they are used in the calculation.		that it needs for those excitations (the batch size). These are collated, and then all the integrals calculated and distributed to each node, where they are used in the calculation.
	This means that all the ffts etc are done in parallel with very little communication (although they themselves do communicate - not sure how much), and the NECI code doesn't need to wait significantly.		This means that all the ffts etc are done in parallel with very little communication (although they themselves do communicate - not sure how much), and the NECI code doesn't need to wait significantly.

		⚫	--[[User:ajwt3\|alex]] 04:54, 29 March 2008 (GMT)

import>Ajwt3 at 04:54, 29 March 2008

2008-03-29T04:54:08Z

New page

= NECI Parallelization =

It seems that there are two main aims of NECI Parallelization
* To Reduce Integral Cache storage needs (and also to therefore compute the stored integrals in parallel)
* To Sum together say 2-v sums in parallel
The two are rather intricately linked in terms of the ordering of storage and generation.
Integrals <ij|ab> are stored as ordered pairs of ordered pairs, ((i,a),(j,b))
where in (x,y), x<=y.
Two-vertex graphs (and indeed all excitations) are generated also in ordered pairs, but instead by excited from to excited to: (i,j)->(a,b). Single excitations are simply i->j.
The integrals required for calculation of a 2-vertex graph are:

Single i->a

<ik|ak>, <ik|ka>

double (i,j)->(a,b)

<ij|ab>, <ij|ba> (and for non-lowdiag, <ia|ia>, <ia|ai> (+i->j, a->b), and <ab|ab>, <ab|ba>, although all these are stored in UMat2D)

In order to parallelize efficiently, the four-index 2-electron (rather than two-index 2-electron) integrals should be used only where they are stored.
--[[User:ajwt3|alex]] 04:54, 29 March 2008 (GMT)
To achieve this it is possible to divide integrals up according to their first (i) index, and store only a subset of (i,j) at each compute node.
It is similarly possible to divide excitations up across compute nodes.

However, if integrals are to be computed in parallel, then the calling code
should execute essentially the same logic for each excitation, such that
when an integral has to be calculated rather than looked up, all compute
nodes are ready to calculate it in parallel. This leads to somewhat of a communication problem:

Parallel communication is slow (slow enough not to wait to do it?), and yet to know whether we need to calculate an integral we need to communicate.
It seems the solution is to batch the integrals:
Each node is assigned a set of excitations, and then lists a number of the integrals
that it needs for those excitations (the batch size). These are collated, and then all the integrals calculated and distributed to each node, where they are used in the calculation.
This means that all the ffts etc are done in parallel with very little communication (although they themselves do communicate - not sure how much), and the NECI code doesn't need to wait significantly.