Backend

Functions
sort_fn	get_sort_function ()
	Return sorting function. More...
sort_with_idx_fn	get_sort_with_idx_function ()
	Return sorting function with associated indices. More...
sort_with_idx2_fn	get_sort_with_idx2_function ()
	Return sorting function with two associated indices. More...
void	assign_idxlist_elements_to_buckets (int *bucket_idxlist, const int *idxlist, int bucket_min_size, int idxlist_size, int nbuckets)
	Assign each element of a given index list to a bucket. More...
void	assign_idxlist_elements_to_buckets2 (int *bucket_idxlist, const int *idxlist, int bucket_min_size, int idxlist_size, int nbuckets, int bucket_stride)
	Assign each element of a given index list to a bucket. More...
int	num_procs_send_to_each_bucket (const int *bucket_idxlist, int nbuckets, int idxlist_size, MPI_Comm comm)
	Return the number of processes that send information to this bucket. More...
void	num_indices_to_send_to_each_bucket (int *n_idx_each_bucket, const int *bucket_idxlist, int idxlist_size, int nbuckets)
	Each process compute the number of indices to send to each bucket. More...
void	senders_to_bucket (int *senders_to_bucket, const int *n_idx_each_bucket, int n_procs_sending_to_bucket, int bucket_max_size, int idxlist_size, MPI_Comm comm, sort_fn sort)
	Each bucket receives an array of size n_procs_sending_to_bucket with the rank ID which send indices to that bucket. More...
void	num_indices_to_bucket_from_each_rank (int *bucket_msg_size_senders, const int *n_idx_each_bucket, const int *senders_to_bucket, int n_procs_sending_to_bucket, int bucket_max_size, int idxlist_size, MPI_Comm comm)
	Each bucket receives an array of size n_procs_sending_to_bucket with the number of indices that each rank send to that bucket. More...
void	bucket_idxlist_procs (int *bucket_ranks, const int *n_idx_each_bucket, const int *bucket_msg_size_senders, const int *senders_to_bucket, int n_procs_sending_to_bucket, int bucket_max_size, int idxlist_size, MPI_Comm comm)
	Each bucket receives an array of size n_procs_sending_to_bucket with the MPI processes owning the index list element in the original decomposition. More...
void	bucket_idxlist_elements (int *bucket_indices, const int *original_idxlist_sorted, const int *n_idx_each_bucket, const int *bucket_msg_size_senders, const int *senders_to_bucket, int n_procs_sending_to_bucket, int bucket_max_size, int idxlist_size, MPI_Comm comm)
	Each bucket receives an array of size n_procs_sending_to_bucket with the indices of the original decomposition belonging to the bucket. More...

Detailed Description

Implementation of distributed directory algorithm

Function Documentation

assign_idxlist_elements_to_buckets()

void assign_idxlist_elements_to_buckets	(	int *	bucket_idxlist,
		const int *	idxlist,
		int	bucket_min_size,
		int	idxlist_size,
		int	nbuckets
	)

Assign each element of a given index list to a bucket.

The assignment of the index list elements to a bucket is straightforward. Given N total global indices and M total buckets, bucket 0 owns elements 0 - N/M-1, bucket 1 owns elements N/M - 2*N/M-1, etc.

Parameters

[out]	bucket_idxlist	integer array with the bucket location for each element of the index list
[in]	idxlist	integer array with the values of the index list (global indices)
[in]	bucket_min_size	the smaller bucket size in the RD decomposition
[in]	idxlist_size	size of the idxlist array
[in]	nbuckets	number of buckets

assign_idxlist_elements_to_buckets2()

void assign_idxlist_elements_to_buckets2	(	int *	bucket_idxlist,
		const int *	idxlist,
		int	bucket_min_size,
		int	idxlist_size,
		int	nbuckets,
		int	bucket_stride
	)

Assign each element of a given index list to a bucket.

The assignment of the index list elements to a bucket is based on a single stride and exatrapolated from that. Given N total global indices in a stride and M total buckets, bucket 0 owns elements 0 - N/M-1, bucket 1 owns elements N/M - 2*N/M-1, etc. Then the full bucket indices can be extrapolated given the total number of indices and the stride size.

Parameters

[out]	bucket_idxlist	integer array with the bucket location for each element of the index list
[in]	idxlist	integer array with the values of the index list (global indices)
[in]	bucket_min_size	the smaller bucket size in the RD decomposition
[in]	idxlist_size	size of the idxlist array
[in]	nbuckets	number of buckets
[in]	bucket_stride	bucket stride

bucket_idxlist_elements()

void bucket_idxlist_elements	(	int *	bucket_indices,
		const int *	original_idxlist_sorted,
		const int *	n_idx_each_bucket,
		const int *	bucket_msg_size_senders,
		const int *	senders_to_bucket,
		int	n_procs_sending_to_bucket,
		int	bucket_max_size,
		int	idxlist_size,
		MPI_Comm	comm
	)

Each bucket receives an array of size n_procs_sending_to_bucket with the indices of the original decomposition belonging to the bucket.

The indices received are the one owned by the bucket, so the RD decomposition but grouped with the ranks owning them in the original decomposition.

Parameters

[out]	bucket_indices	integer array with the processes owning the index list element
[in]	original_idxlist_sorted	integer array with the index list in the original domain decomposition sorted with ascending order of the bucket ID owning the indices
[in]	n_idx_each_bucket	integer array with the number of elements to send to each bucket
[in]	bucket_msg_size_senders	integer array with the number of indices that each rank sends to bucket
[in]	senders_to_bucket	integer array with the list of ranks sending indices to bucket
[in]	n_procs_sending_to_bucket	the number of processes that send info to the bucket
[in]	bucket_max_size	Max size between all buckets
[in]	idxlist_size	size of the idxlist array
[in]	comm	MPI communicator containing all the MPI procs involved in the RD decomposition

bucket_idxlist_procs()

void bucket_idxlist_procs	(	int *	bucket_ranks,
		const int *	n_idx_each_bucket,
		const int *	bucket_msg_size_senders,
		const int *	senders_to_bucket,
		int	n_procs_sending_to_bucket,
		int	bucket_max_size,
		int	idxlist_size,
		MPI_Comm	comm
	)

Each bucket receives an array of size n_procs_sending_to_bucket with the MPI processes owning the index list element in the original decomposition.

This allows to map the indices of the bucket, so RD decomposition, with the indices in the original decomposition.

Parameters

[out]	bucket_ranks	integer array with the processes owning the index list element
[in]	n_idx_each_bucket	integer array with the number of elements to send to each bucket
[in]	bucket_msg_size_senders	integer array with the number of indices that each rank sends to bucket
[in]	senders_to_bucket	integer array with the list of ranks sending indices to bucket
[in]	n_procs_sending_to_bucket	the number of processes that send info to the bucket
[in]	bucket_max_size	Max size between all buckets
[in]	idxlist_size	size of the idxlist array
[in]	comm	MPI communicator containing all the MPI procs involved in the RD decomposition

get_sort_function()

sort_fn get_sort_function

(

)

Return sorting function.

The sorting function is chosen based on the current configuration

Returns

sorting function

get_sort_with_idx2_function()

sort_with_idx2_fn get_sort_with_idx2_function

(

)

Return sorting function with two associated indices.

The sorting function is chosen based on the current configuration

Returns

sorting function

get_sort_with_idx_function()

sort_with_idx_fn get_sort_with_idx_function

(

)

Return sorting function with associated indices.

The sorting function is chosen based on the current configuration

Returns

sorting function

num_indices_to_bucket_from_each_rank()

void num_indices_to_bucket_from_each_rank	(	int *	bucket_msg_size_senders,
		const int *	n_idx_each_bucket,
		const int *	senders_to_bucket,
		int	n_procs_sending_to_bucket,
		int	bucket_max_size,
		int	idxlist_size,
		MPI_Comm	comm
	)

Each bucket receives an array of size n_procs_sending_to_bucket with the number of indices that each rank send to that bucket.

The bucket already knows which ranks send data to it, so it is straightforward to receive info about the number of indices that it will receive from those ranks.

Parameters

[out]	bucket_msg_size_senders	integer array with the number of indices that each rank sends to bucket
[in]	n_idx_each_bucket	integer array with the number of elements to send to each bucket
[in]	senders_to_bucket	integer array with the list of ranks sending indices to bucket
[in]	n_procs_sending_to_bucket	the number of processes that send info to the bucket
[in]	bucket_max_size	Max size between all buckets
[in]	idxlist_size	size of the idxlist array
[in]	comm	MPI communicator containing all the MPI procs involved in the RD decomposition

num_indices_to_send_to_each_bucket()

void num_indices_to_send_to_each_bucket	(	int *	n_idx_each_bucket,
		const int *	bucket_idxlist,
		int	idxlist_size,
		int	nbuckets
	)

Each process compute the number of indices to send to each bucket.

The bucket_idxlist contains the bucket location for each element with the elemnts sending to the same bucket contiguous in memory.

Parameters

[out]	n_idx_each_bucket	integer array with the number of elements to send to each bucket
[in]	bucket_idxlist	integer array with the bucket location for each element of the index list
[in]	idxlist_size	size of the idxlist array
[in]	nbuckets	number of buckets

num_procs_send_to_each_bucket()

int num_procs_send_to_each_bucket	(	const int *	bucket_idxlist,
		int	nbuckets,
		int	idxlist_size,
		MPI_Comm	comm
	)

Return the number of processes that send information to this bucket.

Each process fill an array of size nbuckets with 0 or 1 if they send data to that bucket or not. The number of processes sending info to a bucket is than obtained with a call to MPI_Reduce_scatter.

Parameters

[in]	bucket_idxlist	integer array with the bucket location for each element of the index list
[in]	nbuckets	number of buckets
[in]	idxlist_size	size of the idxlist array
[in]	comm	MPI communicator containing all the MPI procs involved in the RD decomposition

Returns

the number of processes that send info to the bucket

senders_to_bucket()

void senders_to_bucket	(	int *	senders_to_bucket,
		const int *	n_idx_each_bucket,
		int	n_procs_sending_to_bucket,
		int	bucket_max_size,
		int	idxlist_size,
		MPI_Comm	comm,
		sort_fn	sort
	)

Each bucket receives an array of size n_procs_sending_to_bucket with the rank ID which send indices to that bucket.

On the receiver size, the MPI process does not know who is the source of the message, so MPI_ANY_SOURCE is used. This means that the array containing the MPI rank ID can have any order. For this reason, the array is sorted after the exchange.

Parameters

[out]	senders_to_bucket	integer array with the list of ranks sending indices to bucket
[in]	n_idx_each_bucket	integer array with the number of elements to send to each bucket
[in]	n_procs_sending_to_bucket	the number of processes that send info to the bucket
[in]	bucket_max_size	Max size between all buckets
[in]	idxlist_size	size of the idxlist array
[in]	comm	MPI communicator containing all the MPI procs involved in the RD decomposition
[in]	sort	sorting function