comms Module Reference

Module containing EMPIRE coupling data. More...

Public Member Functions
subroutine	allocate_data
subroutine	deallocate_data
subroutine	initialise_mpi
	subroutine to select which mpi comms to use More...
subroutine	initialise_mpi_v1
	subroutine to make EMPIRE connections and saves details into pf_control module More...
subroutine	initialise_mpi_v2
	subroutine to initialise new version of empire More...
subroutine	initialise_mpi_v3
	subroutine to initialise even newer version of empire More...
subroutine	initialise_mpi_v4
	subroutine to initialise empire communicators when the model is to be a subroutine itself More...
subroutine	initialise_mpi_v5
	subroutine to initialise empire communication pattern similarly to v2 but with multiple ensemble members per model process More...
subroutine	send_all_models (stateDim, nrhs, x, tag)
	subroutine to send all the model states to the models More...
subroutine	recv_all_models (stateDim, nrhs, x)
	subroutine to receive all the model states from the models after More...
subroutine	irecv_all_models (stateDim, nrhs, x, requests)
	subroutine to receive all the model states from the models after More...
subroutine	verify_sizes

Public Attributes
integer	cpl_mpi_comm
	the communicator between the empire codes and the model master nodes More...
integer	world_rank
	the rank of this process on MPI_COMM_WORLD More...
integer	cpl_rank
	the rank of this process on CPL_MPI_COMM More...
integer	nproc
	the total number of processes More...
integer	pf_mpi_comm
	the communicator between DA processes More...
integer	pfrank
	the rank of this process on PF_MPI_COMM More...
integer	npfs
	the total number of DA processes More...
integer, dimension(:), allocatable	gblcount
	the number of ensemble members associated with each DA process More...
integer, dimension(:), allocatable	gbldisp
	the displacements of each each ensemble member relative to pfrank=0. VERY useful for mpi_gatherv and mpi_scatterv on pf_mpi_comm More...
integer	nens
	the total number of ensemble members More...
integer	cnt
	the number of ensemble members associated with this process More...
integer, dimension(:), allocatable	particles
	the ensemble members associated with this process More...
integer, dimension(:), allocatable	cpl_mpi_comms
	communicators for if we are using empire v2 or v3 More...
integer, dimension(:), allocatable	state_dims
	state dimensions on each model process for empire v2 More...
integer, dimension(:), allocatable	state_displacements
	displacements of the various parts of the state vector for empire v2 More...
integer, dimension(:), allocatable	obs_dims
	obs dimensions on each model process for empire v3 More...
integer, dimension(:), allocatable	obs_displacements
	displacements of the various parts of the obs vector for empire v3 More...
integer	mdl_num_proc
	number of processes of each ensemble member More...
integer	pf_member_comm
	communicator for empire v3 which contains all processes of individual ensemble members More...
integer	pf_ens_comm
	communicator for empire v3 which contains all ensemble members for that specific part of the state vector More...
integer	pf_ens_rank
	rank of the process on pf_ens_comm More...
integer	pf_ens_size
	size of pf_ens_comm for comms v3 More...
integer	pf_member_rank
	rank of the process on pf_member_comm for empire v3 More...
integer	pf_member_size
	size of pf_member_comm for empire v3 More...
integer, parameter	comm_version =1

Detailed Description

Module containing EMPIRE coupling data.

Todo:

Need to see what happens if some process has no observations in comms_v3

comm_version

The integer parameter comm_version that is defined in comm_version.f90 defines the style of communication pattern used between the model and empire. There are currently 5 different patterns implemented:

• 1 = MPI SEND/RECV pairs between a single model process (single EMPIRE process per ensemble member)
• 2 = MPI GATHERV/SCATTERV between (possibly) multiple model processes (single EMPIRE process per ensemble member)
• 3 = MPI SEND/RECV pairs between multiple model processes and the same parallel process disribution in EMPIRE
• 4 = MODEL AS A SUBROUTINE OF EMPIRE

  Todo:

  Fully document how to specify the model_as_subroutine calls in src/user/model

• 5 = Similar to 2, but with multiple ensemble members for each model process (TOMCAT CASE)

For more information, see the pages Communication Methods and EMPIRE communicators for more information.

Definition at line 57 of file comms.f90.

Member Function/Subroutine Documentation

subroutine comms::allocate_data

(

)

Definition at line 107 of file comms.f90.

subroutine comms::deallocate_data

(

)

Definition at line 113 of file comms.f90.

subroutine comms::initialise_mpi

(

)

subroutine to select which mpi comms to use

Definition at line 119 of file comms.f90.

Here is the call graph for this function:

subroutine comms::initialise_mpi_v1

(

)

subroutine to make EMPIRE connections and saves details into pf_control module

Definition at line 146 of file comms.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

subroutine comms::initialise_mpi_v2

(

)

subroutine to initialise new version of empire

Definition at line 227 of file comms.f90.

subroutine comms::initialise_mpi_v3

(

)

subroutine to initialise even newer version of empire

Definition at line 398 of file comms.f90.

Here is the caller graph for this function:

subroutine comms::initialise_mpi_v4

(

)

subroutine to initialise empire communicators when the model is to be a subroutine itself

Definition at line 607 of file comms.f90.

Here is the caller graph for this function:

subroutine comms::initialise_mpi_v5

(

)

subroutine to initialise empire communication pattern similarly to v2 but with multiple ensemble members per model process

Definition at line 724 of file comms.f90.

Here is the caller graph for this function:

subroutine comms::irecv_all_models	(	integer, intent(in)	stateDim,
		integer, intent(in)	nrhs,
		real(kind=kind(1.0d0)), dimension(statedim,nrhs), intent(out)	x,
		integer, dimension(nrhs), intent(inout)	requests
	)

subroutine to receive all the model states from the models after

Definition at line 1045 of file comms.f90.

Here is the call graph for this function:

subroutine comms::recv_all_models	(	integer, intent(in)	stateDim,
		integer, intent(in)	nrhs,
		real(kind=kind(1.0d0)), dimension(statedim,nrhs), intent(out)	x
	)

subroutine to receive all the model states from the models after

Definition at line 993 of file comms.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

subroutine comms::send_all_models	(	integer, intent(in)	stateDim,
		integer, intent(in)	nrhs,
		real(kind=kind(1.0d0)), dimension(statedim,nrhs), intent(in)	x,
		integer, intent(in)	tag
	)

subroutine to send all the model states to the models

Definition at line 945 of file comms.f90.

Here is the call graph for this function:

Here is the caller graph for this function:

subroutine comms::verify_sizes

(

)

Definition at line 1108 of file comms.f90.

Here is the caller graph for this function:

Member Data Documentation

integer comms::cnt

the number of ensemble members associated with this process

Definition at line 75 of file comms.f90.

integer, parameter comm_version =1

The style of communication between the model and empire. See comm_version for an up-to-date description of the options implemented

Definition at line 41 of file comm_version.f90.

integer comms::cpl_mpi_comm

the communicator between the empire codes and the model master nodes

Definition at line 61 of file comms.f90.

integer, dimension(:), allocatable comms::cpl_mpi_comms

communicators for if we are using empire v2 or v3

Definition at line 79 of file comms.f90.

integer comms::cpl_rank

the rank of this process on CPL_MPI_COMM

Definition at line 64 of file comms.f90.

integer, dimension(:), allocatable comms::gblcount

the number of ensemble members associated with each DA process

Definition at line 69 of file comms.f90.

integer, dimension(:), allocatable comms::gbldisp

the displacements of each each ensemble member relative to pfrank=0. VERY useful for mpi_gatherv and mpi_scatterv on pf_mpi_comm

Definition at line 71 of file comms.f90.

integer comms::mdl_num_proc

number of processes of each ensemble member

Definition at line 91 of file comms.f90.

integer comms::nens

the total number of ensemble members

Definition at line 74 of file comms.f90.

integer comms::npfs

the total number of DA processes

Definition at line 68 of file comms.f90.

integer comms::nproc

the total number of processes

Definition at line 65 of file comms.f90.

integer, dimension(:), allocatable comms::obs_dims

obs dimensions on each model process for empire v3

Definition at line 86 of file comms.f90.

integer, dimension(:), allocatable comms::obs_displacements

displacements of the various parts of the obs vector for empire v3

Definition at line 88 of file comms.f90.

integer, dimension(:), allocatable comms::particles

the ensemble members associated with this process

Definition at line 77 of file comms.f90.

integer comms::pf_ens_comm

communicator for empire v3 which contains all ensemble members for that specific part of the state vector

Definition at line 96 of file comms.f90.

integer comms::pf_ens_rank

rank of the process on pf_ens_comm

Definition at line 99 of file comms.f90.

integer comms::pf_ens_size

size of pf_ens_comm for comms v3

Definition at line 100 of file comms.f90.

integer comms::pf_member_comm

communicator for empire v3 which contains all processes of individual ensemble members

Definition at line 93 of file comms.f90.

integer comms::pf_member_rank

rank of the process on pf_member_comm for empire v3

Definition at line 101 of file comms.f90.

integer comms::pf_member_size

size of pf_member_comm for empire v3

Definition at line 103 of file comms.f90.

integer comms::pf_mpi_comm

the communicator between DA processes

Definition at line 66 of file comms.f90.

integer comms::pfrank

the rank of this process on PF_MPI_COMM

Definition at line 67 of file comms.f90.

integer, dimension(:), allocatable comms::state_dims

state dimensions on each model process for empire v2

Definition at line 81 of file comms.f90.

integer, dimension(:), allocatable comms::state_displacements

displacements of the various parts of the state vector for empire v2

Definition at line 83 of file comms.f90.

integer comms::world_rank

the rank of this process on MPI_COMM_WORLD

Definition at line 63 of file comms.f90.

---

The documentation for this module was generated from the following files:

• src/utils/comms.f90
• comm_version.f90