In this page we briefly describe the steps needed to link a model to EMPIRE. This is most easily done in combination with someone who has linked it previously, so please do contact us and we are happy to help you with this technical aspect.

Compiling the model

This may sound obvious but it typically is the most time consuming part of the whole process: be able to compile the model and run it in a configuration you want.

When you are running EMPIRE this will be in an ensemble so it might be that you are running on a different (bigger) machine than you previously ran your model. It is crucial that you are able to compile the model correctly and that the output you get is what you expect. We recommend that you are able to do this before you attempt any changes relating to setting up a connection to EMPIRE.

Changing the compiler

The compiler for the model that you need to use to connect to EMPIRE has to support MPI. If the model is already parallel using MPI then there should be no changes required here.

Typically, this is done by changing a fortran or C compiler to a wrapper that links to the MPI libraries, such as mpif90 or mpicc. Further, one should make sure that the correct MPI libraries are being used, and not dummy MPI routines as are present in some models.

Understand the model code structure

Look at the code and ask yourself the following questions:

is MPI used?
- if so, where is MPI_INIT called?
- what communicator does the model use? Is it MPI_COMM_WORLD or something else?
where is the main timestep loop of the model?
- where does it start?
- where does it end?
- where is the output made?
what are the model prognostic variables and which do I wish to use as the state vector in data assimilation?

Decide on the communicator pattern that you wish to implement

EMPIRE has different types of communication patterns. You should decide which you want to implement based on your necessities for efficiency and/or ease of implementation of the data assimilation operators. Please see the page Communication Methods for more details. At the moment we generally recommend using version 2.

Changing the communicator

Go to the place in the code where MPI_INIT is called. Copy and paste the code from one of the examples to set up the new communicator.

You will need to replace the communicator that the model uses with the one that is generated by EMPIRE. You may want to put this in a module/common block/global variable so that it is accessible anywhere required.

Further you will want to have the particle identifier available when the output is made.

Modifying the model output files

You have now turned a single model executable into an ensemble of models. You need to make sure that the output from the different models goes into separate places. To do this, you can use the particle identifier (which is simply an integer) from the previous section to rename each output file.

Typically this is done most easily by appending/prepending the output file names with this particle identifier.

Adding the connections to EMPIRE

This happens in two places: – after the model has been initialised/spun up and before the main timestepping loop – at the end of the timestep loop just before the output occurs

Collect the different components of the state vector into a single array. Send this array to EMPIRE. You should emulate the examples for doing this. See models/minimal_model directory for the examples.

Allowing for variational methods (optional)

EMPIRE has a 4DEnVar method which required the model to be run forwards in time repeatedly to find the best set of initial conditions. To do this, the model has to be returned to its state at time 0 an unknown number of times (that depends on the optimization solver).

This is controlled in a reverse communication type manner with the reverse communication flag stored in the MPI_TAG coming from EMPIRE. To make this work, you should investigate how simple it is to reset the model to time 0. This should include the forcing that the model receives (i.e. ensure the model knows that time has gone back to the start).

This is relatively undocumented as we are looking for users with a model who wish to use this method to help us refine the necessary input formats. Please do get in touch.

However, examples containing what to do based on the MPI_TAG can be found in models/linear.