Separate Transient

Motivation

In some instances, in particular with Burst Buffers, it is advantageous to create a new exodus file per timestep. Once the file at time t is closed, it can be migrated to the permanent file store or it can no longer be corrupted by a crash at time t+. If we just close the file and then create a new one, we need to replicate all of the model portion (coordinates, connectivity, BC definition) in each new file. This can result in an overhead especially in the cases where the model topology is not changing.

In this document, we investigate the possibility of extending Exodus such that we can have a "model" output file and then a separate "transient" output file for each time step (or multiple time steps). We would either provide a tool that could "join" selected timesteps onto the "model" file at the end of the run, or we could modify exodus such that it would recognize these "split" files and be able to read the data from separate files.

Mesh Portion of file:

• Assume double precision (64-bit) floats, 32-bit integers
• Assume Hex elements...

coordinates: 3 * number of nodes * 2
map:         1 * number of nodes 

connectivity 8 * number of elements
map:         1 * number of elements

Then the model portion of the exodus file will require (in 4-byte words of data):

• 32-bit integers: 7 * number of nodes + 9 * number of elements
• 64-bit integers: 8 * number of nodes + 18 * number of elements

Transient Portion of File:

	     nvar * number of nodes * 2
	     evar * number of elements * 2

2 * nvar * number of nodes + 2 * evar * number of elements

Therefore, the mesh portion of the file is approximately equivalent to

• about 3.5 nodal and 3.5 element variables with 32-bit integers or
• about 4 nodal and 9 element variables with 64-bit integers

What is needed in "transient" file:

dimensions:
	len_name = 33  -- used for variable names
	time_step = UNLIMITED
	num_nodes = 125 ; -- used for nodal variable sizing
	num_el_blk = 8 ;  -- used to determine number of element blocks
	num_nod_var = 12 ; --
	num_elem_var = 9 ; -- 
	num_el_in_blk? = 8 ; -- for each block, how many values per block

variables:
	int eb_status(num_el_blk) ;  -- is the block active or not... Needed?
	int eb_prop1(num_el_blk) ;   -- block ids used in exodus API
	double time_whole(time_step) ; -- timestep
	double vals_nod_var?(time_step, num_nodes) ; -- for each nodal variable
	char name_nod_var(num_nod_var, len_name) ; -- name of nodal variables
	char name_elem_var(num_elem_var, len_name) ; -- names of element variables
	double vals_elem_var?eb?(time_step, num_el_in_blk?) ; -- for each element variable for each block
	int elem_var_tab(num_el_blk, num_elem_var) ; -- Truth table... Needed?

Proof of concept

• The IOSS and exodus library and the io_shell executable have been modified to permit this type of execution. In io_shell, there is now the -file_per_state option which will put the model data in one file and then each timestep's transient data will be put in a separate "reduced" exodus file. [There is currently no tool to join the files].

• In the IOSS library, there is the property FILE_PER_STATE which defaults to false, but if set to true, then it will create a separate file for the model and then a separate file for each transient state.

• The exodus library was modified such that the ex_create call does not add NetCDF dims and vars that are not needed for the state file. The main change was for QA and Informational records.

TODO

• Determine whether this is a viable and useful extension
• Modify io_shell to be able to join all or a selected subset of the "transient" files onto the "model" file to create an exodus file that is readable to legacy applications.
• Modify the IOSS or exodus libraries to enable them to read the "slit" files without the need to join them into a single file.