Home

The Answer Changing Regression Evaluator (ACRE)

This will not change your answers for you, sorry.

But it is a (hopefully) simple sanity check, first-pass, see what the model is doing type analysis for CLM/ELM/FATES model developers and users. The tool is named as such because verifying changes to model code that have changed answers is much more difficult than verifying code changes that expected to results that are bit-4-bit comparable with previous runs.

Model users that are interested in deeper benchmarking and comparison should use more powerful tools like iLAMB. This is meant to be a first pass, boots on the ground, use before a pull request type of analysis.

Description of ACRE Workflow

Requirements to interpret the code:

1. A python interpreter and the following libraries: numpy, matplotlib, time, and scipy.
2. (Optional) The "code" library for python. This is helpful in debugging this code and is commented out.

Requirements to run the code:

1. CLM/ALM output that contains grid-cells with proximity to the locations listed in acre_controls.xml.
2. History output (h0) files
3. Graphics forwarding to your screen (you need to be able to generate a window if you are on a remote server)
4. (Optional) Restart output
5. (Optional) A folder with CLM/ELM model output which is considered baseline.

Some notes:

1. The tool is designed to evaluate both history type and restart type files. It is ideal to have hourly frequency history output. A bash script with those setting is available.
2. The tool performs single-site evaluations (no gridded analysis), although it should (hehe) work on gridded model output.
3. The tool can be customized by modifying acre_controls.xml, where different sites of interst (SOIs) can be added, and different variables can be added. *However, if a newly added variable contains a dimension composition that has not been added to the code, it will not process the variable.
4. The tool can perform a regression-test like comparison of two datasets.
5. For operating instructions, please execute: python rsc_driver.py -h

PROMPT$> python acre_driver.py --help

=======================================================================

 python acre_driver.py -h --plotmode --regressmode --restartmode
                         --test-hist-pref=<path> --base-hist-pref=<path>
                         --test-rest-pref=<path> --base-rest-pref=<path>
                         --test-name=<test> --base-name=<text>

  This script is intended to diagnose the output of several single site
  runs, as a rapid pass/fail visual analysis on ecosystem response over
  time.  Simulations have on been tested on single site runs, but were
  designed to accomodate gridded output as well.  The key requisit is to 
  have available, multiple time points of history output.


 -h --help 
     print this help message

 --plotmode
     [Optional] logical switch, turns on plotting
     default is False

 --regressmode
     [Optional] logical switch, turns on regression tests
     against a baseline. Requires user to also set --base-rest-pref
     default is False

 --restartmode
     [Optional] logical switch, turns on evaluations of restart type output
     in this mode, the code will search for .r. files and key FATES demogrpahics.
     --------------------------------------- 
     DO NOT USE THIS MODE ON NON-FATES OUTPUT
     --------------------------------------- 

 --test-hist-pref=<path>
     the full path to the folder with history files of the test
     version of output

 --base-hist-pref=<path>
     [Optional]  the full path to history files of a baseline
     version of output

 --test-rest-pref=<path>
     [Optional] the full path to the folder with the restart files of 
     the test version of output

 --base-rest-pref=<path>
     [Optional] the full path to the folder with the restart files of 
     the base version of output

 --test-name=<text>
     [Optional] a short descriptor for the test case that will be used
     for labeling plots. The default for the test case is "test".

 --base-name=<text>
     [Optional] a short descriptor for the base case that will be used
     for labeling plots. The default for the base case is "base".


=======================================================================