Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

edge case in fire code causing divide by zero failure #1165

Closed
ckoven opened this issue Feb 22, 2024 · 0 comments · Fixed by #1166
Closed

edge case in fire code causing divide by zero failure #1165

ckoven opened this issue Feb 22, 2024 · 0 comments · Fixed by #1166

Comments

@ckoven
Copy link
Contributor

ckoven commented Feb 22, 2024

I ran into what I believe is a weird edge in the fire code during a long global 4x5 spinup case, which is that it is possible to have only trunk mass as a fuel on a given patch, and under this condition the model generates a divide by zero error and crashes. the first indications I got were this:

256: PIO: FATAL ERROR: Aborting... An error occured, Writing variables (number of variables = 419) to file (./fates_e3sm_perlmttr_fullmodel_4x5_test_landuse_nocomp_startfrompotentialveg_0005_bgcpostadspinup_v3_5aba095862_7dcb5ef1.elm.h0.0014-09.nc, ncid=294) using PIO_IOTYPE_PNETCDF iotype failed. 
Non blocking write for variable (FATES_FIRE_INTENSITY, varid=141) failed (Number of subarray requests/regions=1, 
Size of data local to this process = 1152). 
NetCDF: Numeric conversion not representable (err=-60). 
Aborting since the error handler was set to PIO_INTERNAL_ERROR... (/global/u2/c/cdkoven/E3SM/externals/scorpio/src/clib/pio_darray_int.c: 395)
256: Obtained 10 stack frames.

turning on debug mode led me to this:

127: forrtl: error (73): floating divide by zero
127: Image              PC                Routine            Line        Source             
127: libpthread-2.31.s  00001478E1DF2910  Unknown               Unknown  Unknown
127: e3sm.exe           0000000003F3DCB8  sfmainmod_mp_char         298  SFMainMod.F90
127: e3sm.exe           0000000003F39B45  sfmainmod_mp_fire         105  SFMainMod.F90
127: e3sm.exe           00000000035F9303  edmainmod_mp_ed_e         219  EDMainMod.F90
127: e3sm.exe           0000000000B06F97  elmfatesinterface        1149  elmfates_interfaceMod.F90
127: e3sm.exe           00000000009B6A3C  elm_driver_mp_elm        1288  elm_driver.F90
127: e3sm.exe           0000000000937F7C  lnd_comp_mct_mp_l         514  lnd_comp_mct.F90
127: e3sm.exe           0000000000491EEB  component_mod_mp_         757  component_mod.F90
127: e3sm.exe           000000000045327B  cime_comp_mod_mp_        2911  cime_comp_mod.F90
127: e3sm.exe           000000000047A81E  MAIN__                    153  cime_driver.F90
127: e3sm.exe           000000000041D23D  Unknown               Unknown  Unknown
127: libc-2.31.so       00001478E1C1A24D  __libc_start_main     Unknown  Unknown
127: e3sm.exe           000000000041D16A  Unknown               Unknown  Unknown

I have a fix that seems to work (PR coming up next). I discussed with @adrifoster about whether this is a plausible edge case to occur in the first place, and she suggested that it could maybe happen under the case that very dry fuels could burn off all fuels other than trunk, leaving only trunk fuel as a result. It seems that it is only happening in a few gridcells in Australia, so that seems consistent with the hypothesis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants