FATES-Hydro restarts fail with drought-deciduous PFT

[jennykowalcz]

As raised in this discussion, FATES-Hydro fails upon restart with a drought-deciduous PFT, with error messages saying is recruitment = F, LAI = 0, stem_water = 0, w_tot_* = NaN, and LWP = NaN. For example,

 42:  Could not find a stable solution for hydro 1D solve
 42:  
 42:  error code:            1
 15:  error diag:   0.000000000000000E+000  0.000000000000000E+000
 15:   0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000
 96:  error diag:   0.000000000000000E+000  0.000000000000000E+000
 96:   0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000
 15:  lat:   -23.6842105263159      longitidue:   302.500000000000     
 15:  is recruitment:  F
 15:  layer:            4
 15:  wb_step_err =                      NaN
 15:  q_top_eff*dt_step =   0.000000000000000E+000
 15:  w_tot_beg =                      NaN
 15:  w_tot_end =                      NaN
 15:  leaf water:                      NaN  kg/plant
 15:  stem_water:   0.000000000000000E+000  kg/plant
 15:  troot_water:   0.000000000000000E+000
 15:  aroot_water:   0.000000000000000E+000
 15:  LWP:                      NaN
 15:  dbh:    3.49675824928844     
 42:  error diag:   0.000000000000000E+000  0.000000000000000E+000
 42:   0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000
 15:  pft:            4
 15:  z nodes:    4.24889198027262        2.09944599013631     
 15:  -0.190993887372315      -0.227329413955693      -0.227329413955693     
 15:  psi_z:                      NaN  2.057457070350210E-002 -1.871740096248686E-003
 15:  -2.227828256764042E-003 -2.227828256765796E-003
 15:  vol,    theta,   H,  Psi,     kmax-
 15:  flux:            0.000000000000000E+000
 15:  l:  0.000000000000000E+000                     NaN                     NaN
 15:                      NaN
 15:                          6.053417126477062E-003
 15:  s:  1.888056051911535E-003  0.000000000000000E+000  -5551.12473567488     
 15:   -5551.14531024558     
 15:                          2.339234678527242E-002
 15:  t:  1.295734246397085E-003  0.000000000000000E+000  -5551.14718198568     
 15:                          1.202241118994975E-002
 15:  a:  2.276191063971552E-004  0.000000000000000E+000  -1845.22052352817     
 15:                     in:  2.402512259636667E-002
 15:                    out:  1.470335964754294E-004
 15:  r1:  0.257044074534647       0.330324016078973      -7.466198803568941E-003
 15:                          0.000000000000000E+000

In discussion with @mpaiao , @ckoven and @JessicaNeedham we think that it may be related to zero leaf volume. This code change prevents the restart failure but also changes the model output somewhat, so maybe there is a better solution?

[mpaiao]

@jennykowalcz Thanks for the follow up on this issue!

If the changes are small and are only happening when deciduous PFTs are enabled, then I don't see any problem, because the original code was not working with restart.

Though in the long-term, I wonder if the target in this case should just assume fully flushed leaves (and potentially no crown damage). Others may have a better sense of this.

[jennykowalcz]

If the changes are small and are only happening when deciduous PFTs are enabled, then I don't see any problem, because the original code was not working with restart.

That's true! I'll take a closer look at the output and share some comparisons here so that we can decide if this is a good way forward.