Slow (re)build

[jedwards4b]

In a cesm coupled test where I am relinking without changing any source code cism is taking an extraordinarily long
time to build - here is an example:

mosart built in 1.548813 seconds
ww built in 1.623209 seconds
cice built in 1.782876 seconds
cam built in 2.021571 seconds
pop built in 6.230793 seconds
Component glc build complete with 39 warnings
cism built in 409.657035 seconds
Building cesm from /glade/work/jedwards/sandboxes/cesm2_x_alpha/components/cmeps/cime_config/buildexe with output to /glade/scratch/jedwards/ERS.ne30_g17.B1850.cheyenne_intel.20221012_134956_nwrlzf/bld/cesm.bldlog.221012-143222 
Total build time: 441.147668 seconds

[Katetc]

Wow. Can you explain how you run a test with only relinking? Is this on Cheyenne, and with Intel?

[jedwards4b]

After the initial test run just go to the case directory and run case.build again. Yes on cheyenne using intel.

[whlipscomb]

@Katetc: Maybe we could share a sandbox to look at this. I have a good idea of the offending CISM module, and I could play with code changes that might speed things up. However, I don't understand why this would happen with only relinking.

[Katetc]

Yes, I'll get a sandbox going and send you info. I was thinking about the python generated files causing this. I'm darn sure that I included code that would prevent regenerating and rebuilding if the files already existed. But maybe something's gone wrong with more recent python libraries or in this case.

[billsacks]

See also ESCOMP/CISM#53 for some discussion / investigation of slow build times. The rebuild issue does seem like an additional problem. Like @Katetc I also feel like we spent time at one point making sure that rebuilds would be fast (not regenerating / rebuilding everything), but maybe that has gotten broken over the last few years.

[jedwards4b]

It seems to me that this only happens under certain circumstances. I think that in this case I updated the esmf module, which should not cause cism to rebuild, but seems to have triggered that.

[billsacks]

Is it possible that cmake would want to rebuild everything given that a change in the esmf module probably results in a change in include paths and potentially other build options?