cce/16.0.1 esmf/8.6.0 module error

[jedwards4b]

the esmf.mk file installed in this case has an incorrect ESMF_DIR setting:

ESMF_F90COMPILECPPFLAGS=-DESMF_NO_INTEGER_1_BYTE -DESMF_NO_INTEGER_2_BYTE -DESMF_MOAB=1 -DESMF_LAPACK=1 -DESMF_NO_ACC_SOFTWARE_STACK=1 -DESMF_NETCDF=1 -DESMF_PNETCDF=1 -DESMF_YAMLCPP=1 -DESMF_YAML=1 -DESMF_PIO=1 -DESMF_NO_OPENACC -DESMF_BOPT_O -DESMF_TESTCOMPTUNNEL -DS64=1 -DESMF_OS_Unicos=1 -DESMF_COMM=mpi -DESMF_DIR=/glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-esmf-8.6.0-iu4mshbv75wnediwfjvd4lrk5dadnk3u/spack-src
ESMF_CXXCOMPILEPATHS= -I/glade/u/apps/derecho/23.09/spack/opt/spack/esmf/8.6.0/cray-mpich/8.1.27/cce/16.0.1/iu4m/include  -I/glade/u/apps/derecho/23.09/spack/opt/spack/netcdf-c/4.9.2/cray-mpich/8.1.27/cce/16.0.1/v5bq/include -I/glade/u/apps/derecho/23.09/spack/opt/spack/parallel-netcdf/1.12.3/cray-mpich/8.1.27/cce/16.0.1/clsq/include -I/glade/u/apps/derecho/23.09/spack/opt/spack/parallelio/2.6.2/cray-mpich/8.1.27/cce/16.0.1/4zrq/include -I/glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-esmf-8.6.0-iu4mshbv75wnediwfjvd4lrk5dadnk3u/spack-src/src/prologue/yaml-cpp/include
ESMF_CXXCOMPILECPPFLAGS=-DESMF_NO_INTEGER_1_BYTE -DESMF_NO_INTEGER_2_BYTE -DESMF_MOAB=1 -DESMF_LAPACK=1 -DESMF_NO_ACC_SOFTWARE_STACK=1 -DESMF_NETCDF=1 -DESMF_PNETCDF=1 -DESMF_YAMLCPP=1 -DESMF_YAML=1 -DESMF_PIO=1 -DESMF_NO_OPENACC -DESMF_BOPT_O -DESMF_TESTCOMPTUNNEL -DS64=1 -DESMF_OS_Unicos=1 -DESMF_COMM=mpi -DESMF_DIR=/glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-esmf-8.6.0-iu4mshbv75wnediwfjvd4lrk5dadnk3u/spack-src -D__SDIR__='' -DESMF_CXXSTD=11 -DESMF_NO_POSIXIPC
ESMF_CCOMPILECPPFLAGS=-DESMF_NO_INTEGER_1_BYTE -DESMF_NO_INTEGER_2_BYTE -DESMF_MOAB=1 -DESMF_LAPACK=1 -DESMF_NO_ACC_SOFTWARE_STACK=1 -DESMF_NETCDF=1 -DESMF_PNETCDF=1 -DESMF_YAMLCPP=1 -DESMF_YAML=1 -DESMF_PIO=1 -DESMF_NO_OPENACC -DESMF_BOPT_O -DESMF_TESTCOMPTUNNEL -DS64=1 -DESMF_OS_Unicos=1 -DESMF_COMM=mpi -DESMF_DIR=/glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-esmf-8.6.0-iu4mshbv75wnediwfjvd4lrk5dadnk3u/spack-src -D__SDIR__='' -DESMF_CSTD=99
ESMF_INTERNAL_DIR=/glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-esmf-8.6.0-iu4mshbv75wnediwfjvd4lrk5dadnk3u/spack-src
ESMF_INTERNAL_MPIRUN="/glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-esmf-8.6.0-iu4mshbv75wnediwfjvd4lrk5dadnk3u/spack-src/scripts/mpirun.srun "
# ESMF_DIR: /glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-esmf-8.6.0-iu4mshbv75wnediwfjvd4lrk5dadnk3u/spack-src
#  ESMF_DIR=/glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-esmf-8.6.0-iu4mshbv75wnediwfjvd4lrk5dadnk3u/spack-src


 ls /glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/
ls: cannot open directory '/glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/': Permission denied

I compared this to the one used for intel and I also see the temp directories in that file but it works fine. The exact error is:

ftn-3013 ftn: LIMIT T_STARTF, File = ../../../../../../work/jedwards/sandboxes/cesm2_dev/cime/CIME/non_py/src/timing/perf_mod.F90, Line = 1 
  Cannot open file "/glade/derecho/scratch/csgteam/temp/spack/derecho/23.09/builds/spack-stage-esmf-8.6.0-iu4mshbv75wnediwfjvd4lrk5dadnk3u/spack-src/mod/modO/Unicos.cce.64.mpi.default/ESMF_TRACEMOD.mod" for "rb" in file "/home/jenkins/crayftn/inl/sources/m_utils.c" at line 101.
  Reason: "Permission denied".

I don't see any direct reference to this temp directory in the build instructions from cesm, I think that the esmf.mk file must be the source.

[jedwards4b]

I think that the solution may be to use the -ef flag in the esmf build?