Density Functional Theory¶
ABINIT¶
ABINIT 9.6.2 on Spruce Knob using GCC 10.3¶
There are 4 builds of GCC 10.3 using various implementations of MPI
Using OpenMPI 3.1.6¶
The modules used:
$> module load lang/gcc/10.3.0 libs/openblas/0.3.19_gcc103 atomistic/wannier90/3.1.0_gcc103 \
libs/fftw/3.3.10_gcc103 parallel/openmpi/3.1.6_gcc103
Create the folder for building inside the sources:
/gpfs/shared/src/abinit-9.6.2/build_gcc103_ompi316
The AC9 file is written below, the last lines are only added after the fallbacks were compiled.
Commment those 6 final lines before executing ../configure:
prefix="/shared/software/atomistic/abinit/9.6.2_gcc103_ompi316"
with_debug_flavor="verbose"
with_optim_flavor="standard"
FCFLAGS_EXTRA="-fallow-argument-mismatch"
#FCFLAGS_EXTRA="-heap-arrays 1 -xAVX"
#FC_LIBS="-lstdc++ -ldl"
#CPP="icc -E"
LINALG_LIBS="-L${MD_OPENBLAS}/lib -lopenblas -lpthread"
WANNIER90_LIBS="-L${MD_WANNIER90}/lib -lwannier"
with_libxml2="/shared/software/lang/gcc/10.3.0"
enable_openmp="no"
with_fft_flavor="fftw3"
FFTW3_FCFLAGS="-I${MD_FFTW}/include"
FFTW3_LIBS="-L${MD_FFTW}/lib -lfftw3 -lfftw3f"
CC=mpicc
CXX=mpicxx
FC=mpif90
with_libxc=/gpfs/shared/src/abinit-9.6.2/build_gcc103_ompi316/fallbacks/install_fb/gnu/10.3/libxc/4.3.4
with_hdf5=/gpfs/shared/src/abinit-9.6.2/build_gcc103_ompi316/fallbacks/install_fb/gnu/10.3/hdf5/1.10.6
with_netcdf=/gpfs/shared/src/abinit-9.6.2/build_gcc103_ompi316/fallbacks/install_fb/gnu/10.3/netcdf4/4.6.3
with_netcdf_fortran=/gpfs/shared/src/abinit-9.6.2/build_gcc103_ompi316/fallbacks/install_fb/gnu/10.3/netcdf4_fortran/4.5.2
with_xmlf90=/gpfs/shared/src/abinit-9.6.2/build_gcc103_ompi316/fallbacks/install_fb/gnu/10.3/xmlf90/1.5.3.1
with_libpsml=/gpfs/shared/src/abinit-9.6.2/build_gcc103_ompi316/fallbacks/install_fb/gnu/10.3/libpsml/1.1.7
Execute:
$> ../configure
After the configure is executed a folder fallbacks, move to that folder and execute:
$> cd fallbacks
$> FCFLAGS=-fallow-argument-mismatch ./build-abinit-fallbacks.sh
After building the fallbacks the lines can be integrated to the file srih0001.ac9.
ABINIT 9.6.2 on Thorny using Intel 2021.4¶
This is the canonical configuration DFT codes for Thorny module dft_intel21:
module load dft_intel21
The ac9 file for this build:
prefix="/shared/software/atomistic/abinit/9.6.2_intel21_impi21"
with_debug_flavor="verbose"
with_optim_flavor="standard"
FCFLAGS_EXTRA="-g -traceback -heap-arrays 1 -xSKYLAKE-AVX512"
FC_LIBS="-lstdc++ -ldl"
CPP="icc -E"
WANNIER90_LIBS="-L${MD_WANNIER90}/lib -lwannier"
with_libxml2="/shared/software/lang/gcc/9.3.0"
CC=mpiicc
CXX=mpiicpc
FC=mpiifort
with_libxc=/shared/src/abinit-9.6.2/build_intel21/fallbacks/install_fb/intel/2021.4/libxc/4.3.4
with_hdf5=/shared/src/abinit-9.6.2/build_intel21/fallbacks/install_fb/intel/2021.4/hdf5/1.10.6
with_netcdf=/shared/src/abinit-9.6.2/build_intel21/fallbacks/install_fb/intel/2021.4/netcdf4/4.6.3
with_netcdf_fortran=/shared/src/abinit-9.6.2/build_intel21/fallbacks/install_fb/intel/2021.4/netcdf4_fortran/4.5.2
with_xmlf90=/shared/src/abinit-9.6.2/build_intel21/fallbacks/install_fb/intel/2021.4/xmlf90/1.5.3.1
with_libpsml=/shared/src/abinit-9.6.2/build_intel21/fallbacks/install_fb/intel/2021.4/libpsml/1.1.7
Run make and make install:
$> make
$> make install
The results of the testsuite:
Suite failed passed succeeded skipped disabled run_etime tot_etime
atompaw 0 0 0 2 0 0.00 0.02
bigdft 0 0 0 19 0 0.00 0.05
bigdft_paral 0 0 0 4 0 0.00 0.00
built-in 0 0 6 1 0 19.79 19.82
etsf_io 0 0 8 0 0 27.45 27.69
fast 0 0 11 0 0 43.82 44.49
gpu 0 0 0 7 0 0.00 0.01
libxc 1 8 26 0 0 457.34 459.65
mpiio 0 0 13 4 0 134.70 140.90
paral 1 8 39 84 0 591.17 595.17
psml 0 2 12 0 0 90.26 90.91
seq 0 0 0 18 0 0.00 0.02
tutomultibinit 0 2 4 0 0 163.75 165.02
tutoparal 0 1 0 27 0 46.55 46.97
tutoplugs 0 4 0 0 0 52.90 53.11
tutorespfn 1 8 20 2 0 922.51 926.44
tutorial 2 10 47 0 0 2210.86 2213.49
unitary 0 0 18 20 0 75.31 75.71
v1 0 1 73 0 0 259.79 262.73
v2 1 14 64 0 0 260.72 263.70
v3 2 12 64 0 0 333.02 337.28
v4 0 10 51 0 0 242.36 245.63
v5 3 12 58 0 0 690.92 696.28
v6 0 8 53 0 0 422.96 426.97
v67mbpt 0 9 16 0 0 185.33 187.75
v7 1 17 47 0 0 706.63 712.59
v8 0 17 50 2 0 620.94 625.90
v9 0 16 41 4 0 501.88 504.46
vdwxc 0 0 1 0 0 7.03 7.05
wannier90 2 5 1 0 0 32.28 32.53
Completed in 2400.30 [s]. Average time for test=10.10 [s], stdev=45.74 [s]
Summary: failed=14, succeeded=723, passed=164, skipped=194, disabled=0
ABINIT 9.4.2 on Spruce Knob using Intel 2019¶
This is the canonical configuration for Spruce module dft_intel19:
module load dft_intel19
The list of modules loaded are:
$> module list
Currently Loaded Modulefiles:
1) lang/gcc/9.3.0 6) libs/fftw/3.3.8_intel19 11) atomistic/elk/7.2.42_intel19
2) lang/intel/2019 7) atomistic/abinit/9.4.2_intel19 12) atomistic/siesta/4.0.2_intel19
3) lang/python/cpython_3.9.6_gcc93 8) atomistic/vasp/6.2.1_intel19 13) dft_intel19
4) libs/libxc/5.1.5_intel19 9) atomistic/octopus/11.0_intel19
5) libs/hdf5/1.12.1_intel19 10) atomistic/espresso/6.8_intel19
The building folder is:
/gpfs/shared/src/abinit-9.4.2/build_intel19
Create a ac9 file for autoconfigure. The file must have the same name as the
headnode with extension .ac9 (For Spruce it will be srih0001.ac9):
$ cat srih0001.ac9
prefix="/shared/software/atomistic/abinit/9.4.2_intel19_impi19"
with_debug_flavor="verbose"
with_optim_flavor="standard"
FCFLAGS_EXTRA="-heap-arrays 1 -axAVX,CORE-AVX2,CORE-AVX-I"
FC_LIBS="-lstdc++ -ldl"
CPP="icc -E"
CC=mpiicc
CXX=mpiicpc
FC=mpiifort
enable_mpi_io="no"
#with_libxc=/gpfs/shared/src/abinit-9.4.2/build_intel19/fallbacks/install_fb/intel/19.0/libxc/4.3.4
#with_hdf5=/gpfs/shared/src/abinit-9.4.2/build_intel19/fallbacks/install_fb/intel/19.0/hdf5/1.10.6
#with_netcdf=/gpfs/shared/src/abinit-9.4.2/build_intel19/fallbacks/install_fb/intel/19.0/netcdf4/4.6.3
#with_netcdf_fortran=/gpfs/shared/src/abinit-9.4.2/build_intel19/fallbacks/install_fb/intel/19.0/netcdf4_fortran/4.5.2
#with_xmlf90=/gpfs/shared/src/abinit-9.4.2/build_intel19/fallbacks/install_fb/intel/19.0/xmlf90/1.5.3.1
#with_libpsml=/gpfs/shared/src/abinit-9.4.2/build_intel19/fallbacks/install_fb/intel/19.0/libpsml/1.1.7
The last 6 lines are commented as they will be used after compiling the fallbacks. Run a first configure with this ac9:
../configure
This first configure will prepare the folders for building the fallbacks:
$> cd fallbacks
$> CC=mpiicc CXX=mpiicpc FC=mpiifort ./build-abinit-fallbacks.sh
After compiling the fallbacks, move one folder up and remove the comments to the last six lines of the ac9 file enabling the fallbacks for the next configure:
$> cd ..
$> tail -n 8 srih0001.ac9
with_libxc=/gpfs/shared/src/abinit-9.4.2/buiild_intel19/fallbacks/install_fb/intel/19.0/libxc/4.3.4
with_hdf5=/gpfs/shared/src/abinit-9.4.2/buiild_intel19/fallbacks/install_fb/intel/19.0/hdf5/1.10.6
with_netcdf=/gpfs/shared/src/abinit-9.4.2/buiild_intel19/fallbacks/install_fb/intel/19.0/netcdf4/4.6.3
with_netcdf_fortran=/gpfs/shared/src/abinit-9.4.2/buiild_intel19/fallbacks/install_fb/intel/19.0/netcdf4_fortran/4.5.2
with_xmlf90=/gpfs/shared/src/abinit-9.4.2/buiild_intel19/fallbacks/install_fb/intel/19.0/xmlf90/1.5.3.1
with_libpsml=/gpfs/shared/src/abinit-9.4.2/buiild_intel19/fallbacks/install_fb/intel/19.0/libpsml/1.1.7
Now proceed to configure again:
$> CC=mpiicc CXX=mpiicpc FC=mpiifort ../configure
Execute make with a appropiated number of compilation threads:
$> make -j16
$> make install
To run the testsuite go to the tests folder and execute:
$> cd tests
$> python3 ../../tests/runtests.py -j4 -n4
On Spruce the results of the testsuite are:
Suite failed passed succeeded skipped disabled run_etime tot_etime
atompaw 0 0 0 2 0 0.00 0.00
bigdft 0 0 0 19 0 0.00 0.03
bigdft_paral 0 0 0 4 0 0.00 0.01
built-in 0 0 5 2 0 24.60 24.61
etsf_io 0 0 8 0 0 24.32 24.91
fast 0 0 11 0 0 45.27 46.23
gpu 0 0 0 7 0 0.00 0.01
libxc 0 9 26 0 0 417.28 419.47
mpiio 0 0 0 17 0 0.00 0.02
paral 11 8 26 76 0 399.02 402.03
psml 0 2 12 0 0 79.79 80.39
seq 0 0 0 18 0 0.00 0.02
tutomultibinit 0 0 3 3 0 29.94 31.10
tutoparal 0 1 0 27 0 33.91 34.53
tutoplugs 0 0 0 4 0 0.00 0.00
tutorespfn 1 9 19 2 0 987.28 991.43
tutorial 4 7 48 0 0 573.78 576.74
unitary 0 0 18 20 0 97.45 97.74
v1 0 0 74 0 0 254.83 257.99
v2 0 12 67 0 0 280.23 283.74
v3 0 12 66 0 0 414.27 420.35
v4 0 10 51 0 0 302.68 306.88
v5 2 14 57 0 0 857.25 864.74
v6 0 8 53 0 0 528.13 533.83
v67mbpt 0 9 16 0 0 251.67 254.97
v7 1 15 49 0 0 929.57 936.62
v8 0 17 50 4 0 1193.84 1199.95
v9 0 15 34 2 0 952.27 956.56
vdwxc 0 0 0 1 0 0.00 0.00
wannier90 0 0 0 8 0 0.00 0.01
Completed in 2381.46 [s]. Average time for test=10.09 [s], stdev=22.42 [s]
Summary: failed=19, succeeded=693, passed=148, skipped=216, disabled=0
Execution completed.
Results in HTML format are available in Test_suite/suite_report.html
Parallel version with GCC 9.3 and MPICH 3.4.1¶
Abinit 9.4.1 was compiled with the following modules:
module load lang/gcc/9.3.0
module load lang/python/cpython_3.9.4_gcc93
module load parallel/mpich/3.4.1_gcc93
module load libs/openblas/0.3.10_gcc93
module load libs/libxc/4.3.4_gcc93
module load libs/xmlf90/1.5.4_gcc93
module load libs/libpsml/1.1.7_gcc93
module load libs/openblas/0.3.10_gcc93
module load libs/hdf5/1.12.0_gcc93
module load libs/netcdf/4.7.4_gcc93
module load libs/netcdf/fortran-4.5.3_gcc93
module load libs/fftw/3.3.9_gcc93
ABINIT uses a configure file, a template can be found at doc/build/config-template.ac9. The lines to be changed from the template are:
prefix="/shared/software/atomistic/abinit/9.4.1_gcc93_mpic341"
with_mpi="${MD_MPICH}"
LINALG_LIBS="-L${MD_OPENBLAS}/lib -lopenblas -lpthread "
with_fft_flavor="fftw3"
FFTW3_LIBS="-L${MD_FFTW} -lfftw3 -lfftw3f"
with_libxc=${MD_LIBXC}
with_libxml2="${MD_GCC}"
with_hdf5="${MD_HDF5}"
NETCDF_FCFLAGS="-I${MD_NETCDF}/include"
NETCDF_LIBS="-L${MD_NETCDF}/lib -lnetcdf"
NETCDF_FORTRAN_FCFLAGS="-I${MD_NETCDF_FORTRAN}/include"
NETCDF_FORTRAN_LIBS="-L${MD_NETCDF_FORTRAN}/lib -lnetcdff"
LIBPSML_FCFLAGS="-I${MD_LIBPSML}/include"
LIBPSML_LIBS="-L${MD_LIBPSML}/lib -lpsml"
with_xmlf90="${MD_XMLF90}"
These lines will use environment variables declared on the corresponding modules loaded above. The configure ac9 is:
build_gcc93_mpic341.ac9
To configure ABINIT is customary to create a build folder, ABINIT was build inside the folder build_gcc93_mpic341:
mkdir build_gcc93_mpic341
cd build_gcc93_mpic341
Execute the configure with:
../configure --with-config-file=../build_gcc93_mpic341.ac9
The resulting summary of the configurations for building ABINIT are these:
Core build parameters
---------------------
* C compiler : gnu version 9.3
* Fortran compiler : gnu version 9.3
* architecture : intel xeon (64 bits)
* debugging : basic
* optimizations : standard
* OpenMP enabled : no (collapse: ignored)
* MPI enabled : yes (flavor: auto)
* MPI in-place : no
* MPI-IO enabled : yes
* GPU enabled : no (flavor: none)
* LibXML2 enabled : yes
* LibPSML enabled : yes
* XMLF90 enabled : yes
* HDF5 enabled : yes (MPI support: no)
* NetCDF enabled : yes (MPI support: no)
* NetCDF-F enabled : yes (MPI support: no)
* FFT flavor : fftw3 (libs: user-defined)
* LINALG flavor : netlib (libs: auto-detected)
* SCALAPACK enabled : no
* ELPA enabled : no
* FCFLAGS : -g -ffree-line-length-none -I/shared/software/libs/netcdf-c/4.7.4_gcc93/include -I/shared/software/libs/netcdf-fortran/4.5.3_gcc93/include -I/shared/software/libs/xmlf90/1.5.4_
gcc93/include -I/shared/software/libs/libpsml/1.1.7_gcc93/include
* CPATH : /shared/software/libs/fftw/3.3.9_gcc93/include:/shared/software/libs/netcdf-fortran/4.5.3_gcc93/include:/shared/software/libs/netcdf-c/4.7.4_gcc93/include:/shared/software/libs/hdf
5/1.12.0_gcc93/include:/shared/software/libs/libpsml/1.1.7_gcc93/include:/shared/software/libs/xmlf90/1.5.4_gcc93/include:/shared/software/libs/libxc/4.3.4_gcc93/include:/shared/software/libs/openblas/0.3
.10_gcc9.3.0/include:/shared/software/parallel/mpich/3.4.1_gcc93/include:/shared/software/lang/python/3.9.4_gcc93/include:/shared/software/lang/gcc/9.3.0/include
* Build workflow : monolith
0 deprecated options have been used:.
Configuration complete.
You may now type "make" to build Abinit.
(or "make -j<n>", where <n> is the number of available processors)
ABINIT can now be build with:
make -j12
Running the testsuite produces these results:
Suite failed passed succeeded skipped disabled run_etime tot_etime
atompaw 0 0 0 2 0 0.00 0.00
bigdft 0 0 0 19 0 0.00 0.01
bigdft_paral 0 0 0 4 0 0.00 0.00
built-in 0 0 5 2 0 18.92 18.93
etsf_io 0 0 8 0 0 71.88 72.11
fast 0 1 10 0 0 114.94 115.72
gpu 0 0 0 7 0 0.00 0.00
libxc 1 7 27 0 0 1217.96 1220.14
mpiio 1 0 12 4 0 2298.48 2306.73
paral 1 11 33 76 0 6497.65 6502.18
psml 0 2 12 0 0 536.08 536.88
seq 0 0 0 18 0 0.00 0.01
tutomultibinit 0 0 6 0 0 248.18 250.11
tutoparal 0 0 1 26 0 154.05 154.50
tutoplugs 0 0 0 4 0 0.00 0.00
tutorespfn 1 8 20 2 0 4046.58 4050.13
tutorial 2 10 47 0 0 1655.70 1659.39
unitary 0 1 17 20 0 107.07 107.41
v1 0 1 73 0 0 529.11 532.30
v2 0 10 69 0 0 601.61 606.15
v3 0 14 64 0 0 597.51 602.62
v4 0 12 49 0 0 559.48 563.87
v5 2 12 59 0 0 2705.48 2712.52
v6 0 7 54 0 0 1491.29 1495.96
v67mbpt 1 9 15 0 0 645.63 648.78
v7 1 14 50 0 0 2800.01 2806.79
v8 0 17 52 2 0 3690.55 3696.26
v9 0 9 42 0 0 1196.94 1200.38
vdwxc 0 0 0 1 0 0.00 0.00
wannier90 0 0 0 8 0 0.00 0.00
Completed in 3760.01 [s]. Average time for test=36.12 [s], stdev=97.31 [s]
Summary: failed=10, succeeded=725, passed=145, skipped=195, disabled=0
CUDA Version with GCC 9.3, MPICH 3.4.1 and CUDA 11.1¶
Similar to the parallel version above with the addition of this module:
parallel/cuda/11.1
The configue file was:
prefix="/shared/software/atomistic/abinit/9.4.1_gcc93_mpic341_gpu"
with_mpi="${MD_MPICH}"
with_gpu="/usr/local/cuda"
with_gpu_flavor="cuda-double"
GPU_CPPFLAGS="-I/usr/local/cuda/include"
GPU_CFLAGS="-I/usr/local/cuda/include"
GPU_CXXFLAGS="-std=c++"
GPU_FCFLAGS="-I/usr/local/cuda/include"
GPU_LDFLAGS="-L/usr/local/cuda/lib64 -lcublas -lcufft -lcudart -lstdc++"
GPU_LIBS="-L/usr/local/cuda/lib64 -lcublas -lcufft -lcudart -lstdc++"
LINALG_LIBS="-L${MD_OPENBLAS}/lib -lopenblas -lpthread "
with_fft_flavor="fftw3"
FFTW3_LIBS="-L${MD_FFTW} -lfftw3 -lfftw3f"
with_libxc=${MD_LIBXC}
with_libxml2="${MD_GCC}"
with_hdf5="${MD_HDF5}"
NETCDF_FCFLAGS="-I${MD_NETCDF}/include"
NETCDF_LIBS="-L${MD_NETCDF}/lib -lnetcdf"
NETCDF_FORTRAN_FCFLAGS="-I${MD_NETCDF_FORTRAN}/include"
NETCDF_FORTRAN_LIBS="-L${MD_NETCDF_FORTRAN}/lib -lnetcdff"
LIBPSML_FCFLAGS="-I${MD_LIBPSML}/include"
LIBPSML_LIBS="-L${MD_LIBPSML}/lib -lpsml"
with_xmlf90="${MD_XMLF90}"
The code must be compiled from a compute node with GPUs as the CUDA toolkit is only present there.
Abinit 9.6.2 on Spruce with GCC 10.3 and OpenMPI 3.1.6¶
- Modules loaded::
- $
$> module load lang/gcc/10.3.0 lang/python/cpython_3.10.2_gcc103 libs/openblas/0.3.19_gcc103 atomistic/wannier90/3.1.0_gcc103 parallel/openmpi/3.1.6_gcc103 libs/fftw/3.3.10_gcc103
Test Suite:
Suite failed passed succeeded skipped disabled run_etime tot_etime
atompaw 0 0 0 2 0 0.00 0.00
bigdft 0 0 0 19 0 0.00 0.01
bigdft_paral 0 0 0 4 0 0.00 0.00
built-in 0 0 6 1 0 26.13 26.15
etsf_io 0 0 8 0 0 112.21 112.56
fast 0 1 10 0 0 182.93 183.29
gpu 0 0 0 7 0 0.00 0.00
libxc 0 13 22 0 0 1462.06 1464.53
mpiio 1 1 11 4 0 3356.26 3372.92
paral 3 8 37 84 0 5626.88 5632.48
psml 0 2 12 0 0 234.75 235.43
seq 0 0 0 18 0 0.00 0.02
tutomultibinit 0 2 4 0 0 612.53 613.89
tutoparal 0 1 0 27 0 42.99 43.59
tutoplugs 0 4 0 0 0 275.10 275.27
tutorespfn 2 16 11 2 0 4887.03 4890.27
tutorial 2 14 43 0 0 2780.47 2783.86
unitary 0 0 18 20 0 182.90 183.27
v1 0 1 73 0 0 700.69 703.41
v2 1 16 62 0 0 742.17 745.30
v3 2 14 62 0 0 1144.97 1150.12
v4 0 14 47 0 0 1147.91 1152.00
v5 3 15 55 0 0 2396.42 2403.60
v6 0 11 50 0 0 1743.95 1748.69
v67mbpt 0 10 15 0 0 1025.92 1029.32
v7 1 26 38 0 0 3385.66 3395.43
v8 0 21 46 2 0 2436.99 2442.48
v9 0 27 30 4 0 1734.56 1738.00
vdwxc 0 0 1 0 0 40.25 40.29
wannier90 2 6 0 0 0 234.82 235.19
Completed in 9245.87 [s]. Average time for test=40.53 [s], stdev=106.36 [s]
Summary: failed=17, succeeded=661, passed=223, skipped=194, disabled=0
CASTEP¶
CASTEP is a leading code for calculating the properties of materials from first principles. Using density functional theory, it can simulate a wide range of properties of materials proprieties including energetics, structure at the atomic level, vibrational properties, electronic response properties etc. In particular it has a wide range of spectroscopic features that link directly to experiment, such as infra-red and Raman spectroscopies, NMR, and core level spectra.
CASTEP can only be compiled with Intel 2018 due to a bug on Intel 2019 MPI implementation. The code was compiled on both clusters with Intel 2018.
Modules used:
$> module purge
$> module load lang/python/intelpython_2.7.16 lang/intel/2018
Compilation line:
$> make ARCH=linux_x86_64_ifort18 COMMS_ARCH=mpi SUBARCH=mpi FFT=mkl MATHLIBS=mkl10 \
INSTALL_DIR=/shared/software/atomistic/castep/19.11-mpi_intel18 \
FFTLIBDIR=${MKLROOT} MATHLIBDIR=${MKLROOT} -j 8
A run of a test suite o both clusters passes all tests.
On Spruce:
$ make ARCH=linux_x86_64_ifort18 COMMS_ARCH=mpi SUBARCH=mpi FFT=mkl MATHLIBS=mkl10 INSTALL_DIR=/shared/software/atomistic/castep/19.11-mpi_intel18 \
FFTLIBDIR=${MKLROOT} MATHLIBDIR=${MKLROOT} -j 8 check
Makefile:595: GNU make version 3.82 or later is recommended: proceeding with Make 3.81
Some modules may be compiled at unnecessarily low optimisation level
make -C "Test" ARCH=linux_x86_64_ifort18--mpi check-simple
make[1]: Entering directory `/gpfs/shared/src/CASTEP-19.11/Test'
rm -f */*/*.{castep,dfpt_wvfn,fd_wvfn,wvfn.*,*.err}
../bin/testcode.py -q --processors=4 --total-processors=16 -e /gpfs/shared/src/CASTEP-19.11/obj/linux_x86_64_ifort18--mpi/castep.mpi -c simple
................................................................................................................................................
................................................................................................................................................
................................................................................................................................................
................................ [464/464]
make[1]: Leaving directory `/gpfs/shared/src/CASTEP-19.11/Test'
On Thorny:
$ make ARCH=linux_x86_64_ifort18 COMMS_ARCH=mpi SUBARCH=mpi FFT=mkl MATHLIBS=mkl10 INSTALL_DIR=/shared/software/atomistic/castep/19.11-mpi_intel18 \
FFTLIBDIR=${MKLROOT} MATHLIBDIR=${MKLROOT} -j 8 check
make -C "Test" ARCH=linux_x86_64_ifort18--mpi check-simple
make[1]: Entering directory `/gpfs20/shared/src/CASTEP-19.11/Test'
rm -f */*/*.{castep,dfpt_wvfn,fd_wvfn,wvfn.*,*.err}
../bin/testcode.py -q --processors=4 --total-processors=48 -e /gpfs20/shared/src/CASTEP-19.11/obj/linux_x86_64_ifort18--mpi/castep.mpi -c simple
..................................................................................................................................................
..................................................................................................................................................
..................................................................................................................................................
.......................... [464/464]
make[1]: Leaving directory `/gpfs20/shared/src/CASTEP-19.11/Test'
VASP¶
VASP 6.2.1 on Thorny Flat with Intel 2021.4¶
The Vienna Ab initio Simulation Package (VASP) is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.
VASP 6.2.1 was compiled with Intel 2021.4 on Thorny Flat. There are two builds of VASP, one compiled with a MKL running the rutines sequential mode (no multithreading) and another build with OpenMP enabled and MKL running in multithreaded mode.
VASP is a proprietary code that require a license to legally run the code.
The downloaded file is called vasp.6.2.1.tar.gz that uncompress into a folder vasp.6.2.1.
Before compiling VASP, you need to edit the file makefile.include for the sequential version:
# Precompiler options
CPP_OPTIONS= -DHOST=\"LinuxIFC\"\
-DMPI -DMPI_BLOCK=8000 \
-DCACHE_SIZE=4000 \
-DscaLAPACK \
-Dvasp6 \
-Duse_bse_te \
-Dtbdyn \
-Dfock_dblbuf
CPP = fpp -f_com=no -free -w0 $*$(FUFFIX) $*$(SUFFIX) $(CPP_OPTIONS)
FC = mpiifort
FCL = mpiifort
FREE = -free -names lowercase
FFLAGS = -assume byterecl -w -traceback -static-libstdc++ -static-libgcc -heap-arrays 1 -xSKYLAKE-AVX512
OFLAG = -O2 -g3
OFLAG_IN = $(OFLAG)
DEBUG = -O0 -g3
MKL_PATH = $(MKLROOT)/lib/intel64
BLAS = -qmkl=sequential -static-intel
LAPACK =
BLACS = -lmkl_blacs_intelmpi_lp64
SCALAPACK = $(MKL_PATH)/libmkl_scalapack_lp64.a $(BLACS)
OBJECTS = fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d.o
INCS =-I$(MKLROOT)/include/fftw
LLIBS = $(SCALAPACK) $(LAPACK) $(BLAS)
OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o
# For what used to be vasp.5.lib
CPP_LIB = $(CPP)
FC_LIB = $(FC)
CC_LIB = icc
CFLAGS_LIB = -O -static-libgcc -xSKYLAKE-AVX512
FFLAGS_LIB = -O1 -static-libgcc -xSKYLAKE-AVX512
FREE_LIB = $(FREE)
OBJECTS_LIB= linpack_double.o getshmem.o
# For the parser library
CXX_PARS = icpc
LLIBS += -lstdc++ -static-libstdc++ -xSKYLAKE-AVX512
# Normally no need to change this
SRCDIR = ../../src
BINDIR = ../../bin
Running tests:
$> make test
VASP 6.2.1 on Spruce using Intel 2019¶
# Precompiler options
CPP_OPTIONS= -DHOST=\"LinuxIFC\"\
-DMPI -DMPI_BLOCK=8000 \
-Duse_collective \
-DCACHE_SIZE=4000 \
-DscaLAPACK \
-Dvasp6 \
-Duse_bse_te \
-Dtbdyn \
-Dfock_dblbuf
CPP = fpp -f_com=no -free -w0 $*$(FUFFIX) $*$(SUFFIX) $(CPP_OPTIONS)
FC = mpiifort
FCL = mpiifort
FREE = -free -names lowercase
FFLAGS = -assume byterecl -w -axSANDYBRIDGE,IVYBRIDGE,HASWELL -static-libgcc -traceback -g
OFLAG = -O2
OFLAG_IN = $(OFLAG)
DEBUG = -O0
MKL_PATH = $(MKLROOT)/lib/intel64
BLAS = ${MKLROOT}/lib/intel64/libmkl_core.a
LAPACK = ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_sequential.a
BLACS = ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_lp64.a
SCALAPACK = ${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a
OBJECTS = fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d.o
INCS = -I${MKLROOT}/include -I$(MKLROOT)/include/fftw
LLIBS = $(SCALAPACK) -Wl,--start-group $(LAPACK) $(BLAS) $(BLACS) -Wl,--end-group -lpthread -lm -ldl
OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o
# For what used to be vasp.5.lib
CPP_LIB = $(CPP)
FC_LIB = $(FC)
CC_LIB = icc
CFLAGS_LIB = -O -static-libgcc -axSANDYBRIDGE,IVYBRIDGE,HASWELL
FFLAGS_LIB = -O2 -static-libgcc -axSANDYBRIDGE,IVYBRIDGE,HASWELL
FREE_LIB = $(FREE)
OBJECTS_LIB= linpack_double.o getshmem.o
# For the parser library
CXX_PARS = icpc
LLIBS += -lstdc++ -static-libstdc++ -static-libgcc -static-intel
# Normally no need to change this
SRCDIR = ../../src
BINDIR = ../../bin
The version that runs MKL with multithreading and enables OpenMP is like this:
# Precompiler options
CPP_OPTIONS= -DHOST=\"LinuxIFC\"\
-DMPI -DMPI_BLOCK=8000 \
-Duse_collective \
-DCACHE_SIZE=4000 \
-DscaLAPACK \
-Dvasp6 \
-Duse_bse_te \
-Dtbdyn \
-Dfock_dblbuf \
-D_OPENMP
CPP = fpp -f_com=no -free -w0 $*$(FUFFIX) $*$(SUFFIX) $(CPP_OPTIONS)
FC = mpiifort
FCL = mpiifort
FREE = -free -names lowercase
FFLAGS = -assume byterecl -w -axSANDYBRIDGE,IVYBRIDGE,HASWELL -static-intel -static-libgcc -traceback -g -qopenmp
OFLAG = -O2
OFLAG_IN = $(OFLAG)
DEBUG = -O0
MKL_PATH = $(MKLROOT)/lib/intel64
BLAS = ${MKLROOT}/lib/intel64/libmkl_core.a
LAPACK = ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a
BLACS = ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_lp64.a
SCALAPACK = ${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a
OBJECTS = fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d.o
INCS = -I${MKLROOT}/include -I$(MKLROOT)/include/fftw
LLIBS = $(SCALAPACK) -Wl,--start-group $(LAPACK) $(BLAS) $(BLACS) -Wl,--end-group -liomp5 -lpthread -lm -ldl
OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o
# For what used to be vasp.5.lib
CPP_LIB = $(CPP)
FC_LIB = $(FC)
CC_LIB = icc
CFLAGS_LIB = -O -axSANDYBRIDGE,IVYBRIDGE,HASWELL -static-libgcc
FFLAGS_LIB = -O2 -axSANDYBRIDGE,IVYBRIDGE,HASWELL -static-libgcc
FREE_LIB = $(FREE)
OBJECTS_LIB= linpack_double.o getshmem.o
# For the parser library
CXX_PARS = icpc
LLIBS += -lstdc++ -static-libstdc++ -static-libgcc -static-intel
# Normally no need to change this
SRCDIR = ../../src
BINDIR = ../../bin
The only module needed to compile VASP is:
module purge
module load lang/intel/2019
VASP includes a testsuite and running it produces this final results:
==================================================================
SUMMARY:
==================================================================
The following tests failed, please check the output file manually:
bulk_SiO2_LOPTICS bulk_SiO2_LOPTICS_nosym bulk_SiO2_LOPTICS_RPR
bulk_SiO2_LPEAD bulk_SiO2_LPEAD_nosym bulk_SiO2_LPEAD_RPR
C_2x2x2_CORE_CON C_2x2x2_CORE_CON_RPR
VASP 6.2.1 on Spruce using Intel 2019¶
This is the canonical configuration for Spruce module dft_intel19:
module load dft_intel19
The list of modules loaded are:
$> module list
Currently Loaded Modulefiles:
1) lang/gcc/9.3.0 6) libs/fftw/3.3.8_intel19 11) atomistic/elk/7.2.42_intel19
2) lang/intel/2019 7) atomistic/abinit/9.4.2_intel19 12) atomistic/siesta/4.0.2_intel19
3) lang/python/cpython_3.9.6_gcc93 8) atomistic/vasp/6.2.1_intel19 13) dft_intel19
4) libs/libxc/5.1.5_intel19 9) atomistic/octopus/11.0_intel19
5) libs/hdf5/1.12.1_intel19 10) atomistic/espresso/6.8_intel19
In case of having a previous build, erase the folder:
rm -rf build/*
Create a file makefile.include with the contents as follows:
# Precompiler options
CPP_OPTIONS= -DHOST=\"LinuxIFC\"\
-DMPI -DMPI_BLOCK=8000 \
-DCACHE_SIZE=4000 \
-DscaLAPACK \
-Dvasp6 \
-Duse_bse_te \
-Dtbdyn \
-Dfock_dblbuf
CPP = fpp -f_com=no -free -w0 $*$(FUFFIX) $*$(SUFFIX) $(CPP_OPTIONS)
FC = mpiifort
FCL = mpiifort
FREE = -free -names lowercase
FFLAGS = -assume byterecl -w -traceback -static-libstdc++ -static-libgcc -heap-arrays 1 -axAVX,CORE-AVX2,CORE-AVX-I
OFLAG = -O2 -g3
OFLAG_IN = $(OFLAG)
DEBUG = -O0 -g3
MKL_PATH = $(MKLROOT)/lib/intel64
BLAS = -mkl=sequential -static-intel
LAPACK =
BLACS = -lmkl_blacs_intelmpi_lp64
SCALAPACK = $(MKL_PATH)/libmkl_scalapack_lp64.a $(BLACS)
OBJECTS = fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d.o
INCS =-I$(MKLROOT)/include/fftw
LLIBS = $(SCALAPACK) $(LAPACK) $(BLAS)
OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o
# For what used to be vasp.5.lib
CPP_LIB = $(CPP)
FC_LIB = $(FC)
CC_LIB = icc
CFLAGS_LIB = -O -static-libgcc -axAVX,CORE-AVX2,CORE-AVX-I
FFLAGS_LIB = -O1 -static-libgcc -axAVX,CORE-AVX2,CORE-AVX-I
FREE_LIB = $(FREE)
OBJECTS_LIB= linpack_double.o getshmem.o
# For the parser library
CXX_PARS = icpc
LLIBS += -lstdc++ -static-libstdc++ -axAVX,CORE-AVX2,CORE-AVX-I
# Normally no need to change this
SRCDIR = ../../src
BINDIR = ../../bin
Execute make, do not try to use multiple compilations threads as this fails.
make
Installation is manual and consists of copying the 3 binaries to the folder that will be added to the $PATH:
$> rsync -av bin/ /shared/software/atomistic/vasp/6.2.1_intel19/bin/
Testsuite can be run by going into testsuite and running:
$> cd testsuite
$> ./runtest
Octopus¶
Octopus 12.2 on Thorny Flat using Intel compilers¶
Octopus 11.3 on Thorny Flat using Intel 2021.4¶
Use the metamodule for DFT codes:
$> module purge
$> module load dft_intel21
Resulting in these modules being loaded:
Currently Loaded Modulefiles:
1) lang/gcc/9.3.0 15) libs/arpack-ng/3.8.0_intel21_impi21
2) tbb/latest 16) libs/xmlf90/1.5.5_intel21
3) compiler-rt/latest 17) libs/libpsml/1.1.10_intel21
4) compiler/latest 18) libs/gridxc/0.9.6_intel21
5) mpi/latest 19) atomistic/wannier90/3.1.0_intel21
6) mkl/latest 20) atomistic/abinit/9.6.2_intel21_impi21
7) lang/python/cpython_3.9.7_gcc93 21) atomistic/vasp/6.2.1_intel21
8) libs/openblas/0.3.17_intel21 22) atomistic/octopus/11.3_intel21_impi21
9) libs/libxc/5.1.7_intel21 23) atomistic/espresso/6.8_intel21
10) libs/hdf5/1.12.1_intel21 24) atomistic/elk/8.3.15_intel21
11) libs/hdf5/1.12.1_intel21_impi21 25) atomistic/siesta/4.1.5_intel21
12) libs/netcdf/4.8.1_intel21_impi21 26) atomistic/siesta/4.1.5_psml_intel21
13) libs/netcdf/fortran-4.5.3_intel21_impi21 27) dft_intel21
14) libs/fftw/3.3.9_intel21_impi21
The building folder is:
/shared/src/octopus-11.3/build_intel21
Run the configure script:
MKL="-L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lm -ldl"
../configure --prefix=/shared/software/atomistic/octopus/11.3_intel21_impi21 \
--with-blas="${MKL}" \
--with-lapack="${MKL}" \
--enable-mpi \
--with-libxc-prefix=$MD_LIBXC \
--with-boost=/shared/software/libs/boost/1.78_gcc93 \
--with-sparskit=/shared/software/libs/sparskit2/lib/libskit.a \
CC=mpiicc CXX=mpiicpc FC=mpiifort FCFLAGS="-g -traceback -heap-arrays 1 -O2 -fp-model=precise"
Compile the code using 12 compilation threads:
$> make -j 12
Run the testsuite:
$> ulimit -s unlimited
$> make check
The results of the testsuite were:
Status: 8 failures
Passed: 223 / 228
Skipped: 1 / 228
Failed: 4 / 228
testfile # failed testcases
------------------------------------------------------------------------------
periodic_systems/17-aluminium.test 4
periodic_systems/18-TiO2.test 2
lda_u/07-noncollinear.test 3
functionals/18-mgga.test 8
Total run-time of the testsuite: 00:10:11
Octopus 11.0 on Spruce Knob using Intel 2019¶
This is the canonical configuration for Spruce module dft_intel19:
module load dft_intel19
The list of modules loaded are:
$> module list
Currently Loaded Modulefiles:
1) lang/gcc/9.3.0 6) libs/fftw/3.3.8_intel19 11) atomistic/elk/7.2.42_intel19
2) lang/intel/2019 7) atomistic/abinit/9.4.2_intel19 12) atomistic/siesta/4.0.2_intel19
3) lang/python/cpython_3.9.6_gcc93 8) atomistic/vasp/6.2.1_intel19 13) dft_intel19
4) libs/libxc/5.1.5_intel19 9) atomistic/octopus/11.0_intel19
5) libs/hdf5/1.12.1_intel19 10) atomistic/espresso/6.8_intel19
The building folder is:
/shared/src/octopus-11.0/build_intel19
Execute this configure line:
$> ../configure --prefix=/shared/software/atomistic/octopus/11.0_intel19 \
--with-lapack="-L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm -ldl" \
--with-blas="-L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm -ldl" \
--with-blacs="${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_lp64.a" \
--with-scalapack="${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a CC=mpiicc CXX=mpiicpc FC=mpiifort
Build the software and install:
$> make
$> make install
Running the tests with:
$> make check
The results were:
Status: 8 failures
Passed: 217 / 223
Skipped: 1 / 223
Failed: 5 / 223
testfile # failed testcases
------------------------------------------------------------------------------
maxwell/02-external-current.test 3
periodic_systems/06-h2o_pol_lr.test 2
linear_response/01-casida.test 16
pseudopotentials/14-carbon_dojo_psp8.test 1
functionals/18-mgga.test 8
Total run-time of the testsuite: 02:16:08
make[3]: *** [Makefile:876: check-base] Error 5
make[3]: Leaving directory '/gpfs/shared/src/octopus-11.0/build_intel19/testsuite'
make[2]: *** [Makefile:865: check-wrapper] Error 2
make[2]: Leaving directory '/gpfs/shared/src/octopus-11.0/build_intel19/testsuite'
make[1]: *** [Makefile:853: check] Error 2
make[1]: Leaving directory '/gpfs/shared/src/octopus-11.0/build_intel19/testsuite'
make: *** [Makefile:545: check-recursive] Error 1
Octopus 10.4 with GCC 9.3¶
Octopus is a Real Space DFT code. This instructions show how to compile Octopus 10.4 (latest version by 2021.04.19). This is the parallel version compiled with GCC 9.3
The modules loaded for compilation were:
module load lang/gcc/9.3.0 libs/libxc/4.3.4_gcc93 \
libs/hdf5/1.12.0_gcc93 \
libs/netcdf/4.7.4_gcc93 \
libs/netcdf/fortran-4.5.3_gcc93 \
libs/openblas/0.3.10_gcc93 \
libs/fftw/3.3.9_gcc93 \
parallel/mpich/3.4.1_gcc93
The sources can be downloaded from the developers and uncompressed with:
wget https://octopus-code.org/download/10.4/octopus-10.4.tar.gz
tar -zxvf octopus-10.4.tar.gz
It is customary to compile codes on a separate folder from the sources.
The foler build_gcc93 is created inside the sources for that purpose:
cd octopus-10.4
mkdir build_gcc93_mpic341
cd build_gcc93_mpich341
The configure line was:
../configure --prefix=/shared/software/atomistic/octopus/10.4_gcc93_mpic341 \
--with-libxc-prefix=${MD_LIBXC} --with-blas=" -L${MD_OPENBLAS} -lopenblas" \
--with-fftw-prefix=${MD_FFTW} --with-netcdf-prefix=${MD_NETCDF_FORTRAN} \
--with-mpi=${MD_MPICH} --enable-mpi
On Thorny Flat the results from the testsuite were:
************************
Passed: 184 / 200
Skipped: 16 / 200
Everything seems to be OK
Total run-time of the testsuite: 00:20:42
Quantum Espresso¶
Quantum Espresso 7.0 on Thorny Flat using Intel compilers¶
Download the sources:
wget https://github.com/QEF/q-e/releases/download/qe-7.0/qe-7.0-ReleasePack.tgz
This is the canonical configuration for module dft_intel22:
module load dft_intel22
The list of modules loaded are:
trcis001:/shared/src/qe-7.0$ module list
Currently Loaded Modulefiles:
1) dev/git/2.29.1 19) libs/arpack-ng/3.8.0_intel22_impi22
2) dev/cmake/3.21.1 20) libs/xmlf90/1.5.4_intel22
3) lang/gcc/9.3.0 21) libs/libpsml/1.1.10_intel22
4) tbb/latest 22) libs/gridxc/0.9.6_intel22
5) compiler-rt/latest 23) libs/elpa/2021.11.002_intel22
6) oclfpga/latest 24) libs/elsi/2.9.1_intel22
7) compiler/latest 25) libs/futile/1.8_intel22_impi22
8) mpi/latest 26) libs/atlab/1.0_intel22
9) mkl/latest 27) libs/nlopt/2.7.1_intel22
10) lang/python/intelpython_3.9 28) atomistic/wannier90/3.1.0_intel22
11) parallel/cuda/11.7 29) atomistic/abinit/9.6.2_intel22_impi22
12) libs/openblas/0.3.20_intel22 30) atomistic/vasp/6.2.1_intel22_impi22
13) libs/libxc/5.2.3_intel22 31) atomistic/octopus/11.4_intel22_impi22
14) libs/hdf5/1.12.2_intel22 32) atomistic/espresso/7.0_intel22_impi22
15) libs/hdf5/1.12.2_intel22_impi22 33) atomistic/elk/8.4.6_intel22_impi22
16) libs/netcdf/4.8.1_intel22_impi22 34) atomistic/siesta/4.1.5_intel22_impi22
17) libs/netcdf/fortran-4.5.4_intel22_impi22 35) dft_intel22
18) libs/fftw/3.3.10_intel22_impi22
Uncompress the sources:
$> tar -zxvf qe-7.0-ReleasePack.tgz
$> cd qe-7.0
Configure the sources to use HDF5, libXC and FFTW 3:
$> ./configure CC=mpiicc CXX=mpiicpc FC=mpiifort --with-hdf5 --with-libxc --enable-parallel \
--prefix=/shared/software/atomistic/espresso/7.0_intel22_impi22 LDFLAGS="-L${MD_FFTW}/lib -lfftw3"
Build the binaries:
$> make all
$> make install
ELK¶
ELK 9.5.14 on Thorny Flat using Intel compilers¶
Sources:
$> wget "https://downloads.sourceforge.net/project/elk/elk-9.5.14.tgz?ts=gAAAAABmJvWjEWBNqcGS8x4psUYJ6SVLd0i3fd3fa72osTpJx3IDNgcg8RSRSWxG6fgxRYdmKjVEB0Z1ebZ2aa6dOuEzaC8duQ%3D%3D&r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Felk%2Ffiles%2Felk-9.5.14.tgz%2Fdownload" --no-check-certificate -O elk-9.5.14.tgz
Uncompress sources:
$> tar -zxvf elk-9.5.14.tgz
$> cd elk-9.5.14
Modules used:
$> module load dft_intel21
Copy the file for intel DFTI (Intel implementation of FFT):
$> cp $MKLROOT/include/mkl_dfti.f90 src
Edit the file make.inc as follows:
MAKE = make
AR = ar
# Elk requires BLAS/LAPACK and fast Fourier transform (FFT) libraries.
# We recommend using optimised, mutithreaded BLAS/LAPACK libaries such as those
# provided by the Intel Math Kernel Library (MKL), OpenBLAS or BLIS.
# Elk can use either the Fastest Fourier Transform in the West (FFTW) or the
# MKL FFT. Both single and double precision versions of the FFT are needed.
#------------------------------------------------------------------------------#
# Libraries #
#------------------------------------------------------------------------------#
#-------------------------------------------------------------------------------
SRC_MKL = mkl_stub.f90
# To enable MKL multithreaded parallelism, uncomment the following line and link
# with the MKL library.
SRC_MKL =
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
SRC_OBLAS = oblas_stub.f90
# To enable OpenBLAS multithreaded parallelism, uncomment the following line and
# link with the OpenBLAS library.
#SRC_OBLAS =
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
SRC_BLIS = blis_stub.f90
# To enable BLIS multithreaded parallelism, uncomment the following line and
# link with the BLIS library.
#SRC_BLIS =
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
# If your compiler does *not* support the Message Passing Interface (MPI) then
# uncomment the line below.
#SRC_MPI = mpi_stub.f90
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
# Elk supports two FFT libraries:
# Fastest Fourier Transform in the West (FFTW)
# Intel Math Kernel Library (MKL) Fast Fourier Transform
#
# FFTW is enabled by default with:
SRC_FFT = zfftifc_fftw.f90 cfftifc_fftw.f90
#
# To enable MKL FFT instead, copy mkl_dfti.f90 to the elk/src directory and
# uncomment the line below.
SRC_FFT = mkl_dfti.f90 zfftifc_mkl.f90 cfftifc_mkl.f90
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
SRC_LIBXC = libxcifc_stub.f90
# To enable Libxc first download and compile version 6.x of the library. Next
# copy the files libxcf90.a and libxc.a to the elk/src directory and uncomment
# the following lines.
LIB_LIBXC = ${MD_LIBXC}/lib/libxcf90.a ${MD_LIBXC}/lib/libxc.a
SRC_LIBXC = libxcf90.f90 libxcifc.f90
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
SRC_W90S = w90_stub.f90
# To enable the Wannier90 library copy libwannier.a to the elk/src directory and
# uncomment the following lines.
SRC_W90S =
LIB_W90 = libwannier.a
LIB_W90 = ${MD_WANNIER90}/lib/libwannier.a
#-------------------------------------------------------------------------------
#------------------------------------------------------------------------------#
# Compilers #
#------------------------------------------------------------------------------#
#-------------------------------------------------------------------------------
# Intel MPI Fortran version 18 and later.
F90 = mpiifort
F90_OPTS = -O3 -xHost -ipo -qopenmp -qmkl=parallel
F90_LIB = -liomp5 -lpthread -lm -ldl
SRC_MKL =
AR = xiar
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
# GNU Fortran compiler with MKL.
#F90 = mpif90
#F90_OPTS = -Ofast -march=native -mtune=native -fomit-frame-pointer -fopenmp -ffpe-summary=none -fallow-argument-mismatch
#F90_LIB = -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -lpthread
#SRC_MKL =
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
# GNU Fortran compiler with BLIS, libflame and FFTW
#F90 = mpif90
#F90_OPTS = -Ofast -march=native -mtune=native -fomit-frame-pointer -fopenmp -ffpe-summary=none -fallow-argument-mismatch
#F90_LIB = -lblis -lflame -lfftw3 -lfftw3f
#SRC_BLIS =
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
# GNU Fortran compiler with OpenBLAS, LAPACK and FFTW
#F90 = mpif90
#F90_OPTS = -Ofast -march=native -mtune=native -fomit-frame-pointer -fopenmp -ffpe-summary=none -fallow-argument-mismatch
#F90_LIB = -lopenblas -llapack -lfftw3 -lfftw3f
#SRC_OBLAS =
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
# Intel Fortran with profiling options.
#F90 = mpiifort
#F90_OPTS = -O3 -qopenmp -mkl=parallel -pg
#F90_LIB = -liomp5 -lpthread -lm -ldl
#SRC_MKL =
#AR = ar
#-------------------------------------------------------------------------------
#-------------------------------------------------------------------------------
# GNU Fortran debugging options.
#F90 = gfortran
#F90_OPTS = -O3 -Wunused -fopenmp
#F90_LIB = -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -lpthread
#SRC_MKL =
#SRC_MPI = mpi_stub.f90
#-------------------------------------------------------------------------------
Execute make to compile the codes:
$> make
Installation folder is manually created and the binaries are copied there:
$> ELK_PREFIX=/shared/software/atomistic/elk/9.5.14_intel21_impi21
$> mkdir -p ${ELK_PREFIX}/bin
$> rsync -av src/eos/eos src/spacegroup/spacegroup src/protex src/elk src/rmspaces src/vimelk ${ELK_PREFIX}/bin
$> rsync -av species ${ELK_PREFIX}
Run the tests:
$> cd tests
$> ./test-mpi.sh
Siesta¶
Siesta 4.1.5 on Thorny flat using Intel compilers¶
Siesta is a electronic structure code using linear scaling algorithms. The version compiled was 4.1.5. The code was compiled with Intel compilers 2023
Download the sources and uncompress them:
$> tar -zxvf siesta-4.1.5.tar.gz
$> cd siesta-4.1.5
$> mkdir Obj_Intel
$> cd Obj_Intel
$> ../Src/obj_setup.sh
To compile the code a arch.make needs to be created. The contents of the file can be copied from Src/intel.make with following changes:
#
# Copyright (C) 1996-2016 The SIESTA group
# This file is distributed under the terms of the
# GNU General Public License: see COPYING in the top directory
# or http://www.gnu.org/copyleft/gpl.txt.
# See Docs/Contributors.txt for a list of contributors.
#
#-------------------------------------------------------------------
# arch.make file for gfortran compiler.
# To use this arch.make file you should rename it to
# arch.make
# or make a sym-link.
# For an explanation of the flags see DOCUMENTED-TEMPLATE.make
.SUFFIXES:
.SUFFIXES: .f .F .o .c .a .f90 .F90
SIESTA_ARCH = Intel_2023.1
CC = icc
FPP = $(FC) -E -P
FC = mpiifort
FC_SERIAL = ifort
FFLAGS = -O2 -g3 -traceback -fPIC -fp-model source -heap-arrays 1
AR = ar
RANLIB = ranlib
SYS = nag
SP_KIND = 4
DP_KIND = 8
KINDS = $(SP_KIND) $(DP_KIND)
LDFLAGS =
#COMP_LIBS = libsiestaLAPACK.a libsiestaBLAS.a
MPI_INTERFACE = libmpi_f90.a
MPI_INCLUDE = .
FPPFLAGS = $(DEFS_PREFIX)-DFC_HAVE_ABORT -DMPI
XMLF90_ROOT=/shared/software/libs/xmlf90/1.5.5_intel21
XMLF90_LIBS=-L$(XMLF90_ROOT)/lib -lxmlf90
#PSML_ROOT=/shared/software/libs/psml/1.1.10_intel21
#PSML_LIBS=-L$(PSML_ROOT)/lib -lpsml
#GRIDXC_ROOT=/shared/software/libs/libgridxc/0.9.6_intel21
#GRIDXC_INCFLAGS=-I $(GRIDXC_ROOT)/include/gridxc
#GRIDXC_LIBS=-L$(GRIDXC_ROOT)/lib -lgridxc
MKL_PATH = $(MKLROOT)/lib/intel64
BLAS_LIBS = -qmkl=sequential -static-intel
LAPACK_LIBS = $(BLAS_LIBS)
BLACS_LIBS = -lmkl_blacs_intelmpi_lp64
SCALAPACK_LIBS = $(MKL_PATH)/libmkl_scalapack_lp64.a $(BLACS_LIBS)
LIBS = $(NETCDF_LIBS) $(SCALAPACK_LIBS) $(LAPACK_LIBS) $(MPI_LIBS) $(COMP_LIBS) $(GRIDXC_LIBS) $(PSML_LIBS) $(XMLF90_LIBS)
# Dependency rules ---------
FFLAGS_DEBUG = -O0 -g3 -traceback -fPIC -check all -fp-model source -heap-arrays 1 # your appropriate flags here...
# The atom.f code is very vulnerable. Particularly the Intel compiler
# will make an erroneous compilation of atom.f with high optimization
# levels.
atom.o: atom.F
$(FC) -c $(FFLAGS_DEBUG) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<
state_analysis.o: state_analysis.F
$(FC) -c $(FFLAGS_DEBUG) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<
.c.o:
$(CC) -c $(CFLAGS) $(INCFLAGS) $(CPPFLAGS) $<
.F.o:
$(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<
.F90.o:
$(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_free_F90) $<
.f.o:
$(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f) $<
.f90.o:
$(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90) $<
Compile the code:
$> make
Finally, copy the resulting binary siesta to the folder in software/atomistic:
$> mkdir -p /shared/software/atomistic/siesta/4.1.5_intel22_impi22/bin
$> cp siesta /shared/software/atomistic/siesta/4.1.5_intel22_impi22/bin