2. Advanced installation
2.1. LAN and cluster licenses
If you purchased the LAN or cluster version of RESCU,
you simply need to run
nano_infogen_linux on a login node to generate
Then upload it on the client portal
and follow the instructions there to complete the activation.
If you are using RESCU < 2.7.0, you need to point the licenser to the
installation node as follows. The information in
license.dat will only unlock RESCU on the
node on which it is installed. To avail RESCU on the entire cluster,
follow these instructions. Use a login or gateway node to install RESCU.
The node’s IP address should be in the ARP cache of all worker nodes (it
most likely is already, so you likely need not worry about this). Then
create the file
ip.dat in the
license directory. Add the inet
addresses of the gateway node in
ip.dat, one by line. For instance,
0.150.188.70 0.16.205.3 0.150.188.73 0.20.205.3
The inet addresses can be listed using the command
ifconfig | grep inet inet addr:192.168.1.29 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe70:e3f5/64 Scope:Link
What you seek here is
RESCU is mostly written in MATLAB, and hence it does not need to be
compiled. Additional functionalities require external programs and
libraries which must be compiled however. MATLAB calls these procedures
through MEX files, which must be compiled. For one, the atomic orbital
calculations require such MEX files. In this instance, external
libraries need not be installed and linked, and hence the
function deals with it by itself. Otherwise,
makeRESCU must be
provided with the library paths. The purpose of the present document is
to demonstrates how to build and install these external libraries and
The following is a list of functionalities that require an installation:
Calls certain routines in the CUDA library. This is not required to run RESCU using GPUs, but it may improve performance in certain calculations;
A set of functions that call LibXC routines. LibXC is an exchange-correlation functional library including hundreds of correlation, exchange and kinetic energy functionals.
A set of functions that call MPI and ScaLAPACK routines. MPI, BLAS, LAPACK and ScaLAPACK libraries are required;
makeRESCU compiles and installs a package listed above
when provided with one of the following flags:
-smi. For example, type the following command to build the SMI
makeRESCU -agreeToLicense y -smi libpaths.m
The last command should compile the SMI interface if the
mpif90 compilers are found in the
PATH environment variable. In
addition, the file
libpaths.m must contain variables providing the
paths of certain external libraries. Specific instructions for each
package lie below.
2.3. Pre-existing MPI installation
Many platforms now have several libraries preinstalled. Loading them is
made easy with modern package managers via the
module command for
instance. Then the OpenMPI installations can be listed as
username@computer:~$ module spider openmpi -------------------------------------------------------------- openmpi: -------------------------------------------------------------- Description: An open source Message Passing Interface implementation Versions: openmpi/3.1.3 openmpi/4.0.1 openmpi/4.0.3 openmpi/4.1.1
OpenMPI is then simply loaded typing
module load openmpi/4.0.3
Similarly, one could seek and load LibXC, BLAS, LAPACK and ScaLAPACK as follows
module load openmpi/4.0.3 libxc/4.3.4 openblas/0.3.9 scalapack/2.1.0
Note that OpenBLAS contains both BLAS and LAPACK. Then, one only need to
specify how to link in the file
xc = '/software/2020/avx512/Compiler/gcc9/libxc/4.3.4'; scalapack = '-lscalapack'; blas = '-lopenblas'; lapack = '';
If you need to install OpenMPI and other libraries yourself, the
simplest way is probably to use a package manager like
section shows how to install the build tools and dependencies using the
spack package manager.
Get spack from the github repository typing
git clone https://github.com/spack/spack.git
and add it to your environment as follows
You may add the last command to your
.bashrc to make this permanent.
spack.yaml file found in
documentation and activate the
cd $RESCU_DIR cp documentation/spack.yaml . spacktivate .
OpenMPI are usually already installed on computing
platforms. You may tell
spack to use preinstalled compilers and
libraries as follows
spack find compiler spack external find OpenMPI
Next, concretize the environment and let
spack build the
dependencies as follows
spack concretize -f spack install -j8
You may adjust the build parallelism using the
-j option (here I use
8 cores). When using rescu, just activate the environment with
If something goes sideways, try to find a local version (e.g. with
perl) and reconcretize
spack external find perl spack concretize -f
Finally, define the file
libpaths.m as at the end of section
1.3 to specify how to link the libraries.
2.5. SMI: ScaLAPACK wrapper
In order to exploit efficiently hundreds of processes, RESCU calls
ScaLAPACK in heavy computational
smi_proxy.c. If ScaLAPACK is already installed on
your computer, you may skip the following subsection. Otherwise, here
are some instructions to download, configure, compile, install and
2.5.1. Building the ScaLAPACK library
Download the ScaLAPACK source from Nanoacademic. It has a couple modifications to make it work with MATLAB and handle large matrix redistribution operations.
wget https://storage.googleapis.com/rescumat/scalapack-2.1.0.tar.gz tar -zxf scalapack-2.1.0.tgz cd scalapack-2.1.0
Then jumps in the ScaLAPACK directory and copy the
SLmake.inc.example file to
cd scalapack-2.1.0 cp SLmake.inc.example SLmake.inc
MATLAB requires position independent code libraries, which means
compiled with the
PIC option. Make sure to add
-fPIC to the
following flags in
NOOPT = -O0 -fPIC FCFLAGS = -O3 -fPIC CCFLAGS = -O3 -fPIC CFLAGS = -O3 -fPIC
Then make the library by typing
make lib and install it by copying
libscalapack.a in the appropriate location.
2.5.2. ScaLAPACK interface
Before compiling the ScaLAPACK interface, paths to BLAS, LAPACK and
ScaLAPACK libraries must be defined. If no BLAS or LAPACK library is
installed, we suggest that you install
OpenBLAS, which contains both. As
mentioned above, this is done in a MATLAB script. Create an M-file and
define three variables called
containing the paths to the BLAS, LAPACK and ScaLAPACK libraries
respectively. For example,
cat > /tmp/installpaths.m <<EOF scalapack = '/usr/lib/libscalapack.a'; lapack = '/usr/lib/libopenblas.a'; blas = '/usr/lib/libopenblas.a'; EOF
Additional flags may be required depending on the compiler. Specify them
using the variable
CFLAGS in the M-file (e.g.
CFLAGS = ’-ifcore’). Finally, compile
smi_proxy.c as follows
makeRESCU -smi /tmp/installpaths.m
2.6. Compiling the LibXC wrapper
A few functionals have been implemented in RESCU (LDA_X, LDA_C_PW92,
GGA_XC_PBE, MGGA_X_TB09). More functionals are available via the
functional library LibXC. LibXC is part of the Easybuild toolchain, and
hence you can skip the installation of LibXC if you executed
install.sh. Both the Linux system and Windows system procedures to
compile and install LibXC are presented in this section.
To install LibXC on a Linux system, follow the following instructions. Download LibXC (version 4.0 and above for full functionality) and decompress the tarball.
tar -xzf libxc-4.3.4.tar.gz
Enter the LibXC directory, configure, build and install the library.
cd libxc-4.3.4 ./configure --prefix=PATH/TO/LIBXC --disable-shared CFLAGS="-O3 -fPIC" make && make check && make install
Note that MATLAB requires position independent code libraries, which
means compiled with the
We also suggest building and linking static libraries in MATLAB, and hence
--disable-shared configure option.
The LibXC library is now installed. Create an M-file and define the
xc as the path to the root of the LibXC library
installation (the path specified using the
cat > /tmp/installpaths.m <<EOF xc = '/usr/software/libxc-4.3.4'; EOF
Then invoke makeRESCU to compile the LibXC package.
makeRESCU -libxc /tmp/installpaths.m
libxc.mexa64 will be created upon successful compilation.
To install LibXC on Windows, follow this procedure. To begin with, complete MathWorks’ instructions to install the MinGW-w64 Compiler. To install the compiler, use the Add-Ons menu, select the Features box and search for MinGW. This may not be possible if your version of MATLAB is anterior to 2016, but don’t worry there is a way nevertheless.
Install the MinGW-w64 Compilers through MSYS2. The MSYS2 project is hosted on Github. Download and execute the installer; MSYS2 will install itself automatically. It is advised to use the default paths.
Open an MSYS2 terminal using the command
and update the MSYS2 packages as follows
pacman -Sy pacman pacman -Syu pacman -Su
Pacman is an Arch Linux packager manager, similar to yum or apt-get on
Linux. You can search a package using the command
pacman -Ss and
install a package using the command
pacman -S. Now, install the
pacman -S --needed base-devel mingw-w64-x86_64-toolchain pacman -S --noconfirm autoconf autoconf-archive automake make libtool
Download LibXC and decompress the tarball. From an MSYS2 shell, configure and make the library.
autoreconf -f -i -I m4 ./configure --prefix '/c/lib/libxc-4.3.4' --disable-shared CFLAGS="-O3 -fPIC" make && make install
If something goes wrong, you may want to refer to the following
Stack Overflow ticket
, which contains instructions on how to build LibXC with
If you use
make check, note that it generally fails on Windows because round-off errors give
slightly different results on Linux, but the error is negligible in
practice. MATLAB will look for libraries with the extension
/c/lib/libxc-4.3.4 must thus be renamed accordingly.
cd /c/lib/libxc-4.3.4 cp libxc.a libxc.lib
The LibXC library is now installed.
If you were able to install the MinGW-w64 Compiler MATLAB application,
you may execute the following instructions. Create an m-file (
and define the variable
xc as the path to the root of the LibXC
library (the path specified using the
--prefix option). Then invoke
makeRESCU to compile the LibXC package.
makeRESCU -libxc /tmp/installpaths.m
libxc.mexw64 will be created upon successful compilation.
220.127.116.11. Unsuccessful mex compilation
makeRESCU fails, you may have to compile
the command line. Open an MSYS2 terminal (64-bit). Add
to your path as follows
export PATH=/c/Program\ Files/MATLAB/R2017b/bin/win64:$PATH export PATH=/c/msys64/mingw64/bin:$PATH
Then change directory to
rescu-x.x.x/src and write
mex CC=/c/msys64/mingw64/bin/gcc libxc4.c -output libxc \ -I/c/lib/libxc-4.3.4/include -L/c/lib/libxc-4.3.4/lib -lxc
CC specifies the path MSYS2’s GCC compiler.
libxc4.c is the LibXC-4 wrapper, replace by
libxc.c if using LibXC-3.
-L are the paths to the LibXC header files and libraries.
18.104.22.168. Unsuccessful MinGW-w64 install
If you were not able to install the MinGW-w64 Compiler MATLAB application, you may execute the following instructions. To compile the mex file, create the file mexopts.bat as follows (taken from Dr Bogdan Roman)
set MINGWPATH=c:\msys64\mingw64 set PATH=%MINGWPATH%\bin;%PATH% set PRELINK_CMDS=echo.>%TEMP%\mexstaticlibs rem set PRELINK_CMDS1=gcc -print-file-name=libwinpthread.a >> %TEMP%\mexstaticlibs rem set PRELINK_CMDS2=gcc -print-file-name=libquadmath.a >> %TEMP%\mexstaticlibs rem set PRELINK_CMDS3=... rem set POSTLINK_CMDS1=upx -9 "%OUTDIR%%MEX_NAME%%MEX_EXT%" rem set POSTLINK_CMDS2=... set COMPILER=g++ set COMPFLAGS=-c -I"%MATLAB%\extern\include" -DMATLAB_MEX_FILE set OPTIMFLAGS=-O3 -funroll-loops -DNDEBUG set DEBUGFLAGS=-g set NAME_OBJECT=-o set LINKER=g++ set LINKFLAGS=-shared -static-libstdc++ -static-libgcc -L"%MATLAB%\bin\win64" -L"%MATLAB%\extern\lib\win64\microsoft" -lmex -lmx -leng -lmat -lmwlapack -lmwblas set LINKFLAGSPOST=@%TEMP%\mexstaticlibs set NAME_OUTPUT=-o "%OUTDIR%%MEX_NAME%%MEX_EXT%"
Copy the file to
%USERPROFILE%\AppData\Roaming\Mathworks\MATLAB\R2015a, for instance.
Then open a MinGW-w64 terminal and add
mex to your path, e.g.
export PATH=$PATH:/c/Program\ Files/MATLAB/R2015a/bin/win64
and compile the mex file typing
mex libxc.c -I/c/lib/libxc-4.3.4/include -L/c/lib/libxc-4.3.4/lib -lxc
Again, the file
libxc.mexw64 will be created upon successful
2.7. Compiling the GPU wrapper
RESCU accelerates certain calculations using GPUs. Doing so requires a
CUDA installation and
compiling a few MEX files. Here is how to do it. Create an M-file in
which the variable
cuda is defined as the path to your CUDA
installation. For example,
cat > /tmp/installpaths.m <<EOF cuda = '/usr/local/cuda-9.2'; EOF
makeRESCU with the GPU option as follows
makeRESCU -gpu /tmp/installpaths.m
2.8. Compiling and using RESCU as a stand-alone executable
With the help of external libraries like OpenMPI and ScaLAPACK, RESCU can use hundreds, even thousands of cores. However, this may require a prohibitive number of MATLAB licenses. This problem is circumvented by compiling RESCU into a standalone program which do not require any license.
Note that MATLAB and its Compiler toolbox are required. Install the
optional packages (e.g. LibXC, SMI) that you intend to use. Call the
compileRESCU in MATLAB’s command window to compile RESCU.
matlab -r "compileRESCU('src','bin')"
The first argument specifies the source directory. It should contain the source code and all relevant MEX files. The second argument is the directory where the RESCU application is installed.
To execute RESCU, the freely available MATLAB Compiler Runtime (MCR) is required. Download and install the MCR that matches the MATLAB version that you used to compile RESCU. Next, follow the MCR manual to set up your environment properly. On Windows and Mac, you likely have nothing to do. On linux, you must modify your environment as follows.
export MCR_CACHE_ROOT=/scratch/johndoe MCRROOT=/software/MCR/v99 export LD_LIBRARY_PATH=$MCRROOT/runtime/glnxa64:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=$MCRROOT/bin/glnxa64:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=$MCRROOT/sys/os/glnxa64:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=$MCRROOT/extern/bin/glnxa64:$LD_LIBRARY_PATH
MCR_CACHE_ROOT is optional but recommended. It tells
the MCR where to store its cache files which may grow quite large.
Finally, add the RESCU standalone application location to the
Suppose that RESCU is found in
/home/software. On Linux you may add
RESCU to your
On Windows, go to System\(>\)System
Properties\(>\)Environment Variables. Select
PATH and click
on the Modify button. Then append