3. Compute Canada examples: calculations on Rorqual

This tutorial follows two real projects that ran on Rorqual, an Alliance (Compute Canada) cluster, from a laptop: a one-calculation connection test, and a multi-step study of a two-dimensional material with a batch of jobs. Both used VASP and the rorqual-gpu profile from Running on Alliance (Compute Canada) clusters; the same steps apply to RESCU and NanoDCAL with a CPU profile and your own installation of those solvers on the cluster. All numbers are the ones those runs reported.

What you need

  • The Alliance setup from Running on Alliance (Compute Canada) clusters finished: an SSH key in CCDB, the Host rorqual entry, and a saved profile (here rorqual-gpu).

  • Your solver on the cluster under your own licence (for VASP, membership of the VASP group or your own build).

  • Today’s shared connection, opened once:

    wsl -d Ubuntu-24.04 -e ssh -fN rorqual
    

3.1. Example 1 — A first job: silicon SCF

Start latticemind in an empty folder, select the cluster, and check the connection:

/remote rorqual-gpu
/remote status rorqual-gpu

/remote status says whether the shared connection is open and lists LatticeMind’s own jobs on the cluster (their names start with lm-). Then ask for one small calculation:

This is a short connectivity test of the remote cluster profile, not a
research question. Create a VASP project here, build bulk silicon in the
diamond structure (primitive 2-atom cell, a = 5.468 Å), and run one SCF:
PBE, ENCUT = 400, PREC = Accurate, EDIFF = 1E-7, ISMEAR = 0 with SIGMA =
0.05, Γ-centred 8 × 8 × 8 k-points, standard Si PAW potential (check it with
the asset inspection tool). Submit it on the configured remote profile.
When it finishes, report the total energy and the band gap from the run
evidence, and say where the run executed. Do not run anything else.

What happens:

  1. Inputs are written and checked on your laptop: INCAR, KPOINTS, POSCAR, and the POTCAR assembled from your local POTCAR library (VASP_PP_PATH). Nothing is sent yet.

  2. The permission question shows exactly what will run: the batch script (#SBATCH lines for the account, time, memory and GPU from the profile, then the modules and the launch line) and the launch script, with the remote folder under $SCRATCH. Answer yes, or no to change something first.

  3. One sbatch is submitted. LatticeMind records its intention to submit before sending it, so a lost reply never leads to a second job: the next check finds the job by its name.

  4. While it waits, the status line shows the Slurm state, for example Waiting in the Slurm queue (Priority) and then Running on the cluster. Checks slow down while the job is pending, so a long queue does not flood the connection.

  5. When it finishes, the outputs are fetched into results/scf/ with their sizes and checksums verified, and the answer is built from them.

The answer:

Quantity

Value

Total energy (2-atom cell)

−10.84587 eV

Band gap (indirect, on the 8 × 8 × 8 mesh)

0.649 eV

Smallest direct gap (Γ)

2.557 eV

SCF steps to EDIFF = 10⁻⁷

11

Where it ran

rorqual, profile rorqual-gpu, one Slurm job

VASP version reported by the run

6.5.1

Solver time

14 s (after a short wait in the queue)

The answer also states its limits: the conduction-band minimum lies between mesh points along Γ–X, so the gap on this coarse mesh is approximate; it is a plain PBE Kohn–Sham gap; and the cutoff and k-mesh were not tested for convergence. A run that finishes is reported separately from a converged or physically adequate result.

3.2. Example 2 — A research study: sliding ferroelectricity in 3R-MoS₂

Li et al., Nat. Commun. 15, 10921 (2024), describe rhombohedral (3R) bilayer MoS₂ as a sliding ferroelectric: its two stackings, XM and MX, have opposite out-of-plane polarization, and sliding one layer over the other switches between them. Reproducing the DFT part needs a monolayer relaxation, two bilayer relaxations with a dipole correction, and a seven-image sliding path: about ten relaxations, too many for a laptop.

Select the cluster and describe the study, with the settings the paper uses and the checks you want. The prompt below condenses the messages of the real study, which took several turns: its first monolayer relaxation was asked to use a dipole correction, did not converge, and was corrected in conversation (LatticeMind’s validator now flags a dipole correction on a symmetric slab).

/remote rorqual-gpu

Reproduce the DFT part of Li et al., Nat. Commun. 15, 10921 (2024), sliding
ferroelectric 3R bilayer MoS2. VASP, PBE with Grimme D3, ENCUT = 520 eV,
PREC = Accurate, EDIFF = 1E-6, Γ-centred 12 x 12 x 1 k-points, 21 Å of
vacuum. First relax the monolayer in-plane (no dipole correction: it has no
net dipole). Then build the XM and MX 3R stackings with the bilayer recipe
and relax only the ions with the dipole correction along z. Report
E(XM) - E(MX), the dipole and the vacuum potential step of each stacking,
and the sliding distance. Then compute the XM -> MX sliding profile with
the path tool: 7 images, each relaxing z only, same settings. Batch the
cluster submissions, check queued jobs rather than waiting idly, and stop
to ask only if a run fails.

How the cluster is used:

  • Relaxations go out as Slurm jobs, several at once. Independent runs, such as the two bilayer stackings and the seven path images, are prepared together and approved as one batch, so you answer one permission question for a whole set of jobs.

  • You can leave. Close the laptop while jobs wait in the queue. The jobs keep running on Rorqual. Tomorrow, open the connection again, start latticemind in the same folder and ask “collect the finished runs and continue”. Jobs that finished while you were away are fetched and checked, and nothing is submitted twice.

  • Failures come back with the cluster’s own words. A job that hits its time limit or runs out of memory is reported with its Slurm state and the end of both logs; LatticeMind does not resubmit it by itself. Raise the limit for later runs with, for example, /compute slurm rorqual-gpu time=06:00:00 mem=64G.

What the study found:

Quantity

Result

E(XM) − E(MX)

0.07 meV per formula unit (degenerate, as symmetry requires)

Out-of-plane dipole, XM and MX

±0.0031 e·Å, equal and opposite

Vacuum potential step ΔV

±0.065 eV

Sliding distance XM → MX

1.827 Å = a/√3

Sliding barrier (7 images, z relaxed)

15.2 meV per cell (7.6 meV per formula unit); symmetric profile, dipole through zero at the midpoint

Two results need care when you compare them with a paper. The sliding distance follows from the relaxed lattice constant and the 3R geometry (a/√3), so check how a published value is defined before comparing it. And the barrier comes from a fixed straight path sampled at seven points, not a nudged-elastic-band calculation, so LatticeMind reports it as a sampled barrier that needs image-spacing and path convergence, not as the minimum-energy barrier.

3.3. Changing resources

The profile holds the default request; change it for later runs of this conversation without editing any file:

/compute slurm rorqual-gpu time=06:00:00
/compute slurm rorqual-gpu mem=64G cpus_per_task=4
/compute slurm rorqual-cpu nodes=1 ntasks=64

Any Slurm resource in the profile can be overridden this way: account, partition, qos, time, nodes, ntasks, ntasks_per_node, cpus_per_task, mem or mem_per_cpu, gpus, constraint, modules and launcher. time is at least 00:10:00 and at most 24:00:00, written as Slurm writes it (90 is 90 minutes, 20:00 is 20 minutes, 1-12 is one day and 12 hours). The batch script in each permission question shows the values that will be used. From a script, the same overrides are --slurm KEY=VALUE options:

latticemind --remote-profile rorqual-gpu --remote-solver vasp --slurm time=04:00:00

/compute slurm is a terminal command; in the browser, edit the profile instead.

3.4. Stopping a job

Ask in words, for example “cancel the queued MX relaxation”. LatticeMind cancels the job with scancel and records it as cancelled; nothing is resubmitted. A job you cancel yourself on the cluster with scancel is reported as cancelled in Slurm, with the end of its logs.

3.5. Good practice

  • Start with a short test such as Example 1 whenever you set up a new profile or a new solver build: it checks the modules, the launcher and output retrieval for a few seconds of compute.

  • Request what the run needs. Long time limits and whole GPUs wait longer in the queue and cost your group more fair-share.

  • Keep results in your project, not on the cluster: $SCRATCH is purged after a period of inactivity, and the copy in results/ is the one you keep.