6. Sample run¶
This section will cover a sample run of ABIN LAUNCHER, to get a concrete example of its operating mode. Our study case here includes three molecules (or geometry files) and two configuration files. Our goal is to optimize the geometry of those three molecules with two different basis sets through ORCA. The profile we will define for this occasion will be called sample_orca.
Tip
Every file presented in this section can be downloaded here. The other files can be fetched directly from ABIN LAUNCHER’s directory. Note that for this example, the renderer.py and clusters.yml file from the abin_sample/sample_files directory replaces the ones present in ABIN LAUNCHER’s directory (which are more complex ones, used by CHAINS).
6.1. Preparation¶
6.1.1. Jinja templates¶
We will simply use the Jinja templates defined in the rendering section and place them in the templates directory of ABIN LAUNCHER:
The Jinja template for the ORCA input file:
sample_orca.inp.jinja¶
! {{ method }} {{ basis_set }} {{ job_type }}
*xyz {{ charge }} {{ multiplicity }}
{% for coordinate_line in coordinates -%}
{{ coordinate_line }}
{% endfor -%}
*
The Jinja template for the job script:
sample_orca_job.sh.jinja¶
#!/bin/bash
#SBATCH --output=slurm_output.log
#SBATCH --job-name={{ profile }}_{{ mol_name }}_{{ config_name }}
#SBATCH --mail-user={{ user_email }}
#SBATCH --mail-type={{ mail_type }}
#SBATCH --time={{ job_walltime }}
#SBATCH --ntasks={{ job_cores }}
#SBATCH --mem-per-cpu={{ job_mem_per_cpu }}
{% if partition != None -%}
#SBATCH --partition={{ partition }}
{% endif %}
echo -e "Renaming the original .xyz file to avoid overwriting it with the new one."
cd $SLURM_SUBMIT_DIR
mv {{ mol_name }}.xyz {{ mol_name }}_ori.xyz
echo -e "\n================= ORCA execution begins now =================="
{% for set_env_line in set_env -%}
{{ set_env_line }}
{% endfor -%}
{{ command }} {{ mol_name }}.inp > {{ mol_name }}.out
echo -e "\n================= ORCA execution ends now =================="
6.1.2. Rendering function¶
We will also use the rendering function we defined in the rendering section, this function is placed in the renderer.py file. For our sample case, this file contains:
renderer.py¶
################################################################################################################################################
## The Renderer ##
## ##
## This script contains the rendering functions for ABIN LAUNCHER, ##
## consult the documentation at https://chains-ulb.readthedocs.io/ for details ##
################################################################################################################################################
import os
from jinja2 import Environment, FileSystemLoader
import abin_errors
def jinja_render(templates_dir:str, template_file:str, render_vars:dict):
"""Renders a file based on its Jinja template.
Parameters
----------
templates_dir : str
The path towards the directory where the Jinja template is located.
template_file : str
The name of the Jinja template file.
render_vars : dict
Dictionary containing the definitions of all the variables present in the Jinja template.
Returns
-------
output_text : str
Content of the rendered file.
"""
file_loader = FileSystemLoader(templates_dir)
env = Environment(loader=file_loader)
template = env.get_template(template_file)
output_text = template.render(render_vars)
return output_text
# =================================================================== #
# =================================================================== #
# Rendering functions #
# =================================================================== #
# =================================================================== #
def sample_orca_render(mendeleev:dict, clusters_cfg:dict, config:dict, file_data:dict, job_specs:dict, misc:dict):
"""Renders the job script and the input file associated with the ORCA program.
Parameters
----------
mendeleev : dict
Content of AlexGustafsson's Mendeleev Table YAML file (found at https://github.com/AlexGustafsson/molecular-data).
Unused in this function.
clusters_cfg : dict
Content of the YAML clusters configuration file.
config : dict
Content of the YAML configuration file.
file_data : dict
Information extracted by the scanning function from the geometry file.
job_specs : dict
Contains all information related to the job.
misc : dict
Contains all the additional variables that did not pertain to the other arguments.
Returns
-------
rendered_content : dict
Dictionary containing the text of all the rendered files in the form of <filename>: <rendered_content>.
rendered_script : str
Name of the rendered job script, necessary to launch the job.
Notes
-----
Pay a particular attention to the render_vars dictionaries, they contain all the definitions of the variables appearing in your Jinja templates.
"""
# Define the names of the templates
template_input = "sample_orca.inp.jinja"
template_script = "sample_orca_job.sh.jinja"
# Define the names of the rendered files
rendered_input = misc['mol_name'] + ".inp"
rendered_script = "orca_job.sh"
# Initialize the dictionary that will be returned by the function
rendered_content = {}
# Render the template for the input file
print("{:<80}".format("\nRendering the jinja template for the orca input file ..."), end="")
input_render_vars = {
"method" : config['method'],
"basis_set" : config['basis_set'],
"job_type" : config['job_type'],
"charge" : config['charge'],
"multiplicity" : config['multiplicity'],
"coordinates" : file_data['atomic_coordinates']
}
rendered_content[rendered_input] = jinja_render(misc['templates_dir'], template_input, input_render_vars)
print('%12s' % "[ DONE ]")
# Render the template for the job script
print("{:<80}".format("\nRendering the jinja template for the orca job script ..."), end="")
script_render_vars = {
"mol_name" : misc['mol_name'],
"config_name" : misc['config_name'],
"user_email" : config['user_email'],
"mail_type" : config['mail_type'],
"job_walltime" : job_specs['walltime'],
"job_cores" : job_specs['cores'],
"job_mem_per_cpu" : job_specs['mem_per_cpu'],
"partition" : job_specs['partition'],
"set_env" : clusters_cfg[job_specs['cluster_name']]['profiles'][job_specs['profile']]['set_env'],
"command" : clusters_cfg[job_specs['cluster_name']]['profiles'][job_specs['profile']]['command'],
"profile" : job_specs['profile']
}
rendered_content[rendered_script] = jinja_render(misc['templates_dir'], template_script, script_render_vars)
print('%12s' % "[ DONE ]")
# Return the content of the rendered files and the name of the rendered job script
return rendered_content, rendered_script
6.1.3. Clusters configuration file¶
For this run, we will be running on the lemaitre3 cluster from the CECI, the job scheduler is SLURM and the chosen scaling function is total_nb_elec.
The clusters configuration file looks like:
clusters.yml¶
lemaitre3:
submit_command: sbatch
profiles:
sample_orca:
rendering_function: sample_orca_render
set_env:
- module --force purge
- module load releases/2018b
- module load ORCA/4.1.0-OpenMPI-3.1.3
command: /opt/cecisw/arch/easybuild/2018b/software/ORCA/4.1.0-OpenMPI-3.1.3/orca
scaling_function: total_nb_elec
job_scales:
-
label: tiny
scale_limit: 50
time: 0-00:10:00
cores: 4
mem_per_cpu: 500 # in MB
-
label: small
scale_limit: 700
partition_name: batch
time: 1-00:00:00
cores: 8
mem_per_cpu: 500 # in MB
-
label: medium
scale_limit: 1000
partition_name: batch
time: 2-00:00:00
cores: 8
mem_per_cpu: 2000 # in MB
delay_command: --begin=now+60
-
label: big
scale_limit: 1500
partition_name: batch
time: 2-00:00:00
cores: 8
mem_per_cpu: 4000 # in MB
delay_command: --begin=now+120
6.2. Execution¶
6.2.1. Geometries and configuration files¶
Our three molecules are simply CH4, C2H6 and C3H8. They are each represented by a XYZ geometry file:
ch4.xyz¶
5
C -2.10791 2.13879 0.00000
H -1.03791 2.13879 0.00000
H -2.46457 1.49933 -0.78025
H -2.46458 3.13423 -0.16366
H -2.46458 1.78280 0.94391
c2h6.xyz¶
8
C -5.27061 0.53724 -0.00000
C -3.75887 0.50632 -0.00000
H -5.66181 0.20733 -0.96702
H -5.63432 1.55129 0.19093
H -5.66857 -0.12330 0.77610
H -3.36090 1.16686 -0.77610
H -3.39516 -0.50772 -0.19093
H -3.36766 0.83624 0.96702
c3h8.xyz¶
11
H 0.92451 0.98680 1.08533
C 0.27964 1.04466 0.19936
H 0.58156 1.93418 -0.36940
H 0.50043 0.16911 -0.42554
C -1.18197 1.09949 0.58346
H -1.44619 0.20868 1.18658
H -1.36524 1.96970 1.24398
C -2.07684 1.18012 -0.63284
H -1.86356 2.07478 -1.23283
H -1.94543 0.30975 -1.28937
H -3.13606 1.22011 -0.34874
For our two configuration files, we want to perform a geometry optimization using the DFT method with the B3LYP functional, and the two considered basis sets are def2-SVP and def2-TZVP. Thus, our two files are:
svp.yml¶
method: B3LYP
basis_set: def2-SVP
job_type: Opt
charge: 0
multiplicity: 1
user_email: your@email.com
mail_type: FAIL
tzvp.yml¶
method: B3LYP
basis_set: def2-TZVP
job_type: Opt
charge: 0
multiplicity: 1
user_email: your@email.com
mail_type: FAIL
Let’s store those files in two distinct directories: one for the molecule geometries, named molecules, and one for the configuration files, named configs. We will also create a directory for the jobs, named orca_jobs.
This is what our directory on the cluster, named abin_docs_sample, now looks like:
abin_docs_sample/
└── abin_launcher/
├── abin_launcher.py
├── geom_scan.py
├── scaling_fcts.py
├── renderer.py
├── abin_errors.py
├── clusters.yml
├── mendeleev.yml
└── templates/
├── sample_orca.inp.jinja
└── sample_orca_job.sh.jinja
└── molecules/
├── ch4.xyz
├── c2h6.xyz
└── c3h8.xyz
└── configs/
├── svp.yml
└── tzvp.yml
└── orca_jobs/
└── currently empty
6.2.2. Running ABIN LAUNCHER¶
Tip
Before executing ABIN LAUNCHER, remember to load your Python 3.5+ distribution (either manually or through your .bashrc or .bash_profile files), which must include PyYAML (version 5.1+) and Jinja2 (version 2.10+).
We can now execute abin_launcher.py by running the command (from abin_docs_sample):
$ python abin_launcher/abin_launcher.py -m molecules/ -cf configs/ -p sample_orca -o orca_jobs/ -cl lemaitre3
This is what appears on the console screen (here captured in a file for ease of reading):
sample_run.log¶
********************************************************************************************************************************************************
EXECUTION OF THE AB INITIO INPUT BUILDER & JOB LAUNCHER BEGINS NOW
********************************************************************************************************************************************************
Codes directory: /home/users/n/i/niacobel/abin_docs_sample/abin_launcher
Loading AlexGustafsson's Mendeleev Table ... [ DONE ]
Loading the clusters configuration file ... [ DONE ]
Cluster name: lemaitre3
Profile: sample_orca
Scaling function for that profile: total_nb_elec
Rendering function for that profile: sample_orca_render
Job scales for that profile:
--------------------------------------------------------------------------------------------------------------------------------------------------
Scale Limit Label Partition Name Time Cores Mem per CPU (MB) Delay Command
--------------------------------------------------------------------------------------------------------------------------------------------------
50 tiny not specified 0-00:10:00 4 500 not specified
700 small batch 1-00:00:00 8 500 not specified
1000 medium batch 2-00:00:00 8 2000 --begin=now+60
1500 big batch 2-00:00:00 8 4000 --begin=now+120
--------------------------------------------------------------------------------------------------------------------------------------------------
Jobs root directory: /home/users/n/i/niacobel/abin_docs_sample/orca_jobs
Looking for .xyz geometry files in /home/users/n/i/niacobel/abin_docs_sample/molecules ... [ DONE ]
Looking for .yml or .yaml files in /home/users/n/i/niacobel/abin_docs_sample/configs ... [ DONE ]
***********************************************
Start procedure for the molecule ch4
***********************************************
Treating 'ch4' geometry with 'tzvp' configuration ... Submitted batch job 69307472 on cluster lemaitre3
Treating 'ch4' geometry with 'svp' configuration ... Submitted batch job 69307473 on cluster lemaitre3
Geometry file archived to /home/users/n/i/niacobel/abin_docs_sample/molecules/launched
***********************************************
End of procedure for the molecule ch4
***********************************************
************************************************
Start procedure for the molecule c3h8
************************************************
Treating 'c3h8' geometry with 'tzvp' configuration ... Submitted batch job 69307474 on cluster lemaitre3
Treating 'c3h8' geometry with 'svp' configuration ... Submitted batch job 69307475 on cluster lemaitre3
Geometry file archived to /home/users/n/i/niacobel/abin_docs_sample/molecules/launched
************************************************
End of procedure for the molecule c3h8
************************************************
************************************************
Start procedure for the molecule c2h6
************************************************
Treating 'c2h6' geometry with 'tzvp' configuration ... Submitted batch job 69307476 on cluster lemaitre3
Treating 'c2h6' geometry with 'svp' configuration ... Submitted batch job 69307477 on cluster lemaitre3
Geometry file archived to /home/users/n/i/niacobel/abin_docs_sample/molecules/launched
************************************************
End of procedure for the molecule c2h6
************************************************
The 'tzvp.yml' config file has been archived to /home/users/n/i/niacobel/abin_docs_sample/configs/launched
The 'svp.yml' config file has been archived to /home/users/n/i/niacobel/abin_docs_sample/configs/launched
6 jobs have been succesfully launched.
********************************************************************************************************************************************************
END OF EXECUTION
********************************************************************************************************************************************************
And if we take a look at the job queue, we can see that our 6 ORCA jobs have indeed been submitted to the job scheduler:
Jobs queue after running ABIN LAUNCHER¶
Tip
If you don’t want to explicitly enter this command each time you want to run ABIN_LAUNCHER, you can always define an alias in your ~/.bashrc file!
6.2.3. Directory structure after execution¶
If we take a look at the directory structure, it now looks like:
abin_docs_sample/
└── abin_launcher/
└── no changes
└── molecules/
└── launched/
├── ch4.xyz
├── c2h6.xyz
└── c3h8.xyz
└── configs/
└── launched/
├── svp.yml
└── tzvp.yml
└── orca_jobs/
└── ch4_svp/
├── ch4.xyz
├── svp.yml
├── ch4.inp
├── orca_job.sh
└── ch4_svp.log
└── ch4_tzvp/
├── ch4.xyz
├── tzvp.yml
├── ch4.inp
├── orca_job.sh
└── ch4_tzvp.log
└── c2h6_svp/
├── c2h6.xyz
├── svp.yml
├── c2h6.inp
├── orca_job.sh
└── c2h6_svp.log
└── c2h6_tzvp/
├── c2h6.xyz
├── tzvp.yml
├── c2h6.inp
├── orca_job.sh
└── c2h6_tzvp.log
└── c3h8_svp/
├── c3h8.xyz
├── svp.yml
├── c3h8.inp
├── orca_job.sh
└── c3h8_svp.log
└── c3h8_tzvp/
├── c3h8.xyz
├── tzvp.yml
├── c3h8.inp
├── orca_job.sh
└── c3h8_tzvp.log
As you can see, since they have been successfully processed, the geometry files and the configuration files have both been archived into a launched directory created by ABIN LAUNCHER. This is the default behavior, allowing you to reuse the same directories for other geometries and configurations, making it easier for example to create an alias for the execution command. If you want to turn it off, just add the -km / --keep_mol and/or -kc / --keep_cf optional arguments to keep the geometry files and/or the configuration files, respectively.
A subdirectory has also been created in orca_jobs for each of the six jobs. Those subdirectories contain the copies of the geometry and the configuration files, the rendered input file and job script, as well as a log file containing the details of the treatment of this geometry-configuration combination by ABIN LAUNCHER. (The output files created by ORCA will also end up in those subdirectories.)
6.2.4. The log file¶
As an example, here is what the c3h8_tzvp.log file looks like:
c3h8_tzvp.log¶
Cluster name: lemaitre3
Profile: sample_orca
***************************************
1. Scanning the geometry file
***************************************
Scanning function: xyz_scan
Loading c3h8.xyz file ... [ DONE ]
-----------------------------------
Atom Type Number of atoms
-----------------------------------
H 8
C 3
-----------------------------------
Total 11
-----------------------------------
11 lines of atomic coordinates have been registered.
************************
2. Job scaling
************************
Scaling function: total_nb_elec
Available job scales:
--------------------------------------------------------------------------------------------------------------------------------------------------
Scale Limit Label Partition Name Time Cores Mem per CPU (MB) Delay Command
--------------------------------------------------------------------------------------------------------------------------------------------------
50 tiny not specified 0-00:10:00 4 500 not specified
700 small batch 1-00:00:00 8 500 not specified
1000 medium batch 2-00:00:00 8 2000 --begin=now+60
1500 big batch 2-00:00:00 8 4000 --begin=now+120
--------------------------------------------------------------------------------------------------------------------------------------------------
A. Scaling function
===================
---------------------------------------------------------------------
Atom Type Atomic Number Number of atoms Number of electrons
---------------------------------------------------------------------
H 1 8 8
C 6 3 18
---------------------------------------------------------------------
Total 11 26
---------------------------------------------------------------------
Scale index: 26
B. Calculation requirements
===========================
--------------------------------------------------
Scale index: 26
--------------------------------------------------
Cluster: lemaitre3
Job scale: tiny
Job scale limit: 50
--------------------------------------------------
Job partition: not specified
Job walltime: 0-00:10:00
Number of cores: 4
Mem per CPU (MB): 500
Delay command: not specified
--------------------------------------------------
************************************
3. Rendering the templates
************************************
Rendering function: sample_orca_render
Loading tzvp.yml file ... [ DONE ]
A. Rendering function
=====================
Rendering the jinja template for the orca input file ... [ DONE ]
Rendering the jinja template for the orca job script ... [ DONE ]
All the templates have been succesfully rendered.
B. Creating the files
=====================
Job directory: /home/users/n/i/niacobel/abin_docs_sample/orca_jobs/c3h8_tzvp
├── The c3h8.inp file has been created into the directory
├── The orca_job.sh file has been created into the directory
└── The geometry file (c3h8.xyz) and the configuration file (tzvp.yml) have been successfully copied into the directory.
*******************************
4. Submitting the job
*******************************
Launching the job ... [ DONE ]