Configuring Caelus Python Library

CPL provides a YAML-based configuration utility that can be used to customize the library depending on the operating system and user’s specific needs. A good example is to provide non-standard install locations for the OpenFOAM or Caelus CML executables, as well as using different versions of OpenFOAM/CML with CPL simultaneously.

The use of configuration file is optional, CPL provides defaults that should work on most systems and will attempt to auto-detect CML installations on standard paths. On Linux/OS X systems, CPL will look at ~/Caelus/caelus-VERSION to determine the installed CML versions and use the VERSION tag to determine the latest version to use. On Window systems, the default search path is C:\Caelus.

Upon invocation, CPL will search and load configuration files from the following locations, if available. The files are loaded in sequence shown below and options found in succeeding files will overwrite configuration options found in preceeding files.

  1. Default configuration supplied with CPL;
  2. The system-wide configuration in file pointed by environment variable CAELUSRC_SYSTEM if it exists;
  3. The per-user configuration file, if available. On Linux/OS X, this is the file ~/.caelus/caelus.yaml, and %APPDATA%/caelus/caelus.yaml on Windows systems;
  4. The per-user configuration file pointed by the environment variable CAELUSRC if it exists;
  5. The file caelus.yaml in the current working directory, if it exists.

While CPL provides a way to auto-discover installed OpenFOAM and CML versions, often it will be necessary to provide at least a system-wide or per-user configuration file to allow CPL to use the right CML executables present in your system. A sample CPL configuration is shown below download caelus.yaml:

# -*- mode: yaml -*-
# Sample CPL configuration file

# Root CPL configuration node
  # Control logging of CPL library
    log_to_file: true
    log_file: ~/Caelus/cpl.log

  # Configuration for Caelus CML or OpenFOAM versions
    # Pick the development version of CML available; use "latest" to choose the
    # latest version available.
    default: "v2012"

    # Versions that can be used with CPL
      - version: "9.04"
        path: ~/Caelus/caelus-9.04

      - version: "10.04"
        path: ~/Caelus/caelus-10.04

      - version: "dev-gcc"
        path: ~/Caelus/caelus-cml            # Use latest git repository
        mpi_path: /usr/local/openmpi         # Use system OpenMPI
        build_option: "linux64gcc++DPOpt"    # Use the GCC version

      - version: "v2012"
        path: ~/OpenFOAM/OpenFOAM-v2012      # Use OpenFOAM
        mpi_path: /usr/local/openmpi         # Use system OpenMPI

The above configuration would be suitable as as a system-wide or per-user configuration stored in the home directory, and the user can override specific options used for particular runs by using, for example, the following caelus.yaml within the case directory:

# Local CPL settings for this working directory
    log_file: cpl_dev.log  # Change log file to a local file

    default: "dev-gcc"     # Use the latest dev version for this run

Note that only options that are being overridden need to be specified. Other options are populated from the system-wide or per-user configuration file if they exist.

Checking current configuration

To aid debugging and troubleshooting, CPL provides a command caelus cfg to dump the configuration used by the library based on all available configuration files. A sample usage is shown here:

$ caelus -v cfg
DEBUG: Loaded configuration from files = ['/home/caelus/.caelus/caelus.yaml']
INFO: Caelus Python Library (CPL) v0.1.0
# -*- mode: yaml -*-
# Caelus Python Library (CPL) v0.1.0
# Auto-generated on: 2018-04-21 17:03:35 (UTC)

    python_env_type: conda
    python_env_name: caelus
      conda_bin: ~/anaconda/bin
    job_scheduler: local_mpi
    always_use_scheduler: false
      join_outputs: true
      shell: /bin/bash
      mail_opts: NONE
    log_to_file: true
    log_file: null
    default: latest
    versions: []

The final configuration after parsing all available configuration files is shown in the output. If the user provides -v (verbose) flag, then the command also prints out all the configuration files that were detected and read during the initialization process. Users can also use caelus cfg to create a configuration file with all the current settings using the -f option. Please see caelus command documentation for details.

CPL configuration reference

CPL configuration files are in YAML format and must contain at least one node caelus. Two other optional nodes can be present in the file, caelus_scripts and caelus_user whose purpose is described below.


The root YAML node containing the core CPL configuration object. This node contains all configuration options used internally by the library.


An optional node used to store configuration for CPL CLI apps.


An optional node node reserved for user scripts and applications that will be built upon CPL.


In the following sections, the configuration parameters are documented in the format root_note.sub_node.config_parameter. Please see the sample configuration file above for the exact nesting structure used for caelus.logging.log_file.

Core library configuration

Python environment options


This section contains options to configure the python environment (either Anaconda/Conda environment or virtualenv settings).


Type of python environment. Currently this can be either conda or virtualenv.


The name of the Python environment for use with CPL, e.g., caelus or caelus-dev.


Extra information for Conda installation on your system.

System configuration


This section provides CPL with necessary information on the system settings, particularly the queue configuration on HPC systems.


The type of job-scheduler available on the system and used by CPL when executing CML executables on the system. By default, all parallel jobs will use the job scheduler, user can configure even serial jobs (e.g., mesh generation, domain decomposition and reconstruction) be submitted on queues.

Name Description
local_mpi No scheduler, submit locally
slurm Use SLURM commands to submit jobs

A Boolean flag indicating whether even serial jobs (e.g., mesh generation) should use the queue system. This flag is useful when the user intends to generate large meshes and requires access to the high-memory compute nodes on the HPC system.


This section contains options that are used by default when submitting jobs to an HPC queue system.

Option Description
queue Default queue for submitting jobs
account Account for charging core hours
stdout Default file pattern for redirecting standard output
stdout Default file pattern for redirecting standard error
join_outputs Join stdout and stderr (queue specific)
mail_options A string indicating mail options for queue
email_address Address where notifications should be sent
time_limit Wall clock time limit
machinefile File used in mpirun -machinefile <FILE>


Currently, these options accept strings and are specific to the queue system (e.g., SLURM or PBS Torque). So the user must consult their queue system manuals for appropriate values to these options.

CPL logging options


This section of the configuration file controls the logging options for the CPL library. By default, CPL only outputs messages to the standard output. Users can optionally save all messages from CPL into a log file of their choice. This is useful for tracking and troubleshooting, or providing additional information regarding bugs observed by the user.

Internally, CPL uses the logging module. For brevity, messages output to console are usually at log levels INFO or higher. However, all messages DEBUG and above are captured in log files.


A Boolean value indicating whether CPL should output messages to the log file. The default value is false. If set to true, then the log messages will also be saved to the file indicated by log_file as well as output to the console.


Filename where the log messages are saved if log_to_file evaluates to True.

CML version configuration


The primary purpose of CPL is to interact with OpenFOAM and CML executables and utilities. This section informs CPL of the various OpenFOAM/CML installations available on a system and the desired version used by CPL when invoking OpenFOAM or CML executables.


A string parameter indicating default version used when invoking OpenFOAM or CML executables. It must be one of the version entries provided in the file. Alternately, the user can specify latest to indicate that the latest version must be used. If users rely on auto-discovery of Caelus versions in default install locations, then it is recommended that this value be latest so that CPL picks the latest CML version. For example, with the following configuration, CPL will choose version 10.04 when attempting to execute programs like pisoSolver.

    default: "latest"

      - version: "9.04"
        path: ~/Caelus/caelus-9.04

      - version: "10.04"
        path: ~/Caelus/caelus-10.04

      - version: "v2012"
        path: ~/OpenFOAM/OpenFOAM-v2012

A list of configuration mapping listing various versions available for use with CPL. It is recommended that the users only provide version and path entries, the remaining entries are optional. CPL will auto-detect remaining parmeters.


A unique string identifier that is used to tag this specific instance of OpenFOAM or CML installation. Typically, this is the version number of the OpenFOAM or Caelus CML release, e.g., 10.04. However, as indicated in the example CPL configuration file, users can use any unique tag to identify a specific version. If this identifier does not follow the conventional version number format, then it is recommended that the user provide a specific version in caelus.caelus_cml.default instead of using latest.


The path to the C++ code/executables install. This is equivalent to the directory pointed by the WM_PROJECT_DIR or CAELUS_PROJECT_DIR environment variable, e.g., /home/caelus_user/projects/caelus/caelus-10.04.


A string parameter identifying the OpenFOAM or Caelus build, if multiple builds are present within a CML install, to be used with CPL. This is an expert only option used by developers who are testing multiple compilers and build options. It is recommended that the normal users let CPL autodetect the build option.


Path to the MPI installation used to compile OpenFOAM or Caelus for parallel execution. By default, CPL expects the MPI library to be present within the project directory.


Directory containing MPI binaries used for mpiexec when executing in parallel mode. If absent, CPL will assume that the binaries are located within the subdirectory bin in the path pointed by mpi_root.


Directory containing MPI libraries used for mpiexec when executing in parallel mode. If absent, CPL will assume that the libraries are located within the subdirectory lib in the path pointed by mpi_root.