Configuring Caelus Python Library¶

CPL provides a YAML-based configuration utility that can be used to customize the library depending on the operating system and user’s specific needs. A good example is to provide non-standard install locations for the OpenFOAM or Caelus CML executables, as well as using different versions of OpenFOAM/CML with CPL simultaneously.

The use of configuration file is optional, CPL provides defaults that should work on most systems and will attempt to auto-detect CML installations on standard paths. On Linux/OS X systems, CPL will look at ~/Caelus/caelus-VERSION to determine the installed CML versions and use the VERSION tag to determine the latest version to use. On Window systems, the default search path is C:\Caelus.

Upon invocation, CPL will search and load configuration files from the following locations, if available. The files are loaded in sequence shown below and options found in succeeding files will overwrite configuration options found in preceeding files.

Default configuration supplied with CPL;
The system-wide configuration in file pointed by environment variable CAELUSRC_SYSTEM if it exists;
The per-user configuration file, if available. On Linux/OS X, this is the file ~/.caelus/caelus.yaml, and %APPDATA%/caelus/caelus.yaml on Windows systems;
The per-user configuration file pointed by the environment variable CAELUSRC if it exists;
The file caelus.yaml in the current working directory, if it exists.

While CPL provides a way to auto-discover installed OpenFOAM and CML versions, often it will be necessary to provide at least a system-wide or per-user configuration file to allow CPL to use the right CML executables present in your system. A sample CPL configuration is shown below download caelus.yaml:

# -*- mode: yaml -*-
#
# Sample CPL configuration file
#

# Root CPL configuration node
caelus:
  # Control logging of CPL library
  logging:
    log_to_file: true
    log_file: ~/Caelus/cpl.log

  # Configuration for Caelus CML or OpenFOAM versions
  caelus_cml:
    # Pick the development version of CML available; use "latest" to choose the
    # latest version available.
    default: "v2012"

    # Versions that can be used with CPL
    versions:
      - version: "9.04"
        path: ~/Caelus/caelus-9.04

      - version: "10.04"
        path: ~/Caelus/caelus-10.04

      - version: "dev-gcc"
        path: ~/Caelus/caelus-cml            # Use latest git repository
        mpi_path: /usr/local/openmpi         # Use system OpenMPI
        build_option: "linux64gcc++DPOpt"    # Use the GCC version

      - version: "v2012"
        path: ~/OpenFOAM/OpenFOAM-v2012      # Use OpenFOAM
        mpi_path: /usr/local/openmpi         # Use system OpenMPI

The above configuration would be suitable as as a system-wide or per-user configuration stored in the home directory, and the user can override specific options used for particular runs by using, for example, the following caelus.yaml within the case directory:

# Local CPL settings for this working directory
caelus:
  logging:
    log_file: cpl_dev.log  # Change log file to a local file

  caelus_cml:
    default: "dev-gcc"     # Use the latest dev version for this run

Note that only options that are being overridden need to be specified. Other options are populated from the system-wide or per-user configuration file if they exist.

Checking current configuration¶

To aid debugging and troubleshooting, CPL provides a command caelus cfg to dump the configuration used by the library based on all available configuration files. A sample usage is shown here:

$ caelus -v cfg
DEBUG: Loaded configuration from files = ['/home/caelus/.caelus/caelus.yaml']
INFO: Caelus Python Library (CPL) v0.1.0
# -*- mode: yaml -*-
#
# Caelus Python Library (CPL) v0.1.0
#
# Auto-generated on: 2018-04-21 17:03:35 (UTC)
#

caelus:
  cpl:
    python_env_type: conda
    python_env_name: caelus
    conda_settings:
      conda_bin: ~/anaconda/bin
  system:
    job_scheduler: local_mpi
    always_use_scheduler: false
    scheduler_defaults:
      join_outputs: true
      shell: /bin/bash
      mail_opts: NONE
  logging:
    log_to_file: true
    log_file: null
  caelus_cml:
    default: latest
    versions: []

The final configuration after parsing all available configuration files is shown in the output. If the user provides -v (verbose) flag, then the command also prints out all the configuration files that were detected and read during the initialization process. Users can also use caelus cfg to create a configuration file with all the current settings using the -f option. Please see caelus command documentation for details.

CPL configuration reference¶

CPL configuration files are in YAML format and must contain at least one node caelus. Two other optional nodes can be present in the file, caelus_scripts and caelus_user whose purpose is described below.

caelus¶: The root YAML node containing the core CPL configuration object. This node contains all configuration options used internally by the library.

caelus_scripts¶: An optional node used to store configuration for CPL CLI apps.

caelus_user¶: An optional node node reserved for user scripts and applications that will be built upon CPL.

Note

In the following sections, the configuration parameters are documented in the format root_note.sub_node.config_parameter. Please see the sample configuration file above for the exact nesting structure used for caelus.logging.log_file.

Core library configuration¶

Python environment options¶

caelus.cpl¶: This section contains options to configure the python environment (either Anaconda/Conda environment or virtualenv settings).

caelus.cpl.python_env_type¶: Type of python environment. Currently this can be either conda or virtualenv.

caelus.cpl.python_env_name¶: The name of the Python environment for use with CPL, e.g., caelus or caelus-dev.

caelus.cpl.conda_settings¶: Extra information for Conda installation on your system.

System configuration¶

caelus.system¶: This section provides CPL with necessary information on the system settings, particularly the queue configuration on HPC systems.

caelus.system.job_scheduler¶

The type of job-scheduler available on the system and used by CPL when executing CML executables on the system. By default, all parallel jobs will use the job scheduler, user can configure even serial jobs (e.g., mesh generation, domain decomposition and reconstruction) be submitted on queues.

Name	Description
`local_mpi`	No scheduler, submit locally
`slurm`	Use SLURM commands to submit jobs

caelus.system.always_use_scheduler¶: A Boolean flag indicating whether even serial jobs (e.g., mesh generation) should use the queue system. This flag is useful when the user intends to generate large meshes and requires access to the high-memory compute nodes on the HPC system.

caelus.system.scheduler_defaults¶

This section contains options that are used by default when submitting jobs to an HPC queue system.

Option	Description
`queue`	Default queue for submitting jobs
`account`	Account for charging core hours
`stdout`	Default file pattern for redirecting standard output
`stdout`	Default file pattern for redirecting standard error
`join_outputs`	Join `stdout` and `stderr` (queue specific)
`mail_options`	A string indicating mail options for queue
`email_address`	Address where notifications should be sent
`time_limit`	Wall clock time limit
`machinefile`	File used in `mpirun -machinefile <FILE>`

Note

Currently, these options accept strings and are specific to the queue system (e.g., SLURM or PBS Torque). So the user must consult their queue system manuals for appropriate values to these options.

CPL logging options¶

caelus.logging¶

This section of the configuration file controls the logging options for the CPL library. By default, CPL only outputs messages to the standard output. Users can optionally save all messages from CPL into a log file of their choice. This is useful for tracking and troubleshooting, or providing additional information regarding bugs observed by the user.

Internally, CPL uses the logging module. For brevity, messages output to console are usually at log levels INFO or higher. However, all messages DEBUG and above are captured in log files.

caelus.logging.log_to_file¶: A Boolean value indicating whether CPL should output messages to the log file. The default value is false. If set to true, then the log messages will also be saved to the file indicated by log_file as well as output to the console.

caelus.logging.log_file¶: Filename where the log messages are saved if log_to_file evaluates to True.

CML version configuration¶

caelus.caelus_cml¶: The primary purpose of CPL is to interact with OpenFOAM and CML executables and utilities. This section informs CPL of the various OpenFOAM/CML installations available on a system and the desired version used by CPL when invoking OpenFOAM or CML executables.

caelus.caelus_cml.default¶

A string parameter indicating default version used when invoking OpenFOAM or CML executables. It must be one of the version entries provided in the file. Alternately, the user can specify latest to indicate that the latest version must be used. If users rely on auto-discovery of Caelus versions in default install locations, then it is recommended that this value be latest so that CPL picks the latest CML version. For example, with the following configuration, CPL will choose version 10.04 when attempting to execute programs like pisoSolver.

caelus:
  caelus_cml:
    default: "latest"

    versions:
      - version: "9.04"
        path: ~/Caelus/caelus-9.04

      - version: "10.04"
        path: ~/Caelus/caelus-10.04

      - version: "v2012"
        path: ~/OpenFOAM/OpenFOAM-v2012

caelus.caelus_cml.versions¶: A list of configuration mapping listing various versions available for use with CPL. It is recommended that the users only provide version and path entries, the remaining entries are optional. CPL will auto-detect remaining parmeters.

caelus.caelus_cml.versions.version¶: A unique string identifier that is used to tag this specific instance of OpenFOAM or CML installation. Typically, this is the version number of the OpenFOAM or Caelus CML release, e.g., 10.04. However, as indicated in the example CPL configuration file, users can use any unique tag to identify a specific version. If this identifier does not follow the conventional version number format, then it is recommended that the user provide a specific version in caelus.caelus_cml.default instead of using latest.

caelus.caelus_cml.versions.path¶: The path to the C++ code/executables install. This is equivalent to the directory pointed by the WM_PROJECT_DIR or CAELUS_PROJECT_DIR environment variable, e.g., /home/caelus_user/projects/caelus/caelus-10.04.

caelus.caelus_cml.versions.build_option¶: A string parameter identifying the OpenFOAM or Caelus build, if multiple builds are present within a CML install, to be used with CPL. This is an expert only option used by developers who are testing multiple compilers and build options. It is recommended that the normal users let CPL autodetect the build option.

caelus.caelus_cml.versions.mpi_root¶: Path to the MPI installation used to compile OpenFOAM or Caelus for parallel execution. By default, CPL expects the MPI library to be present within the project directory.

caelus.caelus_cml.versions.mpi_bin_path¶: Directory containing MPI binaries used for mpiexec when executing in parallel mode. If absent, CPL will assume that the binaries are located within the subdirectory bin in the path pointed by mpi_root.

caelus.caelus_cml.versions.mpi_lib_path¶: Directory containing MPI libraries used for mpiexec when executing in parallel mode. If absent, CPL will assume that the libraries are located within the subdirectory lib in the path pointed by mpi_root.