caelus.utils – Basic utilities

Collection of low-level utilities that are accessed by other packages within CPL, and other code snippets that do not fit elsewhere within CPL. The modules present within utils package must only depend on external libraries or other modules within util, they must not import modules from other packages within CPL.

Struct Dictionary that supports both key and attribute access.
osutils Miscellaneous utilities

Struct Module

Implements Struct.

class caelus.utils.struct.Struct(*args, **kwds)[source]

Bases: collections.OrderedDict, _abcoll.MutableMapping

Dictionary that supports both key and attribute access.

Struct is inspired by Matlab struct data structure that is intended to support both key and attribute access. It has the following features:

  1. Preserves ordering of members as initialized
  2. Provides attribute and dictionary-style lookups
  3. Read/write YAML formatted data

Initialize an ordered dictionary. The signature is the same as regular dictionaries, but keyword arguments are not recommended because their insertion order is arbitrary.

yaml_decoder

alias of StructYAMLLoader

yaml_encoder

alias of StructYAMLDumper

classmethod from_yaml(stream)[source]

Initialize mapping from a YAML string.

Parameters:stream – A string or valid file handle
Returns:YAML data as a python object
Return type:Struct
classmethod load_yaml(filename)[source]

Load a YAML file

Parameters:filename (str) – Absolute path to YAML file
Returns:YAML data as python object
Return type:Struct
merge(*args)[source]

Recursively update dictionary

Merge entries from maps provided such that new entries are added and existing entries are updated.

to_yaml(stream=None, default_flow_style=False, **kwargs)[source]

Convert mapping to YAML format.

Parameters:
  • stream (file) – A file handle where YAML is output
  • default_flow_style (bool) –
    • False - pretty printing
    • True - No pretty printing
class caelus.utils.struct.StructMeta[source]

Bases: abc.ABCMeta

YAML interface registration

Simplify the registration of custom yaml loader/dumper classes for Struct class hierarchy.

caelus.utils.struct.gen_yaml_decoder(cls)[source]

Generate a custom YAML decoder with non-default mapping class

Parameters:cls – Class used for mapping
caelus.utils.struct.gen_yaml_encoder(cls)[source]

Generate a custom YAML encoder with non-default mapping class

Parameters:cls – Class used for mapping
caelus.utils.struct.merge(a, b, *args)[source]

Recursively merge mappings and return consolidated dict.

Accepts a variable number of dictionary mappings and returns a new dictionary that contains the merged entries from all dictionaries. Note that the update occurs left to right, i.e., entries from later dictionaries overwrite entries from preceeding ones.

Returns:The consolidated map
Return type:dict

Miscellaneous utilities

This module implements functions that are utilized throughout CPL. They mostly provide a higher-level interface to various os.path functions to make it easier to perform some tasks.

set_work_dir A with-block to execute code in a given directory.
ensure_directory Check if directory exists, if not, create it.
abspath Return the absolute path of the directory.
ostype String indicating the operating system type
timestamp Return a formatted timestamp for embedding in files
caelus.utils.osutils.abspath(pname)[source]

Return the absolute path of the directory.

This function expands the user home directory as well as any shell variables found in the path provided and returns an absolute path.

Parameters:pname (path) – Pathname to be expanded
Returns:Absolute path after all substitutions
Return type:path
caelus.utils.osutils.backup_file(fname, time_format=None, time_zone=<UTC>)[source]

Given a filename return a timestamp based backup filename

Parameters:
  • time_format – A time formatter suitable for strftime
  • time_zone – Time zone used to generate timestamp (Default: UTC)
Returns:

A timestamped filename suitable for creating backups

Return type:

str

caelus.utils.osutils.clean_directory(dirname, preserve_patterns=None)[source]

Utility function to remove files and directories from a given directory.

User can specify a list of filename patterns to preserve with the preserve_patterns argument. These patterns can contain shell wildcards to glob multiple files.

Parameters:
  • dirname (path) – Absolute path to the directory whose entries are purged.
  • preserve_patterns (list) – A list of shell wildcard patterns
caelus.utils.osutils.copy_tree(srcdir, destdir, symlinks=False, ignore_func=None)[source]

Enchanced version of shutil.copytree

  • removes the output directory if it already exists.
Parameters:
  • srcdir (path) – path to source directory to be copied.
  • destdir (path) – path (or new name) of destination directory.
  • symlinks (bool) – as in shutil.copytree
  • ignore_func (func) – as in shutil.copytree
caelus.utils.osutils.ensure_directory(dname)[source]

Check if directory exists, if not, create it.

Parameters:dname (path) – Directory name to check for
Returns:Absolute path to the directory
Return type:Path
caelus.utils.osutils.ostype()[source]

String indicating the operating system type

Returns:One of [“linux”, “darwin”, “windows”]
Return type:str
caelus.utils.osutils.remove_files_dirs(paths, basedir=None)[source]

Remove files and/or directories

Parameters:
  • paths (list) – A list of file paths to delete (no patterns allowed)
  • basedir (path) – Base directory to search
caelus.utils.osutils.set_work_dir(*args, **kwds)[source]

A with-block to execute code in a given directory.

Parameters:
  • dname (path) – Path to the working directory.
  • create (bool) – If true, directory is created prior to execution
Returns:

Absolute path to the execution directory

Return type:

path

Example

>>> with osutils.set_work_dir("results_dir", create=True) as wdir:
...     with open(os.path.join(wdir, "results.dat"), 'w') as fh:
...         fh.write("Data")
caelus.utils.osutils.timestamp(time_format=None, time_zone=<UTC>)[source]

Return a formatted timestamp for embedding in files

Parameters:
  • time_format – A time formatter suitable for strftime
  • time_zone – Time zone used to generate timestamp (Default: UTC)
Returns:

A formatted time string

Return type:

str

caelus.utils.osutils.user_home_dir()[source]

Return the absolute path of the user’s home directory

caelus.utils.osutils.username()[source]

Return the username of the current user