Defining a new suite

Designing a new workflow encompases two closely tied steps:

Define the suite, which will eventually be composed of tasks, potentially organized in families. The outcome of this step is the suite definition file.

Create the task scripts, that will eventually become job file to be executed.

This section describes how to perform first of these steps.

There are several approaches to defining the suite definition - the two most common are:

Using a plain text file, with a specific syntax.

Using the ecFlow Python API.

This tutorial will give examples for both the plain text and Python methods.

A test-based suite definition is just a plain file, that uses a specific syntax to define the components of a suite (or sets of suites).

To define a simple suite, use a text editor to create a file named test.def, with the following contents:

Listing 1 $HOME/course/test.def

# A simple example suite

suite test
  edit ECF_HOME "{{HOME}}/course"
  task t1
endsuite

Warning

Please replace {{HOME}} so that path {{HOME}}/course is correct and exists.

This file contains the suite definition of a suite named test. The suite test contains a single task called t1. A line by line explanation of the file above is as follows:

Line 1 is a comment line. All characters after the symbol # ignored.
Line 2 is empty.
Line 3 defines a suite named test.
Line 4 defines a variable called ECF_HOME, which defines the directory where all the files that will be used by the suite test will reside.
Line 5 defines a task named t1
Line 6 indicates the end of the suite definition.

Important

This tutorial will consider the {{HOME}}/course path throughout, and all other file paths will be relative to this base path.

Using the ecFlow Python API allows to automate the creation of the suite definition file, by leveraing the expressiveness of the Python language to create complex suites.

Important

Even though the plain text method is the most straightforward, it is also limited. The use of the Python API is more powerful and flexible, and is the recommended approach for defining complex suites.

To define a simple suite, use a text editor to create a Python script named test.py, with the following contents:

Listing 2 $HOME/course/test.py

#!/usr/bin/env python3

import pathlib
from ecflow import Defs, Suite, Task, Edit

if __name__ == '__main__':

    base = pathlib.Path.home() / "course"

    print("[1] Creating suite definition")

    defs = Defs(Suite("test", Edit(ECF_HOME=str(base)), Task("t1")))
    print(defs)

    print("[2] Saving definition to file 'test.def'")
    defs.save_as_defs(str(base / "test.def"))

And then run the Python script:

cd $HOME/course

# Either run by explicitly invoking python
python3 ./test.py

# Or make the script executable, and run it directly
chmod +x test.py
./test.py

Warning

Ensure ecFlow Python API module is available by including it in the Python module search path. This can be done by setting the PYTHONPATH environment variable, e.g.:

export PYTHONPATH=$ECFLOW_ROOT_DIR/lib/pyext3:$PYTHONPATH

Once finished executing, the script will print the suite test definition on the console and also generate the $HOME/course/test.def.

What to do:

Manually create the text-based suite definition file, placing the file at $HOME/course/test.def.
(Optional) Use the Python-based approach to create suite definition file.