Adding a family

Tasks can be logically grouped into families. This section will describe how to group the previously designed tasks t1 and t2 in a new family.

You can consider that a :term`suite` is an hierarchical structure, very similar to a _Unix_ file system, where families are the directories and the tasks are regular files. In this metaphor, the suite is a special family with extra attributes (see Dates and Clocks). Families can themselves contain other families and, like directories/files, there can be many tasks with the same name as long as they are in different families.

Note

In ecFlow nomenclature, suites, families and tasks are called nodes.

The default ecFlow behaviour, regarding finding task specific files (e.g. ecf script file), is to expect the file location to be a reflection of the structure of the suite. For example, if a task is located at /test/f1/t1 (i.e. task t1 is inside family f1, inside suite test), ecFlow will expect the related ecf script file to be found at {BASE}/test/f1/t1.ecf – where {BASE}, by default, is the ecflow_server current working directory. The job file and job output will also be created in this directory.

The following steps are required to create a new family f1:

Update the task scripts
Create a directory structure to reflect the new family
Update the suite definition

Update Task Scripts

Since the task scripts need to be moved to another directory, to match the location in the family, the paths in the %include directories need to be adjusted.

An alternative to adjusting the relative paths to the include files, is to make use of the special ecFlow variable named ECF_INCLUDE. This variable points to a directory used during pre-processing to locate include files.

When angle brackets are used (i.e. %include <>), ecFlow checks if the ECF_INCLUDE variable is specified and if a matching include file exists in it to be used. If an include file is not found using the ECF_INCLUDE variable, it falls back to using the ECF_HOME variable. This has the added advantage that specific includes files can be placed under ECF_INCLUDE, and include files common to many tasks can placed in ECF_HOME – for more details see directives.

This means that task scripts using %include <> can remain unchanged, as long as the ECF_INCLUDE variable is set to point to the directory where the include files can be located.

Update the task scripts to be as follows (notide the use of <>):

%include <head.h>
echo "I am part of a suite that lives in %ECF_HOME%"
%include <tail.h>

Create Directory Structure

As mentioned before, having a family f1, implies creating a directory {HOME}/course/test/f1, and moving existing t1.ecf and t2.ecf files into it.

Update Suite Definition

The suite definition needs to be updated to create a family f1 with two tasks t1 and t2. The ECF_INCLUDE variable also needs to be set to point to the directory where the include files can be found.

Update the suite definition to be as follows:

Listing 18 $HOME/course/test.def

# Definition of the suite test.
suite test
   edit ECF_INCLUDE "{{HOME}}/course"
   edit ECF_HOME    "{{HOME}}/course"
   family f1
      task t1
      task t2
   endfamily
endsuite

The following script generates updated suite definition using the Python API:

Listing 19 $HOME/course/test.py

#!/usr/bin/env python3

import pathlib
from ecflow import Defs, Suite, Family, Task, Edit

def create_family_f1():
    return Family("f1", Task("t1"), Task("t2"))

if __name__ == '__main__':

    base = pathlib.Path.home() / "course"

    print("[1] Creating suite definition")

    defs = Defs(
        Suite("test",
              Edit(ECF_INCLUDE=str(base), ECF_HOME=str(base)),
              create_family_f1()))
    print(defs)

    print("[2] Checking job creation: .ecf -> .job0")
    print(defs.check_job_creation())

    print("[3] Saving definition to file 'test.def'")
    defs.save_as_defs(str(base / "test.def"))

Once loaded into the ecflow_server, the updated suite definition hierarchy is shown as a tree in ecflow_ui.

What to do:

Edit the task scripts to use %include<>
Create the directory struture reflecting the family f1, and move the task scripts into it.
Update the suite definition file, to include family f1 with two tasks t1 and t2.
Replace the suite in the ecflow_server with the new definition.
Inspect and run the suite in ecflow_ui, notice the tree structure. You may have to expand test and f1 nodes to observe the tasks.