Checking job creation

In the previous section we have implemented our first task (the t1.ecf file).

The t1.ecf script needs to be preprocessed to generate the job file. This pre-processing is done automatically by ecflow_server the when the task is about to run. However it is possible to check the job creation before the suite definition is loaded into the ecflow_server.

Text

Automated job creation checking is only available with Python. If the ecflow_server can’t locate the ecf script, please see ecf file location algorithm.

Python

The process of job creation can be checked before the suite definition is loaded into the ecflow_server. The following checks are done:

Locating ecf script files, corresponding to the task in the suite definition.
Performing pre-processing

When the suite definition is large and has many ecf script this checking can save a lot of time.

The following points should be noted about about job creation checking:

It is independent of the ecflow_server. Hence ECF_PORT and ECF_HOST in the job file will have default values.
Job files have a .job0 extension, whereas the server will always generate jobs with a extension .job<1-n>, i.e t1.job1, t1.job2. The numbers correspond to ECF_TRYNO which is never zero.
By default the job file is created in the same directory as the ecf script. See ECF_JOB

Checking is done using ecflow.Defs.check_job_creation

Listing 6 $HOME/course/test.py

import os
from ecflow import Defs, Suite, Task, Edit

print("Creating suite definition")
home = os.path.join(os.getenv("HOME"), "course")
defs = Defs(Suite("test", Edit(ECF_HOME=home), Task("t1")))
print(defs)

print("Checking job creation: .ecf -> .job0")
print(defs.check_job_creation())

# We can assert, so that we only progress once job creation works
# assert len(defs.check_job_creation()) == 0, "Job generation failed"

Note

It is highly advisable that job creation checking is enabled for all subsequent examples.

What to do:

Add job creation checking to $HOME/course/test.py
python3 test.py | ./test.py
Examine the job file $HOME/course/test/t1.job0. In particular note the substitutions made by the ecFlow server such as ECF_PORT, ECF_HOST, etc