Command internals
Warning
This is an internal developer reference, intended for developers and integrators working on
the ecFlow client/server protocol itself, not for end users driving ecflow_client.
It documents the native client/server serialization (the on-the-wire request and reply
payloads) alongside the C++ command classes that implement each command. Unlike the
command line interface reference, this page is not auto-generated from
ecflow_client; it is maintained by hand and may lag behind the current implementation. Treat
the payloads and class references as illustrative of a particular server version rather than a
stable, guaranteed contract.
This page is a reference for the commands understood by the ecFlow server. It provides a per-command reference, so that the description of each command, its options, and its wire representation stay consistent with the implementation.
Introduction
ecFlow follows a client/server model. The server (ecflow_server) holds the suite definition and
the state of every node; the client (ecflow_client) issues commands that either query the
server or change its state. Every interaction is a single round trip:
the client sends a request — a client-to-server command, an instance of a class derived from
ClientToServerCmd(.../libs/base/src/ecflow/base/cts/ClientToServerCmd.hpp);the server processes it and returns a reply — a server-to-client command, an instance of a class derived from
ServerToClientCmd(.../libs/base/src/ecflow/base/stc/ServerToClientCmd.hpp).
Command-line model
The ecflow_client executable exposes each command as a long option. A single invocation carries
exactly one command, optionally followed by that command’s own options and arguments, for example:
ecflow_client --init=$$
ecflow_client --alter change variable FOO bar /suite/family
The registry that maps each option to its command is
.../libs/base/src/ecflow/base/cts/CtsCmdRegistry.cpp; the registration order there also fixes the
order in which commands appear in ecflow_client --help. A few global options (--help,
--version, --debug) apply to every command. A single command class frequently implements
several options: for example CtsCmd covers --ping, --restart, --halt, --shutdown
and --terminate, and PathsCmd covers --suspend, --resume, --kill and others.
Context supplied through the command line versus the environment
Command context reaches the client in two ways:
Command-line options and arguments, parsed with Boost.Program_options. User commands take all of their context this way (paths, attribute names, values).
Environment variables, read by
ClientEnvironment(.../libs/client/src/ecflow/client/ClientEnvironment.cpp). Task commands rely almost entirely on these, because a job script does not know its own path or authentication token; the server writes them into the job at submission time.
Two groups of variables exist, matching the footer that ecflow_client --help prints for every
command. The connection variables are used by every command, whether a user command or a task
command:
Variable |
Meaning |
|---|---|
|
The main server hostname; default value is |
|
The main server port; default value is |
|
Enable secure (TLS) communication between the client and the server. |
|
File that lists alternate hosts to try, if the connection to the main host fails. |
|
The policy ( |
ECF_HOST and ECF_PORT (or the equivalent --host / --port options) must be set for the
client to reach the server; the others are optional.
The task-command variables are used additionally by every task command; the server writes them into the job at submission time:
Variable |
Meaning |
|---|---|
|
Full path name of the task, e.g. |
|
The job password, defined by the server and used to authenticate task commands. |
|
The remote (process) identifier; supports identifying zombies and killing running jobs. |
|
The run number of the job, used in job and output file-name generation. |
|
Maximum time, in seconds, for the client to deliver a message to the server; default is 24 hours. |
|
Lets the task exit with an error on connection failure, instead of waiting for |
|
If set, |
A custom user name may also be supplied through ECF_USER (or --user), in which case the
matching password is read from the custom password file (ECF_CUSTOM_PASSWD); this is what
populates the optional pswd_ and cu_ fields noted in Reading the payloads. The complete set
of constants is declared in .../libs/core/src/ecflow/core/Environment.hpp. The per-command tables
in the reference below highlight the variables each command reads directly; the connection variables
above apply to every command throughout.
Task commands versus user commands
Commands fall into two families, mirrored by the source layout under
.../libs/base/src/ecflow/base/cts/:
Task commands (
cts/task/) are issued from inside a.ecfjob script, through the wrapper functions emitted by the job header. They report the progress of a running job and are authenticated withECF_NAME+ECF_PASS+ECF_RID+ECF_TRYNO, all derived from the environment. All of them share the base classTaskCmd(cts/task/TaskCmd.hpp). The task commands areInitCmd,CompleteCmd,AbortCmd,EventCmd,MeterCmd,LabelCmd,QueueCmdandCtsWaitCmd.User commands (
cts/user/) are issued by an operator or administrator to inspect or steer the server and its suites (begin, alter, force, delete, suspend, and so on). They share the base classUserCmd(cts/user/UserCmd.hpp) and are authenticated by user name.
Note
The request and reply payloads shown in this note are the native serialization used on the
ecFlow client/server connection: the command object encoded as a cereal JSON archive
(.../libs/core/src/ecflow/core/Serialization.hpp). This is distinct from the JSON of the
optional REST API (ecflow_http), which is a separate, hand-written HTTP layer documented in
the REST API reference.
Reading the payloads
On the wire a request is not the bare command but a small envelope. The client wraps the command in
a ClientToServerRequest (member cmd_) and the server wraps its reply in a
ServerToClientResponse (member stc_cmd_). cereal serialises the outermost object under a key
derived from its mangled C++ type name (for example "21ClientToServerRequest"), and it renders a
polymorphic pointer as a polymorphic_name (the concrete command class) plus a ptr_wrapper
holding the object’s data. Inside data, base-class members appear first as nested value0
objects:
for task commands,
cl_host_comes fromClientToServerCmdandpath_to_submittable_,jobs_password_,process_or_remote_id_andtry_no_come fromTaskCmd;for user commands,
cl_host_comes fromClientToServerCmdanduser_comes fromUserCmd(set when the request is dispatched).UserCmdalso carries two optional fields, serialized only when set:pswd_, the user’s password, present when a password file supplies one (ECF_PASSWD, orECF_CUSTOM_PASSWDfor a custom user); andcu_, a flag that istruewhen a custom user name is in use (through--user,ECF_USERor the client API). The user-command examples in this note come from a server without password authentication configured, so they showuser_only; when a password is in play, apswd_field (and, for a custom user,cu_) appears alongside it.
The cereal_class_version and polymorphic_id fields are cereal bookkeeping. The payloads below
are real serializer output, with host, password and user values replaced by placeholders.
Reference
For each command, the table summarises its type, class, action and inputs; the tabs that follow hold the command-line Usage, the serialized Request payload, and the serialized Reply payload. A command whose single class implements several actions adds a table of those actions before the tabs. A few commands carry an unbounded payload (an embedded definition or node subtree); these are described in prose instead of the tabbed layout.
Task commands
Command |
Description |
|---|---|
Mark the task as aborted, with an optional reason. |
|
Mark the task as complete, optionally removing variables. |
|
Block the job until a trigger expression becomes true. |
|
Set or clear an event on the task. |
|
Mark the task as started (active), optionally adding variables. |
|
Update the value of a label on the task. |
|
Update the value of a meter on the task. |
|
Request or update a step of a queue. |
AbortCmd
Type |
Task command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Marks the task as aborted. It is for use inside a .ecf script only, and is normally raised by
the trap installed in the job header. When no reason is supplied the server records
Trap raised in job file. The reason is sanitised on construction (newlines are stripped and
semicolons replaced by spaces) so that it cannot corrupt the output of --migrate or --load.
The zombie handling is the same as for InitCmd.
ecflow_client --abort=<some-reason>
# a bare --abort is valid; the server substitutes a default reason
ecflow_client --abort
Verbatim ecflow_client --abort output (the common environment-variable footer is
omitted):
abort
-----
Mark task as aborted. For use in the '.ecf' script file *only*
Hence the context is supplied via environment variables
Argument(s):
reason: (optional) string
# Optionally provide a reason why the abort was raised
If this task command is a zombie, then the default action will be to *block*.
The default can be overridden by using zombie attributes.
Otherwise the blocking period is defined by ECF_TIMEOUT.
Usage:
--abort=<some-reason>
AbortCmd produced by --abort="disk full" (the reason appears as reason_):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "AbortCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"reason_": "disk full"
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
CompleteCmd
Type |
Task command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Marks the task as complete. It is for use inside a .ecf script only. Where --init can add
variables to a task, --complete can remove them, which is convenient for clearing per-run
variables as the job finishes. The zombie handling is the same as for InitCmd.
ecflow_client --complete
# delete variables name1 and name2 from the task as it completes
ecflow_client --complete --remove name1 name2
Verbatim ecflow_client --complete output (the common environment-variable footer is
omitted):
complete
--------
Mark task as complete. For use in the '.ecf' script file *only*
Hence the context is supplied via environment variables
If this task command is a zombie, then the default action will be to *block*.
The default can be overridden by using zombie attributes.
Otherwise the blocking period is defined by ECF_TIMEOUT.
The init command allows variables to be added, and complete command
allows for them to be removed.
Argument(s):
--remove: (optional)
# A list of space separated variables to removed from this task
Usage:
--complete
--complete --remove name1 name2 # delete variables name1 and name2 on the task
CompleteCmd with no removed variables:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CompleteCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
}
}
}
}
}
}
CompleteCmd produced by --complete --remove A B (the names appear as var_to_del_):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CompleteCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"var_to_del_": [ "A", "B" ]
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
CtsWaitCmd
Type |
Task command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Evaluates a trigger expression and blocks while it is false, returning to the job only once it
becomes true. It is for use inside a .ecf script only, and is the mechanism behind script-level
synchronisation on the state of other nodes.
ecflow_client --wait="/suite/taskx == complete"
Verbatim ecflow_client --wait output (the common environment-variable footer is
omitted):
wait
----
Evaluates an expression, and block while the expression is false.
For use in the '.ecf' file *only*, hence the context is supplied via environment variables
Argument(s):
expression: string
# The expression to be evaluated.
Usage:
--wait="/suite/taskx == complete"
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CtsWaitCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"expression_": "/suite/taskx == complete"
}
}
}
}
}
When the expression evaluates to true the server returns the generic StcCmd reply with
api_ equal to 0:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
While the expression is false the server returns a blocking reply (StcCmd with api_
equal to 2, BLOCK_CLIENT_ON_HOME_SERVER), on which the client waits and retries:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 2
}
}
}
}
}
EventCmd
Type |
Task command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Sets or clears an event on the task. It is for use inside a .ecf script only. Unlike --init,
--complete and --abort, the default zombie action is to fob (allow the client command to
complete without error); this can be overridden with zombie attributes.
ecflow_client --event=ev # set the event (the default)
ecflow_client --event=ev set # set the event, explicit
ecflow_client --event=ev clear # clear the event
Verbatim ecflow_client --event output (the common environment-variable footer is
omitted):
event
-----
Change event. For use in the '.ecf' script file *only*
Hence the context is supplied via environment variables
Argument(s):
event-name: string | int
# The name of the event to be set or cleared.
state: (optional) string
# [ set | clear], default value is set
If this task command is a zombie, then the default action will be to *fob*,
i.e allow the ecflow client command to complete without an error
The default can be overridden by using zombie attributes.
Usage:
--event=ev # set the event, default since event initial value is clear
--event=ev set # set the event, explicit
--event=ev clear # clear the event, use when event initial value is set
--event=ev sets the event; value_ is stored only when it differs from the default,
so the set request omits it:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "EventCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"name_": "ev"
}
}
}
}
}
--event=ev clear adds value_ set to false:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "EventCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"name_": "ev",
"value_": false
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
InitCmd
Type |
Task command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Marks the task as started (state active). It is for use inside a .ecf script only. When the
task command is detected as a zombie the default action is to block; this can be overridden with
zombie attributes, and the blocking period is bounded by ECF_TIMEOUT.
ecflow_client --init=$$
# add or update variables on the task at the same time
ecflow_client --init=$$ --add name=value name2=value2
Verbatim ecflow_client --init output (the common environment-variable footer is
omitted):
init
----
Mark task as started(active). For use in the '.ecf' script file *only*
Hence the context is supplied via environment variables.
Argument(s):
process_or_remote_id: string
# The process id of the job or remote_id.
# Using remote id allows the jobs to be killed.
--add: (optional)
# add/update variables as name value pairs
# a list of space separated variables to add/update on this task
If this task command is a zombie, then the default action will be to *block*.
The default can be overridden by using zombie attributes.
Otherwise the blocking period is defined by ECF_TIMEOUT.
Usage:
--init=$$
--init=$$ --add name=value name2=value2 # add/update variables to task
InitCmd with no added variables:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "InitCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
}
}
}
}
}
}
InitCmd produced by --init=$$ --add A=1 B=2 (the added pairs appear as
var_to_add_):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "InitCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"var_to_add_": [
{ "n_": "A", "v_": "1" },
{ "n_": "B", "v_": "2" }
]
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to OK, i.e. 0), returned
by every task command on success:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
LabelCmd
Type |
Task command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Updates the value of a label on the task. It is for use inside a .ecf script only. The default
zombie action is to fob.
ecflow_client --label=progressed merlin
Verbatim ecflow_client --label output (the common environment-variable footer is
omitted):
label
-----
Change Label. For use in the '.ecf' script file *only*
Hence the context is supplied via environment variables
Argument(s):
label-name
# The name of the label to be set.
value
# The new label value. Can be single or multi-line (space separated quoted strings).
If this task command is a zombie, then the default action will be to *fob*,
i.e allow the ecflow client command to complete without an error
The default can be overridden by using zombie attributes.
Usage:
--label=progressed merlin
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "LabelCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"name_": "progressed",
"label_": "merlin"
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
MeterCmd
Type |
Task command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Updates the value of a meter on the task. It is for use inside a .ecf script only. The default
zombie action is to fob.
ecflow_client --meter=my_meter 20
Verbatim ecflow_client --meter output (the common environment-variable footer is
omitted):
meter
-----
Change meter. For use in the '.ecf' script file *only*
Hence the context is supplied via environment variables
Argument(s):
meter-name: string
# The name of the meter to be set.
value: int
# The new meter value
If this task command is a zombie, then the default action will be to *fob*,
i.e allow the ecflow client command to complete without an error
The default can be overridden by using zombie attributes.
Usage:
--meter=my_meter 20
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "MeterCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"name_": "my_meter",
"value_": 20
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
QueueCmd
Type |
Task command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Cooperates with a queue attribute to hand out steps to a running job. active returns the
first queued or aborted step and advances the index; no_of_aborted returns the count of aborted
steps; reset rewinds the index so steps can be reprocessed; complete and aborted mark the
given step. It is for use inside a .ecf script only, and the default zombie action is to block.
The active and no_of_aborted actions return a string; the others return the generic reply.
QNAME=my_queue
# obtain and activate the next step; returns e.g. "003", or "<NULL>" when exhausted
step=$(ecflow_client --queue=$QNAME active)
# after processing, mark the step complete (or aborted on failure)
ecflow_client --queue=$QNAME complete $step
Verbatim ecflow_client --queue output (the common environment-variable footer is
omitted):
queue
-----
QueueCmd. For use in the '.ecf' script file *only*
Hence the context is supplied via environment variables
Argument(s):
queue-name: string
# The name of the queue to be used.
action: string
# [active | aborted | complete | no_of_aborted | reset ]
# active: returns the first queued/aborted step, the return string is the queue value from the definition
# no_of_aborted: returns number of aborted steps as a string, i.e 10
# reset: sets the index to the first queued/aborted step. Allows steps to be reprocessed for errors
step: string
# The value returned from `step=$(ecflow_client --queue=queue_name active)`
# This is only valid for complete and aborted steps
path: (optional) string
# The path where the queue is defined.
# By default we search for the queue up the node tree.
If this task command is a zombie, then the default action will be to *block*,
The default can be overridden by using zombie attributes.
If the path to the queue is not defined, then this command will
search for the queue up the node hierarchy. If no queue found, command fails
Usage:
step=""
QNAME="my_queue_name"
while [1 == 1 ] ; do
# this return the first queued/aborted step, then increments to next step, return <NULL> when all steps processed
step=$(ecflow_client --queue=$QNAME active) # of the form string i.e "003". this step is now active
if [[ $step == "<NULL>" ]] ; then
break;
fi
...
ecflow_client --queue=$QNAME complete $step # tell ecflow this step completed
done
trap() { ecflow_client --queue=$QNAME aborted $step # tell ecflow this step failed }
--queue=my_queue active (step_ is empty for this action):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "QueueCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"name_": "my_queue",
"action_": "active",
"step_": "",
"path_to_node_with_queue_": ""
}
}
}
}
}
--queue=my_queue complete 003 (step_ carries the step to mark):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "QueueCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"path_to_submittable_": "/suite/family/task",
"jobs_password_": "jZ2Vld",
"process_or_remote_id_": "12345",
"try_no_": 1
},
"name_": "my_queue",
"action_": "complete",
"step_": "003",
"path_to_node_with_queue_": ""
}
}
}
}
}
complete, aborted and reset return the generic StcCmd reply with api_
equal to 0:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
active and no_of_aborted return an SStringCmd whose str_ field holds the
result (for active, the step value; for no_of_aborted, the count):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "SStringCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"str_": "003"
}
}
}
}
}
User commands
Several user commands implement a whole family of operations in a single class, dispatched by an
enumerated discriminator carried in the payload — for example api_ for CtsCmd, CtsNodeCmd,
PathsCmd, LogCmd, ClientHandleCmd and CSyncCmd, the add_attr_type_ /
change_attr_type_ / del_attr_type_ fields for AlterCmd, and user_action_ for
ZombieCmd. For these, the individual actions differ only in that field and in the accompanying
paths, names and values, so each is documented with a table of its actions followed by one or two
representative payloads rather than a payload per action. AlterCmd and CtsCmd are the two
largest. The commands below are listed alphabetically.
Command |
Description |
|---|---|
Add, change, delete or sort an attribute on nodes. |
|
Begin one or all suites. |
|
Retrieve a node’s script, job, output, manual or kill/status file. |
|
Checkpoint the server, optionally changing the checkpoint mode. |
|
Manage client handles (per-client suite subsets) used for synchronisation. |
|
Synchronise a client with the server’s state. |
|
Argument-less server signals (ping, restart, halt, stats, and so on). |
|
Query a node or the definition (get, why, get_state, migrate, and so on). |
|
Delete nodes, or every suite. |
|
Retrieve or submit a task’s script, with optional preprocessing. |
|
Force a node into a state, or force an event. |
|
Free a held dependency so a node may run. |
|
Bundle several commands into a single request. |
|
Load a suite definition into the server. |
|
Inspect or manage the server log file. |
|
Write a message into the server log. |
|
Transfer a node subtree between locations or servers (internal). |
|
Change a node’s position among its siblings. |
|
Node operations (suspend, resume, kill, status, check, archive, and so on). |
|
Move a node subtree to a new parent, possibly on another server. |
|
Read a single piece of node state without changing anything. |
|
Replace or add a node using another definition. |
|
Requeue nodes so they may run again. |
|
Run a task immediately, bypassing its time dependencies. |
|
Return the server version string. |
|
Select the client print style (client-side). |
|
Resolve zombies (fob, fail, adopt, remove, block, kill). |
AlterCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Adds, changes, deletes or sorts an attribute on one or more nodes. It is the primary command for
editing a running suite. The general form is
--alter <operation> <attribute-type> [name] [value] <path> [<path> ...]. The attribute types
accepted by each operation are:
Operation |
Attribute types |
|---|---|
|
|
|
|
|
|
|
|
|
a node flag, carried by |
Every AlterCmd payload carries the same fields; the operation is encoded by the discriminator
integers, and any discriminator not in use holds its “not set” sentinel value (add_attr_type_
6, del_attr_type_ 16, change_attr_type_ 13, flag_type_ 19). For add,
change and delete exactly one of add_attr_type_ / change_attr_type_ /
del_attr_type_ selects the attribute type. For set_flag and clear_flag the flag is
carried by flag_type_, and flag_ selects set (true) versus clear (false). For
sort all attribute discriminators stay at their sentinel and name_ holds the attribute type
(with value_ optionally recursive).
ecflow_client --alter change variable FOO bar /suite/family
ecflow_client --alter add label info "hello world" /suite/family/task
# delete every variable on the node (no name given)
ecflow_client --alter delete variable /suite
# sort the variables of the node and its children
ecflow_client --alter sort variable recursive /suite/family
# set and clear a node flag
ecflow_client --alter set_flag late /suite/family
ecflow_client --alter clear_flag late /suite/family
Verbatim ecflow_client --alter output (the common environment-variable footer is
omitted):
alter
-----
Alter the node according to the options.
arg1 = [ delete | change | add | set_flag | clear_flag | sort ]
arg2 = For delete:
[ variable | time | today | date | day | cron | event | meter | late | generic |
queue | label | trigger | complete | repeat | limit | inlimit | limit_path |
zombie | aviso | mirror ]
For change:
[ variable | clock_type | clock_gain | clock_date | clock_sync | event | meter |
label | trigger | complete | repeat | limit_max | limit_value | defstatus |
late | time | today | aviso | mirror ]
For add:
[ variable | time | today | date | day | zombie | event | meter | late | limit |
inlimit | label | aviso | mirror ]
For set_flag or clear_flag:
[ force_aborted | user_edit | task_aborted | edit_failed | ecfcmd_failed |
statuscmd_failed | killcmd_failed | no_script | killed | status | late |
message complete | queue_limit | task_waiting | locked | zombie | archived |
restored | threshold | log_error | checkpt_error ]
For sort:
[ event | meter | label | variable| limit | all ]
arg3 = [ <name> | <value> ]
arg4 = <new-value>
arg5 = <path> (<path> (...)) - at least one node path required.
*Important Notes*
* All paths must start with a leading '/' character.
* To update, create or remove server variables use '/' for path.
* When updating unnamed attributes (Repeat, Trigger, Complete, ...) the name/arg3 is not necessary.
* After changing the clock the suite needs to be re-queued for the change to take effect.
* When adding or updating node attributes (e.g. variable, meter, event, label, limits, late)
the name (arg3) and value (arg4) must be quoted.
* When sorting attributes, 'recursive' can be used as the value (arg3).
* When adding a meter, the value (arg4) is expected to be a comma-separated triplet
of numerical values the form "<min>,<max>,<value>" (n.b. no spaces are allowed).
* When adding an event, the non-optional value (arg4) must be either "set" or "clear".
* When adding or updating aviso and mirror attributes, the value (arg4) is expected to be
a quoted list of configuration options. For example:
* for aviso, "--remote_path /s1/f1/t2 --remote_host host --polling 20 --remote_port 3141 --ssl)"
* for mirror, "--listener '{ \"event\": \"mars\", \"request\": { \"class\": "od" } }'
--url http://aviso/ --schema /path/to/schema --polling 60"
* For both aviso and mirror, the special value "reload" forces reloading the configuration.
This is typically useful after updating variables used to configure these kind of attributes.
Usage:
--alter=add variable <variable-name> "value" / # add server variable
--alter=add variable <variable-name> "value" /path/to/node # add node variable
--alter=add time "+00:20" /path/to/node
--alter=add date "01.*.*" /path/to/node
--alter=add day "sunday" /path/to/node
--alter=add label <label-name> "label_value" /path/to/node
--alter=add event <event-name> "set"|"clear" /path/to/node
--alter=add meter <meter-name> "<min>,<max>,value" /path/to/node
--alter=add late "-s 00:01 -a 14:30 -c +00:01" /path/to/node
--alter=add limit mars "100" /path/to/node
--alter=add inlimit /path/to/node/withlimit:limit_name "10" /path/to/node
--alter=add inlimit /path/to/node/withlimit:limit_name "-s 10" /path/to/node
--alter=add inlimit /path/to/node/withlimit:limit_name "-n 10" /path/to/node
# zombie attributes have the following structure:
`zombie_type`:(`client_side_action` | `server_side_action`):`child`:`zombie_life_time`
zombie_type = "user" | "ecf" | "path" | "ecf_pid" | "ecf_passwd" | "ecf_pid_passwd"
client_side_action = "fob" | "fail" | "block"
server_side_action = "adopt" | "delete" | "kill"
child = "init" | "event" | "meter" | "label" | "wait" | "abort" | "complete" | "queue"
zombie_life_time = unsigned integer default: user(300), ecf(3600), path(900) minimum is 60
--alter=add zombie "ecf:fail::" /path/to/node # ask system zombies to fail
--alter=add zombie "user:fail::" /path/to/node # ask user generated zombies to fail
--alter=add zombie "path:fail::" /path/to/node # ask path zombies to fail
--alter=delete variable FRED /path/to/node # delete variable FRED
--alter=delete variable /path/to/node # delete *ALL* variables on the given snode
--alter change variable FOO bar /suite/family (change_attr_type_ 0 selects
variable):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "AlterCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite/family" ],
"name_": "FOO",
"value_": "bar",
"add_attr_type_": 6,
"del_attr_type_": 16,
"change_attr_type_": 0,
"flag_type_": 19,
"flag_": false
}
}
}
}
}
--alter add label info "hello world" /suite/family/task (add_attr_type_ 10
selects label):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "AlterCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite/family/task" ],
"name_": "info",
"value_": "hello world",
"add_attr_type_": 10,
"del_attr_type_": 16,
"change_attr_type_": 13,
"flag_type_": 19,
"flag_": false
}
}
}
}
}
--alter delete variable /suite (del_attr_type_ 0 selects variable; an empty
name_ deletes every variable):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "AlterCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite" ],
"name_": "",
"value_": "",
"add_attr_type_": 6,
"del_attr_type_": 0,
"change_attr_type_": 13,
"flag_type_": 19,
"flag_": false
}
}
}
}
}
--alter sort variable recursive /suite/family (all attribute discriminators stay at their
sentinel; name_ holds the attribute type and value_ is recursive):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "AlterCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite/family" ],
"name_": "variable",
"value_": "recursive",
"add_attr_type_": 6,
"del_attr_type_": 16,
"change_attr_type_": 13,
"flag_type_": 19,
"flag_": false
}
}
}
}
}
--alter set_flag late /suite/family (flag_type_ 7 selects the late flag and
flag_ is true). The corresponding clear_flag request is identical except that
flag_ is false:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "AlterCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite/family" ],
"name_": "",
"value_": "",
"add_attr_type_": 6,
"del_attr_type_": 16,
"change_attr_type_": 13,
"flag_type_": 7,
"flag_": true
}
}
}
}
}
On success AlterCmd returns the generic StcCmd reply with api_ equal to 0:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
BeginCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Begins one or all suites, moving them from the definition into a running state so that scheduling
starts. The force flag is stored in force_; the request below is for --begin s1 and the
--begin s1 --force variant differs only in force_ being true.
ecflow_client --begin s1
ecflow_client --begin s1 --force
ecflow_client --begin # begin all suites
Verbatim ecflow_client --begin output (the common environment-variable footer is
omitted):
begin
-----
Begin playing the definition in the server.
Expects zero or a single quoted string.
Argument(s):
suite-name: (optional)
# The name of the selected suite; if not specified, means all suites.
# Including `--force` in the value means reset the begin status and bypass checks.
# Important: using `--force` might cause the appearance of zombies.
Usage:
--begin # will begin all suites
--begin="--force" # reset and then begin all suites, bypassing any checks. Note: string must be quoted
--begin="mySuite" # begin playing suite of name 'mySuite'
--begin="mySuite --force" # reset and begin playing suite 'mySuite', bypass check
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "BeginCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"suiteName_": "s1",
"force_": false
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
CFileCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Retrieves one of the files associated with a node (the script, the generated job, its output, the
manual page, or the output of the kill/status commands). The file type is carried by file_ and
the line limit by max_lines_. The reply is always a string.
ecflow_client --file /suite/family/task jobout 100
ecflow_client --file /suite/family/task script
Verbatim ecflow_client --file output (the common environment-variable footer is
omitted):
file
----
Return the chosen file. Select from [ script<default> | job | jobout | manual | kill | stat ]
By default will return the script.
Argument(s):
path: <path>
# The path to the node
kind: (optional) [ script<default> | job | jobout | manual | kill | stat ]
# `script`: returns the script file, e.g., %ECF_JOB%.ecf
# `job`: returns the pre-processed job script
# `jobout`: returns the job output/log, e.g., %ECF_JOB%.out
# `manual`: returns the job manual
# `kill`: returns the output of ECF_KILL_CMD, e.g., %ECF_JOB%.kill
# `stat`: returns the output of ECF_STATUS_CMD, e.g., %ECF_JOB%.stat
max_lines: (optional)
# default is 10000
--file /suite/family/task jobout 100 (file_ 2 selects jobout):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CFileCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"file_": 2,
"pathToNode_": "/suite/family/task",
"max_lines_": 100
}
}
}
}
}
An SStringCmd whose str_ field holds the file contents:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "SStringCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"str_": "..."
}
}
}
}
}
CheckPtCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Triggers a checkpoint of the server state, and optionally changes the checkpoint mode, interval and save-time alarm. Called with no argument it checkpoints immediately.
ecflow_client --check_pt
ecflow_client --check_pt=always:120:30
Verbatim ecflow_client --check_pt output (the common environment-variable footer is
omitted):
check_pt
--------
Forces the definition file in the server to be written to disk *or* allow mode,
interval and alarm to be changed.
Whenever the check pt file is written to disk, it is measured.
If the time to save to disk is greater than the default of 30 seconds,
then an alarm is raised. This can be seen in the GUI as a late flag on the server.
Once the late flag has been set it will need to manually cleared in the GUI
or by using --alter functionality
Note excessive save times can interfere with job scheduling.
The alarm threshold can be changed. See below.
Argument(s):
mode: (optional) [ never | on_time | on_time:<integer> | always | <integer>]
# never : Never check point the definition in the server
# on_time : Turn on automatic check pointing at interval stored on server
# on_time<integer> : Turn on automatic check point, with the specified interval in seconds
# alarm<integer> : Modify the alarm notification time for check pt saving to disk
# always : Check point at any change in node tree, *NOT* recommended for large definitions
# <integer> : This specifies the interval in seconds when server should automatically check pt.
#
# This only takes effect if mode is on_time/CHECK_ON_TIME
# The default value is 120 seconds, and should be greater than 60 seconds
Usage:
--check_pt
Immediately check point the definition held in the server
--check_pt=never
Switch off check pointing
--check_pt=on_time
Start automatic check pointing at the interval stored in the server
--check_pt=180
Change the check pt interval to 180 seconds
--check_pt=on_time:90
Change mode and interval, to automatic check pointing every 90 seconds
--check_pt=alarm:35
Change the alarm time for check pt saves. i.e if saving the check pt takes longer than 35 seconds
set the late flag on the server.
--check_pt=always:120:30 (mode_ 2 is always):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CheckPtCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"mode_": 2,
"check_pt_interval_": 120,
"check_pt_save_time_alarm_": 30
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
ClientHandleCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
one option per handle operation (see the Actions table below) |
Parameters |
|
Environment variables |
|
Manages client handles: server-side registrations that let a client (typically ecflow_ui)
track a chosen subset of suites and receive only their changes. The operation is carried by api_:
Option |
|
Description |
Reply |
|---|---|---|---|
|
0 |
Register a new handle for a set of suites. |
handle |
|
1 |
Drop a handle. |
|
|
2 |
Drop all handles owned by a user. |
|
|
3 |
Add suites to a handle. |
|
|
4 |
Remove suites from a handle. |
|
|
5 |
Change the auto-add-new-suites flag of a handle. |
|
|
6 |
List registered handles and their suites. |
suites |
ecflow_client --ch_register 1 /s1 /s2
ecflow_client --ch_drop 1
Verbatim ecflow_client --help=<action> output (the common environment-variable
footer shown on every page is omitted). Select an action:
ch_register
-----------
Register interest in a set of suites.
If a definition has lots of suites, but the client. is only interested in a small subset,
Then using this command can reduce network bandwidth and synchronisation will be quicker.
This command will create a client handle, which must be used for any other changes.
The newly created handle can be shown with the --ch_suites command
Deleted suites will stay registered, and must be explicitly removed/dropped.
Note: Suites can be registered before they are loaded into the server
This command affects news() and sync() commands
Argument(s):
auto-add: true | false
# `true` means add any new suites to user list, when they are created
names
# A space separated list of suite names.
# Note: names not in the definition are ignored.
Usage:
--ch_register=true s1 s2 s3 # register interest in suites s1,s2,s3 and any new suites
--ch_register=false s1 s2 s3 # register interest in suites s1,s2,s3 only
--ch_register=false # register handle, suites will be added later on
--ch_register=1 true s1 s2 s3 # drop handle 1 then register interest in suites s1,s2,s3 and any new suites
# The client handle as the first argument is typically used by GUI/python
# When the client handle is no zero, then it is dropped first
To list all suites and handles use --ch_suites
ch_drop
-------
Drop/de-register the client handle.
Un-used handle should be dropped otherwise they will stay, in the server.
Argument(s):
handle: integer
# The handle value (must be an integer > 0)
Usage:
--ch_drop=10 # drop the client handle 10
An error is returned if the handle was not previously been registered.
The handle stored on the local client is set to zero
To list all suites and handles use --ch_suites
ch_drop_user
------------
Drop/de-register all handles associated with the given user.
If no user provided will drop for current user. Client must ensure un-used handle are dropped
otherwise they will stay, in the server.
Argument(s):
user
# The user to be dropped; if empty, means drop current user
Usage:
--ch_drop_user=ma0 # drop all handles associated with user ma0
--ch_drop_user # drop all handles associated with current user
An error is returned if no registered handles
To list all suites and handles use --ch_suites
ch_add
------
Add a set of suites, to an existing handle.
Argument(s):
handle: integer
# The handle value (must be an integer > 0)
names
# A space separated list of suite names.
# Note: names not in the definition are ignored.
Usage:
--ch_add=10 s2 s3 s4 # add suites s2 s3,s4 to handle 10
An error is returned if the handle had not previously been registered
The handle is created with --ch_register command
To list all suites and handles use --ch_suites
ch_rem
------
Remove a set of suites, from an existing handle.
Argument(s):
handle: integer
# The handle value (must be an integer > 0)
names
# A space separated list of suite names.
# Note: names not in the definition are ignored.
Usage:
--ch_rem=10 s2 s3 s4 # remove suites s2 s3,s4 from handle 10
The handle is created with --ch_register command
To list all suites and handles use --ch_suites
ch_auto_add
-----------
Change an existing handle so that new suites can be added automatically.
Argument(s):
handle: integer
# The handle value (must be an integer > 0)
auto-add: true | false
# `true` means add new suites to my list, when they are created
Usage:
--ch_auto_add=10 true # modify handle 10 so that new suites, get added automatically to it
--ch_auto_add=10 false # modify handle 10 so that no new suites are added
The handle is created with --ch_register command
To list all suites and handles use --ch_suites
ch_suites --------- Shows all the client handles, and the suites they reference
--ch_register for suites /s1 and /s2 with auto-add enabled (api_ 0):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "ClientHandleCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 0,
"client_handle_": 0,
"drop_user_": "",
"suites_": [ "/s1", "/s2" ],
"auto_add_new_suites_": true
}
}
}
}
}
--ch_register returns an SClientHandleCmd whose handle_ field is the newly
allocated handle:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "SClientHandleCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"handle_": 42
}
}
}
}
}
--ch_suites returns a string listing of the handles; the remaining actions return the
generic StcCmd reply with api_ equal to 0.
CSyncCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
one option per synchronisation mode (see the Actions table below) |
Parameters |
|
Environment variables |
|
Keeps a client in step with the server. It is issued repeatedly by clients such as ecflow_ui
rather than typed by an operator. The mode is carried by api_:
Option |
|
Description |
Reply |
|---|---|---|---|
|
0 |
Ask whether anything has changed since the given change numbers. |
news |
|
1 |
Return an incremental update. |
sync |
|
2 |
Return the full definition. |
sync |
|
3 |
Return a clock-only update. |
sync |
# issued internally by clients; not normally typed directly
ecflow_client --sync 1 10 5
Verbatim ecflow_client --help=<action> output (the common environment-variable
footer shown on every page is omitted). Select an action:
news ---- Returns true if state of server definition changed. *Important* for use with c++/python interface only. Requires Given a client handle, change and modify number determine if server changed since last call This relies on user calling sync after news to update the locally stored modify and change numbers. These numbers are then used in the next call to news.
sync ---- Incrementally synchronise the local definition with the one in the server. *Important* for use with c++/python interface only. Preference should be given to this method as only the changes are returned. This reduces the network bandwidth required to keep in sync with the server Requires a client handle, change and modify number, to get the incremental changes from server. The change in server state is then and merged with the client definition.
sync_full --------- Returns the full definition from the server. *Important* for use with c++/python interface only. Requires a client_handle. The returned definition is stored on the client.
sync_clock ---------- Incrementally synchronise the local definition with the one in the server. *Important* for use with c++/python interface only. Same as sync, but will *always* sync with suite clock if it has changed. Preference should be given to this method as only the changes are returned. This reduces the network bandwidth required to keep in sync with the server Requires a client handle, change and modify number, to get the incremental changes from server. The change in server state is then and merged with the client definition.
--sync for handle 1 at state change 10 and modify change 5 (api_ 1):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CSyncCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 1,
"client_handle_": 1,
"client_state_change_no_": 10,
"client_modify_change_no_": 5
}
}
}
}
}
The reply is an SSyncCmd (for --sync, --sync_full and --sync_clock) or an
SNewsCmd (for --news). These carry the incremental changes, or the full serialized
definition, needed to bring the client up to date; their payload is unbounded and is not
reproduced here.
CtsCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
one option per server signal (see the Actions table below) |
Parameters |
|
Environment variables |
|
Encapsulates the many simple, argument-less signals to the server in a single class, to limit the
number of global serialisation symbols. Each command-line option maps to one value of the Api
enumeration, carried in the payload as api_. Most actions return the generic success reply; a few
return a string or a structured reply, as noted in the table:
Option |
|
Description |
Reply |
|---|---|---|---|
|
1 |
Restore the definition from the checkpoint file. |
|
|
2 |
Restart the server (resume scheduling). |
|
|
3 |
Stop scheduling new jobs; keep serving requests. |
|
|
4 |
Stop scheduling and stop most request handling. |
|
|
5 |
Terminate the server process. |
|
|
6 |
Reload the white-list file. |
|
|
7 |
Force an immediate dependency evaluation. |
|
|
8 |
Check that the server is reachable. |
|
|
9 |
Retrieve the list of zombies. |
zombie list |
|
10 |
Return server statistics as text. |
string |
|
11 |
Return the list of suites as text. |
string |
|
12 |
Enable server-side debug output. |
|
|
13 |
Disable server-side debug output. |
|
|
14 |
Return the path of the log file for load plotting. |
server load |
|
15 |
Reset the server statistics. |
|
|
16 |
Reload the password file. |
|
|
17 |
Return the statistics structure (test use only). |
stats structure |
|
18 |
Reload the custom-user password file. |
|
ecflow_client --ping
ecflow_client --restart
ecflow_client --suites
Verbatim ecflow_client --help=<action> output (the common environment-variable
footer shown on every page is omitted). Select an action:
ping
----
Check if server is running on given host/port. Result reported to standard output.
Usage:
--ping --host=mach --port=3144 # Check if server alive on host mach & port 3144
--ping --host=fred # Check if server alive on host fred and port ECF_PORT,
# otherwise default port of 3141
--ping # Check if server alive by using environment variables
# ECF_HOST and ECF_PORT
If ECF_HOST not defined uses 'localhost', if ECF_PORT not defined assumes 3141
restore_from_checkpt -------------------- Ask the server to load the definition from an check pt file. The server must be halted and the definition in the server must be deleted first, otherwise an error is returned
restart ------- Start job scheduling, communication with jobs, and respond to all requests. The following table shows server behaviour in the different states. |----------------------------------------------------------------------------------| | Server State | User Request | Task Request |Job Scheduling | Auto-Check-pointing | |--------------|--------------|--------------|---------------|---------------------| | RUNNING | yes | yes | yes | yes | | SHUTDOWN | yes | yes | no | yes | | HALTED | yes | no | no | no | |--------------|--------------|--------------|---------------|---------------------|
shutdown
--------
Stop server from scheduling new jobs.
Argument(s):
confirm: (optional)
# value must be "yes"; bypasses the confirmation prompt
Usage:
--shutdown=yes
The following table shows server behaviour in the different states.
|----------------------------------------------------------------------------------|
| Server State | User Request | Task Request |Job Scheduling | Auto-Check-pointing |
|--------------|--------------|--------------|---------------|---------------------|
| RUNNING | yes | yes | yes | yes |
| SHUTDOWN | yes | yes | no | yes |
| HALTED | yes | no | no | no |
|--------------|--------------|--------------|---------------|---------------------|
halt
----
Stop server communication with jobs, and new job scheduling.
Also stops automatic check pointing
Argument(s):
confirm: (optional)
# value must be "yes"; bypasses the confirmation prompt
Usage:
--halt=yes
The following table shows server behaviour in the different states.
|----------------------------------------------------------------------------------|
| Server State | User Request | Task Request |Job Scheduling | Auto-Check-pointing |
|--------------|--------------|--------------|---------------|---------------------|
| RUNNING | yes | yes | yes | yes |
| SHUTDOWN | yes | yes | no | yes |
| HALTED | yes | no | no | no |
|--------------|--------------|--------------|---------------|---------------------|
terminate
---------
Terminate the server.
Argument(s):
confirm: (optional)
# value must be "yes"; bypasses the confirmation prompt
Usage:
--terminate=yes
reloadwsfile
------------
Reload the white list file.
The white list file (authorisation) is used to verify if a 'user' is allowed to perform a
specific command.
The file path is specified as the ECF_LISTS variable, and loaded only once by the server
(on *startup*). This means that the file contents can be updated, but the file location
cannot change during the server execution.
The ECF_LISTS variable can be used as follows:
- if ECF_LISTS is not specified, or if it is specified with value `ecf.lists`,
then the server will use the value `<host>.<port>.ecf.lists`
- if ECF_LISTS is specified to be a path, such as /var/tmp/ecf.lists,
then the server will use this path to reload the white list file
The server automatically loads the white list file content as part of the startup procedure,
considering that if the file is not present or is empty (i.e., just contains the version
number) then all users have read/write access.
The reload operation will fail if file does not exist or if the content is invalid.
Expected format for this file is:
# all characters after the first # in a line are considered comments and are discarded
# empty lines are also discarded
4.4.14 # the version number is mandatory, even if no users are specified
# Users with read/write access
user1
user2 # comment
* # use this form if you want all users to have read/write access
# Users with read access, must have - before user name
-user3 # comment
-user4
-* # use this form if you want all users to have read access
Usage:
--reloadwsfile
force-dep-eval -------------- Force dependency evaluation. Used for DEBUG only.
zombie_get ---------- Returns the list of zombies from the server. Results reported to standard output.
stats ----- Returns the server statistics as a string.
suites ------ Returns the list of suites, in the order defined in the server.
debug_server_on --------------- Enables debug output from the server
debug_server_off ---------------- Disables debug output from the server
server_load
-----------
Generates gnuplot files that show the server load graphically.
This is done by parsing the log file. If no log file is provided,
then the log file path is obtained from the server. If the returned
log file path is not accessible an error is returned
This command produces a three files in the CWD:
- <host>.<port>.gnuplot.dat
- <host>.<port>.gnuplot.script
- <host>.<port>.png
The generated script can be manually changed, to see different rendering
effects. i.e. just run 'gnuplot <host>.<port>.gnuplot.script'
Argument(s):
log: (optional) path
# The path to the log file.
# If not provided, the log file path is requested from the server.
If the path to log file is known, it is *preferable* to use this,
rather than requesting the log path from the server.
Usage:
--server_load=/path/to_log_file # Parses log and generate gnuplot files
--server_load # Log file path is requested from server
# which is then used to generate gnuplot files
# *AVOID* if log file path is accessible
Now use any png viewer to see the output i.e
> display <host>.<port>.png
> feh <host>.<port>.png
> eog <host>.<port>.png
> xdg-open <host>.<port>.png
> w3m <host>.<port>.png
stats_reset ----------- Resets the server statistics.
reloadpasswdfile
----------------
Reload the server password file.
The password file (authentication) is used by the server to authenticate a 'user' by
verifying if the password provided by the user matches the one held by the server.
The password file is also used on the client to automatically load the password for the
'user' when connecting to the server.
When the server is configured to use a password file, then ALL users must have a password.
The file path is specified as the ECF_PASSWD environment variable, both for the client and
server, and is loaded only by the server on *startup*. This means that the file contents
can be updated (i.e., add/remove users), but the file location cannot change during the
server execution.
The server automatically loads the password file content as part of the startup procedure.
The ECF_PASSWD environment variable is used to specify the password file location,
considering that
- On the server, the default file name is <host>.<port>.ecf.passwd
- On the client, the default file name is ecf.passwd
The format of the file is same for client and server:
4.5.0
# comment
<user> <host> <port> <passwd> # comment
The following is an example
4.5.0 # the version
fred machine1 3142 xxyyyd
fred machine2 3133 xxyyyd # comment
bill machine2 3133 xxyggyyd
Notice that the same user may appear multiple times (associated with different host/port).
This allows the client to use the same password file to contact multiple servers.
For the password authentication to work, ensure the following:
- The password is defined for the client and server
- On the server, add at least the server administrator to the password file
Note: If an empty password file (i.e., containing just the version) is used,
no user is allowed access.
- On the client, the password file should be readable only by the 'user' itself
Usage:
--reloadpasswdfile
stats_server ------------ Returns the server statistics as a struct and string. For test use only.
reloadcustompasswdfile
----------------------
Reload the server custom password file.
The custom password file (authentication) is used by the server to authenticate a 'user' by
verifying if the password provided by the user matches the one held by the server. This
particular file is used for authentication of users that explicitly specify the user name
(either via the environment variable ECF_USER or the --user option).
This mechanism should be used when most users use the machine login name, but a few users
specify their own user name, in which case the password must also be explicitly provided.
The file path is specified as the ECF_CUSTOM_PASSWD environment variable, both for the
client and server, and is loaded only by the server on *startup*. This means that the file
contents can be updated (i.e., add/remove users), but the file location cannot change during
the server execution.
The server automatically loads the password file content as part of the startup procedure.
The ECF_CUSTOM_PASSWD environment variable is used to specify the password file location,
considering that
- On the server the default file name is <host>.<port>.ecf.custom_passwd
- On the client the default file name is ecf.custom_passwd
The format of the file is same for client and server:
4.5.0
# comment
<user> <host> <port> <passwd> # comment
The following is an example
4.5.0 # the version
fred machine1 3142 xxyyyd
fred machine2 3133 xxyyyd # comment
bill machine2 3133 xxyggyyd
Notice that the same user may appear multiple times (associated with different host/port).
This allows the client to use the same password file to contact multiple servers.
For the password authentication to work, ensure the following:
- The password is defined for the client and server
- On the server, add at least the server administrator to the password file
Note: If an empty password file (i.e., containing just the version) is used,
no user is allowed access.
- On the client, the password file should be readable only by the 'user' itself
Usage:
--reloadcustompasswdfile
--ping (api_ 8):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CtsCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 8
}
}
}
}
}
--restart (api_ 2):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CtsCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 2
}
}
}
}
}
--suites (api_ 11):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CtsCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 11
}
}
}
}
}
Actions such as --ping and --restart return the generic success reply (StcCmd
with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
Actions that return text, such as --stats and --suites, reply with an SStringCmd
whose str_ field holds the result:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "SStringCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"str_": "a string reply"
}
}
}
}
}
CtsNodeCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
one option per query (see the Actions table below) |
Parameters |
|
Environment variables |
|
Queries the server about a node or the whole definition. The query is carried by api_ and the
target by absNodePath_:
Option |
|
Description |
Reply |
|---|---|---|---|
|
1 |
Force job generation under the node. |
|
|
2 |
Test job generation without submitting. |
|
|
3 |
Return the definition (or node subtree). |
definition |
|
4 |
Explain why a node is not running. |
string |
|
5 |
Return the definition with node states. |
definition |
|
6 |
Return a representation suitable for migration. |
string |
ecflow_client --why=/suite/family/task
ecflow_client --get=/suite/family/task
Verbatim ecflow_client --help=<action> output (the common environment-variable
footer shown on every page is omitted). Select an action:
get
---
Get the suite definition or node tree in form that is re-parse able
Get all suite node tree's from the server and write to standard out.
The output is parse-able, and can be used to re-load the definition
arg = NULL | arg = node path
Usage:
--get # gets the definition from the server,and writes to standard out
--get=/s1 # gets the suite from the server,and writes to standard out
why
---
Show the reason why a node is not running.
Can only be used with the group command. The group command must include a
'get' command(i.e returns the server defs)
The why command take a optional string argument representing a node path
Will return reason why the node is holding and for all its children.
If no arguments supplied will report on all nodes
arg = node path | arg = NULL
Usage:
--group="get; why" # returns why for all holding nodes
--group="get; why=/suite/family" # returns why for a specific node
get_state
---------
Get state data. For the whole suite definition or individual nodes.
This will include event, meter, node state, trigger and time state.
The output is written to standard out.
arg = NULL | arg = node path
Usage:
--get_state # gets the definition from the server,and writes to standard out
--get_state=/s1 # gets the suite from the server,and writes to standard out
migrate
-------
Used to print state of the definition returned from the server to standard output.
The node state is shown in the comments.
This is the format used in the check point file, but with indentation.
Since this is essentially the defs format with state in the comments,it allows the definition to be migrated to future version of ecflow.
The output includes edit history but excludes externs.
When the definition is reloaded *NO* checking is done.
The following shows a summary of the features associated with each choice:
--get --get_state --migrate
Auto generate externs Yes Yes No
Checking on reload Yes Yes No
Edit History No No Yes
Show trigger AST No Yes No
Usage:
--migrate # show all suites
--migrate=/s1 # show state for suite s1
job_gen
-------
Job submission for chosen Node *based* on dependencies.
The server traverses the node tree every 60 seconds, and if the dependencies are free
does job generation and submission. Sometimes the user may free time/date dependencies
to avoid waiting for the server poll, this commands allows early job generation
arg = node path | arg = NULL
If no node path specified generates for full definition.
checkJobGenOnly
---------------
Test hierarchical Job generation only, for chosen Node.
The jobs are generated independent of the dependencies
This will generate the jobs *only*, i.e. no job submission. Used for checking job generation only
arg = node path | arg = NULL
If no node path specified generates for all Tasks in the definition. For Test only
--why=/suite/family/task (api_ 4):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CtsNodeCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 4,
"absNodePath_": "/suite/family/task"
}
}
}
}
}
--get=/suite/family/task differs only in api_ being 3:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "CtsNodeCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 3,
"absNodePath_": "/suite/family/task"
}
}
}
}
}
--job_gen and --check_job_gen_only return the generic StcCmd reply with api_
equal to 0:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
--why and --migrate return an SStringCmd (the same string-reply shape used by the
other string-returning commands). --get
and --get_state return a DefsCmd whose payload embeds the serialized definition or
node subtree; it is omitted here because its size is unbounded.
DeleteCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Removes nodes from the server. The paths are carried in paths_ and the force flag in
force_. Deleting every suite (--delete _all_) returns a distinct reply.
ecflow_client --delete=/suite/family
ecflow_client --delete force yes /suite/family/task
ecflow_client --delete _all_
Verbatim ecflow_client --delete output (the common environment-variable footer is
omitted):
delete
------
Deletes the specified node(s) or _ALL_ existing definitions( i.e delete all suites) in the server.
Argument(s):
force: (optional) [ force | yes ]
# Use this parameter to bypass checks, i.e. for active or submitted tasks
confirm: (optional)
# value must be "yes"; bypasses the confirmation prompt
target: node paths | _all_
# The value `_all_` means delete all suites.
# The node paths must start with a leading '/'.
Usage:
--delete=_all_ # Delete all suites in server. Use with care.
--delete=/suite/f1/t1 # Delete node at /suite/f1/t1. This will prompt
--delete=force /suite/f1/t1 # Delete node at /suite/f1/t1 even if active or submitted
--delete=force yes /s1 /s2 # Delete suites s1,s2 even if active or submitted, bypassing prompt
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "DeleteCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite/family" ],
"force_": false
}
}
}
}
}
Deleting specific nodes returns the generic StcCmd reply with api_ equal to 0.
Deleting every suite (--delete _all_) instead returns StcCmd with api_ equal to
3 (DELETE_ALL):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 3
}
}
}
}
}
EditScriptCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Retrieves or submits the script of a task, with optional preprocessing and variable substitution. It
underlies the “edit script” workflow in ecflow_ui.
ecflow_client --edit_script /suite/family/task pre_process > script.pp
Verbatim ecflow_client --edit_script output (the common environment-variable footer is
omitted):
edit_script
-----------
Allows user to edit, pre-process and submit the script.
Will allow pre-processing of arbitrary file with 'pre_process_file' option
Argument(s):
path to task
# The path to the task/alias
action: [ edit | pre_process | submit | pre_process_file | submit_file ]
# edit : Returns the script file to standard out. The script will
# include used variables enclosed between %comment/%end at the
# start of the file
#
# pre_process: Returns the script file to standard out. The script will
# include used variables enclosed between %comment/%end at the
# start of the file and with all %include expanded
#
# submit: Extracts the used variables from the supplied file, i.e
# between the %comment/%end and use these them to generate the
# job using the ecf file accessible from the server
#
# pre_process_file: Pre-process the user supplied file,
# expanding includes, performing variable substitution,
# removing manual & comment sections.
#
# submit_file: Like submit, but the supplied file is submitted by the server.
#
# Options `pre_process_file` and `submit_file` facilitate debugging the script file
path_to_script_file: (optional)
# The path to the script file
# Required for options `pre_process_file` and `submit_file`
create_alias: (optional)
# If specified, will create an alias for the task and submit the job using this alias
# Default value is false, to be used in combination with `submit_file` option
no_run: (optional)
# If specified, will create an alias for the task but will not submit the job
# Default value is false, to be used in combination with `submit_file` option
Usage:
--edit_script=/path/to/task edit > script_file
server returns script with the used variables to standard out
The user can choose to edit this file
--edit_script=/path/to/task pre_process > pre_processed_script_file
server will pre process the ecf file accessible from the server
(i.e expand all %includes) and return the file to standard out
--edit_script=/path/to/task submit script_file
Will extract the used variables in the 'script_file' and will uses these
variables during variable substitution of the ecf file accessible by the
server. This is then submitted as a job
--edit_script=/path/to/task pre_process_file file_to_pre_process
The server will pre-process the user supplied file and return the contents
to standard out. This pre-processing is the same as job file processing,
but on a arbitrary file
--edit_script=/path/to/task submit_file file_to_submit
Will extract the used variables in the 'file_to_submit' and will uses these
variables during variable substitution, the file is then submitted for job
generation by the server
--edit_script=/path/to/task submit_file file_to_submit create_alias
Like the the previous example but will create and run as an alias
--edit_script /suite/family/task pre_process (edit_type_ 1):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "EditScriptCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"edit_type_": 1,
"path_to_node_": "/suite/family/task",
"user_file_contents_": [],
"user_variables_": [],
"alias_": false,
"run_": false
}
}
}
}
}
edit and pre_process return an SStringCmd whose str_ field holds the script;
the submit variants return the generic StcCmd reply with api_ equal to 0.
ForceCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Forces a node into a given state, or forces an event, regardless of dependencies. stateOrEvent_
holds the requested state or event, and recursive_ / setRepeatToLastValue_ carry the
recursive and full modifiers.
ecflow_client --force complete /suite/family/t1 /suite/family/t2
ecflow_client --force recursive full complete /suite/family
ecflow_client --force clear /suite/family/t1:myevent
Verbatim ecflow_client --force output (the common environment-variable footer is
omitted):
force
-----
Force a node to a given state, or set its event.
When a task is set to complete, it may be automatically re-queued if it has
multiple future time dependencies. However each time we force a complete it will
expire any time based attribute on that node. When the last time based attribute
expires, the node will stay in a complete state.
This behaviour allow Repeat values to be incremented interactively.
A repeat attribute is incremented when all the child nodes are complete
in this case the child nodes are automatically re-queued.
Argument(s):
state: [ unknown | complete | queued | submitted | active | aborted | clear | set ]
# The state to be set. If the state is either `set` or `clear`, then the event is updated.
recursive: (optional)
# Applies state to node, and recursively to all its children
full: (optional)
# Set repeat variables to last value, only works in conjunction
# with recursive option
path: <path> | <path>:<event>
# The path to a node or event. The path must begin with '/'
Usage:
--force=complete /suite/t1 /suite/t2 # Set task t1 & t2 to complete
--force=clear /suite/task:ev # Clear the event 'ev' on task /suite/task
--force=complete recursive /suite/f1 # Recursively set complete all children of /suite/f1
Effect:
Consider the effect of applying `--force=complete /s1/t1 /s1/t2` at 09:00
suite s1
task t1; time 12:00 # will complete straight away
task t2; time 10:00 13:00 01:00 # will complete on fourth attempt
Specifying a time range, as shown with task t2, means the task is re-queued and the
next time slot is incremented for each complete, until it expires, and the task completes.
Use the Why command, to show next run time (i.e. next time slot)
--force complete /suite/family:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "ForceCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite/family" ],
"stateOrEvent_": "complete",
"recursive_": false,
"setRepeatToLastValue_": false
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
FreeDepCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Frees a node from a held dependency so that it may run, without changing the dependency itself. The flags select which dependency kinds are freed.
ecflow_client --free-dep /suite/family/task
ecflow_client --free-dep trigger date time /suite/family/task
Verbatim ecflow_client --free-dep output (the common environment-variable footer is
omitted):
free-dep
--------
Free dependencies for a node. Defaults to triggers
After freeing the time related dependencies (i.e time,today,cron)
the next time slot will be missed.
Argument(s):
all: (optional)
# Free trigger, date and all time dependencies
date: (optional)
# Free date dependencies
time: (optional)
# Free all time dependencies i.e time, day, today, cron
paths
# A space separated list of paths.
# At least one path must be provided. Paths must start with a leading '/'
Usage:
--free-dep=/s1/t1 /s2/t2 # free trigger dependencies for task's t1,t2
--free-dep=all /s1/f1/t1 # free all dependencies of /s1/f1/t1
--free-dep=date /s1/f1 # free holding date dependencies of /s1/f1
--free-dep /suite/family/task (the default frees the trigger dependency):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "FreeDepCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite/family/task" ],
"trigger_": true,
"all_": false,
"date_": false,
"time_": false
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
GroupCTSCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Bundles several commands into a single request so that they are sent and executed together. Each
member command is serialized as a polymorphic entry of cmdVec_; cli_ records whether the
group originated on the command line.
ecflow_client --group="ping; get=/suite"
Verbatim ecflow_client --group output (the common environment-variable footer is
omitted):
group
-----
Allows a series of ';' separated commands to be grouped and executed as one.
Some commands like halt, shutdown and terminate will prompt the user. To bypass the prompt
provide 'yes' as an additional parameter. See example below.
Argument(s):
arg: string
# A string of ';' separated commands. Each command is the same as if it was
# provided on the command line. The string must be quoted.
Usage:
--group="halt=yes; reloadwsfile; restart;"
# halt server,bypass the confirmation prompt,
# reload white list file, restart server
--group="get; show"
# get server defs, and write to standard output
--group="get=/s1; show state"
# get suite 's1', and write state to standard output
--group containing a single --ping (CtsCmd with api_ 8); further members
appear as additional entries of cmdVec_:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "GroupCTSCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"cmdVec_": [
{
"polymorphic_id": 2147483650,
"polymorphic_name": "CtsCmd",
"ptr_wrapper": {
"id": 2147483650,
"data": {
"cereal_class_version": 0,
"value0": {
"value0": { "cl_host_": "host.example.com" },
"user_": "operator"
},
"api_": 8
}
}
}
],
"cli_": false
}
}
}
}
}
The reply is a GroupSTCCmd bundling the reply of each member command in order; its shape
depends on the members and is not reproduced here.
LoadDefsCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Loads a suite definition into the server. The client reads the file and embeds its serialized form
in defs_; defs_filename_ records the original file name. The defs_ field is a complete
serialized definition and can be large; it is truncated below.
ecflow_client --load=my_suite.def
ecflow_client --load=my_suite.def force
Verbatim ecflow_client --load output (the common environment-variable footer is
omitted):
load
----
Check and load definition or checkpoint file into server.
The loaded definition will be checked for valid trigger and complete expressions,
additionally in-limit references to limits will be validated.
If the server already has the 'suites' of the same name, then a error message is issued.
The suite's can be overwritten if the force option is used.
To just check the definition and not send to server, use 'check_only'
This command can also be used to load a checkpoint file into the server
Argument(s):
path: <path>
# The path to the definition file or checkpoint file.
mode: (optional) [ force | check_only | print | stats ]
# The default value is to apply none of the mode options.
Usage:
--load=/my/home/exotic.def # will error if suites of same name exists
--load=/my/home/exotic.def force # overwrite suite's of same name in the server
--load=/my/home/exotic.def check_only # Just check, don't send to server
--load=/my/home/exotic.def stats # Show defs statistics, don't send to server
--load=host1.3141.check # Load checkpoint file to the server
--load=host1.3141.check print # print definition to standard out in defs format
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "LoadDefsCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"force_": false,
"defs_": "#5.18.0\nsuite s1\n family f1\n task t1\n endfamily\nendsuite\n# ...",
"defs_filename_": "my_suite.def"
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
LogCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Inspects or manages the server log file. get and path return a string; clear, flush
and new return the generic reply.
ecflow_client --log=get 200
ecflow_client --log=clear
ecflow_client --log=path
Verbatim ecflow_client --log output (the common environment-variable footer is
omitted):
log
---
Get, clear, flush or create a new log file.
The user must ensure that a valid path is specified.
Specifying '--log=get' with a large number of lines from the server,
can consume a lot of **memory**. The log file can be a very large file,
hence we use a default of 100 lines, optionally the number of lines can be specified.
Argument(s):
action: [ get | clear | flush | new | path ]
# get: Outputs the log file to standard out.
# defaults to return the last 100 lines
# The second argument can specify how many lines to return
#
# clear: Clear the log file of its contents.
#
# flush: Flush and close the log file. (only temporary) next time
# server writes to log, it will be opened again. Hence it best
# to halt the server first
#
# new: Flush and close the existing log file, and start using the
# the path defined for ECF_LOG. By changing this variable
# a new log file path can be used
# Alternatively an explicit path can also be provided
# in which case ECF_LOG is also updated
#
# path: Returns the path name to the existing log file
value: (optional) [ new_path | optional last n lines ]
# If 'get' is specified, the value specifies lines to get.
# The value must be an integer.
# If 'new' is specified, the value specifies a new path
Usage:
--log=get # Write the last 100 lines of the log file to standard out
--log=get 200 # Write the last 200 lines of the log file to standard out
--log=clear # Clear the log file. The log is now empty
--log=flush # Flush and close log file, next request will re-open log file
--log=new /path/to/new/log/file # Close and flush log file, and create a new log file, updates ECF_LOG
--log=new # Close and flush log file, and create a new log file using ECF_LOG variable
--log=get (api_ 0; get_last_n_lines_ defaults to 100):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "LogCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 0,
"get_last_n_lines_": 100,
"new_path_": ""
}
}
}
}
}
get and path return an SStringCmd whose str_ field holds the requested lines
or the log path:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "SStringCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"str_": "..."
}
}
}
}
}
clear, flush and new return the generic StcCmd reply with api_ equal to
0.
LogMessageCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Writes an arbitrary message into the server log file, useful for annotating operations.
ecflow_client --msg="starting the morning cycle"
Verbatim ecflow_client --msg output (the common environment-variable footer is
omitted):
msg
---
Writes the input string to the log file.
Argument(s):
message: string
# The message to be written to the log file.
Usage:
--msg="place me in the log file"
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "LogMessageCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"msg_": "a user message"
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
MoveCmd
Type |
User command (internal) |
|---|---|
C++ class |
|
Action |
none (internal) |
Parameters |
|
Environment variables |
|
Carries a node subtree from one location (or server) to another. It has no command-line option of
its own: it is generated by PlugCmd and by drag-and-drop in ecflow_ui. The src_node_
field embeds the serialized subtree, so the payload is unbounded; its structure mirrors the other
user commands, with the base-class cl_host_ and user_ followed by the fields above. The reply
is the generic StcCmd success reply with api_ equal to 0.
OrderNodeCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Changes the position of a node among its siblings, which determines display order and, for some
attributes, evaluation order. The ordering is carried by option_.
ecflow_client --order=/suite/family/task top
Verbatim ecflow_client --order output (the common environment-variable footer is
omitted):
order
-----
Re-orders the nodes held by the server
Argument(s):
node: path
# The path to the node to be re-ordered.
sort-type: [ top | bottom | alpha | order | up | down | runtime]
It should be noted that in the absence of triggers and time/date dependencies,
the tasks are submitted in order.
This changes the order and hence affects the submission order::
o top raises the node within its parent, so that it is first
o bottom lowers the node within its parent, so that it is last
o alpha Arranges for all the peers of selected note to be sorted alphabetically (case-insensitive)
o order Arranges for all the peers of selected note to be sorted in reverse alphabet(case-insensitive)
o up Moves the selected node up one place amongst its peers
o down Moves the selected node down one place amongst its peers
o runtime Orders the nodes according to state change runtime
for families by accumulated runtime of its children
useful to submit the task that take longer earlier
This command can fail because:
- The node path does not exist in the server
- The order_type is not does not match one of arg2
Usage:
--order=/suite/f1 top # move node f1 to the top
--order=/suite/family/task top (option_ 0):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "OrderNodeCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"absNodepath_": "/suite/family/task",
"option_": 0
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
PathsCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
one option per node operation (see the Actions table below) |
Parameters |
|
Environment variables |
|
Applies a common node operation to one or more paths. The operation is carried by api_ and the
targets by paths_:
Option |
|
Description |
Reply |
|---|---|---|---|
|
1 |
Suspend the node, preventing it from being scheduled. |
|
|
2 |
Resume a suspended node. |
|
|
3 |
Run |
|
|
4 |
Run |
string |
|
5 |
Check trigger and other expressions resolve. |
string |
|
6 |
Return the edit history of the node. |
string |
|
7 |
Archive the node’s children to disk. |
|
|
8 |
Restore previously archived children. |
|
ecflow_client --suspend /suite/family
ecflow_client --check /suite/family
Verbatim ecflow_client --help=<action> output (the common environment-variable
footer shown on every page is omitted). Select an action:
suspend
-------
Suspend the given node. This prevents job generation for the given node, or any child node.
Usage:
--suspend=/s1/f1/t1 # suspend task s1/f1/t1
--suspend=/s1 /s2 # suspend suites /s1 and /s2
resume
------
Resume the given node. This allows job generation for the given node, or any child node.
Usage:
--resume=/s1/f1/t1 # resume task s1/f1/t1
--resume=/s1 /s2 # resume suites /s1 and /s2
kill
----
Kills the job associated with the node.
If a family or suite is selected, will kill hierarchically.
Kill uses the ECF_KILL_CMD variable. After variable substitution it is invoked as a command.
The command should be written in such a way that the output is written to %ECF_JOB%.kill
as this allow the --file command to report the output: .e.e.
/home/ma/emos/bin/ecfkill %USER% %HOST% %ECF_RID% %ECF_JOB% > %ECF_JOB%.kill 2>&1::
Usage:
--kill=/s1/f1/t1 /s1/f2/t2 # kill the jobs for tasks t1 and t2
--file=/s1/f1/t1 kill # write to standard out the '.kill' file for task /s1/f1/t1
status
------
Shows the status of a job associated with a task, in %ECF_JOB%.stat file
If a family or suite is selected, will invoke status command hierarchically.
Status uses the ECF_STATUS_CMD variable. After variable substitution it is invoked as a command.
The command should be written in such a way that the output is written to %ECF_JOB%.stat
This will allow the output of status command to be shown by the --file command
i.e /home/ma/emos/bin/ecfstatus %USER% %HOST% %ECF_RID% %ECF_JOB% > %ECF_JOB%.stat 2>&1::
If the process id cannot be found on the remote system, then the status command can also
arrange for the task to be aborted
The status command can fail for the following reasons:
- ECF_STATUS_CMD not found
- variable substitution fails
- state is active but it can't find process id, i.e. ECF_RID
- the status command does not exit cleanly
When this happens a flag is set, STATUSCMD_FAILED, which is visible in the GUI
Usage:
--status=/s1/f1/t1 # ECF_STATUS_CMD should output to %ECF_JOB%.stat
--file=/s1/f1/t1 stat # Return contents of %ECF_JOB%.stat file
check
-----
Checks the expression and limits in the server. Will also check trigger references.
Trigger expressions that reference paths that don't exist, will be reported as errors.
(Note: On the client side unresolved paths in trigger expressions must
have an associated 'extern' specified)
arg = [ _all_ | / | list of node paths ]
Usage:
--check=_all_ # Checks all the suites
--check=/ # Checks all the suites
--check=/s1 /s2/f1/t1 # Check suite /s1 and task t1
edit_history
------------
Returns the edit history associated with a Node.
Can also be used to clear the edit history.
Usage:
--edit_history=/s1/f1/t1 # return history of changes for the given node
--edit_history=clear # clear/purge *ALL* edit history from all nodes.
archive
-------
Archives suite or family nodes *IF* they have child nodes(otherwise does nothing).
Saves the suite/family nodes to disk, and then removes the child nodes from the definition.
This saves memory in the server, when dealing with huge definitions that are not needed.
It improves time taken to checkpoint and reduces network bandwidth.
If the node is re-queued or begun, the child nodes are automatically restored.
Use --restore to reload the archived nodes manually
Care must be taken if you have trigger reference to the archived nodes
The nodes are saved to the file ECF_HOME/<hostname>.<port>.<ECF_NAME>.check,
where '/' has been replaced with ':' in ECF_NAME
Nodes like (family and suites) can also to automatically archived by using,
the 'autoarchive' attribute. The attribute is only applied once the node is complete
suite autoarchive
family f1
autoarchive +01:00 # archive one hour after complete
task t1
endfamily
family f2
autoarchive 01:00 # archive at 1 am in morning after complete
task t1
endfamily
family f3
autoarchive 10 # archive 10 days after complete
task t1
endfamily
family f4
autoarchive 0 # archive immediately (upto 60s) after complete
task t1
endfamily
endsuite
Usage:
--archive=/s1 # archive suite s1
--archive=/s1/f1 /s2 # archive family /s1/f1 and suite /s2
--archive=force /s1 /s2 # archive suites /s1,/s2 even if they have active tasks
restore
-------
Manually restore archived nodes.
Restore will fail if:
- Node has not been archived
- Node has children, (since archived nodes have no children)
- If the file ECF_HOME/<host>.<port>.<ECF_NAME>.check does not exist
Nodes can be restored manually(as in this command) but also automatically
Automatic restore is done using the 'autorestore' attribute.
Once the node containing the 'autorestore' completes, the attribute is applied
suite s
family farchive_now
autoarchive 0 # archive immediately after complete
task tx
endfamily
family frestore_from_task
task t1
trigger ../farchive_now<flag>archived
autorestore ../farchive_now # call autorestore when t1 completes
endfamily
endsuite
In this example task '/s/frestore_from_task/t1' is only triggered if 'farchive_now'
is archived, then when t1 completes it will restore family 'farchive_now'
Usage:
--restore=/s1/f1 # restore family /s1/f1
--restore=/s1 /s2 # restore suites /s1 and /s2
--suspend /suite/family (api_ 1):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "PathsCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 1,
"paths_": [ "/suite/family" ],
"force_": false
}
}
}
}
}
--check /suite/family differs only in api_ being 5:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "PathsCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"api_": 5,
"paths_": [ "/suite/family" ],
"force_": false
}
}
}
}
}
--suspend, --resume, --kill, --archive and --restore return the generic
StcCmd reply with api_ equal to 0:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
--status, --check and --edit_history return an SStringCmd whose str_ field
holds the text result:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "SStringCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"str_": "..."
}
}
}
}
}
PlugCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Moves a node subtree to a new parent, possibly on a different server. Internally the server uses
MoveCmd to transfer the subtree to the destination.
ecflow_client --plug /suite/family //host:3141/dest
Verbatim ecflow_client --plug output (the common environment-variable footer is
omitted):
plug
----
Plug command is used to move nodes.
The destination node can be on another server In which case the destination
path should be of the form '<host>:<port>/suite/family/task
Argument(s):
source: path
# The path to the source node
destination: path
# The path to the destination node
This command can fail because:
- Source node is in a 'active' or 'submitted' state
- Another user already has an lock
- source/destination paths do not exist on the corresponding servers
- If the destination node path is empty, i.e. only host:port is specified,
then the source node must correspond to a suite.
- If the source node is added as a child, then its name must be unique
amongst its peers
Usage:
--plug=/suite macX:3141 # move the suite to ecFlow server on host(macX) and port(3141)
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "PlugCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"source_": "/suite/family",
"dest_": "//host:3141/dest"
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
QueryCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Reads a single piece of state from the server without changing anything — for example the state of a node, or the value of an event, meter or variable. The reply is always a string. It is often used inside scripts to branch on a node’s state.
ecflow_client --query state /suite/family/task
ecflow_client --query event /suite/family/task:myevent
Verbatim ecflow_client --query output (the common environment-variable footer is
omitted):
query
-----
Query the status of attributes
i.e state,dstate,repeat,event,meter,label,variable or trigger expression without blocking
- state return [unknown | complete | queued | aborted | submitted | active] to standard out
- dstate return [unknown | complete | queued | suspended | aborted | submitted | active] to standard out
- repeat returns current value as a string to standard out
- event return 'set' | 'clear' to standard out
- meter return value of the meter to standard out
- limit return current value of limit to standard out
- limit_max return limit max value to standard out
- label return new value otherwise the old value
- variable return value of the variable, repeat or generated variable to standard out,
will search up the node tree. When path is '/', the variable is looked up on the
server itself, i.e. 'ecflow_client --query variable /:name'
- trigger returns 'true' if the expression is true, otherwise 'false'
If this command is called within a '.ecf' script we will additionally log the task calling this command
This is required to aid debugging for excessive use of this command
The command will fail if the node path to the attribute does not exist in the definition and if:
- repeat The repeat is not found
- event The event is not found
- meter The meter is not found
- limit/limit_max The limit is not found
- label The label is not found
- variable No user or generated variable or repeat of that name found on node or its parents,
or (when path is '/') no user or server variable of that name found on the server
- trigger Trigger does not parse, or reference to nodes/attributes in the expression are not valid
Argument(s):
attribute: [ state | dstate | repeat | event | meter | label | variable | trigger | limit | limit_max ]
# The kind of the attribute to be queried.
target: <path> | <path>:name
# The path to the node or the node and attribute name.
value: trigger expression | prev | next
# The values `prev` and `next` are only used when the attribute is a repeat
Usage:
--query state / # return top level state to standard out
--query state /path/to/node # return node state to standard out
--query dstate /path/to/node # state that can included suspended
--query repeat /path/to/node # return the current value as a string
--query repeat /path/to/node prev # return the previous value as a string
--query repeat /path/to/node next # return the next value as a string
--query event /path/to/task/with/event:event_name # return set | clear to standard out
--query meter /path/to/task/with/meter:meter_name # returns the current value of the meter to standard out
--query limit /path/to/task/with/limit:limit_name # returns the current value of the limit to standard out
--query limit_max /path/to/task/with/limit:limit_name # returns the max value of the limit to standard out
--query label /path/to/task/with/label:label_name # returns the current value of the label to standard out
--query variable /path/to/task/with/var:var_name # returns the variable value to standard out
--query variable /:var_name # returns the server variable value to standard out
--query trigger /path/to/node/with/trigger "/suite/task == complete" # return true if expression evaluates false otherwise
--query state /suite/family/task:
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "QueryCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"query_type_": "state",
"path_to_attribute_": "/suite/family/task",
"attribute_": "",
"path_to_task_": "/suite/family/task"
}
}
}
}
}
An SStringCmd whose str_ field holds the queried value:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "SStringCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"str_": "complete"
}
}
}
}
}
ReplaceNodeCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Replaces a node (or adds it) using a node of the same path taken from another definition. Like
LoadDefsCmd, the source definition is embedded as a serialized string, here in clientDefs_,
which can be large. The payload therefore has the following structure (clientDefs_ elided):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "ReplaceNodeCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0, "cl_host_": "host.example.com" },
"user_": "operator"
},
"createNodesAsNeeded_": true,
"force_": false,
"pathToNode_": "/suite/family",
"path_to_defs_": "my_suite.def",
"clientDefs_": "#5.18.0\nsuite suite\n family family\n task t1\n endfamily\nendsuite\n# ..."
}
}
}
}
}
The reply is the generic StcCmd success reply with api_ equal to 0.
Verbatim ecflow_client --replace output (the common environment-variable footer is omitted):
replace
-------
Replaces a node in the server, with the given path
Can also be used to add nodes in the server
arg1 = path to node
must exist in the client defs(arg2). This is also the node we want to
replace in the server
arg2 = path to client definition file
provides the definition of the new node
arg3 = (optional) [ parent | false ] (default = parent)
create parent families or suite as needed, when arg1 does not
exist in the server
arg4 = (optional) force (default = false)
Force the replacement even if it causes zombies to be created
Replace can fail if:
- The node path(arg1) does not exist in the provided client definition(arg2)
- The client definition(arg2) must be free of errors
- If the third argument is not provided, then node path(arg1) must exist in the server
- Nodes to be replaced are in active/submitted state, in which case arg4(force) can be used
Replace will preserve the suspended status, if this is not required please re-queue first
After replace is done, we check trigger expressions. These are reported to standard output.
It is up to the user to correct invalid trigger expressions, otherwise the tasks will *not* run.
Please note, you can use --check to check trigger expression and limits in the server.
For more information use --help check.
Usage:
--replace=/suite/f1/t1 /tmp/client.def parent # Add/replace node tree /suite/f1/t1
--replace=/suite/f1/t1 /tmp/client.def false force # replace t1 even if its active or submitted
RequeueNodeCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Requeues nodes so that they may run again. option_ encodes the modifier (0 none, 1
abort, 2 force).
ecflow_client --requeue /suite/family
ecflow_client --requeue abort /suite/family
ecflow_client --requeue force /suite/family/task
Verbatim ecflow_client --requeue output (the common environment-variable footer is
omitted):
requeue
-------
Re queues the specified node(s)
If any child of the specified node(s) is in a suspended state, this state is cleared
Repeats are reset to their starting values, relative time attributes are reset.
Argument(s):
mode: (optional) [ abort | force ]
# abort = re-queue only aborted tasks below node
# force = Force the re-queueing even if there are nodes that are active or submitted
# <null> = Checks if any tasks are in submitted or active states below the node,
# and if so does nothing. Otherwise, re-queues the node.
node: path
# A space separared list of node paths. Paths must begin with a leading '/' character
Usage:
--requeue=abort /suite/f1 # re-queue all aborted tasks of /suite/f1
--requeue=force /suite/f1 # forcibly re-queue /suite/f1 and all its children.May cause zombies.
--requeue=/s1/f1/t1 /s1/t2 # Re-queue node '/suite/f1/t1' and '/s1/t2'
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "RequeueNodeCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite/family" ],
"option_": 0
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
RunNodeCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Runs a task immediately, bypassing its scheduling time. force_ records whether dependencies are
ignored.
ecflow_client --run /suite/family/task
ecflow_client --run force /suite/family/task
Verbatim ecflow_client --run output (the common environment-variable footer is
omitted):
run
---
Ignore triggers, limits, time or date dependencies, just run the Task.
When a job completes, it may be automatically re-queued if it has a cron
or multiple time dependencies. If we have multiple time based attributes,
then each run, will expire the time.
When we run before the time, we want to avoid re-running the task then
a flag is set, so that it is not automatically re-queued.
A repeat attribute is incremented when all the child nodes are complete
in this case the child nodes are automatically re-queued.
Hence this command can be aid, in allowing a Repeat attribute to be incremented
Argument(s):
force: (optional)
# Forcibly run, even if there are nodes that are active or submitted
# This might cause the appearance of zombie
node: path
# A space separared list of node paths.
# The paths must begin with a leading '/' character.
# If the path points to a suite or a family, all tasks will be run recursively.
# When providing multiple paths avoid running the same task twice
Usage:
--run=/suite/t1 # run task t1
Effect:
task t1; time 12:00 # will complete if run manually
task t2; time 10:00 13:00 01:00 # will run 4 times before completing
Specifying a time range, as shown with task t2, means the next time slot
is incremented for each run, until it expires, and the task completes.
Use the Why command, to show next run time (i.e. next time slot)
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "RunNodeCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"paths_": [ "/suite/family/task" ],
"force_": false
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
ServerVersionCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Returns the version string of the running server. The command carries no fields of its own, so the
payload holds only the base-class cl_host_ and user_.
ecflow_client --server_version
Verbatim ecflow_client --server_version output (the common environment-variable footer is
omitted):
server_version
--------------
Returns the version number of the server
Usage:
--server_version
Writes the version to standard output
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "ServerVersionCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
}
}
}
}
}
}
An SStringCmd whose str_ field holds the server version:
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "SStringCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"str_": "5.14.1"
}
}
}
}
}
ShowCmd
Type |
User command (client-side) |
|---|---|
C++ class |
|
Action |
|
Parameters |
|
Environment variables |
|
Selects the print style used when the client renders a definition, and is normally combined with a
retrieval command such as --get in a group. The chosen style is applied on the client side and is
not part of the serialized payload, so the request carries only the base-class cl_host_ and
user_.
ecflow_client --group="get=/suite; show=state"
Verbatim ecflow_client --show output (the common environment-variable footer is
omitted):
show
----
Used to print state of the definition returned from the server to standard output.
This command can *only* be used in a group command, and will only work if it is
preceded with a get command. See examples below.
Argument(s):
output: (optional) [ defs | state | migrate ]
# defs:
# Return the definition structure only, without any state information.
# Necessary `extern`s are automatically added, allowing the output to be
# reloaded into the server e.g., --group="get ; show"
# This is the default if no `output` argument is provided.
#
# state:
# Return definition structure along with all the state information.
# Output includes the trigger expressions' abstract syntax tree (as comment).
# Does not include the edit history
#
# migrate:
# Return definition structure along with all the state information.
# The node state is shown in the comments.
# Use this format to migrate the definition to future version of ecflow.
# The output includes edit history but excludes externs.
# When the definition is reloaded *NO* checking is done.
The following shows a summary of the features associated with each choice
DEFS STATE MIGRATE
Auto generate externs Yes Yes No
Checking on reload Yes Yes No
Edit History No No Yes
trigger AST No Yes No
Usage:
--group="get ; show"
--group="get ; show defs" # same as the previous example
--group="get ; show state" # Show all state for the node tree
--group="get ; show migrate" # Shows state and allows migration
--group="get=/s1; show" # show state for the node only
--group="get=/s1; show state"
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "ShowCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
}
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}
ZombieCmd
Type |
User command |
|---|---|
C++ class |
|
Action |
one option per zombie resolution (see the Actions table below) |
Parameters |
|
Environment variables |
|
Resolves zombies: mismatches between a running job and the server’s record of a task. The action is
carried by user_action_:
Option |
|
Description |
|---|---|---|
|
0 |
Allow the task command to complete without changing state. |
|
1 |
Make the task command fail. |
|
2 |
Adopt the zombie, applying its state change to the task. |
|
3 |
Remove the zombie from the server’s list. |
|
4 |
Block the task command until a decision is made. |
|
5 |
Run the kill command against the zombie’s process. |
ecflow_client --zombie_fob /suite/family/task
ecflow_client --zombie_kill /suite/family/task
Verbatim ecflow_client --help=<action> output (the common environment-variable
footer shown on every page is omitted). Select an action:
zombie_fob
----------
Locates the task in the servers list of zombies, and sets to fob.
This default behaviour of the child commands(label,event,meter) is to fob
Next time the child commands (init,event,meter,label,abort,complete,wait,queue) communicate
with the server, they will complete successfully (but without updating the node tree)
allowing the job to finish.
The references to the zombie in the server is automatically deleted after 1 hour
args = list of task paths, at least one expected
Usage:
--zombie_fob=/path/to/task1 /path/to/task2
zombie_fail
-----------
Locates the task in the servers list of zombies, and sets to fail.
Next time a child command (init,event,meter,label,abort,complete) which matches zombie, communicates with the server, will be set to fail.
Depending on the job setup this may force a abort, the abort will also fail.
Hence job structure should use 'set -e' in the error trapping functions to prevent
infinite recursion.
The references to the zombie in the server is automatically deleted after 1 hour
args = list of task paths, at least one expected
Usage:
--zombie_fail=/path/to/task /path/to/task2
zombie_adopt
------------
Locates the task in the servers list of zombies, and sets to adopt.
Next time a child command (init,event,meter,label,abort,complete,wait queue) communicates with the server, the password on the zombie is adopted by the task.
This is only allowed if the process id matches, otherwise an error is issued.
The zombie reference stored in the server is then deleted.
args = list of task paths, at least one expected
Usage:
--zombie_adopt=/path/to/task /path/to/task2
zombie_remove
-------------
Locates the task in the servers list of zombies, and removes it.
Since a job typically has many child commands (i.e init, complete, event, meter, label, wait, queue)
the zombie may reappear
args = list of task paths, at least one expected
Usage:
--zombie_remove=/path/to/task /path/to/task2
zombie_block
------------
Locates the task in the servers list of zombies, and sets flags to block it.
This is default behaviour of the child commands(init,abort,complete,wait,queue)
when the server cannot match the passwords. Each child commands will continue
attempting to connect to the server for 24 hours, and will then return an error.
The connection timeout can be configured with environment ECF_TIMEOUT
args = list of task paths, at least one expected
Usage:
--zombie_block=/path/to/task /path/to/task2
zombie_kill
-----------
Locates the task in the servers list of zombies, and sets flags to kill
The kill is done using ECF_KILL_CMD, but using the process_id from the zombie
The job is allowed to continue until the kill is received
Can only kill zombies that have an associated Task, hence path zombies
must be killed manually.
arg = list of task paths, at least one expected
Usage:
--zombie_kill=/path/to/task /path/to/task2
--zombie_fob /suite/family (user_action_ 0; other actions differ only in that
value):
{
"21ClientToServerRequest": {
"cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "ZombieCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"value0": {
"cereal_class_version": 0,
"cl_host_": "host.example.com"
},
"user_": "operator"
},
"user_action_": 0,
"process_id_": "",
"password_": "",
"paths_": [ "/suite/family" ]
}
}
}
}
}
The generic success reply (StcCmd with api_ equal to 0):
{
"22ServerToClientResponse": {
"stc_cmd_": {
"polymorphic_id": 2147483649,
"polymorphic_name": "StcCmd",
"ptr_wrapper": {
"id": 2147483649,
"data": {
"cereal_class_version": 0,
"value0": { "cereal_class_version": 0 },
"api_": 0
}
}
}
}
}