REST API

Caution

ecFlow’s REST API is experimental, and its details are subject to change. The documentation reflects its current operation.

Compilation

ecFlow REST API is implemented using an HTTP server. The HTTP server compilation is enabled by default, and can be enabled (ON) or disabled (OFF) using CMake option:

-DENABLE_HTTP=ON

The compilation result is an executable called ecflow_http.

By default, the ecFlow HTTP server is built with support for compression (i.e. allows Accept-Encoding: gzip as header in the request), based on the following CMake option:

-DENABLE_HTTP_COMPRESSION=ON

To disable compression set the ENABLE_HTTP_COMPRESSION to OFF. Notice that compression support requires the presence of zlib – if this dependency is not available the CMake project configuration step will fail.

When launched, the server will by default listen to port 8080. This can be changed with command line option -p, --port. Using option -v, --verbose will increase the logging output of the server.

If ecFlow is compiled with openssl (-DENABLE_SSL=ON), the server will automatically try to start in secure mode (SSL). This can be turned off with option --no_ssl.

Note

SSL is a requirement for any state-altering commands.

When running in SSL mode, the server searches for server.crt and server.key in $HOME/.ecflowrc/ssl. For more details see this page.

Note

Even though the REST API requires SSL for any communication including passwords, the ecFlow server has no such requirement ie. ecFlow server can be run without SSL.

Default interval to check any changes from ecFlow server is 10 seconds. This can be changed with option --polling_interval.

Starting the REST API

The HTTP server is started as so:

ecflow_http --verbose

By default the HTTP server expects ecFlow server to be found from location localhost:3141. This can be changed by setting environment variables ECF_HOST and ECF_PORT before starting the HTTP server.

Command Line Options and Environment Variables

REST API options can be controlled either with command line options or with environment variables. All environment variables with “RESTAPI” are new.

Command line option	Environment variable	Default Value	Description
–http		false	Use HTTP to contact the ecFlow server
–cert_directory	ECF_RESTAPI_CERT_DIRECTORY	$HOME/.ecflowrc/ssl	Directory where SSL certificates (server.crt and server.key) are found
–ecflow_host	ECF_HOST	localhost	ecFlow server hostname
–ecflow_port	ECF_PORT	3141	ecFlow server port
–max_polling_interval	ECF_RESTAPI_MAX_UPDATE_INTERVAL	300 seconds	Maximum interval between ecFlow server updates, set to 0 to disable drift
–no_ssl	ECF_RESTAPI_NOSSL	false	Disable SSL
–polling_interval	ECF_RESTAPI_POLLING_INTERVAL	10 seconds	Interval between ecFlow server updates
–port,-p	ECF_RESTAPI_PORT	8080	REST API port
–tokens_file	ECF_RESTAPI_TOKENS_FILE	api-tokens.json	Location of file where valid tokens (API keys) are found
–verbose,-v	ECF_RESTAPI_VERBOSE	false	Enable verbose mode

REST API uses the existing ecFlow environment variables to determine ecFlow server location:

ECF_HOST: ecFlow server address
ECF_PORT: ecFlow server port

Two new environment variables are introduced:

ECF_RESTAPI_TOKEN_FILE: Location of file where API token hashes are found
ECF_RESTAPI_CERT_DIRECTORY: Directory where server.crt and server.key files are located. If not specified, $HOME/.ecflowrc/ssl is used.

Authentication

For HTTP methods POST, PUT and DELETE, authentication is always required for user commands. Authentication requires that REST API is being run with SSL (the connection between API and ecFlow server isn’t required to be encrypted).

If ecFlow server and REST API are being run on the same server, GET operations can be run without authentication (and without SSL). Similarly, child commands can be run without authentication and SSL, as they are not authentication (in the traditional sense).

Command type	HTTP method	Authentication (SSL) required
user	POST,PUT,DELETE	yes
user	GET	no
child	PUT,GET	no

Basic Authentication

Basic authentication is done with username and password. ecFlow server acts as the authenticator, REST API is only passing the credentials forward.

For instructions how the set the ecFlow server authentication, see Black list file.

The username and password are passed to REST API using standard HTTP basic authentication mechanism. For example, with curl option --user.

Token Based Authentication

REST api supports simple token-basic authentication. In this setting the api will verify a token’s validity and grant access to ecFlow server if token is valid. ecFlow server itself does not know about the token. REST api acts as a proxy for the user.

The token should be passed to the api with http header in a standard fashion:

Authorization: Bearer <TOKEN>

Or with a custom header:

X-API-Key: <TOKEN>

The token itself is just an random alphanumerical string that does not contain any information in itself. REST API does not support JWT’s.

To authenticate the token, the API needs to have a local database of valid tokens. Currently the only supported database backend is file in json format. The API will search the token from current working directory with name “api-tokens.json”. Location can be changed with environment variable ECF_API_TOKEN_FILE. The API will automatically check the file for changes every 20 seconds.

The contents of the json file are:

[
  {
    "hash": "...",
    "description": "...",
    "expires_at": "yyyy-mm-ddTHH:MM:SSZ",
    "revoked_at": "yyyy-mm-ddTHH:MM:SSZ"
  }
]

And the field values are:

hash : hashed and salted token, format is identical to python library ‘werkzeug’: METHOD$SALT$HASH
description: a free-form description of token (application)
expires_at: iso8601 timestamp when this token will expire in UTC time, OPTIONAL: if missing or zero length, no expiration time is set
revoked_at: iso8601 timestamp when this token was revoked in UTC time, OPTIONAL: if missing or zero length, no revoke time is set

Currently supported hashing algorithms are:

sha256 (hmac)
pbkdf2 sha256

A token file can be created with a simple script:

> cat create-token-file.sh
set -eu

desc=$1
pw=$2
salt=$(openssl rand -hex 8)
method=sha256
hash=$(echo -n $pw | openssl sha256 -hmac "$salt")
expires=$(gdate +"%Y-%m-%dT%H:%M:%SZ" -d 'tomorrow') # token expires in 24 hrs

jq --null-input \
  --arg hash "$method\$$salt\$$hash" \
  --arg desc "$desc" \
  --arg exp "$expires" \
  '[{ "hash" : $hash, "description" : $desc, "expires_at" : $exp }]'

Run the script for application “my app” using token “myrandomtokenstring”:

> sh create-token-file.sh "my app" myrandomtokenstring
[
  {
    "hash": "sha256$75f838a880872d20$ca8391ae4e3dc53d68befac3ab0f6f6c13ad2a770fc1e06fb7a7fba87169f21d",
    "description": "my app",
    "expires_at": "2022-10-07T11:23:02Z"
  }
]

API v1 Documentation

The API supports operations using GET, POST, PUT and DELETE methods. Generally the last word of the URL defines the target of the query. For example, https://localhost/v1/suites. There are seven different supported targets:

attributes
definition
output
ping
script
status
suites
tree

A short description of the targets.

API Targets

attributes

Attributes are properties of a node. Supported REST methods are: GET, POST, PUT, DELETE.

definition

Definition is the definition of an ecflow suite or a part of it in the ecflow domain specific language. Supported REST methods are: GET, PUT.

output

Output target is used to retrieve the task output. Supported REST methods are: GET

ping

Pings the ecflow server from the REST api. Supported REST methods are: GET.

script

Scripts are the files that ecflow executes when running tasks. It is possible to view a script content. Supported REST methods are: GET.

suites

This target is used to either list all current suites or to create a new suite. Supported REST methods are: GET, POST.

status

This target is used to access the runtime status of a node. Supported REST methods are: GET, PUT.

tree

This is a special target that shows a tree-view of the ecflow node structure. Supported REST methods are: GET.

API v1 supports the following endpoints – specification for each endpoing can be found in the related subsections.

Table 4 List of Endpoints
Endpoint	Method(s)	Description
/v1/suites	POST, GET	Operations related to Suites
/v1/suites/tree	GET	Operations related to Suite trees
/v1/suites/{path}	DELETE	Operations related to Nodes
/v1/suites/{path}/tree	GET	Operations related to Node trees
/v1/suites/{path}/definition	GET, PUT, DELETE	Operations related to Node definitions
/v1/suites/{path}/status	GET, PUT	Operations related to Node status
/v1/suites/{path}/attributes	POST, GET, PUT, DELETE	Operations related to Node attributes
/v1/suites/{path}/script	GET	Obtain Node script
/v1/suites/{path}/output	GET	Obtain Node job output
/v1/server/status	GET, PUT	Operations related to server status
/v1/server/attributes	POST, GET, PUT, DELETE	Operations related to server attributes
/v1/server/ping	GET	Ping server
/v1/statistics	GET	Obtain REST API statistics

Note

All API v1 endpoints allow the Query Parameters.

Name	Value	Comment
filter	a.b.c[0]	filter returned json
key	abcdf	API key (token), if client is unable to pass the key with HTTP headers

Endpoint `/v1/suites`

Create a new Suite

Endpoint	`/v1/suites`
Method	`POST`
Description	Create a new suite
Parameters	none
Payload	See below for details of the payload used to create a new suite.
Response	`{"ok"}`
Example	`curl -X POST https://localhost:8080/v1/suites -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"definition": "suite test2n family an task an endfamilynendsuite"}'`

Payload to create a new Suite

{
  "definition": "...",
  "auto_add_extern": "true|false"
}

where

definition is the ecFlow node definition
auto_add_extern indicates whether to automatically add external triggers

List all Suites

Endpoint	`/v1/suites`
Method	`GET`
Description	Read the list with all suites
Parameters	none
Payload	empty
Response	`["a", "b", "c"]`

Endpoint `/v1/suites/tree`

Obtain the tree of all Suites

Endpoint	`/v1/suites/tree`
Method	`GET`
Description	Read a tree with all suites
Parameters	`content`, (optional), possible values: `basic`, `full` `with_id`, (optional), possible values: `true`, `false` `gen_vars`, (optional), possible values: `true`
Payload	empty
Response	See below for details.

Response with Suite tree

When query parameter content is not provided, or query parameter is content=basic, the basic Suite tree is provided:

{
  "suite": {
    "family": {
      "task": ""
      }
  }
}

When query parameter content=full is used, the full Suite tree is provided:

{
  "some_suite": {
    "type": "suite",
    "state": {
      "node": "active",
      "default": "complete"
    },
    "attributes": [
      {
        "name": "YMD",
        "value": "20100114",
        "const": false,
        "type": "variable"
      }
    ],
    "children": {
      "some_family": {
        "type": "family",
        "state": {
          "node": "active",
          "default": "unknown"
        },
        "attributes": [],
        "children": {
          "some_task": {
            "type": "task",
            "state": {
              "node": "active",
              "default": "unknown"
            },
            "attributes": [
              {
                "name": "some_label",
                "value": "value",
                "type": "label"
              },
              {
                "name": "some_meter",
                "min": 0,
                "max": 30,
                "value": 0,
                "type": "meter"
              },
              {
                "name": "some_event",
                "value": false,
                "initial_value": false,
                "type": "event"
              }
            ],
            "aliases": {}
          }
        }
      }
    }
  }
}

When query parameters content=full&with_id=true are used, the full Suite tree with ids is provided:

{
  "some_suite": {
    "type": "suite",
    "id": "/some_suite",
    "state": {
      "node": "active",
      "default": "complete"
    },
    "attributes": [
      {
        "name": "YMD",
        "value": "20100114",
        "const": false,
        "type": "variable"
      }
    ],
    "children": {
      "some_family": {
        "type": "family",
        "id": "/some_suite/some_family",
        "state": {
          "node": "active",
          "default": "unknown"
        },
        "attributes": [],
        "children": {
          "some_task": {
            "type": "task",
            "id": "/some_suite/some_family/some_task",
            "state": {
              "node": "active",
              "default": "unknown"
            },
            "attributes": [
              {
                "name": "some_label",
                "value": "value",
                "type": "label"
              },
              {
                "name": "some_meter",
                "min": 0,
                "max": 30,
                "value": 0,
                "type": "meter"
              },
              {
                "name": "some_event",
                "value": false,
                "initial_value": false,
                "type": "event"
              }
            ],
            "aliases": {}
          }
        }
      }
    }
  }
}

When using the query parameter gen_vars=true the generated variables are included in the attributes section. Generated variables can be identified by a generate attribute set to true.

Endpoint `/v1/suites/{path}`

Delete a Node

Endpoint	`/v1/suites/{path}`
Method	`DELETE`
Description	Delete the node at `/{path}`
Parameters	none
Payload	empty
Response	`{"ok"}`

Endpoint `/v1/suites/{path}/tree`

Obtain the tree of a Node

Endpoint	`/v1/suites/{path}/tree`
Method	`GET`
Description	Read a tree view of the node at `/{path}`
Parameters	`content`, (optional), possible values: `basic`, `full` `with_id`, (optional), possible values: `true`, `false` `gen_vars`, (optional), possible values: `true`
Payload	empty
Response	Same as Suite tree.

Endpoint `/v1/suites/{path}/definition`

Obtain the definition of a Node

Endpoint	`/v1/suites/{path}/definition`
Method	`GET`
Description	Read the definition of the node at `/{path}`
Parameters	none
Payload	empty
Response	`{"definition": "<defs>"}`, where `<defs>` is the content of associated .def file

Update the definition of a Node

Endpoint	`/v1/suites/{path}/definition`
Method	`PUT`
Description	Update the definition of the node at `/{path}`
Parameters	none
Payload	See below for details of the payload used to update the definition of an existing node.
Response	`{"message": "Node updated successfully"}`

Payload to update the definition of a Node

{
  "definition": "...",
  "auto_add_extern": "true|false"
}

where

definition is the ecFlow node definition
auto_add_extern indicates whether to automatically add external triggers

Delete a Node

Endpoint	`/v1/suites/{path}/definition`
Method	`DELETE`
Description	Delete the node at `/{path}`
Parameters	none
Payload	empty
Response	`{"message": "Node deleted successfully"}`

Endpoint `/v1/suites/{path}/status`

Obtain a Node status

Endpoint	`/v1/suites/{path}/status`
Method	`GET`
Description	Read the status of the node at `/{path}`
Parameters	none
Payload	empty
Response	`{"status":"aborted"}`
Example	`curl https://localhost:8080/v1/suites/path/to/node/status` `curl https://.../v1/suites/path/to/node/status?filter=default_status`

Update a Node status

Endpoint	`/v1/suites/{path}/status`
Method	`PUT`
Description	Update the status of the node at `/{path}`
Parameters	none
Payload	empty
Response	`{"status":"..."}`

Payload to update Node status (by User)

When updating node status with a user command, with user authentication, the request payload is as follows:

{
  "action": "abort|begin|complete|defstatus|execute|requeue|rerun|resume|submit|suspend",
  "recursive": false
}

where

name: Name of the action that is taken against the given path
recursive: Specify if same action is run recursive through the children of the node. Note: not all actions support recursive operation. Default: false

When action=defstatus, the following additional options are necessary:

{
  ...
  "name": "defstatus",
  "defstatus_value": "aborted|complete|queued|suspended|unknown"
}

Payload to update Node status (by Task)

When updating the node status from a task (i.e. from a script with child command authentication), the request payload is as follows:

{
  "ECF_NAME": "...",
  "ECF_PASS": "...",
  "ECF_RID": "...",
  "ECF_TRYNO": "...",
  "action": "abort|complete|init|wait"
}

where

action is the name of the action to be taken taken
ECF_NAME, ECF_PASS, ECF_RID, and ECF_TRYNO are ecFlow generated parameters

When action=abort, the followign additional parameters are necessary:

{
  ...
  "name": "abort",
  "abort_why": "..."
}

When action=wait, the followign additional parameters are necessary:

{
  ...
  "name": "wait",
  "wait_expression": "..."
}

Endpoint `/v1/suites/{path}/attributes`

Create a new Node attribute

Endpoint	`/v1/suites/{path}/attributes`
Method	`POST`
Description	Create new attribute for the node at `/{path}`
Parameters	none
Payload	`{"type":"...","name":"...","value":"..."}`
Response	`{"message": "Attribute added successfully"}`

Obtain all Node attributes

Endpoint	`/v1/suites/{path}/attributes`
Method	`GET`
Description	Read the attributes of the node at `/{path}`
Parameters	none
Payload	empty
Response	See below for details
Example	`curl https://localhost:8080/v1/suites/path/to/node/attributes`

Response with Node attributes

The response to a request for node attributes has the following JSON format. Notice how the node variables are separated from inherited variables. The inherited variables grouped by ancestor node, each identified by the node name and absolute path.

{
  "path": "..."

  "meters": [ { ... }, ... ],
  "limits": [ { ... }, ... ],
  "inlimits": [ { ... }, ... ],
  "events": [ { ... }, ... ],
  "labels": [ { ... }, ... ],
  "dates": [ { ... }, ... ],
  "days": [ { ... }, ... ],
  "crons": [ { ... }, ... ],
  "times": [ { ... }, ... ],
  "todays": [ { ... }, ... ],
  "repeat": { ... },
  "trigger": { ... },
  "complete": { ... },
  "flag": { ... },
  "late": { ... },
  "zombies": [ { ... }, ... ],
  "generics": [ { ... }, ... ],
  "queues": [ { ... }, ... ],
  "autocancel": { ... },
  "autoarchive": { ... },
  "autorestore": { ... },
  "avisos": [ { ... }, ... ],
  "mirrors" : [ { ... }, ... ],
  "variables": [
    {
      "name": "..."
      "value": "...",
      "const": true|false
    },
    {
      "name": "..."
      "value": "...",
      "generated": true        # only present if the variable is generated
      "const": true|false
    },
    ...
  ]
  "inherited_variables": [
    {
      "name": "...<node-name>..."
      "path": "...<node-path>...",
      "variables: [
        {
          "name": "..."
          "value": "...",
          "type": "variable",
          "const": true|false
        },
        {
          "name": "..."
          "value": "...",
          "type": "variable",
          "generated": true    # only present if the variable is generated
          "const": true|false
        },
        ...
      ]
    },
    ...
  }
}

Update a Node attribute

Endpoint	`/v1/suites/{path}/attributes`
Method	`PUT`
Description	Update the attribute of the node at `/{path}`
Parameters	none
Payload	`{"type":"...","name":"...","value":"..."}`
Response	`{"message":"Attribute changed successfully"}`

Payload to update Node Attribute (by User)

When updating a node attribute using a user command, with user authentication, the request payload is as follows:

{
  "name": "...",
  "type": "event|generic|inlimit|label|late|limit|meter|queue|time|today|trigger|variable",
  "value": "...",
  "old_value": "...",
  "min": "...",
  "max": "..."
}

where

name is the name of the attribute
type is the type of the attribute
value is the value of the attribute, after the operation is complete. This is not necessary when type is delete.
old_value is used when updating attributes that do not have a name. In this case, old_value specifies which attribute(s) are changed.

When type is event, the value must be:

true or set, if the event is to be set
false or clear, if the event is to be cleared

When type is limit, the following keys must be

value, if the value of the limit is to be updated
max, if the upper limit of the limit is to be updated

When type is meter, the following keys are necessary:

value
min
max

When type is today or time, the following keys are necessary:

value when setting the new value
old_value to specify the (possible multiple) attributes to be updated
name is not required

For unnamed attributes, such as repeat or late, consider that

name is not required

Payload to update Node attribute (by Task)

When updating a node attribute from a task (i.e. from a script with child command authentication), the request payload is as follows:

{
  "ECF_NAME": "...",
  "ECF_PASS": "...",
  "ECF_RID": "...",
  "ECF_TRYNO": "...",
  "name": "...",
  "type": "event|label|limit|meter|queue",
  "value": "...",
  "queue_action": "...",
  "queue_step": "...",
  "queue_path": "..."
}

where

name is the name of the attribute
ECF_NAME, ECF_PASS, ECF_RID, and ECF_TRYNO are ecFlow generated parameters

When updating queue attributes, the parameter value is not used, and the following additional parameters are required:

{
  ...
  "queue_action": "...",
  "queue_step": "...",
  "queue_path": "..."
}

The queue_step and queue_path parameters are optional. The parameter queue_step should only be provided when the queue_action is either complete or aborted. If the field queue_path is provided the server will search for the queue only on the named node, otherwise, the absense of the queue_path parameter triggers the upward search of the queue starting from the target node (i.e. the node named by ECF_NAME) through the node tree.

Response to update Node attribute (by Task)

The typical response to updating a node attribute has the following format

{
  "message": "..."
}

However, when updating an attribude of type queue the response body has additional information. When the requesting queue_action is active, the response will contain

{
  "message": "...",
  "step": "<selected-step>"
}

and when the requesting queue_action is no_of_aborted, the response will contain

{
  "message": "...",
  "no_of_aborted": "<current-number-of-queued-or-aborted-steps>"
}

Delete a Node attribute

Endpoint	`/v1/suites/{path}/attributes`
Method	`DELETE`
Description	Delete the attribute of the node at `/{path}`
Parameters	none
Payload	`{"type":"...","name":"..."}`
Response	`{"message":""Attribute deleted successfully"}`

Endpoint `/v1/suites/{path}/script`

Obtain the Node script

Endpoint	`/v1/suites/{path}/script`
Method	`GET`
Description	Read the script associated to the node at `/{path}`
Parameters	none
Payload	empty
Response	`{"script": "..."}`
Example	`curl https://localhost:8080/v1/suites/path/to/node/script`

Update the Node script

Endpoint	`/v1/suites/{path}/script`
Method	`PUT`
Description	Update the script associated to the node at `/{path}`
Parameters	none
Payload	`{"script": "..."}`
Response	`{"message": "Script updated successfully"}`

Payload to update Node script

Warning

A script can only be updated through the REST API if it already exists in the server.

{
  "script" : "..."
}

Endpoint `/v1/suites/{path}/output`

Obtain the Node job output

Endpoint	`/v1/suites/{path}/output`
Method	`GET`
Description	Read the job output associated to the node at `/{path}`
Parameters	none
Payload	empty
Response	`{"job_output": "..."}`
Example	`curl https://localhost:8080/v1/suites/path/to/node/output`

Endpoint `/v1/server/status`

Obtain the Server status

Endpoint	`/v1/server/status`
Method	`GET`
Description	Read the server status information
Parameters	none
Payload	empty
Response	`{"statistics":{...}}`
Example	`curl https://localhost:8080/v1/server/status`

Update the Server status

Endpoint	`/v1/server/status`
Method	`PUT`
Description	Update the server status (i.e. reload configuration)
Parameters	none
Payload	`{"action":"..."}`
Response	`{"message":"Server updated successfully"}`

Payload to update the Server Status

{
  "action" : "reload_whitelist_file|reload_passwd_file|reload_custom_passwd_file"
}

Endpoint `/v1/server/attributes`

Create a new Server attribute

Endpoint	`/v1/server/attributes`
Method	`POST`
Description	Create a new server attribute
Parameters	none
Payload	`{"type":"variable","name":"...","value":".."}`
Response	`{"message":"Attribute added successfully"}`

Obtain all Server attributes

Endpoint	`/v1/server/attributes`
Method	`GET`
Description	Read the server attributes
Parameters	none
Payload	empty
Response	`{"variables":[...]}`
Example	`curl https://localhost:8080/v1/server/attributes`

Update a Server attribute

Endpoint	`/v1/server/attributes`
Method	`PUT`
Description	Update a server attribute
Parameters	none
Payload	`{"type":"variable","name":"...","value":"..."}`
Response	`{"message":"Attribute changed successfully"}`

Delete a Server attribute

Endpoint	`/v1/server/attributes`
Method	`DELETE`
Description	Delete a server attribute
Parameters	none
Payload	`{"type":"variable","name":"..."}`
Response	`{"message":"Attribute deleted successfully"}`

Endpoint `/v1/server/ping`

Ping Server

Endpoint	`/v1/server/ping`
Method	`GET`
Description	Read the rount-trip-time between ecflow_http and ecflow_server
Parameters	none
Payload	empty
Response	`{"host":"...","round-trip-time":"..."}`
Example	`curl https://localhost:8080/v1/server/ping`

Endpoint `/v1/statistics`

Obtain Server statistics

Endpoint	`/v1/statistics`
Method	`GET`
Description	Read the ecflow_http statistics
Parameters	none
Payload	empty
Response	`{"num_requests":"...","num_errors":"..."}`
Example	`curl https://localhost:8080/v1/statistics`

OpenAPI Specification

The OpenAPI specification file is available at openapi.yaml.

Note

The OpenAPI specification allows the graphical visualisation of the supported endpoints, using Swagger UI.

To run Swagger UI in a container, use the following Docker file:

FROM swaggerapi/swagger-ui
ADD openapi.yaml /tmp
ENV SWAGGER_JSON=/tmp/openapi.yaml

More Examples

Authentication options

curl [...] https://localhost:8080/v1/suites -H 'authorization: Bearer <MYTOKEN>'
curl [...] https://localhost:8080/v1/suites -H 'x-api-key: <MYTOKEN>'
curl [...] https://localhost:8080/v1/suites?key=<MYTOKEN>

Create a new family

curl -X PUT https://localhost:8080/v1/suites/test -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"definition": "family b\nendfamily\n"}'

Create a new attribute

curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"autoarchive","value":"+01:00"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"autocancel","value":"+01:00"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"autorestore","value":"/test/a"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"complete","value":"/test/a eq complete"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"cron","value":"-w 0,1 10:00"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"date","value":"1.*.*"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"day","value":"monday"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"event","name":"foo","value":"set"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"label","name":"foo","value":"bar"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"late","value":"-s +00:01 -a 14:30 -c +00:01"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"limit","name":"foo","value":"0"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"meter","name":"foo","value":"10","min":"0","max":"20"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"time","value":"+00:20"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"today","value":"03:00"}'
curl -X POST https://localhost:8080/v1/suites/test/dynamic/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"variable","name":"foo","value":"bar"}'

Update an attribute value

curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"autoarchive","value":"0"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"autocancel","value":"0"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"autorestore","value":"/test"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"complete","value":"/test/a eq active"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"cron","old_value":"-w 0,1 10:00","value":"23:00"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"date","old_value":"1.*.*","value":"2.*.*"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"day","old_value":"monday","value":"tuesday"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"event","name":"foo","value":false}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"label","name":"foo","value":"baz"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"late","old_value":"-s +00:01 -a 14:30 -c +00:01","value":"-c +00:01"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"limit","name":"foo","value":"6"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"meter","name":"foo","value":"15"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"time","old_value":"+00:20","value":"+00:25"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"today","old_value":"03:00","value":"03:00 05:00 01:00"}'
curl -X PUT https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"variable","name":"foo","value":"baz"}'

Update status

curl -X PUT https://localhost:8080/v1/suites/test/status -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"action":"complete"}'
curl -X PUT https://localhost:8080/v1/suites/test/status -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"action":"requeue"}'

Delete an attribute

curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"autoarchive"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"autocancel"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"autorestore"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"complete","value":"/test/a eq active"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"cron","value":"23:00"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"date","value":"2.*.*"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"day","value":"tuesday"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"event","name":"foo"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"label","name":"foo"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"late","value":"-c +00:01"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"limit","name":"foo"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"meter","name":"foo"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"time","value":"+00:25"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"today","value":"03:00 05:00 01:00"}'
curl -X DELETE https://localhost:8080/v1/suites/test/attributes -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>' -d '{"type":"variable","name":"foo"}'

Delete a suite

curl -X DELETE https://localhost:8080/v1/suites/test/definition -H 'content-type: application/json' -H 'authorization: Bearer <MYTOKEN>'

Implementation Details

ecflow_http is basically a wrapper that transforms requests in web-syntax to ecflow syntax, and similarly transforms the results from plain-text to valid json.

ecflow_http is internally using the normal ClientInvoker method to communicate with the server. From the ecFlow servers’ point of view the API is just another client.

ecflow_http can done some things that the command line tool ecflow_client cannot, mostly to enable adding attributes to existing suites. ecflow_client can do to this to some attributes, but the API has a broader support. The API also caches the server state and updates it only in certain configurable intervals.

ecflow_http will keep a cached copy of definitions in its memory. The copy is updated by default every 10 seconds (adjustable with a command line option). This means that when issuing a GET query to API, it will touch the cached copy of definitions and no connection to ecFlow server is made. There are some exceptions to this: when querying output, script, server ping, server status a connections to ecFlow server is opened. Also all altering commands PUT, POST and DELETE result in a connection to ecFlow server.

Method	Endpoint	Will result into a connection to ecFlow server
GET	v1/suites/…/output	YES
GET	v1/suites/…/script	YES
GET	v1/server/ping	YES
GET	v1/server/status	YES
POST	Any	YES
PUT	Any	YES
DELETE	Any	YES
GET	Anything else	NO

ecflow_http uses two external libraries, both libraries are header only and licensed with MIT license:

cpp-httplib: provides http server implementation.
nlohmann/json: provides json encoding/decoding functions

ecflow_http supports the usual REST API versioning, meaning that the current version is “v1” and that version number is a part of the URL. ecflow_http can support multiple different version side-by-side. The v1 code is basically split into two files: ApiV1.hpp/cpp, and ApiV1Impl.hpp/cpp. The first one registers the endpoints used to the HTTP server and deals with all the HTTP specific things. The latter (ApiV1Impl) contains all business logic: contacting ecflow server and formulating requests/responses.

Compiled successfully with following compilers (CMAKE_BUILD_TYPE=Debug):

gnu
- 8.5
- 9.2
- 10.3
- 11.2
- 12.0
clang
- 11.1
- 13.0
- 14.0
- 15.0

Update Interval Drift

By default ecflow_http will increase the update interval length for ecFlow server if the ecflow_http server is inactive. This is called drift.

For every one minutes that goes by without requests from users, the update interval (given with --polling_interval, default value 10 seconds) is increased linearly by one second. The default maximum value is 300 seconds. Whenever ecflow_http receives a request from user, the update interval value is reset to normal value.

The maximum polling interval can changed with command line option --max_polling_interval. If drift is enabled, the minimum value is hard coded to 30 seconds.

REST API

Compilation

Starting the REST API

Command Line Options and Environment Variables

Authentication

Basic Authentication

Token Based Authentication

API v1 Documentation

API Targets

attributes

definition

output

ping

script

suites

status

tree

Endpoint /v1/suites

Create a new Suite

Payload to create a new Suite

List all Suites

Endpoint /v1/suites/tree

Obtain the tree of all Suites

Response with Suite tree

Endpoint /v1/suites/{path}

Delete a Node

Endpoint /v1/suites/{path}/tree

Obtain the tree of a Node

Endpoint /v1/suites/{path}/definition

Obtain the definition of a Node

Update the definition of a Node

Payload to update the definition of a Node

Delete a Node

Endpoint /v1/suites/{path}/status

Obtain a Node status

Update a Node status

Payload to update Node status (by User)

Payload to update Node status (by Task)

Endpoint /v1/suites/{path}/attributes

Create a new Node attribute

Obtain all Node attributes

Response with Node attributes

Update a Node attribute

Payload to update Node Attribute (by User)

Payload to update Node attribute (by Task)

Response to update Node attribute (by Task)

Delete a Node attribute

Endpoint /v1/suites/{path}/script

Obtain the Node script

Update the Node script

Payload to update Node script

Endpoint /v1/suites/{path}/output

Obtain the Node job output

Endpoint /v1/server/status

Obtain the Server status

Update the Server status

Payload to update the Server Status

Endpoint /v1/server/attributes

Create a new Server attribute

Obtain all Server attributes

Update a Server attribute

Delete a Server attribute

Endpoint /v1/server/ping

Ping Server

Endpoint /v1/statistics

Obtain Server statistics

OpenAPI Specification

More Examples

Authentication options

Create a new family

Create a new attribute

Update an attribute value

Update status

Delete an attribute

Delete a suite

Implementation Details

Update Interval Drift

Endpoint `/v1/suites`

Endpoint `/v1/suites/tree`

Endpoint `/v1/suites/{path}`

Endpoint `/v1/suites/{path}/tree`

Endpoint `/v1/suites/{path}/definition`

Endpoint `/v1/suites/{path}/status`

Endpoint `/v1/suites/{path}/attributes`

Endpoint `/v1/suites/{path}/script`

Endpoint `/v1/suites/{path}/output`

Endpoint `/v1/server/status`

Endpoint `/v1/server/attributes`

Endpoint `/v1/server/ping`

Endpoint `/v1/statistics`