At the beginning of the course, the ecFlow server is started in a directory, and tasks/header scripts are created, below the same directory. it will rapidly lead to a situation with many files, in one place (administrative server files, checkpoints files, log-file, white list, tasks wrappers, but also all jobs/output/aliases files).

This may be difficult to maintain or update after some time.

Files Location

For a maintenable operational suite, we recommand to:

Log-Server

It is possible to setup a log-server, to access ‘live’ output from the jobs. ecFlow is provided with the perl script ecflow_logsvr.pl.

Backup Server

it can be useful to have a backup ecFlow server in case of network, disk or host machine crash. The backup server shall be activated on another workstation, with the most recent check-point-file.

The ‘ecf_hostfile’ files can be created with the list of hostnames to contact, when the link with the original ecFlow server is broken.

common task header head.h may be updated with:

export ECF_HOSTFILE=$HOME/.ecf_hostfile

System

As soon as the basic principles of ecFlow are understood and mastered, setting up a project with operational constaints may face several challenges:

Error Handling

A task should abort as close to the problem as possible:

Server Administration

an ‘admin’ suite will be required:

CMD variables

CMD variables shall be set and capable to submit/kill/query a job locally and remotely. They are:

Task Design

most tasks should be re-runnable and they should have an up to date ‘manual section’.

micro

micro character (%) is used as variable delimiter, or to start preprocessing directives (include, manual, end, nopp) in task wrappers.

Python Debugging

Python suite definition files sometimes lead to ‘Memory fault’ message. Error can be understood running it with pdb or gdb:

python -m pdb  <script.py>

gdb python
> set args suite.def
> run
> bt