View Source

SMS triggers check are from the client side only,

ie under care of the user. It is simple to reach a situation where,

from the server side, triggers are no more valid, and related task remains queued (deadlock).

To prevent that, ecFlow triggers check is more robust:

- triggers have to be consistent from the client side.

- 'extern' can only refer to an external suite's node

- server can be requested to confirm that all triggers are referring to an existing node,

in already loaded suites.

Like the SMS, a ecFlow suite can be used for team work. It is suite design and management skill to decide

how to split the suite for a good collaboration within team members.

At ECMWF, operational suites are defined in files describing the full content.

the definition file is split into different python modules for
- observations retrieval
- analysis and forecast
- products generation and dissemination
- plots generation
then a python script is in charge to put all things together and to create the suite
the user decides to load the full suite, only a sub-tree from the suite or to load it as an e-suite (test suite)
a SCM is used to identify the modules edited by different users, merge their updates, resolve the conflicts or allow them to work in an branched version or the suite for the next release.

The following example shows that with ecflow

a master definition file may provide the suite structure (multi1.def), with the right setting of inlimits, limits, triggers,
a developer in charge of adding test task to the suite may update it with a task test2, below an agreed family (/multi/main/00/test),
another tester may add some other tests as a different suite, triggered from 'multi' suite.

ecFlow > intra-suite extern > ecflowviewmap.png

ecflow_client --load multi1.def
ecflow_client --replace /multi/main/00/test/fc_complete/test2 multi2.def
ecflow_client --replace /test20121123 multi3.def
python TestBench.py multi1.def
python TestBench.py  multi3.def

suite limits
  defstatus complete
  limit power6 120
  limit power7 120
  limit prod   10
  limit diss   10
  limit test   10
endsuite

suite multi

  family main
    inlimit /limits:power6
    repeat date YMD 20120101 20301212

  family 00
    family obs; 
      task get; 
    endfamily
    
    family ana
      trigger obs eq complete
      task ifs
    endfamily

    family fc
      trigger ana eq complete
      task model
        meter step -1 120
    endfamily

    family prod
    inlimit /limits:prod
      family 000; 
    trigger /multi/main/00/fc/model:step gt  0 or /multi/main/00/fc/model eq complete
      task gen; endfamily
      family 012;
    trigger /multi/main/00/fc/model:step gt 12 or /multi/main/00/fc/model eq complete
      task gen; endfamily
      family 024; 
    trigger /multi/main/00/fc/model:step gt 24 or /multi/main/00/fc/model eq complete
      task gen; endfamily
      family 048;
    trigger /multi/main/00/fc/model:step gt 48 or /multi/main/00/fc/model eq complete
      task gen; endfamily
      family 096; 
    trigger /multi/main/00/fc/model:step gt 96 or /multi/main/00/fc/model eq complete
      task gen; endfamily
      family 120; 
    trigger /multi/main/00/fc/model:step gt 120 or /multi/main/00/fc/model eq complete
      task gen; endfamily
    endfamily prod

    family diss
    inlimit /limits:diss
      family 000; 
    trigger /multi/main/00/prod/000 eq complete
      task gen; endfamily
      family 012;
    trigger /multi/main/00/prod/012 eq complete
      task gen; endfamily
      family 024; 
    trigger /multi/main/00/prod/024 eq complete
      task gen; endfamily
      family 048;
    trigger /multi/main/00/prod/048 eq complete
      task gen; endfamily
      family 096; 
    trigger /multi/main/00/prod/096 eq complete
      task gen; endfamily
      family 120; 
    trigger /multi/main/00/prod/120 eq complete
      task gen; endfamily
    endfamily diss

    family test
      inlimit /limits:test

      family fc_complete
    trigger /multi/main/00/fc eq complete
    task test1
      endfamily

      family prod_complete
    trigger /multi/main/00/prod eq complete
    task test1
      endfamily

      family diss
    trigger /multi/main/00/diss eq complete
    task test1
      endfamily
    endfamily 

endsuite

extern /limits:test

suite multi
  defstatus suspended
  family main
  family 00
  family test
  family fc_complete
    task test2

    family fam_test
      trigger test2 eq complete
      task prepare

      task process
        trigger prepare eq complete

      task push
        trigger process eq complete

extern /limits:test
extern /multi/main/00/fc/model:step
extern /multi/main:YMD

suite test20121123
  defstatus suspended # so that it can be changed manually to queued
  repeat date YMD 20120101 20301212
  family test
    repeat integer STEP 0 120
    trigger /multi/main:YMD gt /test20121123:YMD or (/multi/main:YMD eq /test20121123:YMD and (/multi/main/00/fc/model:step ge /test20121123/test:STEP or /multi/main/00/fc/model eq complete))
    task run