Page History

 Horizontal Navigation Bar

 Button Group
 Button Hyperlink
titlePrevious
typestandard
urlhttps://confluence.ecmwf.int/display/ECFLOW/Add+a+meter
 Button Hyperlink
titleUp
typestandard
urlhttps://confluence.ecmwf.int/display/ECFLOW/Going+Further
 Button Hyperlink
titleNext
typestandard
urlhttps://confluence.ecmwf.int/display/ECFLOW/Time+Triggers

Sometimes you want a task to run at a given time, or to run every three hours,
or to run only on the first of the month, or on Mondays,...
For that ecFlow supports date and time dependencies.

...

time

time dependencies can be absolute, i.e. they will run at the exact time.
They can also be relative; in this case, we provide the time from the moment the suite is begun.
Time dependencies can be repeated at regular intervals.  The nodes stay complete once all-time instances have run.

 Code Block
time 23:00                  # at next 23:00
time 10:00 20:00 01:00      # every hour from 10am to 8pm
time +00:01                 # one minute after the suite has begun, or 1 minute after re-queue in a presence of a repeat
time +00:10 01:00 00:05     # 10 to 60 minutes after begin every 5 minutes

In the last example, we have a task that runs every five minutes, however, what happens if the task takes longer?

When this happens, the time slot is missed.

cron

Cron dependencies can be specified using the cron keyword. Cron differs from time as when the node is complete it queues again immediately. Cron also only works with a real time clock (not a hybrid clock).

Code Block
cron 23:00 # every day at 23:00 cron 08:00 12:00 01:00 # every hour between 8 and 12 cron -w 0,

is missed

2    11:00       # every sunday and tuesday at 11 am
cron -d 1,15   02:00       # every 1st and 15th of each month at 2 am
cron -m 1 -d 1 14:00       # every first of January at 2 pm
cron -w 5L 23:00           # run on *last* Friday(5L) of each month at 23pm
cron -d 1,L  23:00         # Run on the first and last of the month at 23pm

Info

title	Time,Today,Cron

When the time has expired, the associated node is free to run. The time will stay expired until the node is re-queued.

date or day

Date dependencies can be specified using the date or the day keyword.

Date dependencies are always absolute, but wild cards can be used.

Code Block

date 31.12.2012             # the 31st of December 2012
date 01.*.*                 # every first of the month
date *.10.*                 # every day in October
date 1.*.2008               # every first of the month, but only in 2008
day monday                  # every monday

cron

Cron dependencies can be specified using the cron keyword:

cron 23:00 # every day at 23:00 cron 08:00 12:00 01:00 # every hour between 8 and 12 cron -w 0,2

Mixing time dependencies on the same node

A task can have many time and date dependencies. For example:

Code Block
task tt day monday # Here Day/date acts like a guard over the time. i.e. time is not considered until Monday time 10:00 # run on Monday at 10 am

Code Block
task tt day sunday #

every

On

sunday

the

and tuesday cron -d 1,15 # every 1st and 15th of each month cron -m 1 -d 1

same node, Day/date act like a guard over the time attributes.
   day wednesday
   date 01.*.*                 # The first of

#

every

first of January

A task can have several time and date dependencies. For example:

day sunday day wednesday date 01

month and year
   date 10.*.*                 # The

first

tenth of every month and year

date 10.*.*

   time 01:00                  # The

tenth

time is only set free *if* we are on one of

every month and year time 01:00

the day/dates
   time 16:00

The task will run on sunday’s Sunday’s and wednesday’s Wednesday’s at 1am 1 am and 4pm4 pm, but only if the day is the 1st or the 10th of the month.

Info
With multiple time dependencies on the same node, the dependencies of the same type are or' ed together, then and' ed with the different types.

Mixing time dependencies on different nodes

When time dependencies are placed on different nodes in the hierarchy, the results may seem surprising. Notice the difference between ecflow 4 and ecflow 5

Code Block

title	ecflow 4

family fam
   day monday    # The day attribute no longer guards the time attribute.
   task tt
      time 10:00 # runs on Monday morning at 00:00 ?, and Monday at 10 am

Code Block

title	ecflow 5

family fam
   day monday    # The day STILL guards the time attribute.
   task tt
      time 10:00 # Will run on Monday at 10 am

Code Block
family fam2 time 10:00 task tt day monday # This will run on Monday morning at 00:00 and Monday at 10 am

The example above assumes we have a suite, with an infinite repeat loop. So why does the task run on Monday morning at 00:00?

This is because time dependencies on different nodes act independently of each other. In this case, the time attribute was set free on Sunday at 10 am ( and once free it stays free until it is re-queued). Hence task tt is free to run on Monday morning. After the task has run and re-queued. It will then run on Monday at 10 am.

Like trigger‘s, date and time dependencies can be set for a family. In this case, the tasks of this family will only run according to these dependencies.

Note

All time-related dependencies(like cron, time, today, date and day) are relative to the clock of the suite.

For more information, see Dates and Clocks

Text

Let us modify the definition file to add a family f2.
For brevity, we have omitted the previous family f1

 Code Block
# Definition of the suite test
suite test
 edit ECF_INCLUDE "$HOME/course"  # replace '$HOME' with the path to your home directory
 edit ECF_HOME    "$HOME/course"

 family f2
     edit SLEEP 20
     task t1
         time 00:30 23:30 00:30  # start(hh:mm) end(hh:mm) increment(hh:mm)
     task t2
         day 

sunday

thursday
         time 13:00
     task t3
         date

01

1.*.*              # Date(day,month,year) - * means every day,month,year
         time 12:00              # Time is not considered until date is free
     task t4
         time +00:02.            # + means realative to suite begin/requeue time
     task t5
         time 00:02              # 2 minutes past midnight
 endfamily
endsuite

Python

For brevity, we have left out family f1. In python this would be:

#!/usr/bin/env python2.7

Code Block

language	py
title	$HOME/course/test.py

import os

import

from ecflow import Defs,Suite,Family,Task,Edit,Trigger,Complete,Event,Meter,Time,Day,Date,Edit

def create_family_f2():

f2 =

return

ecflow.

Family("f2"

)

f2.add_variable("SLEEP", 20)

      Edit(SLEEP=20),

f2.add_task

Task("t1"

).add_time(

, Time("00:30 23:30 00:30"

)

)),     # start(hh:mm) end(hh:mm) increment(hh:mm)

f2.add_task("t2").add_day(

"sunday"

)

# for add_date(): day,month,year; here 0 means every month, and every year

Task("t2", Day("thursday"),Time("13:00")),

t3

=

f2.add_task

Task("t3"

) t3.add_date(1, 0, 0)

, Date("1.*.*"), Time("12:00")),  # Date(day

,month,year)

year,

first

of

means every

month or every

day,month,year

t3.add_time(

12,

0

)

# hour, minutes at 12 o'clock f2.add_task("t4").add_time( 0, 2, True ) # hour, minutes, relative to suite start

Task("t4", Time("+00:02")),                # + means realative to suite begin/requeue time

Task("t5", Time("00:02")))

# 2 minutes

after family f2 start f2.add_task("t5").add_time( 0, 2 ) # hour, minutes suite site

past midnight
 
print("Creating suite definition")  
home = os.path.join(os.getenv("HOME"), "course")
defs = Defs( 
        Suite("test",
            Edit(ECF_INCLUDE=home,ECF_HOME=home),

# 2 minutes past midnight return f2 print "Creating suite definition" defs = ecflow.Defs() suite = defs.add_suite("test") suite.add_variable("ECF_INCLUDE", os.getenv("HOME") + "/course") suite.add_variable("ECF_HOME", os.getenv("HOME") + "/course") suite.add_family( create_family_f1() ) suite.add_family( create_family_f2() ) print defs print

create_family_f2()
            ))
print(defs) 

print("Checking job creation: .ecf -> .job0"

)  
print

(defs.check_job_creation())

print

("Checking trigger expressions")

print

errors = defs.check()
assert len(errors) == 0,errors

print

("Saving definition to file 'test.def'")
defs.save_as_defs("test.def")

What to do

Make the changes to the suite definition file
Create all the necessary ecf script‘s by copying the one from /test/f1/t7
Load and begin Replace the suite
Once you start the suite some of the task will be immediately complete. Why?
python: python3 test.py; python3 client.py
text: ecflow_client --suspend=/test ; ecflow_client --replace=/test test.def
ecflow_ui ecflowview has a special window to explain why a task is queued. Select a queued task and press the Image Removed icon

click on the 'Why tab'
Vary the time attributes so that all task runs

Horizontal Navigation Bar

Button Group

Button Hyperlink

title	Previous
type	standard
url	https://confluence.ecmwf.int/display/ECFLOW/Add+a+meter

Button Hyperlink

title	Up
type	standard
url	https://confluence.ecmwf.int/display/ECFLOW/Going+Further

Button Hyperlink

title	Next
type	standard
url	https://confluence.ecmwf.int/display/ECFLOW/Time+Triggers

...

Space shortcuts

Page tree

Versions Compared

Old Version 9

New Version Current

Key

time

cron

date or day

cron

Mixing time dependencies on the same node

Mixing time dependencies on different nodes

Text

Python

What to do