Versions Compared

Old Version 44

changes.mady.by.user Unknown User (ma0)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (ma0)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Horizontal Navigation Bar


Button Group

Button Hyperlink
titlePrevious
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Add+a+meter
Button Hyperlink
titleUp
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Going+Further
Button Hyperlink
titleNext
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/IndentationTime+Triggers


Sometimes you want a task to run at a given time, or to run every three hours,
or to run only on the first of the month, or on Mondays,...
For that ecFlow supports date and time dependencies.

time

time dependencies can be absolute, i.e. they will run at the exact time.
They can also be relative; in this case, we provide the time from the moment the suite is begun.
Time dependencies can be repeated at regular intervals.  The nodes stays stay complete once all-time instances have run. 


Code Block
time 23:00                  # at next 23:00
time 10:00 20:00 01:00      # every hour from 10am to 8pm
time +00:01                 # one minute after the suite has begun, or 1 minute after re-queue in a presence of a repeat
time +00:10 01:00 00:05     # 10 to 60 minutes after begin every 5 minutes
 

In the last example, we have a task that runs every five minutes, however, what happens if the task takes longer? 
When this happens, the time slot , is missed.

cron
Cron dependencies can be specified using the cron keyword.  Cron differs from time as when the node is complete it queues again immediately.  Cron also only works with a real time clock (not a hybrid clock).
 Code Block
cron 23:00                 # every day at 23:00
cron 08:00 12:00 01:00     # every hour between 8 and 12
cron -w 0,2    11:00       # every sunday and tuesday at 11 am
cron -d 1,15   02:00       # every 1st and 15th of each month at 2 am
cron -m 1 -d 1 14:00       # every first of January at 2 pm
 Info
titleTime,Today,Cron
When a time has expired, the associated node is free to run. The time will stay expired, until the node is re-queued.
date or day
Date dependencies can be specified using the date or the day keyword.
Date dependencies are always absolute, but wild cards can be used.
 Code Block
date 31.12.2012  
cron -w 5L 23:00           # run on *last* Friday(5L) of each month at 23pm
cron -d 1,L  23:00         # Run #on the 31stfirst of December 2012
date 01.*.*                 # every first and last of the month at 23pm

 Info
titleTime,Today,Cron
When the time has expired, the associated node is free to run. The time will stay expired until the node is re-queued.
date or day
Date dependencies can be specified using the date or the day keyword.
Date dependencies are always absolute, but wild cards can be used.
 Code Block
date 31.12.2012
date *.10.*                 # every day in October
date 1.*.2008               # everythe first31st of the month, but only in 2008
day mondayDecember 2012
date 01.*.*                 # every first of the # every monday
Mixing time dependencies on the same node
A task can have several time and date dependencies.   For example:
 Code Block
task tt
month
date *.10.*    day monday   # Here Day/date acts like a guard over the time. i.e time is not considered until Monday
# every day in October
date 1.*.2008     time 10:00   # run on Monday at 10 am
 Code Block
task tt
   day sunday    # every first of the month, but only in 2008
day monday                  # every monday
Mixing time dependencies on the same node
A task can have many time and date dependencies.   For example:
 Code Block
task tt
   day monday   # HereOn the same node, Day/date actacts like a guard over the time. attributesi.e.
 time is daynot wednesday
considered until Monday
 date 01.*.* time 10:00   # run on Monday at 10 am

 Code Block
task tt
   day sunday # The first of every month and year
   date 10.*.*      # On the same node, Day/date act like a guard over #the The tenth of every month and yeartime attributes.
   day wednesday
   timedate 01:00 .*.*                 # The timefirst isof onlyevery setmonth free *if* we are on one of the day/datesand year
   date 10.*.*                 # The tenth of every month and year
   time 16:00
 
The task will run on Sunday’s and Wednesday’s at 1am and 4pm, but only if the day is the 1st or the 10th of the month.
01:00                  # The time is only set free *if* we are on one of the day/dates
   time 16:00

The task will run on Sunday’s and Wednesday’s at 1 am and 4 pm, but only if the day is the 1st or the 10th of the month.

 
 Info
With multiple time dependencies on the same node, the dependencies of the same type are or' ed together, then and' ed with the different types.
 

Mixing time dependencies on different nodes
When time dependencies are placed on different nodes in the hierarchy, the results may seem surprising. Notice the difference between ecflow 4 and ecflow 5

 Code Block
titleecflow 4
family fam
   day monday
  
    
 
# The day attribute no longer guards the time attribute.
   task tt
      time 10:00 # runs 
 # This will run on 
on Monday morning at 00:00 ?, and Monday at 10 am 





 Code Block
titleecflow 5
family 
fam2
fam
   day monday 
time 10:00
   # The day STILL guards the time attribute.
   task tt
      
 day monday
time 10:00 # 
This
Will 
will 
run on Monday at 10 am 


 Code Block
family fam2
   time 10:00
   task tt
       day monday # This will run on Monday morning at 1000:00 ?, and Monday at 10 am
The example above assumes we have a suite, with an infinite repeat loop. So why does the task run on Monday morning at  00:00?
This is because time dependencies on different nodes act independently of each other.  In this case, the time attribute was set free on Sunday at 10 am ( and once free it stays free until it is re-queued).  Hence task tt is free to run on Monday  Monday morning.. After the task has run and re-queued. It will then run on Monday at 10 am.

 
Like trigger‘s, date and time dependencies can be set for a family. In this case, the tasks of this family will only run according to these dependencies.
 

 Note
All time-related dependencies(like cron, time, today, date and day) are relative to the clock of the suite.
For more information, see Dates and Clocks
 

Text
 f2.add_variable("SLEEP", 20)
    f2.add_task("t1").add_time( "00:30 23:30 00:30" ) # start(hh:mm) end(hh:mm) increment(hh:mm)
    f2.add_task("t2").add_day( "sunday" )
   
    # for add_date(): day,month,year; here 0 means every month, and every year
    t3 = f2.add_task("t3")
    t3.add_date(1, 0, 0)           # day month year, first of every month or every year
    t3.add_time( 12, 0 )
Let us modify the definition file to add a family f2.
For brevity, we have omitted the previous family f1

 
 Code Block
# Definition of the suite test
suite test
 edit ECF_INCLUDE "$HOME/course"  # replace '$HOME' with the path to your home directory
 edit ECF_HOME    "$HOME/course"

 family f2
     edit SLEEP 20
     task t1
         time 00:30 23:30 00:30
     task t2
         day sunday
     task t3
         date 16.01.2017
         time 12:00
     task t4
         time +00:02
     task t5
         time 00:02
 endfamily
endsuite
 
Python
For brevity we have left out family f1. In python this would be:
 Code Block
languagepy
title$HOME/course/test.py
#!/usr/bin/env python2.7
import os
from ecflow import Defs,Suite,Family,Task,Edit,Trigger,Complete,Event,Meter,Time,Day,Date

def create_family_f2():
    return Family("f2",
            Edit(SLEEP=20),
            Task("t1", Time("00:30 23:30 00:30")),     # start(hh:mm) end(hh:mm) increment(hh:mm)
            Task("t2", Day("sunday")),
            Task("t3", Date("1.*.*"), Time("12:00")),  # Date(day,month,year) - * means every day,month,year
            Task("t4", Time("+00:02")),                # + means realative to suite begin/requeue time
            Task("t5", Time("00:02")))                 # 2 minutes past midnight
 
print("Creating suite definition")  
home = os.path.join(os.getenv("HOME"), "course")
defs = Defs( 
        Suite("test",
            Edit(ECF_INCLUDE=home,ECF_HOME=home),
            #create_family_f1(),
            create_family_f2()
            ))
print(defs) 

print("Checking job creation: .ecf -> .job0")  
#print(defs.check_job_creation())

print("Checking trigger expressions")
errors = defs.check()
assert len(errors) == 0,errors

print("Saving definition to file 'test.def'")
defs.save_as_defs("test.def")
#!/usr/bin/env python2.7
import os
import ecflow  

def create_family_f2():
    f2 = ecflow.Family("f2")
   
 Definition of the suite test
suite test
 edit ECF_INCLUDE "$HOME/course"  # replace '$HOME' with the path to your home directory
 edit ECF_HOME    "$HOME/course"

 family f2
     edit SLEEP 20
     task t1
         time 00:30 23:30 00:30  # start(hh:mm) end(hh:mm) increment(hh:mm)
     task t2
         day thursday
         time 13:00
     task t3
         date 1.*.*              # Date(day,month,year) - * means every day,month,year
         time 12:00              # Time is not considered until date is free
     task t4
         time +00:02.            # + 
#
means 
hour,
realative 
minutes
to 
at
suite 
12 o'clock
begin/requeue time
   


  task 
 f2.add_task("t4").add_time( 0, 2, True ) # hour, minutes, relative to suite start

t5
         time 00:02              # 2 minutes past midnight
 endfamily
endsuite

Python
For brevity, we have left out family f1. In python this would be:
 Code Block
languagepy
title$HOME/course/test.py
import os
from ecflow import Defs,Suite,Family,Task,Edit,Trigger,Complete,Event,Meter,Time,Day,Date,Edit

def create_family_f2():
    return Family("f2",
            
 
Edit(SLEEP=20),
     
#
 
2
 
minutes
 
after
 
family
 
f2
 
start
    f2.add_task
 Task("
t5").add_time( 0, 2 )
t1", Time("00:30 23:30 00:30")),     # 
 # hour, minutes suite site

start(hh:mm) end(hh:mm) increment(hh:mm)
            Task("t2", Day("thursday"),Time("13:00")),
            
           
Task("t3", Date("1.*.*"), Time("12:00")),  # Date(day,month,year) - * means every day,month,year
            
# 2 minutes past midnight

Task("t4", Time("+00:02")),      
return
 
f2

         # + means 

print
realative 
"Creating
to suite 
definition"
begin/requeue time
    

defs
 
=
 
ecflow.Defs()
suite
 
=
 
defs.add_suite("test")
suite.add_variable("ECF_INCLUDE", os.path.join(os.getenv("HOME"),  "course"))
suite.add_variable("ECF_HOME",   
    Task("t5", Time("00:02")))                 # 2 minutes past midnight
 
print("Creating suite definition")  
home = os.path.join(os.getenv("HOME"),
 
 "course"
)
)


suite.add_family( create_family_f1() )
suite.add_family(
defs = Defs( 
        Suite("test",
            Edit(ECF_INCLUDE=home,ECF_HOME=home),
            create_family_f2()
            ))
print(defs) 
defs


print
 
("Checking job creation: .ecf -> .job0")  
 

print
 
(defs.check_job_creation())

print
 
("Checking trigger expressions")

print
errors = defs.check()
assert len(errors) == 0,errors

print
 
("Saving definition to file 'test.def'")
defs.save_as_defs("test.def")


What to do
  1. Make the changes to the suite definition file
  2. Create all the necessary ecf script‘s by copying the one from /test/f1/t7
  3. Replace the suite
    python:  python python3 test.py; python python3 client.py
    text:       ecflow_client --suspend=/test  ;  ecflow_client --replace=/test  test.def
  4. ecflow_ui  has a special window to explain why a task is queued. Select a queued task and press the whyImage Removed icon
         or click on the 'Why tab'
  5. Vary the time attributes so that all task runs

 Horizontal Navigation Bar

 Button Group
 Button Hyperlink
titlePrevious
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Add+a+meter
 Button Hyperlink
titleUp
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Going+Further
 Button Hyperlink
titleNext
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/IndentationTime+Triggers