Horizontal Navigation Bar |
---|
Button Group |
---|
Button Hyperlink |
---|
title | Previous |
---|
type | standard |
---|
url | https://confluence.ecmwf.int/display/ECFLOW/Add+a+meter |
---|
|
Button Hyperlink |
---|
title | Up |
---|
type | standard |
---|
url | https://confluence.ecmwf.int/display/ECFLOW/Going+Further |
---|
|
Button Hyperlink |
---|
title | Next |
---|
type | standard |
---|
url | https://confluence.ecmwf.int/display/ECFLOW/Time+Triggers |
---|
|
|
|
Sometimes you want a task to run at a given time, or to run every three hours,or to run only on the first of the month, or on Mondays,...
time
They can also be relative; in this case, we provide the time from the moment the suite is begun. Time dependencies can be repeated at regular intervals. The nodes stay complete once all-time instances have run.
Code Block |
---|
time 23:00 # at next 23:00
time 10:00 20:00 01:00 # every hour from 10am to 8pm
time +00:01 # one minute after the suite has begun, or 1 minute after re-queue in a presence of a repeat
time +00:10 01:00 00:05 # 10 to 60 minutes after begin every 5 minutes |
In the last example, we have a task that runs every five minutes, however, what happens if the task takes longer?
When this happens, the time slot is missed.
cron
Cron dependencies can be specified using the cron keyword. Cron differs from time as when the node is complete it queues again immediately. Cron also only works with a real time clock (not a hybrid clock).
Code Block |
---|
cron 23:00 # every day at 23:00
cron 08:00 12:00 01:00 # every hour between 8 and 12
cron -w 0,2 11:00 # every sunday and tuesday at 11 am
cron -d 1,15 02:00 # every 1st and 15th of each month at 2 am
cron -m 1 -d 1 14:00 # every first of January at 2 pm
cron -w 5L 23:00 # run on *last* Friday(5L) of each month at 23pm
cron -d 1,L 23:00 |
...
...
...
...
...
...
Info |
---|
|
When the time has expired, the associated node is free to run. The time will stay expired until the node is re-queued. |
date or day
Date dependencies are always absolute, but wild cards can be used. Code Block |
---|
date 31.12.2012 # |
...
...
...
December 2012
date 01.*.* # every first of the |
...
month
date *.10.* # every day in October
date 1.*.2008 # every first of the month, but only in 2008
day monday |
...
...
...
...
...
Mixing time dependencies on the same node
A task can have many time and date dependencies. For example:
Code Block |
---|
task tt
day monday # Here Day/date acts like a guard over the time. i.e. time is not considered until Monday
time 10:00 # run on Monday at 10 am |
Code Block |
---|
task tt
day sunday # On the same node, Day/date act like a guard over the time attributes.
day wednesday
date 01.*.* # The first of every month and year
date 10.*.* # The |
...
tenth of every month and year
|
...
...
...
...
...
...
*if* we are on one of the day/dates
time 16:00 |
The task will run on Sunday’s and Wednesday’s at 1 am and 4 pm, but only if the day is the 1st or the 10th of the month.
Info |
---|
With multiple time dependencies on the same node, the dependencies of the same type are or' ed together, then and' ed with the different types. |
Mixing time dependencies on different nodes
When time dependencies are placed on different nodes in the hierarchy, the results may seem surprising. Notice the difference between ecflow 4 and ecflow 5
Code Block |
---|
| family fam
day monday # The day attribute no longer guards the time attribute.
task tt
time 10:00 # runs on Monday morning at 00:00 ?, and Monday at 10 am |
|
Code Block |
---|
| family fam
day monday # The day STILL guards the time attribute.
task tt
time 10:00 # Will run on Monday at 10 am |
|
---|
Code Block |
---|
family fam2
time 10:00
task tt
day monday # This will run on Monday morning at 00:00 and Monday at 10 am |
The example above assumes we have a suite, with an infinite repeat loop. So why does the task run on Monday morning at 00:00?
This is because time dependencies on different nodes act independently of each other. In this case, the time attribute was set free on Sunday at 10 am ( and once free it stays free until it is re-queued). Hence task tt is free to run on Monday morning. After the task has run and re-queued. It will then run on Monday at 10 am.
Text
Let us modify the definition file to add a family f2. For brevity, we have omitted the previous family f1
Code Block |
---|
# Definition of the suite test
suite test
edit ECF_INCLUDE "$HOME/course" # replace '$HOME' with the path to your home directory
edit ECF_HOME "$HOME/course"
family f2
edit SLEEP 20
task t1
time 00:30 23:30 00:30 # start(hh:mm) end(hh:mm) increment(hh:mm)
task t2
day thursday
time 13:00
task t3
date 1.*.* # Date(day,month,year) - * means every day,month,year
|
...
...
...
...
...
...
# Time is not considered until date is free
|
...
...
...
...
task t4
time +00:02. # + means realative to suite begin/requeue time
task t5
time 00:02 # 2 minutes past midnight
endfamily
endsuite |
Python
For brevity, we have left out family f1. In python this would be:
Code Block |
---|
language | py |
---|
title | $HOME/course/test.py |
---|
|
import os
from ecflow import Defs,Suite,Family,Task,Edit,Trigger,Complete,Event,Meter,Time,Day,Date,Edit
def create_family_f2():
return Family("f2",
Edit(SLEEP=20),
Task("t1", Time("00:30 23:30 00:30")), # start(hh:mm) end(hh:mm) increment(hh:mm)
Task("t2", Day("thursday"),Time("13:00")),
Task("t3", Date("1.*.*"), Time("12:00")), # Date(day,month,year) - * means every day,month,year
Task("t4", Time("+00:02")), # + means realative to suite begin/requeue time
Task("t5", Time("00:02"))) # 2 minutes past midnight
print("Creating suite definition")
home = os.path.join(os.getenv("HOME"), "course")
defs = Defs(
Suite("test",
Edit(ECF_INCLUDE=home,ECF_HOME=home),
create_family_f2()
))
print(defs)
print("Checking job creation: .ecf -> .job0")
print(defs.check_job_creation())
print("Checking trigger expressions")
errors = defs.check()
assert len(errors) == 0,errors
print("Saving definition to file 'test.def'")
defs.save_as_defs("test.def") |
What to do
- Make the changes to the suite definition file
- Create all the necessary ecf script‘s by copying the one from /test/f1/t7
- Replace the suite
python: python3 test.py; python3 client.py
text: ecflow_client --suspend=/test ; ecflow_client --replace=/test test.def - ecflow_ui has a special window to explain why a task is queued. Select a queued task and click on the 'Why tab'
- Vary the time attributes so that all task runs
Horizontal Navigation Bar |
---|
Button Group |
---|
Button Hyperlink |
---|
title | Previous |
---|
type | standard |
---|
url | https://confluence.ecmwf.int/display/ECFLOW/Add+a+meter |
---|
|
Button Hyperlink |
---|
title | Up |
---|
type | standard |
---|
url | https://confluence.ecmwf.int/display/ECFLOW/Going+Further |
---|
|
Button Hyperlink |
---|
title | Next |
---|
type | standard |
---|
url | https://confluence.ecmwf.int/display/ECFLOW/Time+Triggers |
---|
|
| suite definition"</span>
<span class="n">defs</span> <span class="o">=</span> <span class="n">ecflow</span><span class="o">.</span><span class="n">Defs</span><span class="p">()</span>
<span class="n">suite</span> <span class="o">=</span> <span class="n">defs</span><span class="o">.</span><span class="n">add_suite</span><span class="p">(</span><span class="s">"test"</span><span class="p">)</span>
<span class="n">suite</span><span class="o">.</span><span class="n">add_variable</span><span class="p">(</span><span class="s">"ECF_INCLUDE"</span><span class="p">,</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"HOME"</span><span class="p">)</span> <span class="o">+</span> <span class="s">"/course"</span><span class="p">)</span>
<span class="n">suite</span><span class="o">.</span><span class="n">add_variable</span><span class="p">(</span><span class="s">"ECF_HOME"</span><span class="p">,</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s">"HOME"</span><span class="p">)</span> <span class="o">+</span> <span class="s">"/course"</span><span class="p">)</span>
<span class="n">suite</span><span class="o">.</span><span class="n">add_family</span><span class="p">(</span> <span class="n">create_family_f1</span><span class="p">()</span> <span class="p">)</span>
<span class="n">suite</span><span class="o">.</span><span class="n">add_family</span><span class="p">(</span> <span class="n">create_family_f2</span><span class="p">()</span> <span class="p">)</span>
<span class="k">print</span> <span class="n">defs</span>
<span class="k">print</span> <span class="s">"Checking job creation: .ecf -> .job0"</span>
<span class="k">print</span> <span class="n">defs</span><span class="o">.</span><span class="n">check_job_creation</span><span class="p">()</span>
<span class="k">print</span> <span class="s">"Checking trigger expressions"</span>
<span class="k">print</span> <span class="n">defs</span><span class="o">.</span><span class="n">check</span><span class="p">()</span>
<span class="k">print</span> <span class="s">"Saving definition to file 'test.def'"</span>
<span class="n">defs</span><span class="o">.</span><span class="n">save_as_defs</span><span class="p">(</span><span class="s">"test.def"</span><span class="p">)</span>
</pre></div>
</div>
<p><strong>What to do:</strong></p>
<ol class="arabic simple">
<li>Make the changes to the <a class="reference internal" href="/wiki/display/ECFLOW/Glossary#term-suite-definition"><em class="xref std std-term">suite definition</em></a> file</li>
<li>Create all the necessary <a class="reference internal" href="/wiki/display/ECFLOW/Glossary#term-ecf-script"><em class="xref std std-term">ecf script</em></a>‘s by copying the one from <strong>/test/f1/t7</strong></li>
<li>Load and begin the suite</li>
<li>Once you start the suite some of the task will be immediately <a class="reference internal" href="/wiki/display/ECFLOW/Glossary#term-complete"><em class="xref std std-term">complete</em></a>. Why?</li>
<li><a class="reference internal" href="/wiki/display/ECFLOW/Glossary#term-ecflowview"><em class="xref std std-term">ecflowview</em></a> has a special window to explain why a task is <a class="reference internal" href="/wiki/display/ECFLOW/Glossary#term-queued"><em class="xref std std-term">queued</em></a>.
Select a <a class="reference internal" href="/wiki/display/ECFLOW/Glossary#term-queued"><em class="xref std std-term">queued</em></a> task and press the <img alt="why" src="../../../_images/why.jpg" /> icon</li>
</ol>
</div>
</div>
|