Versions Compared

Old Version 45

changes.mady.by.user Axel Bonet

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (ma0)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Horizontal Navigation Bar


Button Group

Button Hyperlink
titlePrevious
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Repeat
Button Hyperlink
titleUp
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Advanced+Topics
Button Hyperlink
titleNext
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Running+the+jobs

 

Limit-submission



Limits provide simple load management by limiting the number of tasks
submitted by a specific ecflow_server
We have learnt from experience that suite designers were using trigger‘s in
two different ways: as data dependency triggers and as courtesy triggers.
Triggers where were designed for the former. The latter are used to prevent too
many jobs running at once and are actually an artificial way of queueing queuing jobs.
Because ecFlow does not distinguish between the two sorts of triggers, suites can become
difficult to maintain after a while. So the concept of limit was introduced.
Limits are declared with the limit keyword
 

inlimit

Limits are used in conjunction with inlimit keyword.
First, a limit must be defined using the ‘limit NAME N’.
The limit definition is typically placed at the suite scope.
Next we create a group of tasks to which we want to apply the limit.
This is done by attaching an ‘inlimit NAME’ attribute to the nodes.
Attaching the attribute to a task adds the task to the group.
Attaching it to a family adds all tasks from that family.
The effect of a limit is that no more than N tasks
of a group will run at once.
A node can be limited by several limits.
    

Ecf script

We will create family f5 with nine tasks.
Create new ecf script s in $HOME/course/test/f5/ directory, each one containing:
 


Code Block
languagebash
title$HOME/course/test/f5/t1.ecf,t2.ecf.....t9.ecf
%include <head.h>
echo "I will now sleep for %SLEEP% seconds"
sleep %SLEEP%
%include <tail.h>

 


Text

Let us modify our suite definition file:

Code Block
# Definition of the suite test.
suite test
 edit ECF_INCLUDE "$HOME/course"
 edit ECF_HOME    "$HOME/course"
 limit l1 2

 family f5
     inlimit l1
     edit SLEEP 20
     task t1
     task t2
     task t3
     task t4
     task t5
     task t6
     task t7
     task t8
     task t9
 endfamily
endsuite
 

Python

#!/usr/bin/env python2.7 

Python
 Code Block
languagepy
title$HOME/course/test.py
import os
from ecflow import
 ecflow  

 Defs,Suite,Family,Task,Edit,Trigger,Complete,Event,Meter,Time,Day,Date,Label, \
                   RepeatString,RepeatInteger,RepeatDate,InLimit,Limit
        
def create_family_f5() :
    
f5 =
return 
ecflow.
Family("f5"
)

,
            
f5.add_inlimit
InLimit("l1"),
    
f5.add_variable("SLEEP",
 
20)

    
for
 
i
 
in
 
range(1, 10):

Edit(SLEEP=20),
          
f5.add_task(
 
"t"
 
+ str
[ Task('t{}'.format(i)) 
)

for i in 
 return f5
range(1,10) ] )
    
print
 
("Creating suite definition")  
 


defs
home = 
ecflow.Defs()
suite = defs.add_suite("test")
suite.add_variable("ECF_INCLUDE", 
os.path.join(os.getenv("HOME")
 + 
,"
/
course")

suite.add_variable("ECF_HOME",    os.getenv("HOME") + "/course")

suite.add_limit

defs = Defs( 
        Suite("test",
            Edit(ECF_INCLUDE=home,ECF_HOME=home),
            Limit("l1",
 
2)

suite.add_family(
,
            create_family_f5())
 
)
print(defs) 
defs


print
 
("Checking job creation: .ecf -> .job0"
 
)  
print
 
(defs.check_job_creation())

print("Checking 
"Check
trigger 
in
expressions 
limit
and 
references
inlimits"

print 
)
assert len(defs.check()) == 0,defs.check() 

print
 
("Saving definition to file 'test.def'")
defs.save_as_defs("test.def")


What to do
  1. Edit the changes
  2. Replace the suite definition
  3. In ecflowview ecflow_ui , observe the triggers of the limit l1
  4. Open the Info panel for l1
  5. Change the value of the limit
  6. Open the Why? panel for one of the queued tasks of /test/f5
  7. Introduce an error in the limits and make sure this error is trapped. i.e. change the Limit.
     Code Block
    languagepy
    titleCheck InLimit/Limit references
    Limit("unknown",2)

           
 Horizontal Navigation Bar

 Button Group
 Button Hyperlink
titlePrevious
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Repeat
 Button Hyperlink
titleUp
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Advanced+Topics
 Button Hyperlink
titleNext
typestandard
urlhttps://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Running+the+jobsLimit-submission