|
|
In the previous exercise we showed how ecflow provides simple load management.
The limits can still allow several hundred jobs to be submitted at once. This can cause problems:
Hence we need a load management that can limit the number of submission. When the Job becomes active the limit token is released.
We can have more than 2 active jobs, since we are only control the number in the submitted state.
If we removed the -s then we can only have two active jobs running at one time
This will allow the configuration of the suite, depending on the load the disk/io and queuing system can sustain.
Here is the simple illustration that modifies the previous example:
Let us modify our suite definition file:
# Definition of the suite test.
suite test
edit ECF_INCLUDE "$HOME/course"
edit ECF_HOME "$HOME/course"
limit l1 2
family f5
inlimit -s l1 # by default consume 1 token from the limit l1
edit SLEEP 20
task t1
task t2
task t3
task t4
task t5
task t6
task t7
task t8
task t9
endfamily
endsuite |
import os
from ecflow import Defs,Suite,Family,Task,Edit,Trigger,Complete,Event,Meter,Time,Day,Date,Label, \
RepeatString,RepeatInteger,RepeatDate,InLimit,Limit
def create_family_f5() :
return Family("f5",
InLimit("l1","",1,False,True), # limit_name(l1),limit_path(""),no_of_tokens_to_consume(1),limit node(False), limit submission(True)
Edit(SLEEP=20),
[ Task('t{}'.format(i)) for i in range(1,10) ] )
print("Creating suite definition")
home = os.path.join(os.getenv("HOME"),"course")
defs = Defs(
Suite("test",
Edit(ECF_INCLUDE=home,ECF_HOME=home),
Limit("l1",2),
create_family_f5()))
print(defs)
print("Checking job creation: .ecf -> .job0")
print(defs.check_job_creation())
print("Checking trigger expressions and inlimits")
assert len(defs.check()) == 0,defs.check()
print("Saving definition to file 'test.def'")
defs.save_as_defs("test.def") |
Introduce an error in the limits and make sure this error is trapped. i.e. change the Limit.
Limit("unknown",2) |
|