...
Horizontal Navigation Bar | ||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
In the previous exercise, we showed how ecflow provides simple load management.
...
- Excessive disk/io in job generation
- Server busy in job generation, and slow to respond to the GUI.
- Overload queueing queuing systems like PBS/SLURM
Hence we need a load management that can limit the number of submissionsubmissions. When the Job becomes active the limit token is released.
This exercise shows the use of '-s' option. Which helps to control the number of tasks in the submitted state.
We can have more than 2 active jobs , since we are we only control the number in the submitted state.
If we removed the -s then we can only have two active jobs running at one time
This will allow The use of -s allows the configuration of the suite, depending on the load the disk/io and queuing system can sustain.
Here is the simple illustration that modifies the previous example:
Text
Let us modify our suite definition file:
Code Block |
---|
# Definition of the suite test.
suite test
edit ECF_INCLUDE "$HOME/course"
edit ECF_HOME "$HOME/course"
limit l1 2
family f5
inlimit -s l1 # by default consume 1 token from the limit l1
edit SLEEP 20
task t1
task t2
task t3
task t4
task t5
task t6
task t7
task t8
task t9
endfamily
endsuite |
...
Code Block | ||||
---|---|---|---|---|
| ||||
import os from ecflow import Defs,Suite,Family,Task,Edit,Trigger,Complete,Event,Meter,Time,Day,Date,Label, \ RepeatString,RepeatInteger,RepeatDate,InLimit,Limit def create_family_f5() : return Family("f5", InLimit("l1","",1,False,True), # limit_name(l1),limit_path(""),no_of_tokens_to_consume(1),limit node(False), limit submission(True) InLimit("l1","",1,False,True), Edit(SLEEP=20), [ Task('t{}'.format(i)) for i in range(1,10) ] ) print("Creating suite definition") home = os.path.join(os.getenv("HOME"),"course") defs = Defs( Suite("test", Edit(ECF_INCLUDE=home,ECF_HOME=home), Limit("l1",2), create_family_f5())) print(defs) print("Checking job creation: .ecf -> .job0") print(defs.check_job_creation()) print("Checking trigger expressions and inlimits") assert len(defs.check()) == 0,defs.check() print("Saving definition to file 'test.def'") defs.save_as_defs("test.def") |
...
- Edit the changes
- Replace the suite definition
- In ecflow_ui , observe the triggers of the limit l1
- Open the Info panel for l1
- effects
- Change the value of the limit
- Open the Why? panel for one of the queued tasks of /test/f5
- and inlimit, observe the effect.
Introduce an error in the limits and make sure this error is trapped. i.e. change the Limit.
Code Block language py title Check InLimit/Limit references Limit("unknown",2)
Horizontal Navigation Bar | ||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|