...
Create a directory for this tutorial so all the exercises and outputs are contained inside:
No Format mkdir ~/batch_tutorial cd ~/batch_tutorial
Create and submit a job called
simplest.sh
with just default settings that runs the commandhostname
. Can you find the output and inspect it? Where did your job run?Expand title Solution Using your favourite editor, create a file called
simplest.sh
with the following contentCode Block language bash title simplest.sh #!/bin/bash hostname
You can submit it with sbatch:
No Format sbatch simplest.sh
The job should be run shortly. When finished, a new file called
slurm-<jobid>.out
should appear in the same directory. You can check the output with:No Format $ cat $(ls -1 slurm-*.out | tail -n1) ab6-202.bullx [ECMWF-INFO -ecepilog] ---------------------------------------------------------------------------------------------------- [ECMWF-INFO -ecepilog] This is the ECMWF job Epilogue [ECMWF-INFO -ecepilog] +++ Please report issues using the Support portal +++ [ECMWF-INFO -ecepilog] +++ https://support.ecmwf.int +++ [ECMWF-INFO -ecepilog] ---------------------------------------------------------------------------------------------------- [ECMWF-INFO -ecepilog] Run at 2023-10-25T11:31:53 on ecs [ECMWF-INFO -ecepilog] JobName : simplest.sh [ECMWF-INFO -ecepilog] JobID : 64273363 [ECMWF-INFO -ecepilog] Submit : 2023-10-25T11:31:36 [ECMWF-INFO -ecepilog] Start : 2023-10-25T11:31:51 [ECMWF-INFO -ecepilog] End : 2023-10-25T11:31:53 [ECMWF-INFO -ecepilog] QueuedTime : 15.0 [ECMWF-INFO -ecepilog] ElapsedRaw : 2 [ECMWF-INFO -ecepilog] ExitCode : 0:0 [ECMWF-INFO -ecepilog] DerivedExitCode : 0:0 [ECMWF-INFO -ecepilog] State : COMPLETED [ECMWF-INFO -ecepilog] Account : myaccount [ECMWF-INFO -ecepilog] QOS : ef [ECMWF-INFO -ecepilog] User : user [ECMWF-INFO -ecepilog] StdOut : /etc/ecmwf/nfs/dh1_home_a/user/slurm-64273363.out [ECMWF-INFO -ecepilog] StdErr : /etc/ecmwf/nfs/dh1_home_a/user/slurm-64273363.out [ECMWF-INFO -ecepilog] NNodes : 1 [ECMWF-INFO -ecepilog] NCPUS : 2 [ECMWF-INFO -ecepilog] SBU : 0.011 [ECMWF-INFO -ecepilog] ----------------------------------------------------------------------------------------------------
You can then see that the script has run on a different node than the one you are on.
If you repeat the operation, you may get your job to run on a different node every time, whichever happens to be free at the time.
Configure your
simplest.sh
job to direct the output tosimplest-<jobid>.out
, the error tosimplest-<jobid>.err
both in the same directory, and the job name to just "simplest". Note you will need to use a special placeholder for the -<jobid>
.Expand title Solution Using your favourite editor, open the
simplest.sh
job script and add the relevant #SBATCH directives:Code Block language bash title simplest.sh #!/bin/bash #SBATCH --job-name=simplest #SBATCH --output=simplest-%j.out #SBATCH --outputerror=simplest-%j.err hostname
You can submit it again with:
No Format sbatch simplest.sh
After a few moments, you should see the new files appear in your directory (job id will be different than the one displayed here):
No Format $ ls simplest-*.* simplest-64274497.err simplest-64274497.out
You can check that the job name was also changed in the end of job report:
No Format $ grep -i jobname $(ls -1 simplest-*.err | tail -n1) [ECMWF-INFO -ecepilog] JobName : simplest
From a terminal session outside the Atos HPCF or ECS your VDI or computer, submit the
simplest.sh
job remotely. What hostname should you use?Expand title Solution You must use hpc-batch for HPCF job submissions, or ecs-batch for remote submissions:
No Format ssh hpc-batch "cd ~/batch_tutorial; sbatch simplest.sh"
No Format ssh ecs-batch "cd ~/batch_tutorial; sbatch simplest.sh"
Note the change of directory so both the job script, the working directory of the job and its outputs are generated in the right place.
An alternative way of doing this without changing directory would be to tell sbatch to do it for you:
No Format ssh hpc-batch sbatch -D ~/batch_tutorial ~/batch_tutorial/simplest.sh
or for ECS:
No Format ssh ecs-batch sbatch -D ~/batch_tutorial ~/batch_tutorial/simplest.sh
...