If you are NOT using the ECMWF Gateways (ecaccess or msaccess) then please refer to the previous user guide for accessing the ECaccess Web Services V1. This documentation is about ECaccess Web Services V2 which is currently only available at ECMWF. At heart, ECaccess Web Services APIs use JSON as a data format and HTTPS as a protocol for data transmission. ECMWF has released a full-featured Perl client library to make interacting with ECaccess Web Services APIs easier. However, if Perl is not available on your platform or you are not able to install additional software then CURL is an alternative to interact with the ECaccess Web Services. CURL is a command-line application for performing requests using a variety of protocols including HTTPS, required to interact with the APIs at a low level. In this document we assume that you are already familiar with the ECaccess Services. If this is not the case then please have refer to other sections in this User Guide to learn about the concepts behind Gateways, Operations, Associations, Protocols, Files, Jobs, Events, Queues and Transfers.
...
| Table of Contents |
|---|
Obtaining CURL
CURL is commonly available on a default install of many UNIX/Linux platforms. Try typing curl in your favorite favourite shell to see if it is installed and in your PATH. If you don't have the tool installed, visit the download page on the CURL website to obtain the official source or a user-contributed binary package: http://curl.haxx.se/download.html
Note that the command-line tool uses the libcurl library, which may be offered as a separate download package, so, if you're not compiling from the source, be sure to download a binary package instead of a libcurl package. The SSL-enabled packages are required to use the ECaccess Web Services.
...
Environment
You need to ensure the following environment variables are set with the correct values:
| Code Block | ||
|---|---|---|
| ||
> export ECACCESS_URLV2="https://ecaccess.ecmwf.int/ecaccess/v2"
> export ECACCESS_OPTS="-w \ncode=%{http_code}\n -3 -k" |
If you are using the RMDCN network to access ECMWF then please use msaccess.ecmwf.int instead of ecaccess.ecmwf.int. The curl options used are the following:
| -w | display the http code on stdout (\ncode=%{http_code}\n) |
| -k | disable certificate checking |
If you don't want or can't setup these environment parameters then just replace $ECACCESS_URLV2 and $ECACCESS_OPTS by their contents in the rest of the document.
...
Input/output
For each ECaccess Service available an example is given in the rest of this document with the CURL syntax to use, the list of parameters and the expected output.
...
Each JSON message should include a field "success" set to "yes". If an error occurs then this field is set to "no" and an "error" field gives the error message returned by the service.
For more informations information concerning JSON see http://www.json.org/
There are a few occasions when an error will not be indicated via a JSON message but with an http error code (e.g. when a binary stream is expected rather than a message):
| 401 Unauthorized | authentication error when trying to download a new certificate |
| 400 Bad request | error when downloading a data file or job output/input file. |
| 403 Forbidden |
| access is only allowed through a secure connection (https). | |
| 412 Precondition failed | missing mandatory URL parameter or unknown JSON parameter. |
| 404 Not Found | the requested service does not exists. |
On a successful request the http status code returned by the server is the following:
| 200 OK | the request has |
| produced a JSON message. |
If you get an HTML document then that could be a problem with a proxy. In this case the HTML message should give you an indication of what the problem is.
6.4 Authentication
...
Authentication
Certificates
Accessing the ECaccess Web Services API requires a valid certificate. Certificates can be created with the CURL command
...
In this request just replace uid by your ECMWF identifier and XXXXXXXX by your PASSCODE obtained from your security token. It generates a certificate in ".eccert.txt" in your user's home directory. The ECaccess Certificate is valid for one week and should be renewed after expiration.
...
Tokens
Requests to the ECaccess Web Services are performed by adding an HTTP header to the request which contains an Authentication token. Authentication tokens are generated for each session and require a valid certificate. To get an Authentication token (e.g. LOGIN) use the command
...
The Authentication token is now released and can't be used for any subsequent requests.
...
Cosinfo
Cosinfo provides ECMWF computer operations system information, such as scheduled or unscheduled downtime of servers, applications or hardware. The command
...
{
"cosinfo" :
"********************************************************************\n*
*\n*
*\n* Welcome to AIX Version 5.3!
*\n*
*\n*
*\n* Please see the README file in /usr/lpp/bos for information pertinent to
*\n* this release of the AIX Operating System.
*\n*
*\n*
*\n********************************************************************\n\n\nDear
colleagues,\n\n\nThe following system sessions have been announced for\n\n
Wednesday 5th of October 2012\n\n\n08:00-10:00 UTC DHS system
session\nImpact: MARS and ECFS may be affected\n\n\n13:00-14:30 UTC
ECPDS system session\nImpact: none\n\n\nHave a nice day,\nSylvia for
Calldesk\n\n\n\nTo reread the message please use: more /etc/motd or cat
/etc/motd\n\n===================================================================\n",
"success" : "yes"
}
code=200
...
Gateways
To display the name of the Gateway you are connected to (please note this command does not need a token), command
...
{
"gateway" : {
"name" : "forte.meteo.fr",
"comment" : "HP-UX B.11.00 - Java HotSpot(TM) Server VM 1.4.2
1.4.2.08-050401-19:15-PA_RISC1.1 PA1.1 mixed mode Hewlett-Packard
Company",
"version" : "3.1.0_2007011001",
"active" : false,
"lastReport" : "Aug 22 13:15"
},
"success" : "yes"
}
code=200
...
Operations
To list all the ECaccess Operations/Services available with the current token use
...
{
"operation" : {
"name" : "submitJob",
"comment" : "job submission",
"duration" : "168h",
"startDate" : "Sep 30 11:40",
"validity" : 7,
"endDate" : "Oct 07 11:40",
"standard" : "168h"
},
"success" : "yes"
}
code=200
...
Associations
All your ECtrans Associations for a Gateway (e.g. the "ecaccess.ecmwf.int" Gateway) can be listed with
...
The table below lists all parameters for the ECtrans Association. For additional information please see the ecaccess-association-put manpage.
| Type | Name | Comment |
| String | name | |
| Boolean | active | default FALSE |
| String | comment | |
| String | grantedUserList | multiple users separated by a ',' |
| String | data | |
| String | directory | |
| String | hostName | |
| String | login | |
| String | password | |
| String | protocol |
...
Protocols
To list all the ECtrans Protocols supported by a Gateway (e.g. ecaccess.ecmwf.int) command
> curl $ECACCESS_OPTS -H "Token: ahs4w294" "$ECACCESS_URLV2/protocols/ecaccess.ecmwf.int"
...
{
"protocol" : {
"name" : "genericFtp",
"value" :
"$msuser[login]:$msuser[passwd]@$msuser[host]/$msuser[dir]$target",
"comment" : "Generic FTP access from the gateway",
"active" : true,
"archive" : "${ecmwf.dir}/gateway/lib/ectrans",
"module" : "ftp",
"activeModule" : true
},
"success" : "yes"
}
code=200
...
Files
The ECaccess files are in the form [domain:][/user-id/]path. Please refer to section 5.4 to File Management for more information on the ECaccess File System. To download a source ecaccess file, e.g. c2a:/tmp/test.tar.gz), use
> curl $ECACCESS_OPTS -H "Token: ahs4w294" -o test.tar.gz \ "$ECACCESS_URLV2/file?source=c2a:/tmp/test.tar.gz&offset=0"
...
This temporary file "tmp:roqZya" can be used to upload a temporary file (e.g. when submitting the input file for an ECaccess Job).
...
Jobs
To submit a new ECaccess Job, e.g. myjob.ksh which is a shell script without Loadleveler directives located in the ECMWF home directory:
...
The table below lists all Job Request parameters. For additional information please see the ecaccess-job-submit manpage.
| Type | Name | Comment |
| String | scheduledDate | in the form 'yyyy-MM-dd HH:mm' |
| String | userMailAddress | |
| Boolean | sendMailOnStart | |
| Boolean | sendMailOnSuccess | |
| Boolean | sendMailOnFailure | |
| Boolean | sendMailOnRetry | |
| Boolean | containsDirectives | default TRUE |
| String | queueName | default is ecgate |
| String | name | |
| String | transferGatewayName | default is current Gateway |
| String | transferRemoteLocation | in the format association-name[@protocol] |
| String | transferOutputFile | |
| String | transferErrorFile | |
| String | transferInputFile | |
| Boolean | transferKeepInSpool | default FALSE |
| String | eventIds | list separated by ';' or ',' |
| Boolean | renewSubscription | default FALSE |
| Boolean | errorToOutput | default FALSE |
| String | manPageContent | |
| Integer | lifeTime | in days |
| Integer | retryCount | default 0 |
| Integer | retryFrequency | in seconds |
| String | inputFile | file on ECMWF file system |
To list all your ECaccess Jobs:
...
{
"success" : "yes"
}
code=200
...
Events
To list all the ECaccess Events which are available to your ECMWF user identifier
...
All parameters for the Event Request are listed in table 12. Table 12: Parameters for an Event Request. For additional information please see the ecaccess-event-create manpage.
| Type | Name | Comment |
| String | name | |
| String | comment | |
| Boolean | isPublic | default FALSE |
| Boolean | overwrite | default FALSE |
| String | title | |
| String | metadata |
To grant an ECaccess Event (e.g. 807):
...
All parameters for the Grant Event Request are listed in table 13. Table 13: Parameters for a Grant Event Request. For additional information please see the ecaccess-event-grant manpage.
| Type | Name | Comment |
| Boolean | subscribe | default FALSE |
| Boolean | notify | default FALSE |
| String | userList | multiple users separated by a ',' |
To trigger an ECaccess Event (e.g. 807):
...
All parameters for the Send Event Request are listed in table 14. Table 14: Parameters for the Send Event Request. For additional information please see the ecaccess-event-send manpage.
| Type | Name | Comment |
| String | env | multiple variables separated by a ';' |
| String | delay | specified in [w]eeks, [d]ays, [h]ours, [m]inutes or [s]econds (e.g. 1w or 2d) |
| String | at | in the format 'yyyy-MM-dd HH:mm' |
| Integer | seq |
To clear an ECaccess Event (e.g. 807):
...
{
"success" : "yes"
}
code=200
...
Queues
To list the available ECaccess Queues:
...
{
"queue" : [ {
"name" : "pvwave",
"comment" : "For jobs using PV-Wave - restricted access"
}, {
"name" : "timecrit",
"comment" : "Suitable for time critical jobs - restricted access"
(...)
} ],
"success" : "yes"
}
code=200
...
Transfers
In order to list all your ECtrans Transfers in the ECtrans Spool:
...
All parameters for the ECtrans Request are listed in table 15. Table 15: Parameters for the ECtrans Request. For additional information please see the ecaccess-ectrans-request manpage.
| Type | Name | Comment |
| String | gatewayName | default current Gateway |
| String | remoteLocation | in the format association-name[@protocol] |
| String | sourceFileName | |
| String | targetFileName | |
| Integer | retryCount | |
| Integer | retryFrequency | frequency of retries in seconds (default: 600 seconds) |
| String | maximumDuration | e.g. 10m or 1h, <0 to deactivate (default 12h) |
| String | minimumDuration | e.g. 30s or 10m, <0 to deactivate (default 10m) |
| Long | minimumRate | bytes/s (default 10240) |
| Integer | priority | |
| String | lifeTime | in [w]eeks, [d]ays, [h]ours, [m]inutes or [s]econds (e.g. 1w or 2d) |
| Boolean | deleteSource | default FALSE |
| Boolean | keepInSpool | default FALSE |
| String | userMailAddress | |
| Boolean | sendMailOnSuccess | |
| Boolean | sendMailOnFailure | |
| Boolean | sendMailOnRetry | |
| String | ifTargetFileExists | 'reject', 'append', 'resume' or 'overwrite' |
To restart an ECtrans Transfer:
...