• We strongly recommend users to read our ECMWF Service Status and the Web-API FAQ before reporting the problem
  
• If the problem you are experiencing is included in the troubleshooting list below, please follow the corresponding instructions.
• If you are still experiencing problems, or if the problem is NOT included in the troubleshooting list below, please feel free to report the problem
  
  

Who to contact: servicedesk@ecmwf.int

What information you will need to pass:

• Your login
• Your job id 
• The problem you are experiencing

Where to find this information:

• All this information can be found on your job list (see next paragraph) or
  
• In the service response (in case of a Web-API  script, look the log messages returned by the server)
• A lot of useful information can be also found on the MARS activity page

Your job list is important for tracing back and debugging your requests. It includes plenty of useful information, like the following:

• job id,
• user login,
• status of your request,
  
• duration of your request,
• details of your request (ie what you have requested),
  
• log messages of your request (ie the feedback from our servers).  
• an option to cancel a request

• Check the status of your request in your job list
• If your request is not in your job list please report the problem
• If your request is in your job list and the status has been "queued", for a long time, we recommend you the following:
  
  • Check the Nr of your "active" and "queued" requests. Please keep in mind that a limitation of  3  "active" requests/per user has been set to improve the service
  • Have a look on the Web-API activity to check the current status of our service.(ie if the service is busy you will see plenty of "queued"  requests). You may also wish to check our  MARS activity
• If your long running request is in your job list and the  status  is "active",  it means that your request is currently handled by MARS but for some reasons it is slow.
  In this case we recommend you the following:
  
  • Do not "cancel" it, if you still need the data.
    

  • Check the MARS activity to see why your request is slow. (You may need to read some helpful information about MARS activity page )

  • If your request is slow because of performance issues read our retrieval efficiency pages

• If your request is "aborted" please check the log messages, find the "error"  and follow the troubleshooting instructions below.
• If the problem is not included  in the troubleshooting list below, please contact us.

'mars - ERROR  - 20160712.064620 - Request validation failed:',
'mars - ERROR  - 20160712.064620 - restricted access to valid data.',
'mars - ERROR  - 20160712.064620 - For more information, look at ',
'mars - ERROR  - 20160712.064620 - https://software.ecmwf.int/wiki/display/UDOC/MARS+access+restrictions#valid.',
'mars - ERROR  - 20160712.064620 - For any queries, please, contact User Support',
'mars - ERROR  - 20160712.064620 - Some errors reported',

• Your request will be  "aborted" if you are trying to access restricted data.
• For more information please have a look at MARS access restrictions

mars - ERROR  - 20160712.133205 - UserError: Cannot retrieve more 
than XXX.XXX fields in one request via the WebAPI. Please, split your 
request [marsode]
mars - ERROR  - 20160712.133205 - Error code is -2
mars - ERROR  - 20160712.133205 - Request failed
mars - INFO   - 20160712.133205 - Request time:  wall: 1 sec
mars - ERROR  - 20160712.133205 - Some errors reported

• On Web-API some limitations have been set to improve the service.  Check the current limits here: News feed 
• In this case, your job has reached the limit of XXX.XXX fields per request. Please split your request into smaller sub requests.

• On Web-API some limitations have been set to improve the service.  Check the current limits here: News feed 
• In this case your job has reached the limit of XXGB per request. Please split your request into smaller sub requests.

if you are using the Access ECMWF Public Datasets service and you get this error it means that the mandatory keyword "dataset" is not set. Review the request and add this keyword.

If you are an authorized user with MARS access you can use WebAPI FAQ#Can I use the Python requests to retrieve data using API MARS service?

The datasets interim_daily and interim_full do not exists any more. Please change your request to ask for the dataset interim instead. Please note that interim is now equivalent to the interim_full, the full resolution dataset. To get the equivalent to the interim_daily dataset, you will need to set "grid" to "1.5/1.5" in your request.

• You are not authorised to use service MARS.
• Make sure you have logged in https://apps.ecmwf.int/auth/login/ and then retrieve your key at https://api.ecmwf.int/v1/key/.
• For this, you will need to have an authorized account on ECMWF. See https://www.ecmwf.int/en/forecasts/accessing-forecasts/order-historical-datasets

Reason: The rate limit of the http requests  on the API service has been exceeded.

Current limit: 10 http requests/sec

Action: Please try again after 1 minute.

WARNING: HTTP received 429
{
  "message": "API rate limit exceeded.",
  "type": "rate_limit_exceeded",
  "documentation_url": "https://software.ecmwf.int/wiki/display/WEBAPI/Web-API+Troubleshooting#Web-APITroubleshooting-HTTP429errors"
}
Error contacting the WebAPI, retrying in 60 seconds ...

Please check the ECMWF Service Status before reporting the problem

If the WEB-SERVICES status is green, it might be an Internal ECMWF server issue. Please report the problem

{
    "url"   : "https://api.ecmwf.int/v1",
    "key"   : "XXXXXXXXXXXXXXXXXXXXXX",
    "email" : "john.smith@example.com"
}

• Make sure you get a key from https://api.ecmwf.int/v1/key/.
• This key needs to be placed in ~/.ecmwfapirc.

Access to ECMWF servers is done over HTTPS. Your Python version does not support SSL (Secure Socket Layer) used by HTTPS.

To verify if SSL is enabled in your python , try the following:

>>> import socket
>>> socket.ssl
<function ssl at 0x4038b0>

If the output is not <function ssl at some_hex_number>, then you should rebuild or reinstall python with ssl enabled.

Symptoms: A Python WebAPI script terminates with "Error contacting the WebAPI, retrying in 60 seconds ...".  The WebAPI retries 10 times, followed by "WARNING: httplib2.URLError received None <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed"

The problem may be that the SSL library used in this Python version doesn't look in the correct certificate path.

• Try from another environment.
• Check which version of Python you are using and how it was installed and linked to openssl.
• Check that ca-certificates are updated on your system. Important, the client, openssl on most systems, maintains a set of trusted root CAs that are used to validate the chain (intermediate cert > server cert) on a ssl connection, so they must be updated on the client.
  • Update OpenSSL, ca-certificates and Python.

• • The following are some references on how to update the ca-certificates package:

Mac OS X
    sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/new-root-certificate.crt
    # Do not update OpenSSL through Homebrew, see https://stackoverflow.com/questions/15185661/update-openssl-on-os-x-with-homebrew

Windows
	# Type mmc.exe on search bar of the windows menu
	# Go to File > Add / Remove Snap In > Select Certificates > Click Add > Click OK
	# Expand Certificates - Current User
	# Right Click on Trusted Root Certification Authorities > All Tasks > Import
	# Follow the steps

	# alternatively
    certutil -addstore -f "ROOT" new-root-certificate.crt

Linux (Ubuntu, Debian)
    apt install ca-certificates
    # Copy your CA to dir /usr/local/share/ca-certificates/
    sudo cp foo.crt /usr/local/share/ca-certificates/foo.crt
    sudo update-ca-certificates

Linux (CentOs 6)
    yum install ca-certificates
    #Enable the dynamic CA configuration feature:
    update-ca-trust force-enable

    #Add it as a new file to /etc/pki/ca-trust/source/anchors/:
    cp foo.crt /etc/pki/ca-trust/source/anchors/

    update-ca-trust extract

Linux (CentOs 5)
    #Append your trusted certificate to file /etc/pki/tls/certs/ca-bundle.crt
    cat foo.crt >> /etc/pki/tls/certs/ca-bundle.crt

Anaconda
	conda update openssl

• • In case you need to add a specific Root Certificate, run the following command (The browser can be used to get the same information, if not on a *nix system, on view certificate):

user@local:~> openssl s_client -connect api.ecmwf.int:443                                                                                  
CONNECTED(00000003)
depth=2 C = BM, O = QuoVadis Limited, CN = QuoVadis Root CA 2 G3
...

Here we can see QuoVadis Root CA 2 G3 is the Root CA for api.ecmwf.int and can be downloaded from https://www.quovadisglobal.com/QVRepository/DownloadRootsAndCRL.aspx and installed locally (as per the commands specified above on the specific OS).

If you have the SSL error in the last step of your request (Transferring from...), you need to install the Root CA for "stream.ecmwf.int": QuoVadis Global SSL ICA G2

• The following is an hack for cases when updates are not possible for whatever reason:

  • Disable SSL Verification, this can be achieved by setting CURL_CA_BUNDLE="" before calling the python api:
    • CURL_CA_BUNDLE="" python main.py

• • Specify the Root CA directly, this can be achieved by setting REQUESTS_CA_BUNDLE="path to ROOT ca QuoVadis Root CA 2 G3" downloaded from the Quovadis Website (that your system cannot find somehow):
    • REQUESTS_CA_BUNDLE="/path_to_cert/QuoVadis_Root_CA_2_G3.pem" python main.py

• Connectivity network issues.
• Local disk space or file systems issues

Check if the results are still available in your job list

If not, you may wish to rerun the request

Check if you have installed the latest api client from our Web-API Downloads

export PERL_LWP_SSL_VERIFY_HOSTNAME=0

BEGIN { $ENV{"PERL_LWP_SSL_VERIFY_HOSTNAME"} = 0; }

Status: rejected

Reason: The total Nr of queued requests per user, allowed, on the API service has been exceeded

Current limit:  20 queued requests per user

Action: Please limit the Nr of your queued requests and try later

Status: rejected

Reason: The total Nr of queued requests in the API service has been exceeded. Presumably the API service is currently busy. Please accept our apologies for any inconvenience this may cause.

Current limit: 2000 requests in total

Action:  try  later you may also wish to check the  Web-API activity

Status: rejected

Reason: The total Nr of queued requests for the C3S ERA 5, in the API service has been exceeded.

Current limit:  4 queued requests per user for C3S ERA 5

Action:  ERA 5 is now available via the CDS, see how to migrate your scripts C3S ERA5: Web API to CDS API

Status: rejected

Reason: The Nr of fields per request  has been exceeded.

Current limit: 600.000 fields per request.

Action:  Split your request

Status: rejected

Reason: The allowed Nr of requests, per user  for this dataset has been exceeded.

Action:  Please limit the Nr of your queued requests for this dataset and try later

Status: rejected

Reason:This dataset has been phased out and it is not any more available from our services.  It is likely that the dataset has been superseded by a new dataset which is currently available.

Action: See the corresponding link for more details or contact us.

Status: rejected

Reason: The dataset is not any more available via the ECMWF public datasets. It has been either migrated to another service or phased-out etc

Action:  see below

Status: rejected

Reason: The dataset has been migrated to Copernicus Climate Data Store (CDS)

Action:  Please migrate your scripts to the new service.

Status: rejected

Reason: The dataset has been migrated to Copernicus Climate Data Store (CDS)

Action:  Please migrate your scripts to the new service.

Status: rejected

Reason: The dataset has been phased out. In most cases a phased out datasets has been superseded by another dataset.

Action: We encourage users to have a look on the project documentation.  Contact servicedesk@ecmwf.int if you have any further questions.

For any kind of ecCodes issues please visit the  ecCodes Frequently Asked Questions

• If you are experiencing a MARS "error" please check also the General MARS Troubleshooting page.
• Some additional information on some specific issues has  also been added in the list  below.

It is very important users to be aware of the MARS messages:

• In this case the application is attempting to connect to one of the MARS  archive servers, which is down.
• Please check the ECMWF Service Status before reporting the problem

• MARS error
• Please report the problem

• Status: aborted
• Reason: The volume of your request is too large 
• Options: Split your request to smaller ones
  

• Status: aborted
• Reason: You are requesting too many fields
• Options: Split your request to smaller ones
  

• Status: aborted
• Reason: You try to access restricted data
• Options: Adapt your request
  

• Status: aborted
• Reason:  Retrieved Nr of fields != expected Nr of fields. Some of the fields that you have requested are not available. eg some S2S parameters are not available from all pressure levels.
• Options:
  • If you have access to the Archive Catalogue, you can check data available here: https://apps.ecmwf.int/archive-catalogue/
  • If you have access to the MARS Catalogue, you can check the data available here: http://apps.ecmwf.int/mars-catalogue/
  • For Public Datasets, check the availability of the dataset using the ECMWF Public Datasets web interface and adapt your request to reflect the availability.

• Status: aborted
• Reason:  This particular period of ERA 5 (1979-1999 )  will be made available only via the Copernicus Climate Data Store (CDS) shortly
  
• Options: Please migrate your scripts to the new service and wait until the data is made available.

For any other MARS problem you can visit the MARS Trouble-shooting

The problem is  that you repeat some MARS keyword values

 see also

https://confluence.ecmwf.int/display/UDOC/Retrieve#Retrieve-Assertionfailed

Messages starting with Failed HPSS call: n = ::hpss_Read(fd_,buffer,... are HPSS errors. The actual message text can vary but they usually mean that the data is unavailable from tapes for some time.

 see also

Retrieve#FailedHPSScall