Aviso is a system developed at ECMWF with the aim of:

• Notifying for data availability
  • Real-Time Model Output Data
  • Product dissemination via ECPDS
• Meant for automated systems
• Based on an Event system supporting a mechanism When <this> … Do <that> …

It allows users to:

• Define which events to be notified of.
• Define the triggers to be executed once a notification is received.
• Dispatch notifications to the notification server.

Content

Install AVISO and configure for ECMWF events

Aviso can be used as a Python API or as Command-Line Interface (CLI) application. Here a few steps to quickly get a working configuration listening to notifications.

Note, this guide assumes the user to have a ECMWF account.

1. Install AVISO following instructions on: https://pyaviso.readthedocs.io/en/latest/guide/getting_started.html#installing 
   
   
2. Configure MARS credentials following instructions on Access MARS
   
   

3. Create a configuration file in the default location ~/.aviso/config.yaml

   Code Block
   language yml
   username_file: ~/.marsrc/mars.email key_file: ~/.marsrc/mars.token notification_engine: type: etcd_rest host: aviso.ecmwf.int port: 443 https: true

Install AVISO and configure for ECMWF events

Aviso can be used as a Python API or as Command-Line Interface (CLI) application. Here a few steps to quickly get a working configuration listening to notifications.

Note, this guide assumes the user to have a ECMWF account.

1. Install AVISO following instructions on: https://pyaviso.readthedocs.io/en/latest/guide/getting_started.html#installing 
2. Configure MARS credentials following instructions on Access MARS

3. Create a configuration file in the default location ~/.aviso/config.yaml

   Code Block
   language yml
   username_file: ~/.marsrc/mars.email key_file: ~/.marsrc/mars.token notification_engine: type: etcd_rest host: aviso.ecmwf.int port: 443 https: true configuration_engine: type: etcd_rest host: aviso.ecmwf.int port: 443 https: true schema_parser: ecmwf remote_schema: True auth_type: ecmwf

   
   

4. Create ~/.marsrc/mars.email and ~/.marsrc/mars.token files. mars.email file should contain “email” listed here: https://api.ecmwf.int/v1/key and mars.token file should contain “key” shown on the same page.
   
   
5. Crate a Create your listener configuration file(s) with following content:
   1. Example for dissemination:
      

      Code Block
      language yml title listener_diss.yaml
      listeners: - event: dissemination request: destination: <user_destination> class: od expver: 1 domain: g stream: enfo step: [1,2,3] triggers: - type: echo

      Note the dissemination event listener. request describes for which dissemination event users want to execute the triggers. It is made of a set of fields. Users have to specify only the fields they wants to use as filters. destination is a mandatory field and it is associated to one or more destinations which are linked to the user's ECMWF account. Only the notifications complying with all the fields defined will execute the trigger. The trigger in this example is echo. This will simply print out the notification to the console output.
      
      
   2. Example for MARS:
      

      Code Block
      language yml title listener_mars.yaml
      listeners: - event: mars request: class: od expver: 1 domain: g stream: enfo step: [1,2,3] triggers: - type: echo

      
      Note, in the MARS event listener, destination field is not present.
      
      

      Warning

      Aviso will notify when data exists in MARS, but most users will not be able to retrieve this data before the scheduled time (Dissemination schedule). Triggering an automated retrieval of data will require adding logic to ensure you only retrieve data when you have respective permissions to do so. For automation of downstream time-critical workflows, this should mainly be based on dissemination events.

      
      
      
6. If AVISO is installed in Python virtual environment, activate the environment first:
   If AVISO is installed in conda or Python virtual environment, activate the environment first:Python virtual environment: "myenv"
   Code Block

   language	bash

   Code Block
   language bash

   source {PATH_TO_MY_ENV}/myenv/bin/activate

   Conda:
   

   Code Block
   language bash
   conda activate

   
   
   
7. Launch the AVISO application Launch the AVISO application to listen ECMWF defined in your configuration file:
   

   Code Block
   language bash
   aviso listen listener_diss.yaml #or aviso listen listener_mars.yaml

Once in execution this command will create a process waiting for notifications. Users can terminate the application by typing CTRL+C .
Note, the configuration file is only read at start time, therefore every time users make changes to it they need to restart the listening process.

Upgrading pre-existent installation

To upgrade Aviso to a newer version once it is available run the following command:

...

...

pip3 install --upgrade pyaviso

 Defining your listeners

Aviso configuration file allows the definition of multiple listeners. Alternatively, the listeners configuration can be indicated as an independent file or multiple files:

...

triggers:
  - type: command
    working_dir: $HOME/aviso/examples
    command: ./script.sh --datestream ${request.datestream} -s-date ${request.streamdate}
 --time ${request.time} --step ${request.step}
    environment:
      STEP: ${request.step}
      TIME: "The time is ${request.time}"

...

Here an example file of a Python script running Aviso and executing a MARS request after a notification is received.

Dealing with past notifications

Before listening to new notifications, Aviso by default checks what was the last notification received and it will then return all the notifications that have been missed since. It will then carry on by listening to new ones. The first ever time the application runs however no previous notification will be returned. This behaviour allows users not to miss any notifications in case of machine reboots.

notification is received.

Dealing with past notifications

Before listening to new notifications, Aviso by default checks what was the last notification received and it will then return all the notifications that have been missed since. It will then carry on by listening to new ones. The first ever time the application runs however no previous notification will be returned. This behaviour allows users not to miss any notifications in case of machine reboots.

To override this behaviour by ignoring the missed notifications while listening only to the new ones, run the following:

aviso listen --now

This command will also reset the previous history.

Users can also explicitly replay past notifications. Aviso can deliver notifications from the ECMWF server up to 14 days in the past. This can also be used to test the listener configuration with real notifications.​

Here an example, launch Aviso with the following options:​To override this behaviour by ignoring the missed notifications while listening only to the new ones, run the following:

aviso listen --now

This command will also reset the previous history.

Users can also explicitly replay past notifications. Aviso can deliver notifications from the ECMWF server up to 14 days in the past. This can also be used to test the listener configuration with real notifications.​

Here an example, launch Aviso with the following options:​

aviso listen --from 2020-01-20T00:00:00.0Z --to 2020-01-21T00:00:00.0Z

It will replay all the notifications sent from 20 January to 21 January and the ones complying with the listener request will execute the triggers.

Note, the dates must be in the past and --to can only be defined together with --from. The dates are defined in ISO format and they are in UTC.

...

 listen --from 2020-01-20T00:00:00.0Z --to 2020-01-21T00:00:00.0Z

It will replay all the notifications sent from 20 January to 21 January and the ones complying with the listener request will execute the triggers.

Note, the dates must be in the past and --to can only be defined together with --from. The dates are defined in ISO format and they are in UTC.

In absence of --to, the system after having retrieved the past notifications, it ​will continue listening to future notifications. If --to is defined Aviso will terminate once retrieved all the past notifications.

Examples

There is a GitHub repository with collection of example Python scripts and listener configuration files specifically designed for the AVISO software and notifications generated by ECMWF. These examples are intended to illustrate how AVISO can be effectively used with data distributed by ECMWF and we strongly recommend going through the examples if you intend to use AVISO to get notifications about availability of ECMWF data.

Upgrading AVISO installation

To upgrade Aviso to a newer version once it is available run the following command:

pip3 install --upgrade pyaviso

Running as a service on Linux machines

...