Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Define which events to be notified of.
  • Define the triggers to be executed once a notification is received.
  • Dispatch notifications to the notification server.


Content

Table of Contents

Children Display

Install AVISO and configure for ECMWF events

...

  1. Install AVISO following instructions on: https://pyaviso.readthedocs.io/en/latest/guide/getting_started.html#installing 

  2. Configure MARS credentials following instructions on Access MARS

  3. Create a configuration file in the default location ~/.aviso/config.yaml

    Code Block
    languageyml
    username_file: ~/.marsrc/mars.email
    key_file: ~/.marsrc/mars.token
    notification_engine:
      type: etcd_rest
      host: aviso.ecmwf.int
      port: 443
      https: true
    configuration_engine:
      type: etcd_rest
      host: aviso.ecmwf.int
      port: 443
      https: true
    schema_parser: ecmwf
    remote_schema: True
    auth_type: ecmwf 


  4. Create ~/.marsrc/mars.email and ~/.marsrc/mars.token files. mars.email file should contain “email” listed here: https://api.ecmwf.int/v1/key and mars.token file should contain “key” shown on the same page.

  5. Crate Create your listener configuration file(s) with following content:
    1. Example for dissemination:
      Code Block
      languageyml
      titlelistener_diss.yaml
      listeners:
        - event: dissemination
          request:
            destination: <user_destination>
            class: od
            expver: 1
            domain: g
            stream: enfo
            step: [1,2,3]
          triggers:
            - type: echo
      Note the dissemination event listener. request describes for which dissemination event users want to execute the triggers. It is made of a set of fields. Users have to specify only the fields they wants to use as filters. destination is a mandatory field and it is associated to one or more destinations which are linked to the user's ECMWF account. Only the notifications complying with all the fields defined will execute the trigger. The trigger in this example is echo. This will simply print out the notification to the console output.

    2. Example for MARS:
      Code Block
      languageyml
      titlelistener_mars.yaml
      listeners:
        - event: mars
          request:
            class: od
            expver: 1
            domain: g
            stream: enfo
            step: [1,2,3]
          triggers:
            - type: echo

      Note, in the MARS event listener, destination field is not present.


      Warning

      Aviso will notify when data exists in MARS, but most users will not be able to retrieve this data before the scheduled time (Dissemination schedule). Triggering an automated retrieval of data will require adding logic to ensure you only retrieve data when you have respective permissions to do so. For automation of downstream time-critical workflows, this should mainly be based on dissemination events.



  6. If AVISO is installed in Python virtual environment, activate the environment first:
    If AVISO is installed in conda or Python virtual environment, activate the environment first:Python virtual environment: "myenv"
    Code Block
    languagebash
    source {PATH_TO_MY_ENV}/myenv/bin/activate
    Conda:
    Code Block
    languagebash
    conda activate


  7. Launch the AVISO application Launch the AVISO application to listen ECMWF defined in your configuration file:
    Code Block
    languagebash
    aviso listen listener_diss.yaml
    #or
    aviso listen listener_mars.yaml


Once in execution this command will create a process waiting for notifications. Users can terminate the application by typing CTRL+C .
Note, the configuration file is only read at start time, therefore every time users make changes to it they need to restart the listening process.

Upgrading AVISO installation

To upgrade Aviso to a newer version once it is available run the following command:

...

languagebash
themeDJango

...



 Defining your listeners

Aviso configuration file allows the definition of multiple listeners. Alternatively, the listeners configuration can be indicated as an independent file or multiple files:

...

Code Block
languageyml
triggers:
  - type: command
    working_dir: $HOME/aviso/examples
    command: ./script.sh --datestream ${request.datestream} -s-date ${request.stream}
   date} --time ${request.time} --step ${request.step}
    environment:
      STEP: ${request.step}
      TIME: "The time is ${request.time}"

...

In absence of --to, the system after having retrieved the past notifications, it ​will continue listening to future notifications. If --to is defined Aviso will terminate once retrieved all the past notifications.


Examples

There is a GitHub repository with collection of example Python scripts and listener configuration files specifically designed for the AVISO software and notifications generated by ECMWF. These examples are intended to illustrate how AVISO can be effectively used with data distributed by ECMWF and we strongly recommend going through the examples if you intend to use AVISO to get notifications about availability of ECMWF data.


Upgrading AVISO installation

To upgrade Aviso to a newer version once it is available run the following command:

Code Block
languagebash
themeDJango
pip3 install --upgrade pyaviso

Running as a service on Linux machines

...