Page History

...

File Formatting

• The output files are written through the NetCDF API
• The NETCDF4 _CLASSIC model will be adopted
• Recommended compression level deflate=6
• Shuffling=True
• Fletcher32=True is strongly recommended
  

File Structure

• Each netCDF4 file contains a single output variable (along with coordinate/grid variables, attributes and other metadata) from a single model and a single simulation (i.e., from a single ensemble member and a single start date)There is flexibility in specifying how many time slices (samples) are stored in a single file. A single file can contain all the time-samples for a given variable and climate experiment, or the samples can be distributed in a sequence of files.
• Recommended maximum file size of 4GB

• A file containing a hash created with sha256sum should be created for each file

  Code Block
  language bash title Create hash files
  sha256sum filename.nc > filename.sha256

  
  

File Naming

<institute>_<stream><institute_id>_<model_id tag>_<forecast_type>_<start date identifier>_<modeling realm>_<frequency>_<level>_<production date and start date identifier>_<data year><data month><data day>[-<data year><data month><data day>]_<variable MARS <level_type>_<variable name>_<ensemble member>

examples:

egrr_enfh_atmos_day_sfc_201601A_19950417-19950418_ta_r3.ncegrr

<model_enfh_atmos_month_plev_201601A_199504-199505_ta_r3.nc

"201601A" is  a placeholder while a form for representing the model version, production year and startdate is determined:

egrr_enfh_atmos_month_plev_P2016_M1A_S19950401_199504-199505_ta_r3.nc

P=production year

M=model version

S=startdate

e.g. could the filename (alternatively) be something like:

...

Metadata

...

id_tag> as it is defined in the description of the "source" global attribute
<institute_id>, <forecast_type>, <modeling_realm>, <frequency> and <level_type> coming from the global attributes of the same name

<start_date_identifier> being an string "SYYYYMMDDHH"
<variable_name> from the netCDF name of the variable (short name)
<ensemble_member> from the 'realization' coordinate value

NOTE: The file name should be able to be rebuilt from the contents of the file

Metadata

• The metadata is constrained by the CF convention (NetCDF Climate and Forecast (CF) Metadata Convention)
• This is also based on information from the SPECS_standard_output.pdf document, the CMIP5 tables and the Attribute Convention for Data Discovery.
  • Attributes currently are a significantly reduced subset of the SPECS/CMIP5 requirements with minor adaptations

...