Page tree
Skip to end of metadata
Go to start of metadata

This is a recipe for first time users of ERA5 data and ECMWF's data server.

1. Learn about ERA5 data

  1. Read What is ERA5.
  2. Browse the ERA5 catalogue and make sure the data you are interested in is available in ERA5
  3. Read the ERA5 data documentation and take note of the parameters (fields) you want to use, and of the corresponding values for data type, stream, level type (surface, pressure levels, model levels, etc) base time and forecast steps.

2. License agreement and user account

  1. You need an account on www.ecmwf.int, so we have a record of you accepting the license agreement. If you do not have an account, register an account now.
  2. Read the Copernicus license agreement and accept the agreement at the bottom the page.

3. Technical prerequisites

You need:

  • A computer with a *nix operating system is highly recommended. Microsoft Windows is known to work too, but is unsupported, see here for details.
  • The Python programming language. If you do not have it installed yet, install Python now.

4. Set up your computer

This is to configure your computer to programmatically retrieve data from ECMWF. You only need to do this once, and only if you have never retrieved ECMWF data using Python before. For more information refer to Access ECMWF Public Datasets of ECMWF WebAPI.

1. Install the ECMWF WebAPI client library by running:

 

sudo pip install https://software.ecmwf.int/wiki/download/attachments/56664858/ecmwf-api-client-python.tgz

2. Get and install an ECMWF API key:

  1. Go to https://apps.ecmwf.int/auth/login/ to verify you are logged in. If you are not, log in with your ECMWF account details (as registered in step 2).

  2. Go to https://api.ecmwf.int/v1/key/ . You will see something like below:

 

{
    "url"   : "https://api.ecmwf.int/v1",
    "key"   : "XXXXXXXXXXXXXXXXXXXXXX",
    "email" : "john.smith@example.com"
}

3. Copy the bit with the curly brackets as shown above into a text file and save the file as $HOME/.ecmwfapirc (on Unix/Linux) or %USERPROFILE%\.ecmwfapirc (on Windows)

5. Run a test

This is to verify that your computer is set up correctly.

1. Copy the following Python script to a text file and save it, for example as 'my_ERA5_test_script.py'

#!/usr/bin/env python
from ecmwfapi import ECMWFDataServer

server = ECMWFDataServer()
server.retrieve({
    "class": "ea",
    "dataset": "era5",
    "expver": "1",
    "stream": "oper",
    "type": "an",
    "levtype": "sfc",
    "param": "165.128/166.128/167.128",
	"date": "2016-01-01/to/2016-01-02",
    "time": "00:00:00",
    "step": "0",
    "target": "my_ERA5_test_file.grb",
 })

2. Run the script. On most computers you would do this by opening a command prompt and typing

python my_ERA5_test_script.py

The test request will download data from ECMWF and save as file 'my_ERA5_test_file.grb' in the directory you issued the command from.

Note that the data retrieval is not designed to be instant. The test request above usually takes a few minutes to complete, but depending on demand can take longer.

3. With your favourite GRIB file reader verify that you can read the file.

If you get an error message or the output is not as specified in to the Python script, most likely your computer setup is wrong, or you did not accept the ERA5 license, so please go back to the previous steps.

6. Create your data retrieval script

  1. Browse the ERA5 catalogue for the data you are interested in, and in the last step make a selection in all boxes and click 'View the MARS request'. This shows a template Python script with your selected options.
  2. Copy the Python script to a text file and save it, for example as 'my_ERA5_script_v1.py'.
    1. If you specify "format" as "netcdf", then you will need to set "grid". Otherwise, GRIB to netCDF conversion will fail.
  3. Adapt the Python script to your requirements,e.g. change date, time and grid. Here is some guidance:
#!/usr/bin/env python
from ecmwfapi import ECMWFDataServer

server = ECMWFDataServer()
server.retrieve({
    "class": "ea",						# Do not change
    "dataset": "era5",					# Do not change
    "expver": "1",						# Do not change
    "stream": "oper",					# can be "oper", "wave", etcetera; see ERA5 catalogue (http://apps.ecmwf.int/data-catalogues/era5) and ERA5 documentation (https://software.ecmwf.int/wiki/display/CKB/ERA5+data+documentation)
    "type": "an",						# can be an (Analysis) or fc (forecast) or 4v (4D variational analysis)
    "levtype": "sfc",					# can be "sfc", "pl", "ml", etcetera; see ERA5 documentation
    "param": "10u/10v/2t",				# Parameters you want to retrieve. For available parameters see the ERA5 documentation. Specify here using shortName or paramID, and separated by '/'. 
	"date": "2016-01-01/to/2016-01-31",	# Set a single date as "YYYY-MM-DD" or a range as "YYYY-MM-DD/to/YYYY-MM-DD".
    "time": "00:00:00/12:00:00",		# If above you set "type":"an", "time" is the time of analysis. If above you set "type":"fc", "time" is the initialisation time of the forecast. 
    "step": "0",						# The forecast step. If above you set "type":"an", set "step":"0". If above you set "type":"fc", set "step" > 0.
    "grid": "0.3/0.3",					# Optional. The horizontal resolution in decimal degrees. If not set, the archived grid as specified in the data documentation is used.
    "area": "75/-20/10/60",				# Optional. Subset (clip) to an area. Specify as N/W/S/E in Geographic lat/long degrees. Southern latitudes and western longitudes must be
                                        # given as negative numbers. Requires "grid" to be set to a regular grid, e.g. "0.3/0.3".
    "format": "netcdf",					# Optional. Output in NetCDF format. Requires that you also specify 'grid'. If not set, data is delivered in GRIB format, as archived.
    "target": "CHANGEME",				# Change this to the desired output path and file name, e.g. "data1.nc" or "./data/data1.grib". The default path is the current working directory.
})

For available "keyword":"value" combinations please browse the ERA5 catalogue for the data you are interested in, and in the last step make a selection in all boxes and click 'View the MARS request'. This shows a template Python script with your selected options. For ERA5 ensemble data, the 'number' field identifies the 10 ensemble members.

Limitations:

  • You can have a maximum of three active requests at a time.
  • The maximum data volume is 20GB per request.
  • The maximum number of fields to be retrieved is 600,000 fields per request.

Note that the data retrieval is not designed to be instant. A larger request can take hours and even days to complete.

To retrieve data efficiently (and get your data quicker!) you should retrieve all the data you need from one tape, then from the next tape, and so on. In most cases this means retrieving all the data you need for one month, then for the next month, and so on. See ERA5 retrieval efficiency.

7. Run your data retrieval script

On most computers you would do this by opening a command prompt and typing

python my_ERA5_retrieve_script_v1.py

This will retrieve the data as specified and download it as a single file to your computer into the current directory (or whichever "target" directory you specified).

For long running processes you can check the progress of your request in your job list.

8. Check results

Check that the data you downloaded meets your requirements.

To report an issue or bug please contact Copernicus Support at ECMWF