Skip to end of metadata
Go to start of metadata

Click on the question or the small triangle to the left to show the answer.

ECMWF related questions
 Q. How do I get an account on the ECMWF computer system?

A. All OpenIFS users will be given full access to this website. Access to other ECMWF systems may not always be required and your needs should be discussed with the OpenIFS team or User Support. Member state users should always apply for an account via their national Met Service representative. If you want to contact ECMWF prior to applying to your national Met Service then please see this list for the appropriate ECMWF User Support contact to ask: http://www.ecmwf.int/en/computing/get-support/user-support-contact-points

If you are a user from a non ECMWF member state you should contact openifs-support@ecmwf.int to discuss your needs.

 Q. Where can I find information about the different IFS cycles?

A. Information about the changes between IFS cycles can be found on the main website 'Evolution of the ECMWF analysis and forecast system'. The ECMWF newsletter publish details of changes to the operational system.

 Q. I can't retrieve files from the ECMWF ftp server. I get an error: ftp:bind: Address already in use.

A. This is usually caused by a firewall at the user site blocking ftp connections. There are two types of ftp connections, so-called 'active' and 'passive'. The essential difference is that 'active' mode tries to open multiple connections to the ECMWF ftp server. This behaviour is undesirable for sites running firewalls and is usually blocked. Although passive mode ftp is usually the default, if you see this error try using the '-p' option 'ftp -p' or, if it's available the command 'pftp' which invokes a passive mode only ftp.

Compiling OpenIFS / FCM related questions
 Q. I get the error: Can't locate Time/Piece.pm in @INC

This error occurs because some modules are missing from the perl installation on your computer. We use the FCM software to compile OpenIFS which uses Perl.  Please see the Prerequisites page for more information about this and how to resolve the error.

 Q. I get the error: [FAIL] master.exe: target not found after an update:

 A. When compiling this error message appears:

[FAIL] make oifs           # 94.3s
[FAIL] make                # 94.8s
[FAIL] master.exe: target not found after an update:
[FAIL] /nfs01/n02/n02/emgdc/git/openifs/make/noopt/oifs/bin/master.exe: expect target file

Although make finished, the executable file 'master.exe' was not created. This means an error occurred at the final 'linking' stage when all the compiled object files are combined to make the final executable.

To see what actually caused the problem, look in the logfile generated by the fcm make command. It will be file make/$OIFS_BUILD/.fcm-make/log. Note the '.' in front of the .fcm-make directory name. This directory is 'hidden'. e.g. If you have: OIFS_BUILD=noopt, then the log file from the build will be in make/noopt/.fcm-make/log.

If you look in the logfile, go down to the bottom and look for any errors such as 'undefined reference' which prevented the final link step. A undefined reference means a subroutine/function could not be found either because a library is not included or some code files are missing.

 Q. The command 'fcm make' gives the error: mpif90: command not found?

A. This is because the FCM configuration files assume the use of a 'compiler wrapper' which is typically provided by MPI (e.g. OpenMPI or MPICH) installations. Make sure you have a flavour of MPI installed and you have the 'bin' directory included on your PATH environment variable. For example, on linux use 'apt-get install libopenmpi-dev' to get OpenMPI. HPC machines may require you to use a 'module' command e.g 'module load openmpi': check your local help information.

 Q. I get this error, what does it mean? "[FAIL] cpg1c.intfb.h: missing dependency (type=include)"

A. FCM has found an include statement: "#include cpg1c.intfb.h" in the source code but can't find the source file cpg1c.F90. Make sure the file exists or FCM is configured to look at all the sources.

 Q. How do I change where FCM builds the library archive (libmaster.a) file before the link stage?

A. FCM queries the TMPDIR environment variable. If set it uses that location, otherwise it defaults to /tmp. On HPC systems, /tmp may not be the best choice if it is in memory.

 Q. What does the $HERE variable in FCM configuration files mean?

A. The $HERE variable is a special variable that FCM interprets as the directory in which the configuration file resides. Unlike most of the other variables in the FCM configuration files, $HERE cannot be overridden by an environment variable of the same name.

 Q. How do I change the compiler used by my MPI compiler wrappers?

A. For more information on how to correctly set your environment for compiling OpenIFS, see: HowTos: How to set your user environment.

It varies depending on your MPI implementation and your local environment. As an example, for OpenMPI, set the environment variables:
export OMPI_FC=pgf90
export OMPI_CC=pgcc
to use the Portland compilers instead of the default GNU (gfortran) compilers. Check the 'man page' of the mpif90 command to see what other options are possible. Some implementations allow you to use an option -fc to set the underlying compilers.

For Intel installations there are several ways in which the compiler can be set. Normally, the mpif90 command will invoke the gfortran compiler, the mpiifort will invoke the ifort compiler and the mpifc command can be used with a specified compiler on the command line. However, this behaviour can be changed by setting environment variables which these wrapper scripts will read. For more details consult the Intel compiler documentation.

Some HPC centres provide the 'module' command for loading and unloading particular compiler suites. Consult your local documentation for more details.

 Q. I get an error about missing shared object file: error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory.

A. This can arise with the Intel compiler. The MKL library is Intel's version of the LAPACK and BLAS libraries which should be supplied with the compiler. The .so file extension means this is a shared library found at runtime. In order to find this library, the environment variable LD_LIBRARY_PATH must be set correctly. You will need to find the location of this library and add the directory to the LD_LIBRARY_PATH. Normally this is done for you by a small script supplied with the Intel compiler script. Please check your support information.

 Q. I get undefined references to 'dgemm_', 'dgemv_', 'sgecon_' etc.

A. These routines come from the LAPACK and BLAS libraries. You may have one or other of these libraries missing from your .cfg config (or OIFS_LAPACK_LIB environment variable). Alternately you have specific both libraries but the compiler can't find them; check the pathnames to the libraries specified on the compile link carefully.

 Q. I get undefined references to: 'jas_stream_memopen', 'jpc_decode', 'jas_matrix_create', 'jas_image_destroy', 'jpc_decode'

A. These are references to the JASPER library and arise because the grib_api library referred to in the OpenIFS build configuration was compiled with the Jasper library, but the LD_LIBRARY_PATH environment variable does not list the directory containing the library so the compiler can't find it. Either amend the LD_LIBRARY_PATH or add the -ljasper library to your compilation options.

 Q. I get the error : There is no specific subroutine for the generic 'grib_write_bytes' when compiling grib_api_interface...

A. This error is most likely because the version of grib-api is too old. For example, OpenIFS 40R1 requires a minimum version of grib-api version 1.11.0. Please upgrade your version of grib-api.

 Q. Should I use the same compiler for OpenIFS, LAPACK, OpenMPI etc?

A. It's recommended that the same compiler suite is used to compile OpenIFS, your MPI installation and LAPACK. This may not always be practical but it's strongly recommended as some compilers use a different default size of LOGICAL variables which can cause crashes due to memory alignment problems in subroutine calls.

 Q. Where is the log file from compiling OpenIFS kept?

A. The log output from compiling OpenIFS can be found in the 'make' directory, then in the build directories 'opt', 'noopt' etc. The FCM software keeps the log in a hidden directory called '.fcm-make'. So if OIFS_BUILD=opt was set, the logfile for this compilation would be : make/opt/.fcm-make/log.

OpenIFS questions: general and runtime
 Q. What does the version numbering, CY38R1v01 mean?

A. The 'CY' refers to the model 'cycle'. The 'R' refers to the IFS model release. The 'v01' refers solely to OpenIFS and is the version of OpenIFS based on that IFS cycle and release.

ECMWF and Meteo-France jointly develop the IFS model. A new cycle is formed when the two centres merge their code changes, typically about once a year. A model release is specific to ECMWF (the 'R' actually represents 'Reading,UK') and there are usually at most 2 or 3 releases of the model before a new cycle is produced from the next merge with Meteo-France.

The version number refers just to OpenIFS. As OpenIFS model versions have a much longer lifetime than IFS versions, OpenIFS is periodically updated with: technical changes for new compiler versions, new hardware, bugfixes from more recent IFS releases, new functionality and so on.

Changes to the version number will not alter the model's results. Only changes to the cycle and release number will alter the scientific results from the model.

 Q. What does the 'T' mean in 'T511', 'T1279' etc?

A. The 'T' refers to the spectral truncation of the model and the number following is the highest retained wavenumber. A spectral method, based on spherical harmonic expansion is used in the model, for example to compute some horizontal derivatives. The 'T' means the model uses a 'triangular' truncation in spectral space to decide the retained spherical harmonics. The number following the 'T' is the highest retained wave number. For more information on the spectral truncation and resolution of the model and the model grid, the IFS documentation is recommended reading.

T159 is approximately 125km resolution, T255 - 80km, T511 - 40km, T799 - 25km, T1279 - 16km.

 Q. How do I know the grid from from the 'T' number?

A. This depends on the grid used. For medium and high resolutions, the model uses a 'reduced Gaussian grid' with reduced numbers of gridpoints along a latitude circle towards the poles. Full quadratic and linear grids are also possible. The following table shows the correspondence between spectral resolution and lat/lon resolution. 'TL' means linear grid with a triangular spectral truncation. 'T' means a quadratic grid with triangular spectral truncation. 'N' is the number of latitudes between equation and poles. The grid definitions can also be found here: reduced gaussian grid definitions.

SpectralGaussianLat/lon
T63N481.875
TL95N481.875
T106N801.125
TL159N801.125
T213N1600.5625
TL255N1280.7
TL319N1600.5625
TL399N2000.450
TL511N2560.351
TL639N3200.28125
TL799N4000.225
TL1023N5120.176
TL1279N6400.141
TL2047N10240.088
 Q. What is the difference between IFS/OpenIFS model versions?

A. The OpenIFS model is based on the IFS code though it is only capable of forecasts, no data assimilation is provided. The first OpenIFS version was based on cycle 38 release 1 (known as CY38R1). More information is found on the page comparing OpenIFS .v. IFS. OpenIFS is released less frequently than the IFS so model versions are not always consecutive.

Information on scientific and technical changes for operational cycles is available on the main ECMWF website.

 Q. What format is the model output?

A. The model writes its output in GRIB format using WMO FM-92 GRIB edition 1 and edition 2 messages. ECMWF provides the GRIB_API software for handling and manipulating grib data using either command line tools and a programming library for C, Fortran and Python.

For plotting GRIB data, ECMWF provides the Metview software, which is freely available under the Apache license.

Note that the OpenIFS model outputs a mix of GRIB edition 1 & 2 messages in a single file. Unlike Metview, not all GRIB software available on the Internet can read such files and extra steps may be necessary to separate the messages before use if you want to use 3rd party software.

 Q. How do I check the model is working correctly on my system after I've installed it?

A. The model can be run in 'reference' mode where it computes statistical numerical norms and compares against values calculated at ECMWF. Please see 'Testing the Installation' for more details.

 Q. What timestep should I use?

A. The timestep depends on the horizontal and vertical resolution, the type of grid (full, linear, cubic reduced Gaussian), whether Eulerian or semi-Lagrangian (SL) transport is used and whether the 2 or 3 time-level SL scheme is used. The recommended default is the 2 time-level SL scheme. For this configuration, timesteps that give a good combination of run-time and forecast are: T21 - T159 : 60mins;  T255: 45mins;  T399: 20mins;  T511: 15mins;  T1279 10mins.

If the 2 time-level scheme is not used, these timesteps should be halved. If Eulerian, not semi-Lagrangian advection is used, these timesteps should be a quarter. For more details, see the code in the routine sudyn.F90 which has approximate values for the timestep (n.b. the NAMDYN namelist variable TSTEP is in units of secs).

Note the model will calculate the optimum timestep for its particular configuration. This value is written to the NODE_001.01 output file. Look for the line:
    PTSTEP_OPT (optimal timestep computed in SUTSTEP) 0.36000000E+04
to see a suggested value in seconds. However, please use these values as a rough guide only. This code has been removed in later versions of IFS.

With longer timesteps, it is possible to get errors in the NODE.001.01 log file regarding trajectories going underground. Whilst some warnings may appear with the recommended timestep values above, the timestep should be reduced with excessive numbers of warnings.

 Q. OpenIFS fails with ABORT! 1 SU_MCICA:ERROR OPENING FILE MCICA

A. The file 'MCICA' is a climatological input file that the model reads on the first time step. The model expects to read this file and all the climatological files from a sub-directory called 'ifsdata' in the same directory containing the binary executable (e.g. master.exe). See Downloading OpenIFS for more details on how to download and install the data files. It is recommended that the data files are kept separate from the directories where model runs are done and a link to the climatology data directory is made. e.g. if the ifsdata tar files have been unpacked in a directory $HOME/ecmwf/ifsdata then use the command ln -s $HOME/ecmwf/ifsdata/climatology ifsdata in the run directory to make sure the MCICA file is available. Contact OpenIFS support for further help.

 Q. OpenIFS fails with an error message from GRIB_API.

A. This usually indicates an error in the GRIB file or GRIB_API configuration rather than the model. Try using the 'grib_ls' or 'grib_dump' commands on the file causing the problem to try to reproduce the error. See the GRIB FAQ on this page, or email openifs-support@ecmwf.int and include the model output file (NODE_001.01) and the job log.

 Q. OpenIFS fails with GRIB_API_ERROR: unable to locate sample file gg_sfc_grib1.tmpl

A. This error means the model can't find the GRIB templates it needs to write out the model results. These are contained in a directory called 'ifs_samples' which is provided with the grib_api library. First check that the directory pointed to by OIFS_GRIB_API_DIR exists and is readable. Then check if the directory $OIFS_GRIB_API_DIR/share/grib_api/ifs_samples exists and is readable. If it does then it's possible that the library may have been moved (or a link to it changed) and because these paths are hardcoded into the grib_api library at compile time, it means the model can't find it. To overcome this, set the environment variable:

GRIB_SAMPLES_PATH - Set to the folder containing the set of samples you want grib_api to use instead of the default one.

e.g.

export GRIB_SAMPLES_PATH=$HOME/ecmwf/grib_api/share/grib_api/ifs_samples/grib1_mlgrib2

 Q. OpenIFS fails (segmentation violation) when compiled with optimized compiler options ('opt' configuration) but works without any optimization ('noopt' configuration).

A. This could be for a number of reasons. However, a common one is that when compiled optimized the model requires more memory.  With the optimized settings OpenMP is enabled. This causes the model to require more what's known as 'stack memory'. To increase your stack memory to the maximum allowed on your system, use the command 'ulimit -s unlimited'. If that does not solve your problem, email openifs-support@ecmwf.int.

 Q. OpenIFS fails with error: Dr.Hook has detected an invalid key-pointer/handle while leaving the routine 'RPASSF'

A. This normally means that the OpenMP stacksize is not large enough. The default per process stack size limit is too low and needs to be increased. Check the value of the OMP_STACKSIZE environment variable. It might also mean the model was not compiled correctly. RPASSF is a fortran77 routine and must be compiled with auto-double options to promote REAL*4 to REAL*8 to match the rest of the model. This is set by default in the compiler configuration files but check this is compiled correctly. Contact openifs-support@ecmwf.int for further help.

 Q. OpenIFS fails with error: MPL_BUFFER_METHOD: KPROCIDS NOT CORRECT FROM PROCESSOR 1

A. This means that the number of MPI processes (NPROC) specified in the job control language (PBS, SLURM, etc) in the job script and the value in the fort.4 namelist file are different. They must be the same.Contact openifs-support@ecmwf.int for further help.

 Q. OpenIFS hangs and I don't get any output?

A. This could be for a number of reasons.

One possibility is that the buffer space for the MPI communication is not big enough. If you get a NODE.01_001 output file (the model log), look for the line containing "MBX_SIZE" e.g.

 MP_TYPE=          2 MBX_SIZE=  128000000

If your value of MBX_SIZE is less than this then change it in the fort.4 namelist file:

&NAMPAR0
 MP_TYPE=2,
 MBX_SIZE=128000000,

Another possibility is that the model is configured as if running at ECMWF where model I/O is via a different method to a special database. Check that the NODE_001.01 output file and make sure that in namelist NAMCT0, the variable LFDBOP is .FALSE. and in namelist NAMPAR1, the variable NOUTTYPE=1.If not, change these in the fort.4  namelist file.

If this does not solve the problem, please contact openifs-support@ecmwf.int.

 Q. OpenIFS runs but does not generate any output.
 Q. What are the standards for writing code for OpenIFS?

Coding standards for IFS (and it's sister models) are provided by MeteoFrance here: http://www.umr-cnrm.fr/gmapdoc//spip.php?article213&lang=en

GRIB/GRIB_API related questions
 Q. I get an error message from GRIB_API when I run the model saying 'boot.def' not found?

A. The 'boot.def' file is the GRIB definitions file. The location of the boot.def file is hardcoded into the GRIB_API library when it is compiled. Make sure that the directory where GRIB_API was installed has not moved. If it has, this hardcoded location will be wrong. You can override this location by using the environment variable: GRIB_DEFINITION_PATH and set this to the directory containing the boot.def file. e.g.
export GRIB_DEFINITION_PATH=/usr/local/apps/grib_api/1.9.18/share/definitions

This error can arise when grib_api is compiled in HPC environments where it's compiled on the front-end but the batch system has different mount points for the frontend directories causing the path to the grib_api installation to be different.

 Q. OpenIFS fails with an error message from GRIB_API.

A. This usually indicates an error in the GRIB file or GRIB_API configuration rather than the model. Try using the 'grib_ls' or 'grib_dump' commands on the file causing the problem to try to reproduce the error. See the GRIB FAQ on this page, or email openifs-support@ecmwf.int and include the model output file (NODE_001.01) and the job log.

 Q. OpenIFS (or grib_api) reports: Wrong size for bitmap / passed array too small

A. When you run the model (or using the grib_ls/grib_dump tools on the GRIB file), you see this error:

GRIB_API ERROR   :  Wrong size for bitmap it contains 1373624 values
GRIB_API ERROR   :  unable to get bitmap as double array (Passed array is too small)

This occurs because sometimes GRIB files are incorrectly encoded for the property 'longitudeOfLastGridPoint'. This is known to happen for the 'stl1' field (surface temperature layer 1). grib_dump shows this:

  longitudeOfLastGridPointInDegrees = 340;

To correct it, use a command similar to:

grib_set -w shortName=stl1 -s longitudeOfLastGridPoint=359824 ICMCLabcdINIT ICMCLabcdINIT.fixed

where 'stl1' is the field to be corrected, 359824 is the last longitude point of (359.824degrees) of (in this case) the correct resolution, and ICMCLabcdINIT is the file to be corrected.

Email openifs-support@ecmwf.int for additional assistance.

 Q. What are the GRIB_API environment variables and what do they mean?

A. The environment variables for GRIB_API are described on the GRIB_API software support page.

 Q. How can I trace grib_api calls?

 A. Try:

export GRIB_API_DEBUG=-1

This enables brief debug-level logging messages to be displayed by the library.

If set to 1, you will get very verbose output.

See GRIB_API environment variables for more information.

 Q. How can I check that GRIB_API was installed correctly / How do I find out where GRIB_API installed the definitions and sample files?

A. If grib_api installed correctly, the command grib_info will print out the location of the sample files and definitions. If you get 'grib_info not found' make sure you have added the grib_api installation bin directory to your PATH environment variable.

 Q. How do I convert from GRIB to netCDF?
 Q. When I use grib_ls on OpenIFS output, I get an error: GRIB_API_ERROR : unable to represent the step in h

A. If you see this type of error:

% grib_ls ICMGGepc8+000004
ICMGGepc8+000004
edition  centre typeOfLevel level  dataDate  stepRange dataType  shortName packingType

GRIB_API ERROR   :  unable to represent the step in h
1       ecmf    surface     0      20041015 Decoding invalid stepRange

it is most likely because the model step isn't expressible as an integer number of hours (for example if the model output is every timestep with a timestep less than 1hr). Change the units to either minutes or seconds:

 grib_ls -s stepUnits=m ICMGGepc8+000004
 grib_ls -s stepUnits=s ICMGGepc8+000004
 Q. OpenIFS fails with GRIB_API_ERROR: unable to locate sample file gg_sfc_grib1.tmpl

A. This error means the model can't find the GRIB templates it needs to write out the model results. These are contained in a directory called 'ifs_samples' which is provided with the grib_api library. First check that the directory pointed to by OIFS_GRIB_API_DIR exists and is readable. Then check if the directory $OIFS_GRIB_API_DIR/share/grib_api/ifs_samples exists and is readable. If it does then it's possible that the library may have been moved (or a link to it changed) and because these paths are hardcoded into the grib_api library at compile time, it means the model can't find it. To overcome this, set the environment variable:

GRIB_SAMPLES_PATH - Set to the folder containing the set of samples you want grib_api to use instead of the default one.

e.g.

export GRIB_SAMPLES_PATH=$HOME/ecmwf/grib_api/share/grib_api/ifs_samples/grib1_mlgrib2

Mac OS X specific questions
 Q. When compiling GRIB_API on Mac OS X I get an error: ld: duplicate symbol _grib_file_not_found in .libs/action_class_alias.o and .libs/action.o

A. This is a known issue with compiling on Mac OS X. See this page on the GRIB_API Installation FAQ for the workaround.

 Q. I'm using the Intel compiler and the MKL library but get this error: dyld: Library not loaded: libmkl_intel_lp64.dylib when I run OpenIFS.

A. On Mac OS X, the MKL library is a dynamic (shared) library which is loaded at runtime and not when the model is compiled. To ensure the library is found you need to add the directory containing the library to your dynamic library path environment variable (DYLD_LIBRARY_PATH). The Intel compiler supplies a script that correctly sets the compile environment for you which should be run. This is normally found in the Intel compiler install directory: /opt/intel/Compiler/11.0/054/Frameworks/mkl/tools/environment/mklvars.sh.