Community GSI V3 On-Line tutorial

Download and Building the GSI system (version 3.0)

The following directions will produce the gsi executible. It is assumed that the WRF code (V3.2 or V3.3) has already been built. If that is not the case, download and build the WRF code first.

  1. From the community download page, download the GSI V3 tar file and CRTM coefficient tar file.

  2. Move the tar file to your working directory and unpack the file : 
    3.   tar -zxvf comGSI_v3.tar.gz 
      tar -zxvf REL-2.0.2.Coeffs.JCSDA_CRTM_forGSI.tar.gz

  3. You will see the directory comGSI_v3. Go into it 
    4.            cd comGSI_v3

  4. To build GSI, set the build environment:
    • after compiling WRF (either EM or NMM core) 
           setenv WRF_DIR "path for WRF"
    • Check $NETCDF using: 
             echo $NETCDF

      if it's empty or doesn't link to the libraries that were installed using compiler you are using, set it using: 
            setenv NETCDF  "path for netCDF"
  5. Create GSI configuration file using: 
    6.          ./configure
    The configure script will ask you to select a compiler option specific to the platform you are building on. Currently the following platform/compiler combinations are supported:
    • AIX with xlf
    • LINUX with pgi
    • LINUX with ifort
    • Mac OSX with pgi
    Running the configure script will create a new file configure.gsi. This configure file sets the paths and build options for the compile script. Should the build fail, it may be necessary to edit the configure file.

  6. To compile, run: 
    7.           ./compile >& compile_gsi.log

  7. Confirm that the executible has built:
    8. Searching the log file compile_gsi.log for errors and then checking executable: 
    
          cd ./run  
          ls gsi.exe

    If no executable gsi.exe exists under the run/ directory, check the log file compile_gsi.log and configure.gsi.

After successfuly compiling GSI, user can follow the following steps to run GSI practice cases:

practice case one: May 11th, 2008 east US cyclone

This is a small domain case. We will show how to run this case with NMM background and conventional data on a single processor Linux workstation.

  1. create a directory (for example: casedata under directory ./run) to save data (anywhere you have write permit)
    2. 
        mkdir casedata 
        cd casedata
  2. download the data
    3. goto Download/Practical Case page (link) and download tar file for this case to casedata directory.

  3. untar the data file 
    4.  
   tar -zxvf 2008051112.tar.gz
    The following files are under directory ./2008051112:
    
  ./wrfout_d01_ARW_2008-05-11_12:00:00     (ARW background)
  ./wrfinput_d01_NMM_2008-05-11_12:00:00   (NMM background)
  ./obs/little_endian.gdas1.t12z.prepbufr.nr (observation in Little Endian)
  ./obs/gdas1.t12z.prepbufr.nr               (observation in Big Endian)
  4. create a work directory or use an existing directory as working directory
    5. In this case, we use comGSI_v3/run as working direcotry.

  5. edit running scripts: run_gsi.ksh
    6. Example running script (run_gsi.ksh) is in the GSI system under ./run directory. The following changes to the script needed to run the case:
    • Job head:
      • Depends on the computer system, the head of the script should be changed. This is the same as you run other jobs. In this case, we will run it on a linux desktop. So, no change is needed.

    • Define machine type (pick one close to your computer)
      • Pick one close to your computer. Here, we use a Linux worstation with the pgi compiler. So, we pcik: 
           ARCH='LINUX_PGI'
    • define the following variables in run_gsi.ksh based on previous setup:

      • ANAL_TIME 2008051112
      WORK_ROOT ./run_${ANAL_TIME}
      BK_FILE comGSI_v3/run/casedata/2008051112/wrfinput_d01_NMM_2008-05-11_12:00:00
      OBS_ROOT comGSI_v3/run/casedata/2008051112/obs/
      PREPBUFR ${OBS_ROOT}/little_endian.gdas1.t12z.prepbufr.nr
      CRTM_ROOT CRTM_Coefficients
      FIX_ROOT comGSI_v3/fix
      GSI_EXE comGSI_v3/run/gsi.exe

      NOTE:
      • The path before directory comGSI_v3 and CRTM_Coefficients has been deleted
      • for IBM: please link the observation file : gdas1.t12z.prepbufr.nr
      • for ARW: please link the background file : wrfout_d01_ARW_2008-05-11_12:00:00
    • Define background and background error covariance
      • In previous table, we pick the NMM background file. Here we also pick NAM background error covaraince: 
        bk_core=NMM
  bkcv_option=NAM
  6. run GSI
    7. The example script for linux workstation can be found : here
    How to run job dependes on computer system. On Linux workstation, run GSI by: 
       run_gsi.ksh
  7. check the results
    8. cd run_2008051112/.

    check stdout and compare to reference stdout here
    check files in run directory and compare to reference file list here

practice case two: single observation test

The single observation test can be done by editing the above runnig script:

  1. change the follow variable in run_gsi.ksh 
    2.     &SETUP
      oneobtest=.true.,

   &SINGLEOB_TEST
      maginnov=1.0,magoberr=0.8,oneob_type='t',
      oblat=38.,oblon=279.,obpres=500.,obdattim=${ANAL_TIME},
      obhourset=0.,
   $SINGLEOB
  2. run gsi
    3. the same as step 6 in above case
    The example script for linux workstation can be found: here

  3. check results
    • Check the stdout and compare to reference stdout. here
    • Check the analysis increment plot
      • An NCL script under comGSI_v3/util/Analysis_Utilities/plots_ncl was used to make analysis increment plot. The script can be found here and the example plot can be checked here

practice case three: 12z analysis on March 22th, 2011

This is a big domain case covering North America. It was used as the exmaple case in the Chapter 5 of the User's Guide to illustrate how to run GSI with different data types and diagnose the results. We will show the steps to run this case with conventional, radiance, and GSPRO data on NCAR's IBM machine, Bluefire. The ARW background file was used for this case. We have compiled GSI version 3 on Bluefire.

  1. create a directory ./casedata under directory ./run to save data (anywhere you have write permit)
    2. 
        mkdir casedata 
        cd casedata
  2. download the data
    3. goto Download/Practical Case page (link) and download tar file for this case to casedata directory.

  3. untar the data file 
    4.  
   gzip -d 2011032212.tar.gz 
   tar -xvf 2011032212.tar
    The following file and directory should show up under directory ./2011032212:
    
  ./wrfinput_d01_ARW_2011-03-22_12     (ARW background)
  ./obs     (many observation BUFR files in Big Endian)
  4. create a work directory or use an existing directory as working directory
    5. In this case, we use comGSI_v3/run as working direcotry.

  5. edit running scripts: run_gsi.ksh
    6. Example running script (run_gsi.ksh) is in the GSI system under ./run directory. The following changes to the script needed to run the case:
    • Job head:
      • Depends on the computer system, the head of the script should be changed. This is the same as you run other jobs. In this case, we will run it on IBM supercomputer with LSF. So, the follow job header needs to be added: 
      
## Below (IBM queueing system) commands
#BSUB -P ????????
#BSUB -a poe
#BSUB -x                  # exlusive use of node (not_shared)
#BSUB -n   4             # number of total tasks
#BSUB -R "span[ptile=2]"  # how many tasks per node (up to 8)
#BSUB -J gsi              # job name
#BSUB -o gsi.out          # output filename (%J to add job id)
#BSUB -e gsi.err          # error filename
#BSUB -W 00:15
#BSUB -q regular          # queue

set -x

# Set environment variables for IBM
export MP_SHARED_MEMORY=yes
export MEMORY_AFFINITY=MCM
export BIND_TASKS=yes


# Set environment variables for threads
export SPINLOOPTIME=10000
export YIELDLOOPTIME=40000
export AIXTHREAD_SCOPE=S
export MALLOCMULTIHEAP=true
export XLSMPOPTS="parthds=1:spins=0:yields=0:stack=128000000"

# Set environment variables for user preferences
export XLFRTEOPTS="nlwidth=80"
export MP_LABELIO=yes
    • define processor number and machine type
      • Pick one close to your computer. Here, we use IBM with LSF. So, we pcik: 
           GSIPROC=4
     ARCH='IBM_LSF'
    • define the following variables in run_gsi.ksh based on previous setup:

      • ANAL_TIME 2011032212
      WORK_ROOT ./gsiprd_${ANAL_TIME}
      BK_FILE comGSI_v3/run/casedata/2011032212/wrfinput_d01_ARW_2011-03-22_12
      OBS_ROOT comGSI_v3/run/casedata/2011032212/obs
      PREPBUFR comGSI_v3/run/casedata/2011032212/obs/nam.t12z.prepbufr.tm00.nr
      CRTM_ROOT GSI/CRTM/CRTM_Coefficients
      FIX_ROOT comGSI_v3/fix
      GSI_EXE comGSI_v3/run/gsi.exe

      NOTE:
      • The path before directory comGSI_v3 and CRTM_Coefficients has been deleted
    • define background and its error matrix
      • In previous table, we use the ARW background file. Here we also pick NAM background error covaraince: 
        bk_core=ARW
  bkcv_option=NAM
    • add links to the radiance and GPSRO observation
      • In this case, radiance data and GPS RO data are used with conventional data. The following links need to be added in the run script in data link section: 
      ln -s ${OBS_ROOT}/gdas1.t12z.1bamua.tm00.bufr_d amsuabufr
ln -s ${OBS_ROOT}/gdas1.t12z.1bamub.tm00.bufr_d amsubbufr
ln -s ${OBS_ROOT}/gdas1.t12z.1bhrs4.tm00.bufr_d hirs4bufr
ln -s ${OBS_ROOT}/gdas1.t12z.gpsro.tm00.bufr_d gpsrobufr
  6. run GSI
    7. The example script for IBM can be found : here
    How to run GSI job dependes on computer system. On Bluefire (IBM with LSF), run it by: 
       bsub < run_gsi.ksh
  7. check the results
    8. cd gsiprd_2011032212/.

    • check stdout and compare to reference stdout here
    • check files in run directory and compare to reference file list here
    • check fit files and compare to reference fit files:
      • surface pressure; temperature; wind; moisture; radiance
    • check cost function and the norm of gradient:
      • Use the tool in util/Analysis_Utilities/plot_cost_grad to make plot: 
        copy fort.220 to util/Analysis_Utilities/plot_cost_grad
  cd into util/Analysis_Utilities/plot_cost_grad
  ./filter_fort220.ksh
  ./ncl GSI_cost_gradient.ncl
      compare result GSI_cost_gradient.pdf to refrence plot: here
    • check analysis increment
      • Use ncl script under util/Analysis_Utilities/plots_ncl scripts to make analysis increment plots and compare to refrence plots at:
      level 2; level 20; level 40