====== Bash Script for postprocessing with CDO ======

The shell (bash) script below is an example for postprocessing using multiple CDO commands including the following general parts:\\
- Selecting time periods\\
- Selecting variables\\
- Temporal resampling (e.g. to daily means, if the data is already at daily means, then this step would be idle but harmless)\\
- Interpolation to a longitude-latitude grid (see section "[[https://wiki.mpimet.mpg.de/doku.php?id=analysis:postprocessing_icon:regridding:shell:start|Regridding the data]]" using CDO within the shell)\\
\\
The script assumes the existence of a grid-description and a weight-file (see also "[[https://wiki.mpimet.mpg.de/doku.php?id=analysis:postprocessing_icon:regridding:shell:start|Regridding the data]]").\\

Since the required computational performance may exceed what is available at the interactive prepost nodes, it may be useful to read the Wiki introduction section about [[https://wiki.mpimet.mpg.de/doku.php?id=analysis:postprocessing_icon:start|SLURM workload manager]] and allocating an exclusive computational environment.

As an example, the script selects surface temperatre (ts) for the months January-March of the 1 year-long coupled dyamond simulation), calculates daily mean values and interpolates the data onto a 1°x1° longitude-latitude grid. This script makes use of the [[https://wiki.mpimet.mpg.de/doku.php?id=analysis:postprocessing_icon:start|SLURM workload manager]] and can be submitted via ''sbatch scriptname.sh''. If it is intended to execute the script without SLURM, just delete the '''SBATCH ...'' lines at the top (leave the ''#!/bin/sh'') and execute the script via ''bash scriptname.sh''.

<code bash>

#!/bin/sh 
#SBATCH --job-name=myjobname
#SBATCH --partition=gpu
#SBATCH --nodes=1
#SBATCH --time=01:00:00
#SBATCH --mail-type=FAIL
#SBATCH --account=project_name_to_be_charged
#SBATCH --output=job_log.o%j
#SBATCH --error=job_log.e%j

set -xe

## example of how to postprocess 2d (lon-lat and time) atmospheric variable. 

### START USER INPUT
CDO="/home/mpim/m214003/local/bin/cdo" #path for cdo version (here Uwes local version)
var="ts" #variable
years="2020"
exp="dpp0016"
grid="1" #in degree (i.e. 1x1): other option: grid="005" for 0.05x0.05 or define own grid description file and weight file
months="01 02 03"
model_files="${exp}_atm_2d_ml_" #adjust for variable of interest
path_to_scratch="$SCRATCH/${exp}"
path_to_model_raw_data="/work/mh0287/k203123/GIT/icon-aes-dyw_albW/experiments/${exp}"
griddes_file="griddes_${grid}.txt" #specifiy grid description text file here (see Wiki on this: https://wiki.mpimet.mpg.de/doku.php?id=analysis:postprocessing_icon:regridding:shell:start)
weight_file="weight_${exp}_${grid}.nc" #specifiy weight file here (see Wiki on this: https://wiki.mpimet.mpg.de/doku.php?id=analysis:postprocessing_icon:regridding:shell:start)
out_dir="$SCRATCH/${exp}" # this is the path where the final file will be stored. In this example, this is also the $SCRATCH dir of the user, but could be changed to the $WORK dir of the user for instance
griddes_file="griddes_${grid}.txt" #specifiy grid description text file here (see Wiki on this: https://wiki.mpimet.mpg.de/doku.php?id=analysis:postprocessing_icon:regridding:shell:start)
weight_file="weight_${exp}_${grid}.nc" #specifiy weight file here (see Wiki on this: https://wiki.mpimet.mpg.de/doku.php?id=analysis:postprocessing_icon:regridding:shell:start)
### END USER INPUT

#####################################
### creating directory structure on scratch (if non existing)

for yy in ${years}; do
   for mm in ${months}; do

      ${CDO} -O -P 24 -remap,${griddes_file},${weight_file} -daymean -cat -apply,"-selname,${var}" [ ${path_to_model_raw_data}/${model_files}*${yy}${mm}*.nc ] ${path_to_scratch}/${var}_tmp_${mm}.nc

   done  #mm
done #yrs

## cat and del monthly chunk files
${CDO} -O -P 24 cat ${path_to_scratch}/${var}_tmp_*.nc ${out_dir}/${exp}_${var}_${grid}x${grid}.nc
rm ${path_to_scratch}/${var}_tmp_*.nc

</code>