====== Access reanalysis data ======
Some data is stored on Levante and can be easily accessed there. All other reanalysis data can be downloaded from the sources (see following paragraphs), where you typically have to create an account.

===== Data browsing =====
You can use **Freva** to browse data on Levante. If you're lucky, what you need already exists somewhere and you don't need to download anything.
Somehow it should be possible to use Freva as well to access Reanalysis (the DKRZ documentation is still for mistral). The analysis-part of Freva is only available for project members of climxtreme or regiklim-nukleus, but data browsing seems to be possible for everyone with a DKRZ account.

[[https://gems.dkrz.de/solr/data-browser/?project=reanalysis|Freva browser]]\\
[[https://freva-clint.github.io/freva/|Freva Documentation]]\\
Try:
<code>
module load clint gems
</code>
Or using your own conda environment:
<code>
conda install -c conda-forge freva
</code>

Then you can do
<code python>
import freva

freva.config("/work/ch1187/clint/nextgems/freva/evaluation_system.conf")
os.environ["EVALUATION_SYSTEM_CONFIG_FILE"] = "/work/ch1187/clint/nextgems/freva/evaluation_system.conf"

files = freva.databrowser(experiment="era5",
                          institute="ecmwf",
                          product="reanalysis",
                          project="reanalysis",
                          time_frequency="mon",
                          time="1993-01-01 to 1993-12-01",
                          time_select="flexible",
                          )
ds = xr.open_mfdataset(files)  # unless you mix 2D and 3D files this should work
</code>


Also check out the data from the Integrated Climate Data Center (ICDC) on levante. The ICDC provides a variety of datasets including atmospheric reanalysis. Data is available in NetCDF format and can be accessed via levante in:\\
<code>/pool/data/ICDC/reanalyses/ </code>

At the moment data from NCEP/NCAR Reanalysis 1 (1948 to present) and NCEP Climate Forecast System Reanalysis CFSR (1979-2010) are available for surface variables. More information can be found [[http://icdc.cen.uni-hamburg.de/1/daten/reanalysis-atmosphere.html|here]].


===== ERA5 =====
ERA5 is the new reanalysis product from ECMWF, which replaces ERA-INTERIM.
You have several possibilities:
=== Catalogued grib files from the DKRZ === 
Most surface and pressure-level ERA5 data is catalogued by the DKRZ. Note that this is still a prototype so the final catalog-address might change.

<code python>
import intake 
cat = intake.open_catalog("https://gitlab.dkrz.de/data-infrastructure-services/era5-kerchunks/-/raw/main/main.yaml")
list(cat)
</code>

Global maps can be plotted relatively easy.

<code python>
import matplotlib.pyplot as plt
import cartopy.crs as ccrs

ds = cat['surface_analysis_monthly'].to_dask()

fig, ax = plt.subplots(subplot_kw=dict(projection=ccrs.Robinson()))
ax.set_global()
ax.scatter(ds.lon, ds.lat, c=ds['2t'].sel(time='1998-01-01'), transform=ccrs.Geodetic())
ax.coastlines()
</code>


=== Remapped ERA5 data on HEALPix ===

In addition, a small subset of ERA5 between the years 2010--2023 is available on HEALPix.
<code python>
import healpy as hp
import intake


cat = intake.open_catalog("https://tcodata.mpimet.mpg.de/internal.yaml")
ds = cat["HERA5"](time="P1D").to_dask()
</code>

=== Download ===
ERA5 is available for download on the Climate Data Store (CDS) cloud server. The CDS offers different ways to access ERA5 data:\\
  * Web interface ([[https://cds.climate.copernicus.eu/api-how-to|documentation]]). Simple operations on the data (calculations, plotting, etc.) can also be performed online by using the CDS Toolbox.
  * [[reanalysis:dataaccess:era5-python:start|Download ERA5 data with python using the CDS Application Programming Interface (API)]]
  * [[reanalysis:dataaccess:era5-julia:start|Download ERA5 data with julia using the CDS Application Programming Interface]]

=== grib files on Levante (ERA-INTERIM and earlier products) ===

[[https://cera-www.dkrz.de/WDCC/ui/cerasearch/q?hierarchy_steps_ss=ERA_INTERIM]|ERA Interim]] and previous ERA products from ECMWF are available after registration at DKRZ (filling [[https://www.dkrz.de/up/services/data-management/projects-and-cooperations/era/era40-1/documents|ECRANA form]] and sending it to data@dkrz.de). More information can be found [[https://www.dkrz.de/up/services/data-management/projects-and-cooperations/era|here]]\\
Data from ERA5 are freely available at **/work/bk1099/data/** with about 3 months delay\\
**Available parameters:**\\
On //model and pressure levels//: 075 (Specific rain water content), 076 (Specific snow water content), 129 (Geopotential/..height), 130 (temperature), 131 (U), 132 (V), 133 (specific humidity), 135 (vertical velocity), 138 (vorticity), 155 (divergence), 203 (Ozone mass mixing ratio), 246 (Specific cloud liquid water content), 247 (Specific cloud ice water content), 248 (Fraction of cloud cover)\\
For //model levels//: 077 (Eta-coordinate vertical velocity), 152 (Logarithm of surface pressure at sea level), too.\\
For //pressure levels//: 060, 157, too.\\
For //intensive surface data// 46 2D surface variables:  027 028 029 030 031  032 033 034 035 036, 037 038 039 040 041  042 129 134 136 137, 139 141 151 160 161  162 163 164 165 166, 167 168 170 172 183  186 187 188 198 206, 235 236 238 243 244  245\\
For //extensive surface data// 47 2D surface variables, aggregated, forecasted, etc:  034 044 045 049 050  057 059 078 079 129, 137 142 143 144 145  146 147 151 159 164, 169 175 176 177 178  179 180 181 182 195, 196 197 201 202 205  208 209 210 211 212, 228 239 240 243 244  245  and 251.228.\\
For the latter data (except 049, 201, 202), monthly means are available, too.\\
See the parameter table at [[https://confluence.ecmwf.int/display/CKB/ERA5+data+documentation#ERA5datadocumentation-Dataorganisation|ECMWF]]. The most reliable and up-to-date description you always find in the README in the base directory given above. See the [[https://modvis.dkrz.de/bm0146/ERA5/statusE5.html|status page]] for the progress of the data transfer.\\


===== MERRA2 =====

  - select variables and spatio-temporal coverage of MERRA2 from [[https://disc.gsfc.nasa.gov/datasets?page=1&keywords=MERRA-2|GES DISC]]
  - get a list of download links for the selected data in .txt file (Subset/Get Data --> Download Method --> "Get File Subsets using the GES DISC Subsetter")
  - make sure a .netsrc and .urs_cookies exist in <home>
  - create .dodsrc in the target directory containing
    <code>
    HTTP.NETRC=< YourHomeDirectory >/.netrc  
    HTTP.COOKIEJAR=<YourHomeDirectory >/.urs_cookies </code>
  - use wget (or curl)
    <code bash>
     wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --content-disposition -i "<url.txt>" </code>
     

===== JRA-3Q =====
Currently (April 2024) there is no subsetting possible; files are only available in grib.\\
You can download them from the following two sources (both need registration):

[[https://data.diasjp.net/dl/storages/filelist/dataset:645|Download JRA-3Q via DIAS (Japanese system)]]\\
[[https://rda.ucar.edu/datasets/ds640.0/dataaccess/#|Download JRA-3Q via NCAR/UCAR]]\\

[[https://jra.kishou.go.jp/JRA-3Q/document/JRA-3Q_TL479_format_en.pdf|Documentation (model grid)]]\\
[[https://jra.kishou.go.jp/JRA-3Q/document/JRA-3Q_LL125_format_en.pdf|Documentation (1.25° lat/lon grid)]]\\


===== NCEP =====
[[https://rda.ucar.edu/datasets/ds090.2/#|Download NCEP from UCAR]] (probably not complete)