-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
96 lines (65 loc) · 3.74 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
THORPEX Interactive/International Grand Global Ensemble
NCEP-EMC component
Code Manager: Walter Kolczynski ([email protected])
Based substantially on previous version created by NCEI
=========================================================
Repository location
https://github.com/WalterKolczynski-NOAA/TIGGE
=========================================================
Build and installation
The build and installation has been combined in to a single scripts that can
be run in the src directory:
> cd src
> ./build_install_all.sh
This will build and install the ECMWF grib_api library as well as the NCEP TIGGE
executable. External library paths are loaded using LUA modules, with a single
module located in modulefiles that loads all of the necessary other modules.
Currently the system is only configured to run on NCEP operational machine
(WCOSS-Dell). Porting to additional systems will require an alterantive module
file and modification of the build script to load the new module file.
=========================================================
Configuration
This system uses a rocoto workflow to control job submission to the scheduler.
Most modifyable settings are located in the workflow definition, tigge.xml.
CYCLE_THROTTLE: Maximum number of cycles to run at once
TASK_THROTTLE: Maximum number of tasks to run at once
For a system running in real-time, these settings will rarely have an impact.
Effect will be limited to situations where the system is trying to 'catch up'.
SDATE: Start time
EDATE: End time
INCYC: Forecast cadence
These control which dates will be run. For a real-time system, just set the end
date in the far future. There is a file dependency that will prevent the system
from running a cycle before it is time.
TIGGE_PTMP: Temporary location to store input, output, and logs
GEFS_COM: Location of 1-degree pgrb2a and pgrb2b files
ACCOUNT: Job account for the scheduler
SCHEDULER: Scheduler type (i.e. lsf, slurm, or moabtorque)
SHARED_QUEUE: Job queue to use for compute jobs
TRANSFER_QUEUE: Job queue to use for transfer jobs (make external connections)
Since rocoto is a passive workflow system that has to be run to update job status
and handle new job submission, an entry should be added to your crontab that calls
the rocoto/cron_rocoto.sh script. Recommend frequency is every five minutes.
=========================================================
Tasks
This system has three tasks. All are serial jobs.
link_input: Creates symlinks in $TIGGE_PTMP/input to the files in $GEFS_COM
process_tigge: The main job; performs QC and creates files needed for TIGGE
transfer_output: Transfers the tigge tar file to the FTP server (on EMC rzdm) and
also archives a copy on HPSS
=========================================================
Delivery to ECMWF:
The end goal of this project is to provide the NCEP component of the THORPEX Grand
Global Ensemble to the master archive maintained by and at ECMWF.
Links:
master archive status * http://apps.ecmwf.int/datasets/tigge
master archive access * http://apps.ecmwf.int/datasets/data/tigge/levtype=sfc/type=cf/
Delivery % is in the upper 90% per year - meaning a few missed cycles are OK.
Currently the transfer is done through EMC's FTP site (on rzdm).
Link: ftp://ftp.emc.ncep.noaa.gov/ufs/global/ensemble/tigge/
The ECMWF Master archive is the only entity that needs the output data, so IP restrictions can
be implemented but thats not really needed as the data is rather harmless
should someone randomly discover the directory.
ECMWF Contact (general): [email protected]
ECMWF Contact (technical): [email protected]
ECMWF Contact (?): [email protected]