Setup and Configuration

This section provides details on setting up a scenario folder. Due to privacy concerns (mgra land-use file) and large file sizes (networks) not all model inputs are available on the GitHub repository. In addition, a scenario folder structure is different than the folder structure on the repository. To automate the process, SANDAG built a tool that creates a scenario folder with necessary directory structure by assembling files from a locally cloned GitHub repository and in-house input database. Note that scenario tool can be used only on a SANDAG machine.

• Create a Scenario Folder
• Create Networks (SANDAG Staff)
• Configure the CT-RAMP and JPPF Services

Create a Scenario Folder

Follow the steps below to create a model scenario folder using SANDAG’s tool:

1. Check for an open server using the DOS tool mascot-abm.
2. Log into server
3. Using the show hidden icons arrow on the task bar, click the INRO Software Manager to check if EMME license is operating. If it's not, log into license server on Mustang by typing in the Host box: mustang.sandag.org:5170 (automatic)
4. Using the server’s Windows Explorer, navigate to T:\ABM\release\ABM\version_14_0_1\dist
5. Double click on createStudyAndScenario.exe and click Run.
6. On pop-up, enter fields in Create an ABM scenario. Make sure to enter the correct version and year and know your Scenario Folder (where you will run model from; typically study area/abm_runs) and your Network Folder (input networks). Browse to an existing folder or copy (ctrl + c) and paste (ctrl + v) paths to minimize chance of typos. After fields are entered, click Create.

7. A DOS window will open and necessary files will be copied. If you have a server with a valid license, it will pause for about 30-45 seconds on init emme folder. You will get a pop-up window that says you have successfully created the scenario. Click Quit. Exit the GUI by clicking on Quit.
8. Go to your newly created abm_run folder and double check on input files/date stamps to verify they match expected date stamps. Make any modifications to input files (ie parametersByYears.csv for adjusting auto operating costs, or mgra_based_inputXXX.csv file for adjusting parking costs). Refer to confluence page to know how and where to make changes.

Create Networks (SANDAG staff)

Follow the steps below to create a new network:

1. Navigate to T:\ABM\release\version_14xxxx\dist and click on createStudyAndScenario.exe
2. Click Yes and fill in the new study location and where to copy initial networks from.
3. Click Create.
4. If you want the latest TCOV, run copytcovnew 14 in your new covs directory from an arcinfo server (if you will be making changes to networks that you want carried forth for all future modeling, edit master tcov first and run postarc).
5. If there are any base AT network changes, please see Data Solutions (DS) group to scope out the edits needed.
6. Make any necessary changes to highway and transit network in local tcov, hwyproj.prn, hwyproj.col, headways files, etc.
7. Address any unique tolling, capacity, transit fare policies, etc that can be done pre-network generation (ie. Special_fares.txt, timexfer files).
8. Run tchc and tctr amls (or specialty amls) to build highway and transit networks.
9. Address any unique tolling, capacity, transit frequencies, transit fare policies, etc that have to be done post-network generation (ie. trrt.csv)
10. Timed Transfer files must match transit network for routes, configurations, and time periods. If any of those changed for Coaster or routes connecting to Coaster, the time xfer files must be changed too.
11. Special_fares.txt must be updated if Coaster configuration changed. Change configuration IDs and, if necessary, the station names at the mid-point (usually around Sorrento Valley station).
12. Check networks, emphasizing any areas that have changed.
13. If base AT network changed, wait for DS group to finish changes and add to local AT_files/bike_editing_sr14.gdb and re-create centroid connectors. If any TAPs were added, deleted, or changed locations from the time the last AT shapefiles were created, re-create the AT network by following steps outlined in the Active Transportation Network Quick Guide.
14. If highway, transit, and AT networks and supporting files are built and look good proceed to running the model steps.

Configure the CT-RAMP and JPPF Services

If Maven was set up correctly, users do not need to edit any of the files mentioned in this subsection.

To execute a model run, the files controlling the CT-RAMP and JPPF services must be configured. The Household Manager and Matrix Manager section provides a broad overview of these services. When executing the travel model, the user first executes the bin/runMtxMgr.cmd file which sets the project drive, directory, kills existing java processes, and specifies the location of the conf directory and the sandag_abm.jar file. The runDriver.cmd file sets the java driver properties.

The StartHHandNodes.cmd file kills all java processes currently running, starts the household manager (runHHMgr.cmd) and specifies the nodes to be used for the model run (runSandag01.cmd, runSandag02.cmd, runSandag03.cmd). The sandag_abm.properties file specifies the location of all the inputs, outputs, UEC files, and other tokens that need to be set. It also specifies which model components to run.

The CTRampEnv.bat file sets the ports, the machine names, IP addresses, drive letters to be used for machines, account settings (user name and password), and location of JAVA used for the model run. Variables in CTRampEnv.bat are described in the table below, with the SANDAG setup provided as an example.

Statement	Purpose
set MATRIX_MANAGER_PORT = 1991 set HH_MANAGER_PORT = 1129	set ports
set SNODE = yes set MAIN = AZTEC set NODE1 = HAWKEYE set NODE3 = AGGIE set NODE3 = MUSTANG	set machine names, which vary depending on cluster
set MAIN_IP = 172.16.34.40 set HHMGR_IP = 172.16.34.30	set main IP addresses, which vary depending on cluster
set MEMORY_MTXMGR_MIN = 90000m set MEMORY_MTXMGR_MAX = 100000m set MEMORY_HHMGR_MIN = 1000m set MEMORY_HHMGR_MAX = 15000m set MEMORY_CLIENT_MIN = 30000m set MEMORY_CLIENT_MAX = 70000m set MEMORY_SPMARKET_MIN = 30000m set MEMORY_SPMARKET_MAX = 120000m set MEMORY_BIKELOGSUM_MIN = 26000m set MEMORY_BIKELOGSUM_MAX = 26000m set MEMORY_WALKLOGSUM_MIN = 26000m set MEMORY_WALKLOGSUM_MAX = 26000m set MEMORY_BIKEROUTE_MIN = 26000m set MEMORY_BIKEROUTE_MAX = 26000m set MEMORY_DATAEXPORT_MIN = 30000m set MEMORY_DATAEXPORT_MAX = 120000m set MEMORY_EMFAC_MIN = 10000m set MEMORY_EMFAC_MAX = 10000m set MEMORY_VALIDATE_MIN = 10000m set MEMORY_VALIDATE_MAX = 10000m	set JVM allocations
set PROPERTIES_NAME = sandag_abm	set main property file name
set MAPDRIVE = T:	all nodes need to map the scenario drive, currently mapped as T:
set MAPDRIVEFOLDER = \\sandag.org\transdata	set MAPDRIVEFOLDER
set USERNAME = XX set PASSWORD = XX	set account for remote access using psexec
set MAPANDRUN = mapAndRun.bat	location of mapAndRun.bat on remote machines
set JAVA_64_PATH = c:\\program files\\java\\jre7	specity the 64-bit Java path

After preparing the computing environment by setting paths, runDriver.cmd, runHhMgr.cmd, and runMtxMgr.cmd then start up three separate Java programs, which need to be initiated prior to the travel model being executed. These three programs are called using the following commands:

Statement	Purpose
start %JAVA_64_PATH%\bin\java -server -Xmx16m -cp "%CLASSPATH%" -Dlog4j.configuration=log4j-driver.properties -Djppf.config=jppf-driver.properties org.jppf.server.DriverLauncher	Starts the Java DriverLauncher class using a Java virtual machine with no more than 16 MB of memory (Xmx16m); using the log4j logging service as configured by log4j-driver.properties; using the JPPF configuration file jppf-driver.properties. This program starts up the JPPF manager, as described in System Design.
start %JAVA_64_PATH%/bin/java -server -Xms%MEMORY_HHMGR_MIN% -Xmx%MEMORY_HHMGR_MAX% -cp "%CLASSPATH%" -Dlog4j.configuration=log4j_hh.xml org.sandag.abm.application.SandagHouseholdDataManager2 -hostname %HOST_IP_ADDRESS% -port %HOST_PORT%	Starts the Java SANDAGHouseholdDataManager class using a Java virtual machine using the log4j logging service as configured by log4j_hh.xml; defining the host as the machine with the IP address in HOST_IP. This program starts up the Household manager, as described in System Design.
start %JAVA_64_PATH%\bin\java -server -Xms%MEMORY_MTXMGR_MIN% -Xmx%MEMORY_MTXMGR_MAX% -Dlog4j.configuration=log4j_mtx.xml -Djava.library.path=%JAR_LOCATION% org.sandag.abm.ctramp.MatrixDataServer -hostname %HOST_IP_ADDRESS% -port %MATRIX_MANAGER_PORT% -ram 1500 -label "SANDAG Matrix Server"	Starts the Java MatrixDataServer class using a Java virtual machine using the log4j logging service as configured by log4j_mtx.xml.

After preparing the computing environment by setting paths, the runSandagXX.cmd (where XX is 01, 02, 03, or 04 in SANDAG’s application) instructions then start up a Java program, which needs to be initiated prior to the travel model being executed. Note that these scripts need to be called from each machine on the cluster individually. The following command is used to start the CT-RAMP nodes:

Statement	Purpose
start %JAVA_64_PATH%\bin\java -server -Xms16m -Xmx16m -cp "%CLASSPATH%" -Dlog4j.configuration=log4j-sandag{X}.properties -Djppf.config=jppf-sandag{X}.properties org.jppf.node.NodeLauncher	Starts the psExec program that starts the node on machine {X}, where {X} is 01, 02, 03, or 04 in SANDAG’s application.

Most of the JPPF-related configuration parameters have been optimized for the SANDAG travel model application and, as such, need not be modified. There are, however, a handful of parameters described in the table below that may need to be modified to meet the specifications of the computing environment upon which the model is being executed. Each of the files listed below can be found in the conf directory.

File Name	File Function	Statement	Purpose
jppf-clientDistributed.properties	JPPF Client Driver Control file	driver1.jppf.server.host = 172.16.34.40	IP address of the main computer (MUSTANG, CHARGER, AZTEC, or GAUCHO)
jppf-driver.properties	JPPF Driver Control file	jppf.server.host = 172.16.34.40	IP address of the main computer (MUSTANG, CHARGER, AZTEC, or GAUCHO)
jppf-node{x}.properties	Remote JPPF Node Control file	jppf.server.host = 172.16.34.40	IP address of the main computer (MUSTANG, CHARGER, AZTEC, or GAUCHO)
		processing.threads = 18	Number of computing cores on node {X}
		other.jvm.options = -Xms45000m -Xmx50000m -Dlog4j.configuration=log4j-sandag{X}.xml -Dnode.name=sandag02	Maximum amount of memory, in MB, to allocate to node {X}

The final configuration file that needs to be edited prior to executing a model run is the sandag_abm.properties file located in conf. This file serves as the general control module for the entire model application. Descriptions of the properties can be found in Properties File.

Go To Top

Next Section: Scenario Directory