Computing Environment
This section describes the hardware and software SANDAG uses to execute the model and its sub-sections are divided up as such:
It is important to note that both the software and model structure are highly configurable and flexible; depending on the analysis needs, the required computing power could vary dramatically. SANDAG currently has the model configured to run on a single machine. The hardware and software are described below.
- Operating system: Microsoft Windows Server 2012 Release 2 with Service Pack 1 or Win-7, 64-bit Enterprise edition;
- Processors: Two Intel Xeon processors with 8-16 cores (i.e., two processors with hyper-threading capability);
- Memory (RAM): 256 GB recommended (160 GB is minimum required memory)
Refer to Specifications and Configurations of Clusters for detailed information on all clusters.
The travel model currently uses Version 4.3.7 64-bit Emme software. The Emme software is used to build skims, manipulate networks, manipulate matrices, perform assignments, and run the aggregate special market models. The Java software, as discussed below, needs to access matrix data in OMX format. The details of setting up and configuring the software are presented in Setup and Configuration.
SANDAG’s travel model operates on the Coordinated Travel - Regional Activity-based Modeling Platform (or CT-RAMP) developed by Parsons Brinckerhoff. The software is written in the Java programming language and therefore requires the 64-bit Java Development Kit (version 8 or higher) to be installed on each computer running the CT-RAMP software. The Java Development Kit includes the Java Runtime Environment. The 64-bit version of the software allows CT-RAMP to take advantage of larger memory addresses.
The output of the travel model is handled by Microsoft SQL Server 2014 Enterprise products. ABM output data is loaded via SQL Server Integration Services into a SQL database for management, manipulation, storage, and reporting. It is planned to further leverage these products, along with SQL Server Analysis Services and Reporting Services, to create a full reporting suite accessible internally at SANDAG via report flat files or Microsoft Power Pivot and externally via the web.
The CT-RAMP software allows discrete choice models to be specified via spreadsheets, referred to as a Utility Expression Calculator or UECs. These files are Excel-based. It is helpful to have Excel installed so that the spreadsheets can be opened, though it is not essential for running the model system.
A Python virtual environment is a setup for a Python environment such that the Python interpreter, libraries and scripts installed into it are separated from other Python environments, and (by default) any libraries installed in a “system” Python, i.e., one which is installed as part of your operating system.
Using a virtual environment allows users to specify which library and their respective versions are required to run the SANDAG ABM model (including Emme APIs). It improves reproducibility of results as a version of the SANDAG ABM model is tied to a specific environment: the model is controlled for AND the environment on which to run the model.
The following steps describe how to create a Python virtual environment properly set up to run the SANDAG ABM model and how to install this virtual environment in Emme.
Virtual environments for the SANDAG ABM model are stored on the server under C:\python_virtualenv
From the ABM repository (and desired branch), copy the following files into C:\python_virtual_env
Check add_python_virtualenv.bat to be similar to the example shown below. Verify that the batch file refers to the proper ABM version (e.g. abm_14_2_0).
Open an Emme Shell in admin mode and run add_python_virtualenv.bat
Upon completion, you should now see a new directory:
-
C:\python_virtualenv\<ABM_VERSION_HERE:abm14_2_0>with subdirectories Lib and Scripts
The following libraries and their respective versions described in requirements.txt are installed in this virtual environment:
Copy the python_virtualenv.pth in the Emme distribution (file may be found under src/main/emme/python_virtualenv.pth in the GitHub repository) at:
C:\Program Files\INRO\Emme\Emme 4\Emme-4.4.X\Python27\Lib\site-packages\python_virtualenv.pth
The PTH file contains the following:
Note that every time a new Emme version is installed, this step needs to be reproduced on every machine (computer/server) where the new version is installed.
When creating a new scenario from the PythonGUI, it will contain the file:
%SCENARIO_FOLDER%\emme_project\start_emme_with_virtualenv.bat
This file makes sure that:
- The virtual environment selected at scenario creation will always be used
- The Emme version selected at scenario creation will always bse used
Here is an example of the file content:
When a new Emme version is installed on a machine, step 2 needs to be completed again (i.e. install the virtual environment in this new Emme distribution).
The Scenario generation GUI also needs to be updated to include the new Emme version:
src/main/python/pythonGUI/createStudyAndScernario.py- Update the
CreateScenarioGUI.emme_versiondefault value - Update the
optionListto include the new Emme version number