Introduction
Teletraan services are the backend RESTful services providing the core deployment support. There are basically 3 major function categories, Teletraan service, Teletraan worker and Teletraan agent service. They share the same code base and could be configured to run either as one single service or multiple dedicated services. At Pinterest, we run Teletraan service and Teletraan Agent Service separately. See Run Services in Production for more details.
- Teletraan Service
The RESTful service that interacts with deploy board UI and command line requests. It handles most of the Teletraan service API calls such as maintain environment, deploys, etc..
- Teletraan Agent Service
The RESTful service that interacts with Deploy Agents. Every Deploy Agent queries Teletraan (Agent) service periodically for deploy instructions. Even though the Agent Service handles only one API call, ping(), but it is usually the most busy one. Teletraan Agent Service can be configured to run along with Teletraan Service in the same process.
- Teletraan Worker
This is a collection of java processes to handle Teletraan async jobs, such as keeping auto deploy schedules, cleaning up obsoleted hosts, etc.. Teletraan Workers can be configured to run along with Teletraan service in the same process.
Deploy agent is a simple Python script running on every host. It pulls deploy instructions from Teletraan server periodically, performs the actual deploy by executing Deploy Scripts, saves the status locally and reports back to server. Deploy Agent is designed to follow Teletraan server's instructions at every step of the way. Not only this makes the agent's logic simple and its code stable, but also it gives Teletraan server more control over the agent's behaviors. See Run Deploy Agent for more details on setup Deploy Agents in production.
Deploy Board is the web UI of Teletraan. See the step-by-step instructions for how to use Deploy Board UI.
- Environment
An environment is an abstract concept corresponding to a service that will be deployed. An environment has one or many stages, which are the concrete server entities containing deploy configs and actual deploys.
- Stage
An environment stage contains one currently active deploy, all the past deploys and deploy related configs (Success Threshold, Max Concurrent Deploy Hosts, Build Name, Capacity, Auto Deploy). It is recommended to create as many stages as needed. Teletraan makes it very easy to create a new stage. At Pinterest, services usually have beta, canary and prod stages.
- Deploy
A deploy contains all the information for a particular service deployment, such as build, deploy status, operator, dates etc.
- Build
A build represents a particular code change and build artifact. It has an url where the build artifact can be downloaded, and information such as repo, commit, and build url, so we can fetch more code info needed.
- Build Name
Build also contains Build Name. It is very important to match the Build Name in Environment Stage configuration for builds to show up in that stage. See Publishing a build and Creating a new environment for more details.
- Capacity
Capacity represents the hosts to deploy, or deploy targets. Capacity could be a collection of individual hosts or groups. See Setting up capacity configs for more details.
We recommend deploying code changes to pre-production stages first, such as beta or canary. You could hook up tests using pre/post webhooks to qualify the new code changes. Alternatively, you can also invoke tests in your deploy scripts, for example:
>cat teletraan/POST_RESTART
...
if [ "$STAGE_NAME" == "canary" ] && [ "$DEPLOY_TYPE" != "HOTFIX" ]]
then
# Run my test
fi
...
All services deployed at Pinterest have at least the canary stage, which really is just one or two hosts borrowed from prod hosts. We found it very effective exposing issues in new code changes. See Setting up Capacity Config for details on how to setup canary stage.
The following state machine diagram describes how Deploy State changes:
