What is totem?

Totem is continous delivery pipline tool which is aimed in simplifying delivery of the code for stateless application to any environment. Under the hood, it utilizes containerization tools and technology (Docker).

It was specifically designed for micro services/frameworks by utilizing practices and patterns that are used for deploying the same (gitflow, service discovery, containerization, centralized logging, and more...)

Reporting Issues

Issues can be reported at : https://github.com/totem/totem.github.io/issues Issues may include help requests, bugs or feature requests.

Architecture

High Level flow

• Developer commits file to git and pushes the changes to scm (e.g: github)
• Github triggers webhooks for push events to image builder tool e.g: "Image Factory" and CI tools like Travis, Bamboo.
• Once travis (or any other ci) build and image builder build finishes it notifies orchestrator about the status using post build webhooks.
• Orchestrator on receving any hook, loads/caches the configuration for the build (totem.yml). This file is loaded from Amazon S3 (or etcd).
• Orchestrator on receving all the hooks with status as "success", it invokes deployer api to deploy the built image to CoreOS cluster.
• Deployer creates fleet unit files for the application and submits the job to fleet daemon (running on CoreOS cluster). On successul deploy, deployer promotes the current deployment by updating the yoda proxy configuration stored in etcd.

Application Deployment

Here is a sample application deployed using Totem in Amazon EC2 infrastructure.

Components

The "Totem Ecosystem" comprises of several components each responsible to perform discrete set of jobs. One may choose to deploy some/all of the components in Totem , depending upon his/her needs. It is also possible to have additional components for deploying applications using Totem (for e.g.: sidekicks for registering with SkyDNS)

• Image Factory: Responsible for building docker images using "Dockerfile" and push these to a central registry (e.g: Quay)
• Orchestrator: Responsible for consuming hooks from different services (like Travis, Image Factory etc) and creating deployment requests to Deployers for configured clusters.
• Deployer: Schedules the deployment to a cluster (currently CoreOS) and sets up the proxy hosts/listeners.
• Yoda Proxy: Dynamic Haproxy backed by etcd.
• Yoda Discover: Acts as side kick for registering application unit to yoda proxy using etcd as backing store. Also provides ability to register yoda hosts to etcd.
• Yoda Route53: Registers the yoda hosts with Amazon Route 53 for DNS based load balancing for hosts.
• Fleet Templates: Jinja based fleet unit files that gets deployed to CoreOS cluster. Some of the default templates are already provided, but one may choose to have his/her own custom templates.

Totem Events

Overview

The different totem components (deployer, orchestrator) publishes events to elasticsearch (if enabled). Here is an example event:

{
    "date": "2015-04-01T06:25:06.389047",
    "type": "NEW_JOB",
    "component": "orchestrator",
    "meta-info": {
      "git": {
        "owner": "meltmedia",
        "repo": "totem-demo",
        "ref": "master",
        "commit": "87c4419d3e278036055ca2cd022583e0397d3a5d"
      },
      "job-id": "9be083a1-9fc2-48b3-83bc-e11c8bbab305"
    },
    "details": {
    }
}

Event fields

Every event consists of following fields:

• date: Timestamp for event
• type: Event type
• component: Totem component that generated the event (orchestrator, deployer).
• meta-info: Meta info associated with the event. Typically it comprises of git meta-data and job identifier.
• details: The detailed json object describing the event.