DC/OS is the obvious choice for deploying fast growing web based sites and services, with support for the SMACK stack delivered out of the box. But how can you run more traditional workloads, like the LAMP stack? In this post we’ll explore some of the challenges by putting together a simple ‘movie community’ web app that combines traditional web technologies like PHP and MySQL together with Cassandra, all deployed onto a DC/OS cluster.

 

Intro

Distributed computing and SMACK (Spark, Mesos, Akka, Cassandra and Kafka) are a marriage made in heaven and the setup is quickly solidifying its position as the dominant stack for big data. Meanwhile, DC/OS brings all these great tools together for you, with enterprise grade, robust and secure container management to ensure your applications stay running with reduced effort on your part. But there is more. DC/OS is a technology capable of handling computing applications old and new; for example the LAMP stack (Linux, Apache, MySQL and PHP), which continues to power web applications requiring a solid and reliable foundation. On its own this setup is vulnerable to high server loads and machine downtime. Hosting your application on a DC/OS cluster helps your applications to handle these risks, by empowering on-demand scale out (and in again), over multiple machines as the need arises in a cloud-native, elastic and failure-resilient manner.

 

This idiom is called application clustering. Instead of a statically deployed server park, you now have a pool of multiple servers working and acting as one. Each of the servers maintains the same information and collectively they perform tasks and assignments. If one server fails, the others automatically pick up the pace. Workload is automatically rebalanced as needed, for example the platform automatically responds to application crashes or server outages by restarting the failed components on another server. DC/OS enables you to build self-managing infrastructure – and that simplifies SLA compliance, which means fewer 3am callouts. It’s pretty easy to work with, and it’s free open source software you can try out for yourself.

 

DC/OS builds the cluster on servers running an operating system like Red Hat Enterprise Linux or CoreOS. CoreOS is an open-source, lightweight operating system that has garnered a lot of attention since its initial release in 2013. The essence is this: instead of installing packages via apt or yum, CoreOS uses Docker containers to manage your services at a higher level of abstraction. Each service and its dependencies are packaged within a docker container.

 

Containers provide similar benefits as complete virtual machines, but focus on applications instead of entire virtualized hosts. Since these containers don’t run their own Linux kernel or require a hypervisor, they have minimal performance overhead. This makes them fast, lightweight and highly suitable for cluster environments.

 

CoreOS has a few other nice features. For instance, CoreOS systems configuration is managed via etcd, a distributed, centralised configuration management system; and most of CoreOS runs on a read-only file-system.

 

Let’s get started.

 

Prerequisites:

 

Cluster Spin-up + SSH

To avoid making this post too long, we will assume you already set up a DCOS cluster. If you need help with this, check out the documentation on dcos.io

 

Wordra: Transfer videos through WordPress and Cassandra

Our sandbox app will have four big components. WordPress will provide the web interface. Cassandra will store videos for us.MariaDB Galera will handle data transfers and Marathon-lb will balance the loads on the nodes.

 

Let’s start with Cassandra, which is available as part of the DC/OS distribution.

 

dcos package install cassandra

 

Installing Cassandra will take a little while. While that’s going on, open up the DC/OS web interface. You can find it running on the cluster’s master. Once Cassandra is finished installing you can verify its health by checking the ‘Services’ tab.

 

Next we have to create the required tables. In order to do so we have to access the node that is running Cassandra. But which one? Since you are already on the Services page, it isn’t too hard to find out.

cassandra-dcos

Be aware however that in a dynamic distributed environment this IP can (and will!) change as new nodes are spun up. If you rather have this address straight on the command line, it is sufficient to ssh to one of the nodes (any will do) and retrieve the IP of the node running ‘cassandra.dcos.mesos’:

 

sudo docker run --net=host tutum/dnsutils dig cassandra.dcos.mesos

 

Using service discovery this will return the internal IP address of the node that is running Cassandra as well. Write it down somewhere, then navigate to <your lb-hostname>/mesos. In the ‘Slaves’ tab, look up the internal IP address of the node and copy the corresponding mesos id.

 

Using this Mesos ID we can now SSH to this node and access the Cassandra database.

dcos node ssh --master-proxy --mesos-id=<mesos id>

 

We will now generate the tables we need, using some help from the Docker container. In the shell, enter the following (long) line of code:

docker run -i -t --net=host --entrypoint=/usr/bin/cqlsh spotify/cassandra -e
"CREATE KEYSPACE video WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
USE video;
create table movies(movie_id INT PRIMARY KEY,uploadTime TIMESTAMP,title TEXT,username TEXT,filesize INT,chunkcount INT,ftype TEXT);
create table data(movie_id INT,chunkID INT,chunkData BLOB,PRIMARY KEY(movie_id,chunkID));
create table moviesnum(id INT PRIMARY KEY);insert into moviesnum(id) values(0);" cassandra.dcos.mesos 9160

 

This creates 3 tables.

  • “movies” holds the file metadata,
  • “data” holds the chunks of data,
  • “moviesnum” tracks the movie_id

 

Cassandra is now set up and ready to receive our movies.

 

Marathon-lb

Marathon-lb is based on HAProxy, a fast proxy and load balancer. It already comes included in DC/OS and takes care of load balancing for TCP and HTTP based applications, with SSL support, HTTP compression, health checking and more.

 

Installing marathon-lb package is as simple as another single line:

 

dcos package install marathon-lb

 

To check if the marathon-lb is running, navigate to…

http://<public slave ip>:9090/haproxy?stats

haproxy-dcos

Here you can find a statistics report to show that marathon-lb is running.

 

You will need to configure the public slave loadbalancer to make the loadbalancer “InService”:

Select the public slave loadbalancer in the loadbalancers and in the healthcheck tab, edit the healthcheck like this:

Pingport:9090,
pingpath:/_haproxy_health_check

 

With that, the instance status should be ‘InService’.

More information on marathon-lb can be found in this Marathon-lb dedicated blog post

 

Deploying the dockerized applications using the definition json file

Moving on to third piece of the puzzle. We will now deploy a containerized WordPress + MariaDB Galera database. Traditionally WordPress’ backbone database has always been MySQL. The distributed environment counterpart of this would be MariaDB, with MariaDB Galera being a synchronous multi-master cluster for MariaDB. It is a faster replacement of MySQL which enables master-master replication clustering. The main advantage of MariaDB Galera is that if the server crashes other nodes still hold the data.

 

To use WordPress + MariaDB Galera, you’ll have to create a definition json file for your app. In our case this file holds the definition of a group (‘galerawordpress’), containing three applications:

  • Galera seed
  • Galera node
  • WordPress

 

The services in DC/OS find each other using built-in DNS name service discovery. The DNS name of any application in our json file needs to have the following structure:

 

appName-groupName.serviceName.mesos

 

For example, the DNS name of the Galera seed would be…

 

seed-galerawordpress.marathon.mesos

 

Time to put that into practice. Create a json file and give it the following content:

 


{
   "id":"galerawordpress",
   "apps":[
      {
         "id":"seed",
         "container":{
            "type":"DOCKER",
            "docker":{
               "image":"sttts/galera-mariadb-10.0-xtrabackup",
               "network":"BRIDGE",
               "portMappings":[
                  {
                     "containerPort":4567,
                     "hostPort":4567,
                     "protocol":"tcp"
                  },
                  {
                     "containerPort":8080,
                     "hostPort":0,
                     "protocol":"tcp"
                  }
               ]
            },
            "volumes":[

            ]
         },
         "env":{
            "SERVICE_NAME":"galera",
            "SERVICE_TAGS":"seed",
            "XTRABACKUP_PASSWORD":"3240fd7as9f8798",
            "MYSQL_ROOT_PASSWORD":"wordpress"
         },
         "args":[
            "seed"
         ],
         "cpus":0.5,
         "mem":512,
         "instances":1,
         "maxLaunchDelaySeconds":10,
         "healthChecks":[
            {
               "protocol":"HTTP",
               "portIndex":1,
               "gracePeriodSeconds":30,
               "intervalSeconds":5,
               "maxConsecutiveFailures":6,
               "ignoreHttp1xx":true
            }
         ],
         "upgradeStrategy":{
            "minimumHealthCapacity":0
         }
      },
      {
         "id":"node",
         "container":{
            "type":"DOCKER",
            "docker":{
               "image":"sttts/galera-mariadb-10.0-xtrabackup",
               "network":"BRIDGE",
               "portMappings":[
                  {
                     "containerPort":4567,
                     "hostPort":0,
                     "protocol":"tcp"
                  },
                  {
                     "containerPort":8080,
                     "hostPort":0,
                     "protocol":"tcp"
                  },
                  {
                     "containerPort":3306,
                     "hostPort":3306,
                     "protocol":"tcp"
                  }
               ]
            },
            "volumes":[

            ]
         },
         "env":{
            "SERVICE_NAME":"galera",
            "XTRABACKUP_PASSWORD":"3240fd7as9f8798",
            "MYSQL_ROOT_PASSWORD":"wordpress"
         },
         "args":[
            "node",
            "seed-galerawordpress.marathon.mesos"
         ],
         "cpus":1,
         "mem":512,
         "instances":0,
         "maxLaunchDelaySeconds":10,
         "healthChecks":[
            {
               "protocol":"HTTP",
               "portIndex":1,
               "gracePeriodSeconds":30,
               "intervalSeconds":5,
               "maxConsecutiveFailures":6,
               "ignoreHttp1xx":true
            }
         ],
         "upgradeStrategy":{
            "minimumHealthCapacity":0.8,
            "maximumOverCapacity":0.2
         },
         "dependencies":[
            "seed"
         ]
      },
      {
         "dependencies":[
            "node"
         ],
         "id":"wordpress",
         "container":{
            "type":"DOCKER",
            "docker":{
               "image":"bigindustries/wordra:latest",
               "network":"BRIDGE",
               "portMappings":[
                  {
                     "containerPort":80,
                     "hostPort":0,
                     "protocol":"tcp"
                  }
               ]
            }
         },
         "env":{
            "SERVICE_NAME":"wordpress",
            "WORDPRESS_DB_HOST":"galerawordpress-node.marathon.mesos:3306",
            "WORDPRESS_DB_PASSWORD":"wordpress",
         },
         "Labels":{
            "HAPROXY_GROUP":"external",               "HAPROXY_0_VHOST":"dcos-ElasticLoadBa-8SZKUUCZOES7-1382204081.eu-west-1.elb.amazonaws.com",
            "HAPROXY_0_PORT":"80"
         },
         "cpus":1,
         "mem":256.0,
         "instances":1,
         "healthChecks":[
            {
               "path":"/",
               "portIndex":0,
               "protocol":"HTTP",
               "gracePeriodSeconds":120,
               "intervalSeconds":10,
               "timeoutSeconds":40,
               "maxConsecutiveFailures":5
            }
         ]
      }
   ]
}

 

Explanation:

  • The json file starts with the definition of its docker image and its ports.
  • Nodes look up the seed using the DNS name defined in “args”.
  • Port 4567 is the default port of MariaDB Galera, and will be used for node syncing.
  • Port 3306 is the default port used in mysql. WordPress app will be connecting to the node using this port.
  • Haproxy group labels, which are arguments passed to the underlying HAProxy, allowing services to keep in communication.

We also specify the resources that the app requires (cpu and memory), define the environmental variables of the database and add a health check.

 

In the definition of the WordPress application we specify the DNS name of the node as the DB_HOST_NAME. This is how the WordPress application will find the MariaDB Galera database. Database credentials are passed as environmental variables. Moreover we defined some HAProxy parameters for setting marathon load balancer.

 

Save the file as galerawordpress.json.

 

Time for the big moment: we’re ready to deploy our app. Again, this is pretty easy. All you need to do is run the following command.

 

dcos marathon group add galerawordpress.json

galera-dcos

Go back to the DC/OS Web UI to see the effect. Our app just started one instance of “seed” (and temporarily suspended the “node”).

 

Check if the seed is healthy. If it looks OK, add a node manually by scaling it up to one instance. Check if this node turns a healthy green. If all checks out we can then scale it further up and suspend the seed. The seed can and should be suspended. The reason for this is that the seed node will spawn a new cluster again when restarted (e.g. by Marathon), even though the other nodes still form a valid cluster.

marathon-allloadedup-dcos

Now that we have our database cluster up and running we can scale our WordPress instance up to 1. As soon as it turns healthy, WordPress will now be accessible via the public DNS name. We will now setup WordPress.

 

Setting up WordPress

Navigate to the public DNS name. Once there you will be greeted by a WordPress install page. After you made your way to the admin dashboard, choose the ‘Installed plugins’ tab from the left sidebar menu and activate “Wordra”.

 

Next, go to the ‘Settings’ tab.

In the Wordra settings page pass the DNS name of Cassandra. If you set up Cassandra as described in this post then the dns name will be

 

cassandra.dcos.mesos

 

The plugin is now ready to use . Add a page or post and insert the short code ‘[upload]’. Publish the page. You will see the upload template of our plugin appearing. If you wish to have a list of videos to watch, create a new page, add the shortcode ‘[list]’ and publish it.

wordpress-dcos

With that we’ve finished our awesome new movie community web app! It is now ready to face the world, backed by DC/OS. If any traffic spikes appear, DC/OS will handle it. If anything breaks, new instances of our application will spring up. Isn’t technology beautiful?

 

Conclusion

DC/OS offers an effective way to dynamically scale and run web-based sites and services in production with tight SLA compliance. In this post we showed you how to set up a simple ‘movie community’ web application, using technologies like PHP, WordPress and MySQL deployed to DC/OS. Using DC/OS tied the components together, addressing availability and scalability concerns in a flexible way that is relatively straightforward to set up.