First, let’s discuss what’s suboptimal with the previously described approach. The basic idea was to provide a custom Chef recipe which is executed on our instances whenever our Symfony2 application gets deployed. This recipe took care of

• executing the tasks which need to be done whenever the application is deployed, like installing the Symfony2 vendors or cleaning the application cache
• configuring Apache to correctly serve the application

Well, the problem is that these are really two very different tasks which shouldn’t be mixed together.
Updating the web server configuration every time you release a new version of your application simply doesn’t make sense.

Thanks to Chef deployment hooks, we can separate these tasks. Whatever needs to be done upon application deployment can be provided within the application itself, making it much more self-contained. This way, application business logic and application deployment logic live in the same source tree.

On the other hand, system configuration steps which aren’t specific to deploying a new version of your application, but are specific to hosting your application in a given system context, shouldn’t be bundled with your application, but with your system context.

Thus, we are going to separate the deployment recipes and the system setup recipes. We will provide the deploy recipes from our application, and the system setup recipes through a custom Chef cookbook, just as we did in the first version of this tutorial.

Let’s look at the old version of our deploy.rb recipe, and decide which parts are related to the deployment of our application, and which parts are related to hosting our application.

Well, it’s actually quite simple – everything from the beginning of the file through line 44 is stuff that needs to be done upon every single deployment, or else we wouldn’t end up with a working application.

Let’s move this part of the deployment recipe into our application. Where does it belong? When Scalarium’s Chef deploys our application, it looks for certain scripts in the /deploy directory of our application:

• /deploy/before_migrate.rb
• /deploy/before_symlink.rb
• /deploy/before_restart.rb
• /deploy/after_restart.rb

As their names imply, these scripts are triggered at certain points of the deployment lifecycle. They are closely related to the steps that are necessary when deploying Ruby on Rails applications, and thus not all of them are useful for us when deploying a Symfony2 application.

For the steps we want to execute with our recipe (installing the vendors, clearing the app cache, executing db migrations, installing the assets, and chowning app cache and log dirs), the before_symlink.rb is just fine – it hooks into the deployment process the moment before Chef, after downloading the application source code from Github, changes the symbolic link at /srv/www/symfonyexample/current to the newly downloaded release. At this moment, we have all the source code available, but it is not yet put into production, thus it’s the most sensible moment for additional setup steps.

Moving the deployment-specific parts of our recipe into this hook gives us a before_symlink.rb script as it’s available in our ScalariumExampleSymfony2Application repository on GitHub.

The rest of the original recipe is all about configuring Apache in a way that gives us a working vhost for serving our application – this configuration may change, but it isn’t related to any specific deployment. Thus, these steps should be done only when our Scalarium/AWS instance is set up, and not on every deployment.

And as with the deployment steps, there is a better way to set up Apache for our application, too. Turns out, we don’t need a recipe at all. The reason is that Scalarium implements a very convenient substitution logic – if our own cookbook provides a file with the same name at the same location as one from the Scalarium-provided cookbooks, the file from our own cookbook “wins” and is used instead.

The Scalarium file we are going to substitute is located at /mod_php5_apache2/templates/default/web_app.conf.erb. By providing this file in our own cookbook repository, it’s used as the template for our Symfony2 application vhost – we don’t even need to define “symfony2::deploy” as a Custom Recipe anymore.

And that’s it. As described in my previous post on this topic, we need to configure our Scalarium cloud with the information on our custom cookbook and our application, but instead of manually hooking our custom recipe into the configure and deploy events in Scalarium, our newly created deploy hook now lives in the application itself, and is automatically triggered on every deployment, and our newly created web_app.conf.erb Apache vhost template is automatically used by Scalarium’s Chef when setting up new PHP application server instances.

- (id)outlineView:(NSOutlineView *)outlineView
objectValueForTableColumn:(NSTableColumn *)tableColumn
byItem:(id)item

never get’s called. The reason might very simple: in Interface Builder, in the attributes inspector for your OutlineView, you can define the Content Mode as View Based or Cell Based.

If you simply want to display the content of NSStrings, choose Cell Based, and objectValueForTableColumn will be called.

Only took me 3 hours to find out.

This guy with the psychopathic look standing in a room full of rubbish in front of something that vaguely looks like computers is me standing in front of the first MyHammer server cluster, only days before the whole system went into production.

It was 2005, and I had set up the whole system in a way I’m still quite proud of. Being the only systems administrator at the company at that time, I knew that I wouldn’t face happy times if I would set up the cluster in a traditional way, that is, machine by machine. Although it started small, the system was supposed to grow, especially regarding the web servers, and even I knew that while managing 13 servers individually was doable, it wasn’t fun, and managing several dozen machines just sounded like total nightmare.

Which is why I thought about better ways to setup and manage the whole cluster. I ended up with a central installation server running FAI – Fully Automatic Installation, and hosting a cfengine server. This way, new machines could be automatically provisioned with the operating system (Debian GNU/Linux) they needed by simply booting from their network adaptor, and new as well as existing systems could easily be maintained, configured, and updated via cfengine. No need to ever log in on a specific machine to change configuration or install software. A centralized syslog server completed the picture.

cfengine was a major pain in the ass at times, and with Chef and Puppet there a way more sophisticated tools available today, but the overall system ran extremely well and could easily scale over time.

Years later, the term Devops was coined, and I couldn’t help congratulating myself for getting it right so long ago.

The methodology explained here is outdated, please read Deploying Symfony2 Apps via Scalarium: Improved Methodology for an updated version.

About

This article describes how to use the cloud-based cluster-management platform Scalarium in order to automatically mass-deploy Symfony2 applications with a MySQL database backend onto clusters of Amazon EC2 virtual machines by creating a special Symfony2 environment, using a custom Chef recipe, and making use of Doctrine migrations.

Target audience

Current or soon-to-be Scalarium customers, devops interested in deploying Symfony2 applications using centralized configuration management tools like Chef or Puppet, PHP developers.

Prerequisites

In order to get your Symfony2 application running on Amazon AWS, you need an AWS account, a Scalarium account (correctly set up with your AWS credentials), and a Github account to host your Symfony2 application and your custom Chef recipes. This tutorial assumes that you are already familiar with managing EC2 clusters with Scalarium. It’s tested to work with Symfony 2.0.4 on Ubuntu 11.04 instances.

Overview

The above diagram is here to illustrate how the different parts are working together in order to deploy a running application. The virtual machines hosting our Symfony2 application and the database are EC2 instances running in the Amazon AWS cloud. We will use Scalarium to define the layout of our cluster (called a “cloud” in Scalarium), to configure the application which will be deployed onto the application servers within this cluster, and to configure the custom Chef recipes which need to be applied to our application server instances in order to finalize the deployment and setup of our application.

Looking at the diagram, we could say that Scalarium manages, Amazon runs, and Github provides – Scalarium uses what Github provides in order to run the application on Amazon.

Because Scalarium already provides the setup logic and recipes that allow us to set up basic Apache/PHP application servers and MySQL database servers on AWS, we will only need to provide what is needed in order to finalize the deployment and setup of our specific Symfony2 application – setting up a basic Ubuntu server, installing software packages like PHP, Apache, MySQL, and checking out our Symfony2 application code from Github is all done by Scalarium without the need to provide additional means.

The example application

It makes sense to follow this tutorial using your own Symfony2 application. However, an example application hosted at Github.com → MyHammer → ScalariumExampleSymfony2Application is used to illustrate the process – you are free to use this repository to deploy the application to your Scalarium cloud. The commit history of this repository also shows step by step which changes need to be applied to a Symfony2 application to prepare it for running on a Scalarium cloud.

Setting up a Scalarium cloud for our application

Our very first step is to set up a new cloud in Scalarium which manages the EC2 instances, server roles, application configuration and custom Chef cookbooks needed to deploy and run our Symfony2 application.

In order to do so, click on Add Cloud in the upper right corner of the main Scalarium administration interface. Choose whatever name you like (I will stick with “Symfony”), define a region that makes sense for you (Europe in my case), and choose “Ubuntu 11.04″ as the default operating system. Please choose “Role Dependent” as the Hostname Theme, because this will make it much easier to follow my explanations whenever I’m talking about specific machine instances.

Assuming that you have already set up Amazon AWS in your Scalarium account, please choose the AWS credentials and SSH keys accordingly.

You now have an empty cloud. Let’s define these three things in order to host our application:

• An application
• Custom Cookbooks
• An application server and a database server role

Let’s start with our application. Later, we are going to make some modifications to our application that are necessary for hosting it with Scalarium, but we can already configure it into our cloud. Clicking on Add Application gives us a dialogue in which we can configure where EC2 instances will be able to pull our application from upon bootup or application deployment.

I’m going to choose “SymfonyExample” as the name. Please choose “PHP” as the Application Type. If you are going to use the sample application that resides on Github, please choose a Repository Type of “Git”, and enter the following Repository URL: “git://github.com/MyHammer/ScalariumExampleSymfony2Application.git”. All other parameters are not important for now – however, if you are configuring your own private repository, you will need to provide a deploy SSH key.

Although this is sufficient to deploy the code from Github onto our instances, it’s not enough to actually get a running application. Additional steps are necessary after the code has been pulled from Github, and those steps need to be performed using a custom Chef recipe. We are going to write this recipe later, but we can already configure the cookbook that contains the recipe into our cloud. On the cloud overview page, select Manage Cookbooks from the Actions dropdown.

In the new dialogue, check Enable custom cookbooks, and configure as follows:

• Repository Type: “Git”
• Repository URL: “git://github.com/MyHammer/ScalariumExampleSymfony2ChefRecipes.git”

Deploy SSH Key and Branch / Revision don’t need to be filled out.

As a last step, our cloud needs two machine roles: one for our application servers, and one for our database server. We will start with the application server role. On the cloud overview page, select Add role, and choose the PHP Application Server role.

Once you have added it, an additional configuration is necessary: In Custom Recipes, please add the recipe “symfony2::deploy” for the Scalarium actions configure and deploy. This makes sure that our recipe is run everytime our application is deployed to an instance, or whenever the cloud changes.

Last but not least, create an additional role MySQL Master. Additional configuration is not needed here. If you like, you can already add and boot an instance for this role.

Modifying the Symfony2 application

Our Symfony2 application doesn’t need to be re-programmed in any way in order to run via Scalarium – however, its configuration needs to be specifically modified. You can either modify your existing “prod” environment configuration, or you may want to create a new environment called “scalarium”, which is what I did for the example application.

The nice thing about Scalarium and Symfony2 is that the former provides a very nice PHP interface to the layout of the current cluster, and the latter provides an easy way to make use of this interface.

This allows us to dynamically configure our application’s database setup, avoiding the need to hard code e.g. the IP address of our MySQL master into the application. Here is a step by step description on how to achieve this:

First, create a new and empty file /app/config/config_scalarium.yml. This is going to be the main configuration file for the new “scalarium” environment. Fill the file with the following content:

imports:
- { resource: parameters_scalarium.php }
- { resource: config.yml }


This makes our new environment use 99% of it’s settings based on those from the “prod” environment – however, an additional file is imported, /app/config/parameters_scalarium.php – this is where we are going to dynamically define our database settings using the power of Scalarium.

Let’s fill /app/config/parameters_scalarium.php with life:

<?php

unset($scalarium);


What happens here is that the database parameter placeholders like %database_host%, which are used in /app/config/config.yml to configure the database for our application, are programmatically defined based on the current state of the Scalarium cloud. Leveraging the power of the scalarium.php script, which is available in the root directory of our application per default (a symlink is created whenever our application is deployed on an instance), this makes our database config dynamic and therefore always up to date.

The creators of Scalarium point out that using scalarium.php is just a convenience. The full-stack alternative would be to write a Chef recipe that generates the configuration file and register this recipe on the configure and deploy events. Inside the Chef recipe Scalarium provides you with all necessary information like which hosts exist in the cloud and what their configuration is (e.g. IP addresses, roles, etc.). See http://support.scalarium.com/kb/custom-instance-setup for further information on how to use the Scalarium cloud structure information in your own recipes.

One important thing to consider here is that the settings in /app/config/parameters.ini overwrite settings with the same name in /app/config/parameters_scalarium.php – we therefore need to remove all lines containing “%database” from parameters.ini!

The last step is to create a Frontend Controller for our new “scalarium” environment at /web/app_scalarium.php:

<?php

$kernel = new AppKernel('scalarium', false);
$kernel->loadClassCache();
//$kernel = new AppCache($kernel);
$kernel->handle(Request::createFromGlobals())->send();


You can look at a visual before-and-after comparison of all these changes on Github:

Putting the parts together

Our cloud now has everything in place it needs. Its roles are defined, our application servers know where to pull our application from, instances of role “PHP Application Server” are configured to use our custom Symfony2 deploy recipe, and our application has been modified to make use of the dynamic cloud configuration in a new environment called “scalarium”.

Let’s take a moment to look at our custom Chef recipe.

What this recipe does is it checks for each application that is going to be deployed if it’s actually a Symfony2 application, and if yes, it starts by updating the vendors (or installing them from scratch if none exist yet), clears the application’s cache, applies database migrations (if any), makes sure that cache and log files and directories are read and writable for those users that need it, installs and configures an Apache vhost file for the app, and disables the default vhost.

Ok, let’s deploy our application! All you need to do is to add an instance of role PHP Application Server and boot it up. Everything else happens automatically:

• Scalarium requests an EC2 instance from Amazon AWS which is booted with a fresh Ubuntu 11.04 image
• Scalarium sets this machine up with some basic packages like PHP, Apache etc., and executes its basic setup routines (which are Chef recipes, too)
• Our Symfony2 application is pulled from Github and deployed to /srv/www/symfonyexample/current
• The instance executes our custom Chef recipe symfony2::deploy which installs the vendors, clears the app cache, applies the database migrations, installs the web assets, sets users and permissions on cache and log files, and configures an Apache vhost for our application

Once this is done, you can browse to the public IP of the booted application server instance, which will display the Symfony2 welcome page.

Being the BDD fanatic that I am, I wanted to start off behaviour-driven right from the beginning, and Vows looked like a good choice.

However, I quickly came to the point where I needed to stub out a dependency in one of my modules, and as far as I can see, Vows doesn’t provide mocking/stubbing. But https://github.com/felixge/node-gently does, and here is my approach at combining these two:

This is the Vows spec:
var gently = global.GENTLY = new (require("gently"));

vows.describe("My Module").addBatch({
"when calling its foo() method": {
topic: myModule,
"it triggers a console message": function (topic) {
gently.expect(gently.hijacked.sys, "puts", function(str) {
assert.equal(str, "Hello World");
});
topic.foo("Hello World");
}
}
}).export(module);


And this is the implementation of MyModule:
if (global.GENTLY) require = GENTLY.hijack(require);

exports.foo = foo;


No idea if this makes any sense in the long run – I will tell you when I’m about 14 hours or so into BDD node.js development…

The following roots in something I heard from Jon Jagger at QCon London 2011 after his fantastic talk about Deliberate Practice. It was related to Test Driven Development. He asked “Why do cars have brakes?”. It made us think “To stop!”, but he said “No – to go faster”.

(Jon points out he didn’t invent it, he heard it from Kevlin Henney).

I have been thinking about this ever since, and here is what I came up with.

Imagine I would give you the keys to my car. I would tell you “here are the keys, you can drive wherever you want, including the highway, have fun!”

How fast would go? My car is not exactly a sports car, but it can do around 200 km/h. I guess we both agree that you would drive around 50 km/h within cities (the maximum allowed in Germany), and probably up to 200 km/h on the highway, as long as there is no limit.

Fine. Now image this: I would give you the keys to my car. I would tell you “here are the keys, you can drive wherever you want, including the highway, have fun! Oh, just one thing, the brakes don’t work.”

Now let’s forget for a moment that in reality, you probably wouldn’t start at all, if you had to drive, then how fast would you go? 10 km/h, maybe 20? Driving extremely cautious, always trying to look as far ahead as possible if you are going to need to halt? Yeah, I thought so.

But why is that? The brakes don’t have anything to do with the speed of my car – it’s still capable of doing 200 km/h just fine!

It’s because the ability to stop is what enables you to go real fast. With only a bit of exaggaration you could say that having a brake allows for a very “iterative” way of driving – no cars within the next 300 meters, let’s accelerate a bit – oh, there’s a car coming over from the right, let’s brake a bit – ok, now I can accelerate again – ah, there is a signal that suddenly turned red, no problem, I will stop here.

For me, this metaphor is the best I could find by now to explain to myself (and in the future, to others), why I really want to develop test-driven, and why it actually makes me faster, not slower, although I’m doing more.

Just as the brake doesn’t directly influence your driving speed, but does so indirectly, your tests won’t influence your coding speed directly, but indirectly. It’s because once they are in place, they allow you to iterate over your code and refactor it at what I, from my own experience, can only describe as the speed of light compared to conventional programming.

With tests in place, it’s like: Mh, what if I would split this rather long method into two? – ok, works; What if I put a bit more of dependency injection into this class? – ah, now this test here fails, no problem, I will have it back to green within minutes, I know exactly where to go to fix this; Hey, I could give this method here a better name – ok, still green; There’s this performance bottleneck deep inside this one class that is heavily used by a lot of other classes, let’s see if I can fix this – my tests will tell me if I accidently changed behaviour.

Compare this to conventional programming: You will never know for sure what breaks somewhere else if you change something. If you want to find out, you need huge amounts of manpower to have your webpage or UI tested for regression. What really happens is that you slow down to a near halt: because you don’t know what’s around the next corner when developing, and you know there is nothing that will immediately stop you and save you from harm if you take that next corner, you will drive, err, code so cautious, you won’t make any real progress.

On which Jon commented: “Yes. As the pragmatic programmers say, paraphrasing – you don’t know why it’s broken because you didn’t know why it worked in the first place.”

So, I had this (rather obvious) gut feeling that working iterative in software projects makes a lot of sense, I’ve heard all the arguments and explanations and examples and stuff like that, and I probably already “got it”, but I thought it could make sense to clearly work out why exactly it makes sense, by visualizing it. Surely it’s no rocket surgery what happens here, but I kind of like it and would like to share it.

So here’s my approach:

The customer tells a product manager what he wants him to build. Let’s see how a waterfall approach leads to the product team failing at this:

The reason why what is finally delivered isn’t what the customer expected is that the project goes through the hands of different people and different stages, and every time the project is given to another person or team, the amount of misunderstanding grows. That’s only natural because we cannot copy ideas from one brain to another in a 1:1 manner.

Here is why an iterative approach makes sense:





The project really starts the same: There is a certain amount of misunderstanding, and the team does things wrong. But due to the regular feedback from the customer, this wrong direction can be corrected. It might then move into another direction which is still a bit wrong, but then comes the next correction, and finally everything is on track.

Once stable, it will allow users to easily collect and visualize different types of statistical data which is typically produced by internet platforms, like CPU performance, user logins, HTTP requests etc.

The main application is build on Rails, the server used for data collection is written in node.js, the web interface makes heavy use of jQuery and uses Raphaël to create SVG graphs. Mass data is saved in a SQL db, other data is stored using CouchDB.

The project’s code is hosted on Github at https://github.com/ManuelKiessling/PlatformHealthViewer.

This video is a short introduction to the current alpha version of the project. A funny voice and lots of grammatical shortcomings are included for free:

Transcription of the video:

Hi. Platform Health Viewer – or PHV – is my current pet project.

I need an easy and lightweight way to collect and visualize the different key performance indicators of the web platform I’m responsible for – stuff like CPU performance of important systems, user logins, http requests.

So I started to play around with Ruby on Rails, jQuery, CouchDB and node.js, and here is an early alpha I would like to demonstrate.

My primary goal was to make the process from feeding data into the system to visualizing that data as simple as possible.

In order to get data into the system, all you need to make is an HTTP call, which makes it very easy to collect data from very different sources.

Let’s try an example. I would like to visualize the cpu usage of my local machine.

I’m going to collect this data using a standard unix command, sar.

That’s an important aspect of my approach: It doesn’t play any role for the Platform Health Viewer where the data comes from, you’re completely free to choose how to collect it.
This way you can feed really anything into the system, from generic data like CPU load to highly individual stuff like the user logins of your specific web site.

Ok, here is how I can get my cpu “usr” value on my OS X command line:

sar 1 1| grep Average| cut -b 14-15

Great, that will do.

How do we push these values into the system? It’s a simple http post request using curl:

curl –data “event[value]=`sar 1 1| grep Average| cut -b 14-15`&event[source]=macbook&event[name]=cpu_usr_percentage” http://localhost:3000/queue_event

As you can see, the payload of the post requests is just 3 parameters: the source of the event, the name of the event, and its value.

Again, you’re completely free here, you don’t need to configure event names and sources inside PHV – just define them when pushing data into the system, it will happily accept it. We will see in a moment how to make sense of different events that were pushed into the system.

Ok, let’s use a small helper script I wrote in order to feed the CPU sys, idle, usr and nice values into my system:

cat script/agents/macosx/cpu_overview_percent.sh

As you can see, this is all done using only standard unix commands.

Let’s start the script:

bash ./script/agents/macosx/cpu_overview_percent.sh http://localhost:3000/ macbook

I’m just providing two parameters here, the URL to my platform health viewer installation, which resides on the same host for this demo, and the source name, which I call “macbook”.

As you can see, my script pushes all four CPU usage values into the system. Now let’s have a look at this data within platform health viewer.

Well, the Dashboard is still empty, because we did not yet define any visualizations. But the “Tageditor” doesn’t show any events, too. This is because the events I pushed into the system have not yet been normalized to event-types.

This is an additional step, because it will allow the system to push incoming events into the database as quickly as possible without the need to normalize those events regarding their name and source. This normalization is done using a rake task:

rake queue:convert

This task reads the events from the incoming queue, creates new event-types as needed, or connects the event values with existing event types if they already exists. It then deletes the incoming queue.

Getting back to our tageditor, we can now see our 4 event types.

An event type is the combination of an event source and an event name, so “macbook – cpu_idle_percentage” is one event type.

Let’s see how we can use the tag editor to create something useful. Grouping one or more event types into a tag is what makes our data suitable for visualization. I’m not quite happy with the term “tag” by the way, maybe I will come up with something better.

Anyway, let’s create a very simple tag which we can use to visualize exactly one value.

I’m going to name my tag “macbook_cpu_usr”. It will hold all events whose source matches “macbook”, and whose name matches “cpu_usr_percentage”. I could type those parameters into the text box, but it’s easier to just drag’n'drop them there.

Ok, let’s add this tag.

Now we have this first tag, and to check if it works as expected, I can preview the values of the matching events.

Let’s push some new values into our system and check if they are visible here.

Ok, I’m starting my helper skript again in order to post new values to the server, and I start my rake task in order to normalize these values.

Clicking again on “Show latest events” now shows these values.

I will now start data push and normalization in a loop in order to get a lot of values.

Ok, we still have no data visualization, so let’s do this now. Let’s switch to the Dashboard and add a frame, which is a container that will hold our graph.

A frame is the visualization of all values connected to a tag, so I need to provide the name of the tag I want to visualize with this frame.

“Add frame”, and here we go. A simple line graph representing one of my CPU values. The graph is actually an SVG, created using Raphael, an awesome JavaScript library.

And thanks to jQuery, I can freely move and resize the graph.

Let’s create a graph with all my CPU values in it. Back to the Tageditor, I’m going to drag all my values together.

I can also create tags by combining event-sources and -names with already existing tags, as you can see here.

Let’s check the values of my new tag, and there are all the different CPU values my script collects.

Back to the Dashboard, I’m going to create another frame for my new tag. As you can see, this one contains 4 linegraphs and gives me a nice overview of my system’s CPU performance. Of course, a graph legend is needed, something that’s not yet implemented.

Well, that’s it, that’s the current state of this project, I would love to hear your feedback, you can fork the code on github and drop me an e-mail.

Thanks for your interest.