Buildbot is a powerful and flexible tool used by many sophisticated products like Python, Firefox, Webkit, Google Chrome, Twisted, Node.js and many more.
Buildbot, the Continuous Integration Framework Buildbot is primarily used to build software for different architectures covering the process from code compilation to packaging, but it can also be used to do continuous integration of less sophisticated projects where tools and services like Jenkins and Travis-CI are more popular but less flexible.
In this story I introduce Buildbot to run test suites for two products, a sample web application and a sample dependent web project. The web app is a fully functional open source example of a Django pluggable application. While the project is an implementation of the Django tutorial plus a few simple test cases.
The configuration will face the following final scenario:
Test suites for both the app and the project will run under several versions of Django and Python in order to be sure that the app is usable for every supported Django release and that the project might be easily migrated from one release to the next one.
The web application is hosted in a public repository in GitHub while the project is in a private repository in an in-house server. Doing so I cover two ways to send notifications to Buildbot. The webapp will notify Buildbot through a web hook, while the project will do the same through a post-receive hook created in the self hosted repository.
I want Buildbot to build the web project when changes hit project's repository. And given that the project depends on the app, I will tell Buildbot to build the project when the app's repository gets updated too.
Buildbot is a Twisted based distributed service customizable through a Python script. The service is divided into several components that have to be put together to run the desired configuration.
Let's take a quick look at the components and their relationship before addressing the setup.
Buildbot first class citizens are masters and slaves. You can have several of both types, but the typical configuration uses one master and several slaves. In this example I use only one master and three slaves, all them running under the same OS instance.
Think on the master as the service that runs the web, gets notifications from repositories, orchestrates operations with slaves and sends notifications to users.
Think on slaves as environments. When Buildbot is used to build software for different hardware architectures slaves live on appliances powered by such architectures. When it's about testing Django code, slaves may well be virtualenvs running each a different combination of Python/Django.
Before the slaves start running the builds Buildbot needs to know what to build and when. Something you do by declaring the following components:
These components are the core of Buildbot's flexibility. Let's see their functionality and how they are related to each other.
The ChangeSource object is responsible of getting notifications on source code changes from repositories. Source code changes are the usual starting point of the whole process. They lead Buildbot to wake up the next component: Schedulers.
A Scheduler will run a list of Builders. You can associate a Filter to each Scheduler to decide whether the Builder has to run or not. Only those schedulers that pass their filter will trigger their list of Builders.
A Builder will run the sequence of Build Steps that make up the build. A Build Step might be a shell command, a SCM command, a trigger for a scheduler or any other step you might want to run as part of the build process. A set of Build Steps are grouped together under a Build Factory.
Finally, each Builder object is created with a Build Factory and a Slave. The Builder runs in the given Slave by executing the sequence of Build Steps represented by the Build Factory.
There are three pictures in the following section that help to put these concepts in clear.
To make an easier introduction I will approach the configuration in three steps:
I first detail the configuration regarding the core components of the master. Read the complete story before doing any actual configuration. The complete Buildbot installation and setup are covered later, including
master.cfg files for each step.
Buildbot configuration layout to build a Django web project. First Buildbot will build the web project. The steps to build the project are going to be the same in each slave. The goal is to prove that the project builds successfully under the supported releases of Python and Django.
The configuration uses the following components:
Look at the image to see how these components are connected.
By adding the web app to the configuration Buildbot will have to distinguish when to build the project and when the app. A situation easily handled by a couple of filters.
change_hook_dialectsparameter in order to receive notifications from GitHub.
The important additions here are the two filters and the
change_hook_dialect parameter in the status object.
This extra status parameter tells Buildbot to allow incoming notifications from GitHub in the web path
So far, with the current configuration, a source code change will make Buildbot either build the project or the app but not both. However, given that the project depends on the app it seems reasonable to check whether a source code change in the app would eventually break the project.
Buildbot configuration layout to build a Django web project and a Django web app. Apps' Builders trigger Projects' Builders through Trigger build steps and Triggerable Schedulers (blue lines). In other words, each of the project's builders could run right after Buildbot runs each of the app's builders.
Say the app builds successfully under Python 3.2 and Django 1.5. Buildbot could then build the project within the same slave to verify whether the project also builds under the same Python/Django conditions without errors.
Buildbot has a special build step called Trigger. Such a build step triggers a special type of scheduler called Triggerable, that in turn runs the list of builders associated with the scheduler.
For the case of this story the Trigger step can be used as the last step of each of the app builders. The trigger step will run a triggerable scheduler that in turn will run the project builder for that very same slave.
Both functions are covered with test cases:
do_somethingchecks whether the function returns the integer value 10.
do_otherthingverifies that the function returns the integer value 11.
Change 'do_something' and its corresponding test case to make the function return a string and to make the test validate that result.
Such a change comes to represent one of the many changes we could face during the development cycle of any application. When pushing the changes to the repository the project won't build.
The Buildbot configuration for this example uses 'master' branches for both the app and the project, but to run a full continuous integration environment you might want to run builds on development branches too, as to be certain that new code and patches don't break anything before it lands in production.
All the files and steps-to-reproduce the complete Buildbot configuration are available here.
You can adapt the configuration to cover other cases, extend the build steps and/or builders to do additional tasks like code checking, build documentation, packaging and deployment. Buildbot is flexible enough as to cover any corner case.