Continuous integration for non-trivial applications, by nature, involves a significant amount of orchestration, and can become quite complex. This kind of problem is a natural fit for the Cloudify (http://cloudify.co) open source orchestrator, which excels both at complex automation tasks, and at integrating with other tools relevant to the space. This article discusses a CI toolchain that includes Cloudify and other tools such as Ansible and Terraform.

Continuous Integration Challenges

For many development organizations, continuous integration is a desirable yet difficult-to-achieve goal. The age of virtualization has brought with it the ability to spin up arbitrarily complex environments quickly, bringing within reach the ability to perform realistic integrations and tests on demand. Tools like Cloudify simplify the automation of virtual environments by modeling environments as a composite of building blocks that can be tailored for user requirements.

Simplification is a must once the CI/CD process evolves beyond building and unit testing. Performing quality, repeatable integration (and performance) tests are a must, but very complex for non-trivial deployments. CI/CD pipelines look something like this:

CI servers such as Jenkins, Travis, Team City, etc. detect an approval step in software repositories (commit) and run a build and unit test regime, based on the technology stack being used. Integration testing requires the construction of a simulated production environment (or environments) in order to evaluate a system and/or subsystem operation as a whole. Automated performance testing is more rarely pursued, and is challenging because meaningful performance tests require infrastructure very similar to production. Next comes the promotion to a QA environment, traditionally the home of manual testing. Next comes staging, which is as close to production as possible, and possibly home to user acceptance testing. Finally comes production, which depending on acceptable downtime, may be phased or ‘big bang’.

Automating this sequence beyond unit testing requires the ability to stand up and tear down complex environments on demand. It may also require the deployment and configuration of software packages. This is a task that Cloudify is ideally suited for.

Cloudify Abstractions

Blueprints

Cloudify has several abstractions from which parameterized deployments can be designed, and subsequently plugged into a CI pipeline. The foundational abstraction is the blueprint, which contains the description of elements to be orchestrated. These elements may be network elements, compute servers, software, hardware, etc… The common thread holding together the elements in a blueprint is that they will share a lifecycle: they will be deployed and undeployed together. A blueprint is (at its simplest) a YAML file, which can be versioned and released like any piece of software (i.e. “infrastructure as code”). It also can be consumed by a CI server to construct environments.

Plugins

Plugins are extensions to the Cloudify server that provide declarative components for building blueprints. These extensions can provide bridges to other technologies, including but not limited to Ansible, Chef, Puppet, and Terraform. The plugins provide a broad scope of infrastructure possibilities, including various public and private clouds and Kubernetes. This permits Cloudify to become somewhat of an integration platform for the purposes of test, and lets you use the tools you are comfortable with.

Blueprint Composition

Blueprints can be composed of other blueprints, and assembled at runtime for optimal environment construction. Consider a multi-phase test that requires various compute node configurations, but a common network infrastructure. The network infrastructure can be defined in a separate blueprint from compute, and critical information about the network can be advertised to the compute layer, avoiding repeated construction and tear down of the network.

Since the integration tests and integration testing infrastructure (blueprints) are closely coupled, they can/should be versioned together in the code repository. Groups of tests can be related to specific infrastructure deployments and consumed by the CI server at test time.

CI Server Integration

Cloudify can be integrated with a CI server either by a purpose built plugin, as in the case of Jenkins, or by exploiting the Cloudify server’s northbound REST API.

In the diagram, the integration test blueprints are versioned along with the integration test code. In simple cases, perhaps just one environment configuration is needed. In others, many. Each blueprint can also be parameterized to pass in relevant parameters such as network address ranges, virtual instance image ids and flavors, and whatever else is necessary. The test environments are taken from the source code repository and passed to a Cloudify server ( in simple cases no server is needed ), which deploys an environment for the related tests. When the environment has been constructed, the CI server can access critical non-static information from the Cloudify server such as DHCP assigned server IPs. Then the CI server can execute the related integration tests for the infrastructure configuration. After testing (not in diagram) the CI server can command Cloudify to tear down the environment. Alternatively, it might command Cloudify to scale the environment up or down and rerun tests, or command Cloudify to construct a new environment for other tests.

In another twist, the containerized version of Cloudify can be created and destroyed as needed by the testing regime. This requires a bit more work on the CI server side, since Cloudify itself will need to be provisioned as part of the testing process.

It should be noted that Cloudify can start and manage the CI server itself, independently from and prior to CI operation, to provide high availability.

Promotion

In most cases, your testing environment will attempt to mirror your production environment. Careful design of testing blueprints can yield blueprints and plugins that can be used to stand up other environments like QA, UAT, and ultimately production. Cloudify is an active orchestrator, not just a fire and forget command line tool. It can not only deploy complex environments, but also scale, heal, and run complex workflows on them. So the same framework that automates the creation of environments for testing can be used to actively manage environments in production, including blue/green and canary deployments. Cloudify can also be used to create (and tear down) CI build workers on demand to optimize cost efficiency.

Conclusion

Cloudify brings infrastructure automation to continuous integration in a comprehensive and opinionated way. Cloudify can tie together CI tools, node configuration tools, and virtualized environments both public and private, to provide a high level of test automation. Cloudify can also automate the promotion of a code base from dev to production, using versionable blueprints that support the “infrastructure as code” paradigm. This high order of automation advances CI goals to provide build quality feedback in the minimum time (and cost) possible.