Automating Meaningful Automated Tests
One of the challenges of meaningful testing is the construction of target environments suitable for the task. Cloudify excels in the construction and configuration of virtual environments, and can be leveraged with a build server like Jenkins to provide testing that more closely reflects production conditions.
Beyond the standard “stick this tool in your CI pipeline”, because there certainly many other tools that can spin up VMs, Cloudify has an extraordinarily rich set of capabilities that can be adapted to simplify the automation of meaningful integration tests. Besides having robust native orchestration capabilities, Cloudify has a rich set of plugins to integrate with common tool chains consisting of tools like Ansible, Terraform, Azure ARM templates, Cloud Formation templates, etc. As such Cloudify provides cross platform/tool automation support, as well as serving as an integration platform.
DevOps toolchains can include dozens of unintegrated tools, which makes automation a technically complex and arduous task. … The toolchain should focus on removing execution barriers and automating the development and continuous delivery process. .. There’s a cost attached to learning, integrating and .. replacing them - Gartner via Nati Shalom on LinkedIn
On the simplest end of the spectrum, consider a typical CI process that runs integration tests on an environment. The top level Cloudify descriptor is called a ‘blueprint’, which contains a YAML based TOSCA dialect that models the resources needed in the test environment including networking, storage, and compute for various clouds.
In this model, Cloudify is invoked in a Jenkins pipeline in order to construct the target environment on one or more clouds. The blueprint then provides Jenkins inputs required for an integration test suite which is executed. Finally, Jenkins (via Cloudify) tears down the virtual infrastructure after retrieving testing logs and errors (not shown). The assumption here is that the test code is separate from the infrastructure blueprint.
A more integrated approach is possible as well, where the test suite is packaged as part of the blueprint. This makes sense when the testing is sophisticated enough to require knowledge of the deployment topology. For example, if all your test code does is pump in data and then make some calls to verify it was processed correctly, it can be largely independent of the topology. On the other hand, if your tests require scaling a cluster and measuring effects at granular level (for example), or does chaos monkey style testing, it’s best to package the code and the blueprint together as a complete representation of a test suite. This actual simplifies the original picture.
The inclusion of a declarative, general purpose orchestrator like Cloudify makes sophisticated tests of distributed systems more approachable and repeatable. This elevates the scope of devops to areas that have traditionally required human participation. It also directly addresses a key challenge in devops, and that is flexible tool integration using an open (and open source) platform as a stable basis. The result is a platform that enables meaningful, repeatable tests that can assist in automating the creation of high quality software.