One of the best fit use cases for Cloudify is the automated creation of operational environments. This capability is great for automating integration testing, and is related to the Environment As A Service concept. Gitlab provides an alternative platform to Github, and includes features such as source code control and CI/CD. In this article I’ll investigate extending the typical continuous integration (CI) process beyond unit testing to fully automated integration testing, using the CI feature of the Gitlab source code management system in concert with the Cloudify orchestrator using an on-prem installation of Gitlab. This combination supports a high level of testing automation, and high levels of software quality.
Gitlab CI, on a superficial level, triggers a script as the result of a repository commit. Gitlab uses a ‘runner’ as the platform to run the script. Docker (and other executors) are also available for running tests, but not covered here. A ‘runner’, for our purposes, is a compute host that Gitlab can ssh to, install code under test, and then run our test script.
Typically this test script, encoded in a repository file named .gitlab-ci.yml, sets up the unit testing environment by installing necessary packages, and then executes testing commands (e.g. junit or tox ). The details of .gitlab-ci.yml are beyond the scope of this article, but available in the Gitlab documentation if you’re curious.
The overall process is pictured above. The Runner, triggered by Gitlab CI, runs ‘ install’ on a blueprint which creates the test environment. Note that a best practice is to have the test blueprint included in the repo so it is under version control as well. In this case, an additional, unpictured step is required; a simple call to
cfy blueprints upload. Once the environment is up and available, the blueprint will publish any needed connection information (e.g. IP addresses) of the test environment, which is fed to the integration tests.
While it is possible to install and configure the Cloudify client with every test run, it is much more convenient and performant to have the CLI pre-installed on the runner. As mentioned before, this article deals with using a Gitlab ssh executor on a Linux server. To prepare the runner, ssh into the runner server and install Cloudify client for the gitlab-runner user. Instructions are in the Cloudify documentation. Once that is done, create a profile for connecting to the desired Cloudify server. Run
cfy status as the gitlab-runner user to verify all is well.
In addition to the Cloudify CLI, the jq utility will prove useful for extracting information from the Cloudify deployment. The install of jq will vary by the Runner OS, and is available from yum and apt package managers.
Now that the Cloudify CLI is available to the runner, creating the environment(s) for automated integration tests is simple. The install workflow can be run with a single step. This will look something like the following:
cfy install -b testenv -d testenv -i <your inputs> tests/integration/resources/testenv.yaml
Once this step is complete, a deployment with the name testenv exists on the Cloudify server and we can fetch the outputs. This is done with the
cfy deployment outputs CLI command:
cfy deployment outputs -d testenv --json
Of course your outputs will differ based on the system under test (and possibly the tests themselves), but the same principle applies. The output from the above command can be piped to jq to grab relevant information. A .gitlab-ci.yml example excerpt follows. Note the addition of the
cfy uninstall command, which cleans up the test environment:
- tox -e py27,flake8 #unit tests
- cfy install -b testenv -d testenv -i 'input1=something' test/integration/resources/testenv/yaml
- IP=`cfy deployment outputs -d testenv --json|jq 'ips.value.ip1'`
- python tests/integration/test1.py $IP #integration test
- python tests/integration/test2.py $IP #integration test after_script:
- cfy uninstall testenv
Automated integration testing is a high value activity that takes software quality to the next level. Cloudify provides a convenient way to define test environments and then construct and tear them down on demand with simple CLI commands. This article discussed a simple example scenario that hopefully is enough to get anyone started bringing their automated testing game to the next level.