“DevSecOps” can be defined as integrating software security and compliance into the software development process, as opposed to bolting it on the end as a separate testing phase. This is where the “shift left” terminology comes from; the movement of testing traditionally performed late in the development process to earlier phases. At a minimum, this approach aims to catch security issues as early as practical, minimizing costly rework (and possibly redesign) that results from late discovery. It also emphasizes the inseparability of security from the development process. This article describes the DevSecOps movement background, ecosystem, and a simple example process.
Software security vulnerability testing is as old as network connectivity, but has really exploded in importance in the internet era. Historically, security testing has been a post-development activity, performed during QA (quality assurance) and staging phases. This fits naturally with typical waterfall processes. As the DevOps (and Agile) movement has emphasized incrementalism (i.e. rapid iteration), the need to automate as much traditional QA as possible has shifted testing “left” in the process. One of the key benefits of rapid iteration is the early discovery of errors, which has led to the prevalence of automated unit tests triggered by development team events (such as pull/merge requests). Consider also that the “infrastructure as code” practice on its own puts security into the hands of developers, and as a result requires extensive automated security testing.
Despite this flow of former QA responsibilities into development teams in recent years, security testing has mostly remained the domain of non-automated manual processes. To some degree, this lack of movement is driven by the fact that security expertise is a specialized, rich, and ever changing skill set, which makes the creation of adequate automated tests by developers unlikely. This often leads to the hiring of a separate security specialist or team, depending on organization size. Embedding such experts into development teams is a common approach that avoids the dubious goal of training all developers to become security/compliance experts.
SecOps is the security evaluation and monitoring of the operational system to combat attackers and assure compliance. SecDevOps is specifically the application of cross functional teams and automated security testing to the development lifecycle. Standard DevOps pipelines are enhanced with security and compliance tests, to enhance existing coding standards, unit, and integration testing. Due to robust growth in the supporting product ecosystem, much can be achieved with automated testing without a huge investment in development resources.
Security testing tools suitable for DevSecOps can be divided into “static” and “dynamic” types, with static tests being applied to source code and/or static configuration, and dynamic tests applied to an execution environment (not necessarily operational).
Static tools, despite their limitations, are a natural fit for CI pipelines, and can be triggered by developer events such as commit or pull request. These tools are typically language specific, and looks for vulnerabilities to attacks like:
- SQL injection
- Credentials in code
- Buffer overflows
- Cross site scripting
Dynamic tools evaluate running code and environments for vulnerabilities and tend to be oriented towards webapps. These tests are most valuable in integration, and QA/staging phases. They look for vulnerabilities like:
- Access controls
- Network/VNF security
- Password security
- Session management
To make the above discussion more concrete, let’s take a look at an example DevSecOps pipeline. The stack consists of Jenkins driving the build process, Ansible performing configuration, and Cloudify preparing the test infrastructure. Including an orchestrator like Cloudify is critical for creating repeatable infrastructure that security tests can be run against, especially with regards to network security.
In this scenario, Jenkins retrieves release artifacts, executes unit tests, and executes static security analysis tests. If there are no errors, the integration phase begins. The integration tests first require a realistic test environment be constructed, which is where Cloudify comes in. Ideally, the same Cloudify blueprints used in production are used during integration and dynamic security testing, probably parameterized to create a smaller scale deployment. Cloudify also interfaces with Ansible, making environment construction seamless from the Jenkins perspective.
Once the environment is running, which can include full network, storage, compute, and service configuration, Cloudify returns outputs describing various hosts and network configurations for the security tests. Jenkins then runs automated network penetration and application tests against the environment to identify vulnerabilities. Once complete, Jenkins can tell Cloudify to tear down the environment.
A reliable DevSecOps process requires the repeatable construction of secure environments. Those test environments must be representative of the production environments to be reliable indicators of compliance. An open source orchestrator like Cloudify can connect the test environment directly to production, the only way to create a truly secure DevSecOps process.