Four Best Practices of CI Builds

Continuous integration builds (also called CI builds) are an absolute process game-changer, turning a local ‘works on my box’ coding and deployment strategy into an automatic and reliable process. Once a team gets a taste of an automated and reliable build process, it is easy to fall into a trap of adding all of the things to your CI build. Integration tests, security scans, performance tests, etc, until eventually, the CI build has lost the features that make a CI build implicitly valuable. The best development teams keep continuous integration builds simple.

Best Practice: With no changes to code or steps, the result of CI builds should be the same.

Responsibilities of the CI Build

A CI build’s job is to collect all developer changes, and create a releasable artifact, ensuring changes from all developers on a project are included in the product, and don’t cause conflicts with each other. A CI build must be fast, so that changes that break a build can be immediately corrected; a quick feedback loop is essential to the CI process. A CI build should perform the following steps:

1. Collect all source code and changes.
2. Restore dependencies.
3. Compile / construct the build.
4. Run unit tests and build verification tests.
5. Create the artifact.

Assuming the same code input from step one, and no changes to any of the steps, a CI build’s result should be the same, every time.

A new CI build should be executed if the state of any of the five items above are modified. If there’s a code change anywhere, step one has changed. If any dependencies are updated, in the framework or libraries the product uses, step two has changed. If a build setting has changed, like going from 32 bit to 64 bit, or from ‘debug’ mode to ‘release’ mode, that is step three. Unit test changes typically are manifested as code changes, but any test settings changed would be an item for step four. If the artifact construction process changes at all, that’s step five.

Best Practice: Execute a new build on any individual change to those steps.

Tempting Steps You Should Avoid in CI Builds

• Code Security Checks
• Integration Tests

Code security checks are tempting items to add to a CI build, but fail the ‘no changes to code or steps’ rule. Most code security checks are rule-based in nature, getting updates when there is a new known vulnerability. The process is similar to a malware database updates. With no code change, and no change to steps, but an invisible-to-the-developer modification to the security changes, builds can succeed or fail seemingly randomly. Also, the scope of secure code changes can be larger than common unit / build verification failures. A security change can be very small, but more commonly is larger and design-related.

Best practice: Include code security checks on a regular schedule, not directly connected to the CI build, and have those check failures create backlog items the developers can research and correct.

Integration tests are tempting to add to a CI build, but fail the rule of failing fast. Any system large enough to have integration tests in the first place will be a fairly complex set of tests to run, which may take quite a while to execute fully. They also tend to rely on a lot of scripting and setup work.

Best practice: Include integration tests as a post-CI step in an automated pipeline separate from the CI build.

---

To recap: Continuous Integration builds should be quick, automated, and provide a fast feedback loop. They should contain the following steps.

1. Collect all source code and changes from the repo.
2. Restore any dependencies.
3. Compile / construct the build.
4. Execute Unit Tests and Build Verification tests.
5. Create the artifact.

Remember the best practices of good CI builds:

1. With no changes to code or steps, the result of CI builds should be the same.
   
2. Execute a new CI Build on any changes to the five steps.
   
3. Include code security checks on a regular schedule, not directly connected to the CI build, and have those check failures create backlog items the developers can research and correct.
   
4. Include integration tests as a post-CI step in an automated pipeline separate from the CI build.

Happy building!