Continuous Integration (CI) is a process that is done periodically (ideally after each new commit) during the development of a product. CI reduces the integration time and helps to address errors and issues in the early stages of product development.
With our vast experience of following the CI process, we have developed a list of some best practices that should be helpful to anyone following the practice. This list addresses challenges you may possibly encounter.
- Time of execution – For a CI process to be effective, it should finish its runs and report the results as quickly as possible (ideally, no more than a few minutes). For this reason, the builds and tests should be divided into segments to allow you to run on different pipelines.
At HARMAN, we test on every push to the repository and check if the code can be compiled for various toolchains. Consequently, the CI process will invoke three parallel subtasks, which test the builds on these different platforms.
It is good to run several unit tests per commit — basically, another set of parallel tasks that will build the unit tests and run them. It is also essential to split the unit tests into multiple executables, as they can be compiled independently into separate tasks and run them on the same task. Fundamentally, the unit tests code and builds are changed to achieve better compilation and execution time per unit test suite. This can be further improved by splitting a unit test suite into two suites and running them in parallel.
- Stability – To avoid false alarm triggers it is necessary to disable any unstable tests. If a test is yielding inconsistent results every time it is run, this may indicate that this test is defective. Instability in tests influences the results and harms the effectiveness of the CI process. So, beyond hesitation, unstable tests should be disabled or removed if necessary from the process.
- Informative results – In case a CI run fails, an alert supplying short and useful information should be sent to the author. It should include the following:
- Name of the branch,
- Commit ID and
- Links to the jobs that failed
This way, the author can focus on quickly identifying what breaks the product. Also, the approach of reverting the commit shouldn’t be taken as it might cause other issues or conflicts.
What should be run? – Not every commit needs to trigger all the CI processes. At HARMAN, we developed a script that validates that we have covered all the functions in our design flows with Unit tests. This script is only activated when we push the code to the main branch and not when we are working on a feature branch.
The reason being, during the SCRUM cycle of development, we begin by writing our design for the new story on a side branch (or a feature branch). At this time – the design and the code are not the same – so running the new script will undoubtedly fail. It is later when we start to implement the code and cover the new functions with unit tests. Only when we have fully developed and tested the feature, we merge the side branch to the main branch.
A simple 6-step process:
These are some of the best practices which helped us optimize the application of the CI process. Hope you found them useful. The CI process not only increases the product development process efficiency but also provides the developers with a “safety net” that allows them to push new code without fear.
If you have any comments, please feel free to write to us here or on LinkedIn/Twitter.
