Jenkins and Continuous Integration
When hundreds of engineers check-in code to a large codebase, continuous integration becomes a necessity for managing the health and scalability of the software. Thanks to automated testing, code integration is much more efficient and systematic.
One great tool that supports automated testing is Jenkins. Jenkins is a Java-based open source server that supports test automations and continuous integration by creating the software pipeline as code. When used with some source control management tool (ie. Git, Perforce, etc.), engineers can commit code to the shared codebase and see the network topology of all projects influenced by the commit. Specific project configurations can be defined on Jenkins through code as well.
Jenkins runs jobs on individual nodes. In order to set up the proper workflow, we need a Jenkins master. This master orchestrates Jenkins slave nodes which host the different jobs that are used in the software and that need to be tested regularly. Suppose we write a new script that produces log reports based on data retrieved from a cloud server and gets published onto a network share. We want to run this script everyday at 5 PM PST. How do we automate this process using Jenkins?
1. Set up the Jenkins slave
This step requires obtaining a slave node that can host the project. After provisioning the new slave node with the required Jenkins pre-requisites, we then create a project on the Jenkins master itself that indicates which node is hosting the project.
2. Set up the project configurations
Now we indicate how this script is going to be run. We specify which source control management tool to sync the project with, what environment variables are injected into the script at build time, how other projects are influenced by this new project, what the schedule should be, and much more. While we can use the Jenkins GUI to specify this, we also have the option of defining project configurations through code. In my experience, I’ve used Groovy to specify project configurations simply by using the DSL job option which calls upon my Groovy script each time the project is run.
3. Build the project
Once the project configurations are set, we can then build the project based on specific parameters. If we choose to run the project by a schedule, we can specify the default parameters in the project configurations. This step allows us to see how the overall codebase is influenced by the new project we created.
4. Scaling up
When repeating this process with thousands of automations, the Jenkins master serves as a useful platform for helping engineers navigate through successful and broken projects systematically and quickly.