Background
A few weeks back, while sitting with my team, I learned that they run a Jenkins job which has to execute the automation scripts using the robot framework, daily around 1:00 a.m. The primary purpose of the job is to place all the necessary files and folders over the node machine of Jenkins (after copying them from the machine pointed as master on Jenkins), execute those scripts accordingly, generate the reports, and finally copy the output files back to the master machine to analyze the logs for the overall reporting and analysis purposes. This whole efficient job starts with a slack manual step which was enough to tag this activity 'not fully automated' because they had to place the updated work regularly on the specified path over the master machine before leaving the office (this includes taking the updated code, files, and automation scripts, building the project to take the DLLs, etc.). This may sound tedious but more than that, there is no sense to do it manually.
My intentions for the team
I thought to automate this thing as well and make a Jenkins job which will perform all the steps I defined a moment ago. So, the entire article is about automating the building process of a project after taking the updates from any version control system.
Solving the problem
Before leading and implementing this concept directly into the production, I thought to try it personally on my machine where I don’t have two machines like master and node machine but the concept and the overall process can be applied in the same way. Here, for this purpose, I have used two different directories. One for the code check-in purpose and the other one to get the updates, build the project, and copy the files.
Why Jenkins?
Definitely, there are many other continuous integration tools available for making automation a bit more automated and fast enough, like Circle CI, Eclipse Hudson, JetBrains TeamCity, etc. but Jenkins is the widely used CI tool because of its handy user experience and extensive availability of plugins. You can read more about Jenkins from my previous post.
Creating a build definition
Now onwards, I will be quickly describing each of the steps involved in this particular Jenkins job but to get the fundamentals of how Jenkins works and what are the must-know options and fields in Jenkins, you can refer to the article I have mentioned above.
Firstly, you need to create a fresh Jenkins Job, give it a relevant name, and write the description of the job stating what you want to do inside this job, just for future reference for the people who will be modifying the job later on.
Figure 1: Creating New Job and Naming It
Now, here is the description of the job and the steps it tends to perform.
Figure 2: Giving the job description
Secondly, you will download the MSBuild plugins from Jenkins Manager. Select Jenkins from the left menu and then Manage Plugin tab, we need this plugin to build our C# project because in our case, after getting the updates from SVN, we will be using DLL files to run out automation scripts (of course with many other files which we will be known in later steps).
Figure 3: Downloading the MSBuild Project to run MSBuild Commands
MSBuild is one of the build platforms which enables all the sub-activities needed to build your project inside Visual Studio. You must use it when you are building your MS project using the command line. You never know that MSBuild makes possible many of the core steps since you hit F5 to generate the DLLs in the debug or the release folder. It will take care of all this. But you need to know the path of its exe and other stuff mostly when you are automating the building process.
Thirdly, you will have to write the batch commands in order to take the updates from the SVN, then build the project, copy the respective needed files and then finally copy the files you required in the respective folders. I will be explaining each command separately, rather it will be more helpful if you know the basics of writing batch command.
Figure 4 Commands to take SVN updates to the entire folder to get the fresh code and content
- echo Project build started
- E:
- cd E:\FOO\
- C:\"Program Files"\TortoiseSVN\bin\svn.exe upgrade
- C:\"Program Files"\TortoiseSVN\bin\svn.exe update
- echo Updated the SVN
- echo
We use echo just to print out something on to the console; it can be for logging purposes or for ease in the error handling, the next command is to change the directory because, inside E, FOO is the folder where I have checked in all the stuff from SVN once and I want to take an update in here before building the projects with the latest commits and changes if anyone has made some.
Before executing the update command, you need to run the upgrade command where your svn.exe resides. In my case the svn.exe was present inside the Program Files – TortoiseSVN – Bin folder and most probably it will be in the same location if you have followed the simple wizard of installing the SVN.
Note
I have enclosed the
Program Files folder inside the quotes because this folder name in the whole path has the space that’s why it needed to be treated like this. You can try removing the quotes and writing the path simply then you will know the exact use case for it.
Just after taking the updates, we will be building our C# project using the MSBuild which we have just configured above; moreover, you can see that I have selected the
Default MSBuild option from the MSBuild version drop-down menu, this is the name given by me which refers to the path of MSBuild inside my machine. This name could be anything -- you can name it with any specific .NET version etc. To set this you will go to Manage Jenkins from the left sidebar and then select the Global Tool Configuration. After scrolling a bit, you will see the button
MSBuild Installation, this is where you can specify the path I’ve just talked about.
Figure 5: Selecting our custom MS to build Environment Variable
Figure 6: Giving path to MSBuild of your machine
That’s too much to be good so far, now we have to write some ugly but useful commands to copy the files from different folders and place them where we want them.
- echo Moving the content to another folder
- rmdir E:FOO\Copied /s
- robocopy E:\FOO\JobManagerScripts E:\Copied\JobManagerScripts /CopyAll /E
- robocopy E:\FOO\PythonScripts E:\Copied\PythonScripts /CopyAll /E
- robocopy "E:\FOO\visual studio\FoodiesInn\FoodiesInn-Responsive\bin\Release" E:\Copied\Binaries\bin\Release *.dll /CopyAll /E
- robocopy "E:\FOO\visual studio\FoodiesInn\FoodiesInn-Responsive\bin\Debug" E:\Copied\Binaries\bin\Debug *.dll /CopyAll /E
- echo Done with the build.
- exit 0
Figure 7: Commands to copy and paste the required files and folder
Again, after some log messages and deleting the pre-present files from the destination location, we will copy the needed files from the source location. You might wonder why I have used robocopy instead of simple copy command then here I will tell you then using robocopy can be good up to some extent since it gives better logs, multiple recursive attempts if first, it fails due to some errors to copy the content, also copy command faces an error when filename exceeds a limit – 254 or 255 if I remember correctly. So, in short, robocopy is far more suitable to use for copying files.
Additionally, you can see there are multiple commands with the same sort of format; this is because I need to copy three different types of files from three different folders which include python files, DLLs (which we have gotten after building the project in one of the previous steps) and our automation script files along with the resource files they needed. You can alter these commands according to your files, folder locations, and other things as well.
Here, after this step, we have completed our target. Now we see how to build the job manually, debug the error if we have any, and then we will configure them to run sometime later automatically by defining the time.
Running build manually
Once you have completed all the steps you can proofread the commands and steps properly before building the job, then build your job by clicking Build Now option from the left menu, meanwhile you can go to the console output by clicking on the running job instance and where you can read about every step’s details over there, know the insights what is happening and exactly how.
Figure 8: Running the build manually
Figure 9: Looking at the console output
Setting timer
Besides, the manual build option is just for testing purposes, to check either if your job doing that you wanted and when it comes to the actual use case of it we automate the time of execution of the job by setting up the timer. To set up the timer so that your job will be running automatically at the defined time you need set the time in the Build Trigger option, you have to do it periodically hence you make these asterisks ahead to hour 1, which is the specified syntax to tell the build time in Jenkins.
Note
Suppose you have to build your job twice a day, then your time defining syntax somehow will be like this;
H 3, 19 * * *
This will run your Jenkins job once at 3 AM and then 7 PM according to the defined periodic timing.
Figure 10: Setting up the timer
Generalizing support for version controls
For the sake of achieving my target I’ve used SVN as a version control throughout the article, you can use any of the version control systems according to your requirements since there are no boundaries Jenkins demands to work with for the version control system; furthermore, there is one thing to keep in mind, which is that you need to install the respective plugin of that version control, or need to take care of the command syntax accordingly. For instance, if you will be using GIT instead of SVN then from the Manager Plugin tab you will download the GIT plugin and write the command to take updates from your GIT repository folders.
What's next?
Thus, up to this point we have familiarity with the basic structure, tabs, scriptwriting, and other relevant fields of Jenkins, we have seen from where we can build the job manually, how we can see the console output to know either if our job is passed or if it fails then what causes its failure, etc. Apart from this, we have seen how to trigger the job timer periodically according to the job demand, server timings on which our job is supposed to run, etc.
You can do more stuff and further tasks other than this job or try automating other things which you do manually. Because Jenkins is an easy to use continuous integration tool and is not for just professionals at the personal level but the organizations at a larger scale throughout the world are using Jenkins as one of the core automation tools.