Continous Integration Tools Comparison

This article only covers the comparison of some of the renowned tools used for Continuous Integration.

What Continuous Integration is

Continuous Integration (CI) implements a continuous processes of applying quality control; small pieces of effort, applied frequently. The purpose of Continuous Integration is to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.

Continuous Integration

The practice of frequently integrating one's new or changed code with the existing code repository should occur frequently enough that no intervening window remains between commit and build, and such that no errors can arise without developers noticing them and correcting them immediately. The normal practice is to trigger these builds by every commit to a repository, rather than a periodically scheduled build. The practicalities of doing this in a multi-developer environment of rapid commits are such that it's usual to trigger a short time after each commit, then to start a build when either of the times expire, or after a rather longer interval since the last build.

It helps in running all unit and acceptance tests, publishes build results, notifies developers if a build breaks, labels successful builds in the source repository.

Some of the practices that should be followed sincerely once the project is in the Development phase are:

• Maintain a code repository
• Automate the build
• Make the build self-testing
• Every commit should be buildable
• Keep the build fast
• Test in a clone of the production environment
• Make it easy to get the latest deliverable
• Everyone can see the results of the latest build
• Automate deployment

Some of the advantages:

• When unit tests fail or a bug emerges, developers might revert the codebase back to a bug-free state, without wasting time debugging
• Developers detect and fix integration problems continuously, avoiding last-minute chaos at release dates, (when everyone tries to check in their slightly incompatible versions).
• Early warning of broken/incompatible code
• Early warning of conflicting changes
• Immediate unit testing of all changes
• Constant availability of a "current" build for testing, demo, or release purposes
• Immediate feedback to developers on the quality, functionality, or system-wide impact of code they are writing

There are many tools already present in the market for Continuous Integration, one can choose the tool according to the requirements, and here is the comparison for some of commonly used tools:

• Cruise Control is a free and open source build scheduler; it works with any build tool that produces parse-able output. Some of the features of Cruise Control are:
   
  • It is cross-platform and works with many of the VCSs
  • It is configured in a XML file that maps the build cycles to certain tasks
  • The results are published on the system tray or in the form of emails
     
• Hudson Build is a Continuous Integration Server that integrates build, unit tests, code coverage, and analysis and provides instant knowledge of the status of builds. Some of the features of Hudson Build are:
   
  • Easy Installation
  • Hudson can be configured entirely from its friendly web GUI with extensive on-the-fly error checks and inline help. There's no need to tweak XML manually (optional) anymore
  • Hudson can generate a list of changes made into the build from CVS/Subversion. This is also done in a fairly efficient fashion, to reduce the load on the repository
  • Monitor build results by RSS or e-mail to get real-time notifications on failures and many more...
     
• TFS is a tool that integrates with Visual Studio in the dockable panel "Team Explorer". In Team Explorer, it lists all opened projects and provides various options in context menu on right-clicking the project. It builds on every check-in and provides instant knowledge of the status of builds. Some of the features of TFS are:
   
  • Team Foundation Server 2010 supports teams doing parallel development and includes many powerful features like atomic check-ins, branching and merging, shelving, labelling, concurrent check-outs, check-in policies, and the association of check-ins with work items
  • As mentioned, it can help in associating the check-ins with work items. Work items can consist of requirements, tasks, bugs, issues, and test cases
  • It provides a robust and fully featured build automation server. You can customize Team Foundation Build and configure triggers for manual build, Continuous Integration, rolling builds, or scheduled builds
  • Provides features to write scripts to run test cases on each build. There are check-in policies, with those in place you can forbid checking-in something without all unit tests passing
  • Powerful reports and dashboards in TFS helps you to analyse and track progress
  • provides built-in support for Agile and Waterfall methodologies
     
• TeamCity is a tool that automates code analysing, compiling, and testing processes, providing instant feedback on build progress, problems and test failures, all in a simple and intuitive web-interface. Some of the features of TeamCity are:
  
  
  • Run multiple builds and tests under various configurations and platforms simultaneously
  • Perform Pre-tested commits, helping the rest of a team sustain an uninterrupted workflow
  • Have build history insight with customizable statistics on build duration, success rate, code quality and custom metrics

Comparison:

Name	Platform	License	Windows Builder	Java Builder	Source Code management System	Notification	IDE Integration
Cruise Control	Cross-Platform	Free	MSBuild, NAnt, Visual Studio	Phing, Apache Ant, Maven	AccuRev SCM, Alienbrain, BitKeeper, CVS, File system, FTP, Git, Mercurial, MKS Integrity, Perforce, Plastic SCM, PVCS, Rational ClearCase, Robocopy, Surround SCM, Sourceanywhere, SourceGear Vault, StarTeam, Subversion, Telelogic Synergy, Visual Source Safe, TFS	E-mail, CCTray, RSS	Eclipse
Hudson	Servlet Container	MIT	MSBuild, NAnt	Ant, Maven 2, Kundo	AccuRev SCM, Bazaar, BitKeeper, Clearcase, CMVC, CVS, Git, Mercurial, Perforce, PVCS, SourceSafe, StarTeam, Subversion, TFS	Android, E-mail, Google Calendar, IRC, XMPP, RSS, Twitter	Eclipse, IntelliJ IDEA, NetBeans
TeamCity	Windows, VSTM	Proprietary	MSBuild, NAnt, Visual Studio	Ant, Maven 2/3	Subversion, CVS, Git, Mercurial, AccuRev SCM, Clearcase, Perforce, SourceSafe, StarTeam, TFS, Vault	E-mail, XMPP, RSS, IDE, SysTray	Eclipse, Visual Studio, IntelliJ IDEA, RubyMine, PyCharm, PhpStorm, WebStorm
TFS	Servlet Container	Proprietary	MSBuild	Unknown	Team Foundation Server	E-Mail, SOAP	Visual Studio

PS: Hope you will see how to implement all of these in my next article.