Agile Development: Part I

Introduction

This article is a quick FAQ of Agile. By reading this you will understand fundamentals of Agile and different ways of implementing Agile.

What does Agile mean?
 
Dictionary meaning of Agile is quick moving. Now how does that apply to software? Agile development methodology considers software as the most important entity and accepts user requirement changes. Agile advocates that we should accept changes and deliver the same in small releases. Agile accepts change as a norm and encourages constant feedback from the end user.

Figure: - Agile

Below figure shows how Agile differs in principles from traditional methodologies.

Figure: - Change of Agile thinking

• It's not necessary to have hi-fi tools and process but a good team interaction can solve lot of problems.
• Working software is more important than documentation.
• Management should not pay attention to only customer contract rather interact with customer and analyze the requirements.
• In traditional methodologies we pledge to stick our plans but agile says "If the customer wants to change, analyze and change your plan accordingly".

Below are principles of Agile methodology:-

• Welcome change and adapt to changing requirements
• Working software is the main measure of progress.
• Customer satisfaction is the most important thing and that can be attained by rapid, continuous delivery of useful software
• Day to day meetings between business people and development team is a must.
• Business and developers must work together. Face to face to communication is the most important thing.
• Deliver and update software regularly. In Agile we do not deliver software in one go, but rather we deliver frequently and deliver the important features first.
• Build projects around teams of motivated and trustful people.
• Design and execution should be kept simple.
• Strive for technical excellence in design and execution.
• Allow team to organize themselves.

Can you explain Agile modeling?
 
Agile modeling is an approach to the modeling aspects of software development. It's a practice for modeling and documentation for software systems. In one line

It's a collection of best practices for software modelling in light-weight manner.

In abstraction we can say it augments other software processes. For instance let's say your company is using UML and then Agile applies approach practices on UML.For example "Keep things simple" is Agile approach. So it means that we do not need to use all diagrams in our project, use only which are needed. If we summarize then in one word we can say Agile modeling says "Do only what's needed and nothing more than that".

Figure: - Agile Modeling

What are core and supplementary principles in Agile modeling?
 
Agile modeling defines set of practices which can show us the way towards becoming successful Agile modelers. These practices are divided in to two sections one the "Core Principles" and other "Supplementary Principles". Below figure 'Agile Model Principles' shows the same in a pictorial format.

Figure: - Agile Model Principles

Let's understand one by one what those principles mean.

Core Principles

• Simplicity: - Do not make complex model keep it simple. When you can explain your team with a pen and paper do not complex it by using modeling tool like rational rose. Do not add complexity to show off something. If the developer understands only flow chart then explain him with a flow chart, if he understand pseudo-code then use pseudo-code and so on. So look at your team what they understand and prepare document in a similar fashion.
• Welcome Change: - Requirements grow with time. Users can change the requirements as the project moves ahead. In traditional development cycle you will always hear the word "Freeze the requirement", this has changed with Agile coming in. In Agile we welcome change and the same is reflected in the project.
• Incrementally change: - Nothing can be right at the first place itself. You can categorize your development with "The most required", "Needed features" and "Luxury features". In the first phase try to deliver the "The most required" and then incrementally deliver the other features.
• Model exists with a purpose: - Model should exist for a purpose and not for the sake of just existing. We should know who our target audience for whom the model is made. For instance if you are making a technical documents it's for the developers, a power point presentation it's for the top management and so on. If the model does not have target audience then it should not exist probably. In short "just deliver enough and not more".
• It should be light: - Any document or artifact you create should be also updated over a period of time. So if you make 10 documents then you should note that as the source code changes you also need to update those documents. So make it light as possible. For instance if your technical document is made of all diagrams existing in UML, it becomes important to update all diagrams over a period of time, which is again a pain. So keep it light weight make a simple technical document and update the same when you have logical ends in the project, rather than updating it periodically.
• Keep multiple models: - Project issues vary from project to project and the same project behavior can vary from organization to organization. So do not think one model can solve all issues keep yourself flexible and think about multiple models. Depending on situation apply the model. For instance if you are using UML for technical documentation then every diagram in UML can reflect the same aspects in different way. For instance a class diagram shows the static view of project while a flow chart a dynamic view. So keep yourself flexible by using different diagrams and see which best fits your project or the scenario.
• Software is the most important thing:- The main goal of a software project is to produce high quality software which can be utilized by your end customer in a effective manner. Many projects end up with bulky documents and management artifacts. Documentation is for the software and not software for the documentation. So any document or activity which does not add value to the project should be questioned and validated.
• Get Rapid and regular feedbacks: - Software is finally made for the user. So try to get feedback on regular basis from the end user. Do not work in isolation involve the end user. Work closely with the end customer, get feedback, analyze requirements and try to meet there need.

Supplementary principles

• Content is important than presentation: - The look and feel is not important rather the content or the message to be delivered by the content is important. For instance you can represent project architecture using complex UML diagrams, simple flow chart or by using simple text. It will look fancy that you can draw complex UML diagrams but if the end developer does not understand UML then it ends no where. A simple textual explanation could have met the requirement for communicating your architecture to the end developer / programmer.
• Honest and open communication: - Take suggestion, be honest and keep your mind open to new model. Be frank with the top management if your project is behind schedule. An open and free environment in project keeps resources motivated and the project healthy.

What is the main principle behind Agile documentation?
 
The main deliverable in Agile is a working software and not documentation. Documentation is a support to get the working software. In traditional delivery cycle lot of documentation where generated in design and requirement phase. But we are sure many of documentation where either created just for the sake of it or it was just created. Below are the some of the key points to make documentation Agile:-

• Before creating any document ask a question do we need it and if we who is the stake holder. Document should exist only if needed and not for the sake of existence.
• The most important thing is we need to create documentation to provide enough data and no more than that. It should be simple and should communicate to stakeholders what it needs to communicate. For instance below figure 'Agile Documentation' shows two views for a simple class diagram. In the first view we have shown all the properties for "Customer" and the "Address" class. Now have a look at the second view where we have only shown the broader level view of the classes and relationships between them. The second view is enough and not more. If the developer wants to get in to details we can do that during development.

Figure: - Agile documentation

• Document only for the current and not for future. In short whatever documentation we require now we should produce and not something we need in the future. Documentation changes its form as it travels through every cycle. For instance in the requirement phase it's the requirement document, in design it's the technical documentation and so on. So only think which document you want to create now and not something in the future.

What are the different methodologies to implement Agile?
 
Agile is a thinking approach to software development which promises to remove the issues we had with traditional waterfall methodology. In order to implement Agile practically in projects we have various methodologies. Below figure 'Agile Methodologies' shows the same in more detailed manner.

Figure: - Agile Methodologies

Note: - We will cover each methodogly in detail in the coming sections.

What is XP?
 
Extreme Programming (also termed as XP) is an agile software development methodology. XP focuses on coding of the software. XP has four core values and fourteen principles.

XP has four core values:-

• Communication: - Team should communicate on a regular basis, share information, discuss solutions and so on. Teams who communicate very often are able to solve problems more efficiently. For instance any kind of issues which are resolved in a cryptic fashion send an email to the whole team. This ensures that knowledge is shared with every one and in your absence some other developer can solve the problem.
• Simplicity: - Keep things simple. Either it's from a process angle, technical angle or from a documentation point of view. An over complicated process or a technical architecture is only calling for problems.
• Feedback: - Regular feedback from end user helps us to keep the project on track. So regular feedbacks should be enabled from end user and testing team.
• Courage: - To bring change or to try something new, needs courage. When you try to bring change in an organization you are faced with huge resistance. Especially when your company is following traditional methodologies applying XP will always be resisted.

From the above four core values 14 principles are derived. Values give a broader level view while the 14 principles go deep in to how to implement XP.

• Rapid feedbacks: - Developers should receive rapid feedbacks from the end user. This avoids confusion in the last minute of delivery. In water fall model feedbacks are received in late intervals. This is minimized in XP. 
• Keep it Simple: - Encourage simplicity in project design and process. For instance rather than using complex tools probably simple handwritten flowcharts on board can solve the problem.
• Give incremental changes: - Whenever you update patches and updates, release it in small pieces. If you are updating numerous patches in one go and if there is a defect, it will be difficult to track the same.
• Embrace Change: - Do not be rigid with the customer saying that we have already signed the requirement so we can not change the architecture. End customer or users are finally human beings so they can change as the project moves ahead....Accept it if it's logical.
• Light Weight: - Keep documentation and process as simple as possible. Do not overdose the developer with unnecessary documentation. Developer's main work is coding and ensuring that the code is defect free, so he should be more concentrating on the code rather than documentation. 
• Deliver Quality: - Any code you deliver should be defect free. Be committed to your work and deliver defect free code.
• Start small and grow big: - Many times the end customer wants to start with a big bang theory. He can start with a big team, wants all the functionalities at the first roll out and so on. Start with small team and the "must have" features to be delivered. As we add features and the work load increases gradually increase your team strength.
• Play to win: - Take all steps which are needed to make a project success. Any type of deadline and commitment try to meet the same with true spirit.
• Encourage honest communication: - Promote honest communication. If communication happens face to face then there is less leakage of requirement. Encourage end user to sit with developers and give feedbacks; this makes your project stronger.
• Conduct testing honestly: - Test plans should not be created for the sake of creation. Test plan should prove actually that you are on track record.
• Adapt according to situation: - No two projects are same, no two organization are same and behavior of people from person to person. So it's very essential that our approach also adapts according to situations.
• Metric honesty: - Do not gather metrics for the sake of gathering or showing off to external people how many metrics your project derives. Pick metrics which makes sense to your project and helps you measure your project health.
• Accept responsibility: - Do not impose or assign people on task which they do not like. Rather question the resource once which tasks he likes and assign accordingly. This will increase productivity to a huge level and maintains your project enthusiasm high.
• Work with people's instincts: - Normally in a project team there are highly motivated people, moderately motivated and people with less motivation. So give power to your motivated team members and encourage them.

What are User Stories in XP and how different are they from requirement?
 
Use story is nothing but end users requirement. What differentiates a user story from a requirement is that they are short and sweet. In one sentence they are just enough and nothing more than that. User story ideally should be written on index cards. Below figure 'User Story Index Card' shows the card. Its 3 x 5 inches (8 x 13 cm) card. This will keep your stories as small as possible. Requirement document go in pages. As we are keeping the stories short its simple to read and understand. Traditional requirement documents are verbose and they tend to loose the main requirement of the project.

Note: - When I was working in a multinational company I remember first 50 pages of the requirement document having things like history, backtracking, author of the document etc. I was completely drained till I started reading the core requirement.

Every story has a title, short description and estimation. We will come to the estimation part later.

Note: - Theoretically it's good to have cards, but in real scenario you will not. We have seen in actual scenario project manager keeping stories in document and every story not more than 15 lines.

Figure: - User Story Index Card

Who writes User stories?
 
It's written and owned by the end customer and no one else.

When do we say a story is valid?
 
Story is valid if it can be estimated.

When are test plans written in XP?
 
Test plans are written before writing the code.

Can you explain the XP development life cycle?
 
XP development cycle consists of two phases one is 'Release Planning' and the other is 'Iteration Planning'. In release planning we decide what should be delivered and in which priority. In iteration planning we break the requirements in to tasks and plan how to deliver those activities decided in release planning. Below figure 'Actual Essence' shows what actually these two phases deliver.

Figure: - Actual Essence

If you are still having the old SDLC in mind below figure 'Mapping to Traditional Cycle' shows how the two phases map to SDLC.

Figure: - Mapping to Traditional Cycle

So let's explore both these phases in a more detailed manner. Both phases "Release Planning" and "Iteration Planning" have three common phases "Exploration", "Commitment" and "Steering". 

Figure: - XP Planning Cycle

Release Planning

Release planning happens at the start of each project. In this phase project is broken in to small releases. Every release is broken down in to collection of user stories. Now let's try to understand the three phases in release planning.

Exploration: - In this phase requirement gathering is done by using user story concept (Please read the previous question on user story to understand the concept of user story). In this phase we understand the requirement to get higher level understanding. Please note only higher level. User story card is size normally 3 X 5 inch, so you can not really go detail in that size of card. We think it's absolutely fine rather than writing huge documents it sounds sense to have to the point requirement paragraphs. So here is a step by step explanation of how the exploration phase moves :-

• So the first step is user writes the story on the user card. 
• Once the story is written the developer analyzes it and determines can we estimate the user story?. If the developer can not estimate then it's again sent back to user to revise and elaborate the user story.
• Once the user story is clear and can be estimated, ideal day or story (read about story point, ideal day and estimation in the coming questions) are calculated. 
• Now its time to say the user, ok we can not deliver everything at one go, so can you please prioritize. So in this phase the end user gives ranking to the user stories (In the next section we will deal with how a user story is ranked). 
• Once the user is done with story prioritization, its time to calculate velocity determination (In the coming section we have one complete question on velocity determination).
• Agile is all about accepting end customer changes. In this phase we give a chance to the end user to decide if they want to change anything. If they want to change we again request the user to update the story.
• If everything is ok we go ahead for iteration planning.

Below figure "Release planning" shows the above discussed steps in a pictorial format.

Figure: - Release Planning

Iteration Planning

Iteration planning is all about going deep in to every user story and breaking the same in to tasks. This phase can also be termed as detailing of every user story. Iteration planning is all about translating the user story in to task. Below are the steps in details for iteration planning:-

• User stories which need to be delivered in this iteration are broken down in to manageable tasks. 
• Every task is then estimated. The result of the estimation is either ideal hours or task points (we will discuss about task point and ideal hours in the coming section). 
• After the tasks are estimated we need to assign the task to developers. Each programmer picks a task and own responsibility to complete the task.
• Once he owns the responsibility he should estimate the same and commit to complete the same.
• In XP on any development task two developers should work. In this phase the developer makes partner of his choice for developing this task.
• In this phase we do designing of the task. We should not make lengthy and comprehensive design plans; rather it should be small and concentrated on the task. In traditional SDLC we have full devoted phase for designing and the output is a lengthy and complicated design document. One of the important characteristic of a software project is that as we come near execution we are clearer. So it's best to prepare design just before execution.
• Now that you and your partner are familiar with the design plan its time to write a test plan. This is one of the huge differences as compared to original traditional SDLC. We first right the test plan and then start execution. Writing test plans before coding gives you a clear view of what is expected from the code.
• Once the test plan is completed its time to execute the code.
• In this phase we run the test plan and see that if all test plan pass.
• Nothing is perfect it has to be made perfect. Once you are done with coding, review the code to see if there is any scope of refactoring (Refactoring is explained in more depth in the coming sections).
• We the run the functional test to ensure everything is up to the mark.

Figure: - Iteration Planning

One of the important points to realize is project is broken down in to set of releases a which is further analyzed using short user stories à user stories are further broken in to task ,which is estimated and executed by the developer. Once one release is done the next release is taken for delivery. For instance the below project shown in figure 'Release, Stories and Task' has two releases one and two.

Figure: - Release, Stories and Tasks

Can you explain how planning game works in Extreme Programming?
 
The above question answers the question.

How do we estimate in Agile?
 
If you read the Agile cycle carefully (explained in the previous section) you will see Agile estimation happens at two places.

• User Story Level Estimation: - In this level a User story is estimated using Iteration Team velocity and the output is Ideal Man days or Story points. 
• Task Level Estimation: - This is a second level of estimation. This estimation is at the developer level according to the task assigned. This estimation ensures that the User story estimation is verified.

Estimation happens at two levels one when we take the requirement and one when we are very near to execution that's at the task level. This looks very much logical because as we are very near to complete task estimation is more and more clear. So task level estimation just comes as a cross verification for user story level estimation.

Figure: - Agile Estimation

User Story Level Estimation

Estimation unit at user story in Agile is either "ideal days" or "Story points".

Ideal days are nothing but the actual time the developer spent or will spend on only coding. For instance attending phone calls, meetings, eating lunch and breakfast etc are not included in the ideal days. In old estimation technology we estimate eight hours as the complete time a developer will do coding. But actually a developer does not code continuously for eight hours, so the estimates can be very much wrong if we consider the full eight day hours.

Estimation units can also be represented in story points. Story Points are abstract units given to represent the size of the story. In normal scenario one story point equals to one ideal day. Story point is a relative measure. If one story is one story point and the other is two story points that means the second story will take twice the effort as compared to the first story.

Velocity determination defines how many user stories can be completed in one iteration. So first the user decides the length of the iteration. Length of iteration is decided depending on the release dates. Velocity is normally determined from history. So what ever was the last team history velocity same will be used in the further estimation. But if there is no history then the below formulae will be used:-

Figure: - Velocity Determination

There are two formulas in the above figure the first formula is used when we do not have history about the project and the second formulae is when we have a history of the iteration. Below are the details of all the parameters in the formulae:-

• Number of developers: - Total Number of developers in the iteration.
• Load factor: - This means how much productive time a developer will spend on the project. For instance if the load factor is 2 then developers are only 50% productive.
• How long is the iteration in business days: - One iteration is of how many man days.

Below figure 'Iteration and Release calculation' shows a simple sample with a team size of 5, load factor of 2, one iteration takes 11 business days and there two releases in the project.

Figure: - Iteration and Release calculation

Task Level Estimation

As the Agile cycle moves ahead user story is broken down in to task and assigned to each developer. Level of effort at the task level is a form of Task points or Ideal hours. Ideally one task point represents one ideal hour. Ideal hour is the time when developer spends only on coding and nothing else.

Individual Velocity determination defines how many how many ideal hours a developer has within one iteration. Below figure 'Individual Velocity Calculation' shows in detail how to get the number of ideal hours in iteration for a developer. Below is a sample calculation which shows with 8 hours a day , iteration of 11 days and load factor of 2 ( i.e. developer code for only 50% time i.e. 4 hours) , we get 44 ideal hours for developer in that iteration.

Figure: - Individual Velocity Calculation

On What basis can stories be prioritized?
 
User story should normally be prioritized from the business importance point of view. In real scenarios this is not the only criteria. Below are some of the factors to be accounted when prioritizing user stories:- 

• Prioritize by business value: - Business user assigns a value according to the business needs. There three level of ratings for business value:-
  • Most important features: - With out these features the software has no meaning.
  • Important features: - It's important to have features. But if these features do not exist there are alternatives by which user can manage.
  • Nice to have features: - These features are not essential features but rather it's over the top cream for the end user.
• Prioritize by risk: - This factor helps us prioritize by risk from the development angle. Risk index is assigned from 0 to 2 and are classified in three main categories :-
  • Completeness
  • Volatility
  • Complexity

Below figure "Risk Index" shows the values and the classification accordingly.

Figure: - Risk Index

Can you point out simple differences between Agile and traditional SDLC?
 
Below figure "Agile and Traditional SDLC" points out some of the main differences. If you have worked practically on both these you can point out more differences.

• Lengthy requirement documents are now simple and short user stories.
• Estimation unit man days and man hours are now ideal days and ideal hours respectively.
• In traditional approach we freeze the requirement and complete the full design and then start coding. But in Agile we do designing task wise. So just before the developer starts a task he does design.
• In traditional SDLC we used always hear this voice 'After signoff nothing can be changed', in Agile we work for the customer, so we do accept changes.
• Unit test plans are written after coding or during coding in traditional SDLC. In Agile we write unit test plans before writing the code.

Figure: - Agile and Traditional SDLC

Continue to next part >>