The Sprint Methodology in Agile Project Management

Abstract

Working in sprints facilitates a quick and continuous review of results allowing regular feedback and thereby keeping on the right path towards a successful project for all parties involved. This eventually also helps ensuring that the project continues being profitable as well as maintaining the interest and momentum towards stakeholders, which can be critical towards the funding of the project . This rapid iterativeness thus challenges the traditional way of conducting projects, also known as waterfall project management. Traditional project management is characterized by more robustness and formalities involving very thorough preliminary planning as well as protracted requirements definition. The long lasting requirements definition often results in outdated requirements before the project development even has begun. Agile project management (APM) was thus developed to meet the needs of a new type of project where the development phase is more crucial than the planning phase, namely within information systems and technology projects .

Sprints are part of the APM framework scrum, which is typically applied within IT projects, software development projects and projects of a more exploratory nature like data science projects. These types of projects are affected by the constant technological development, which makes sprints an appropriate way of managing them, as this methodology helps responding to high levels of change and uncertainty .

This article investigates relevant aspects of the sprint methodology within data science projects. It provides concrete and hands-on recommendations to project managers or scrum masters of data science projects who are about to plan and conduct sprints with their team.

Motivation

According to PRINCE 2, a project is "a temporary organization that is created for the purpose of delivering one or more business products according to an agreed Business Case.". To deliver such business products, a project needs to be managed with respect to many aspects, defined as "the planning, delegating, monitoring and control of all aspects of the project, and the motivation of those involved, to achieve the project objectives within the expected performance targets for time, cost, quality, scope, benefits and risks.", according to PRINCE 2 .

Projects go through a series of phases, also known as a project's life cycle. These phases can differ from project to project but are typically broken down by deliverables or milestones. In general, the phases can, according to the PMBOK® Guide, be summed down to the starting of the project, the organizing and preparing, the carrying out of project work and eventually the closing of the project . In the traditional predictive life cycle, the products and projects' deliverables are clearly defined at the beginning with little room for changes in the scope of the project without needing careful managing. The project scope, costs and time are thus defined as early in the process as possible. This requires that a project is definable within all these aspects at an early stage and is characterized by low levels of change and uncertainty. However, some projects are characterized by such an amount of uncertainty that the early defining of such aspects simply is too difficult, which makes traditional project management and predictive life cycles inadequate. This is where APM and adaptive life cycles like sprints become relevant and challenge the traditional project life cycle as the one defined by the PMBOK® Guide . New demands to projects thus require new methodologies and frameworks.

Taking offset in agile software development, APM takes emphasis in two main concepts that helps defining how project teams can adapt rapidly to these unpredictable and changing requirements. First, risk is minimized when the team focuses on short iterations with clearly defined deliverables. Secondly, the feedback loop created by these short iterations facilitates the direct communication with partners during the development process, which helps avoiding unnecessary documentation and bureaucracy .

Agile Project Management

Scrum Roles and Process

Figure 1: Agile Project Management Frameworks

Adaptive life cycles like sprints are characterized by being very rapid iterations of approximately 2-4 weeks. Each sprint is fixed in time and cost at its beginning. Here, the product backlog list, also known as project requirements, is also reviewed in order to determine, which of the requirements that can be delivered within the next iteration .

Within APM, a large number of different but useful frameworks exists. Three of the most frequently used ones, namely the scrum, lean and kanban frameworks, are illustrated in figure 1. Sprints are part of the scrum framework, which is mainly used within IT projects . The scrum consists of three major components: the roles, the process and the artifacts.

Figure 2: Scrum Activities

Starting with the roles, the scrum team is cross-functional and consists of a product owner, a scrum master and a scrum team. This cross-functional constellation works full-time on the project in order to get through the product backlog list in due time. The second major scrum component, namely the process, counts five overall activities: the kick-off, the sprint planning meeting, the sprint, the daily scrum, and the sprint review meeting, which all are illustrated in figure 2 . The third major scrum component, the artifacts, include the product backlog, the sprint backlog and the burn-down charts, which will be elaborated further in the Scrum Artifacts section.

At the beginning of a project, the product owner, the scrum master and the scrum team all meet together to kick it off. This meeting occurs only once during a project. At this kick-off meeting, the high-level backlog for the project, which is the overall project scope and requirements, as well as its major goals are defined. Subsequently, the same participants all meet again at the beginning of each sprint for a sprint planning meeting. Here, they define the product backlog list i.e. the decomposition of the high-level backlog. Furthermore, they determine the sprint goal, which is the formal outcome of the sprint. Unlike the kick-off meeting, the sprint planning meeting is held at the beginning of each sprint within a project.

After defining these two main parts, the participants consequently outline the sprint backlog. These sprint planning meetings will usually take up to a day . Having planned the sprint, its backlog and goals, the team is now ready to explore the sprint’s research topics.

Scrum Artifacts

Figure 3: Sprint burn-down chart and Release burn-down chart

As mentioned, the scrum artifacts include the product and the sprint backlog as well as the burn-down charts. The product and sprint backlogs are defined during the sprint planning meetings. The product backlog is defined with the product owner whereas the sprint backlog is defined only by the scrum team, since this type of backlog is more specific to the individual sprint .

The use of burn-down charts is intentionally applied within scrum unlike in traditional project management. Burn-down charts are simple two-dimensional charts showing the progress of a given process. The purpose of using burn-down charts is to provide relevant information about these progresses in an easy-to-comprehend manner. This is especially relevant during scrum and sprints, where time is a limited resource and needs to be micro-managed carefully . Within burn-down charts, three types are commonly used. The sprint burn-down chart documents the progress of the sprint. The release burn-down chart documents the progress of the release and the product burn-down chart documents the overall project progress . An illustration of how these three types of burn-down charts can look like can be seen in figure 3.

As the examples in figure 3 illustrate, each task is typically represented on the x-axis in terms of time with the duration on the y-axis. The sprint burn-down chart is on a very operational level, showing the day-to-day progress by depicting the total backlog hours remaining in the sprint per day. These total backlog hours are estimated based on the amount of time left in the sprint. The sprint burn-down chart would thus decrease to zero hours remaining by the end of the sprint.

The release burn-down chart works in a similar way, however, it represents the remaining time until the release will be done. Finally, the product burn-down chart is thus used to indicate the overall project progress on a higher level .

These burn-down charts can thus help the scrum team and especially the scrum master keeping an easy overview of how the project is progressing and on a more specific level also get a day-to-day overview of how each sprint is progressing.

Sprint Methodology

What characterizes sprints, besides being short iterations of maximum four weeks, is that no external interference should influence the work of the scrum team. This also means that project requirements cannot be changed during a sprint. In general, it is recommended to hold short daily stand-up meetings of 15 minutes, known as scrum meetings, during a sprint. Here, relevant matters such as what has been done since last scrum meeting and what will be done within next scrum meeting can be discussed.

The purpose of these daily meetings is to hold track of how the sprint and thereof the project is progressing. This can help the scrum master keeping track of the team and ensure as efficient a sprint as possible . The meetings are short and informal leaving no room for problem solving or thorough explanations, hence the importance of standing up during these meetings. Standing up can help the team being focused and remove any obstacles to progress .

Within data science projects, a number of five additional meetings are recommended to be held within each sprint. These meetings will be discussed further in the Application section. After each sprint is finished, it is reviewed during the sprint review meeting. Here, the sprint product is demonstrated to the product owner.

Application

Data Science Life Cycle

Depending on the project type, a number of different adaptive life cycles exist. Data science projects differ from software development and data mining projects as they are of a more experimental and exploratory nature and thus need a less rigid project life cycle. In data science projects, the team learns from data and adjusts its work to the results accordingly. The amount of planning and interdependencies between the project phases in the software development life cycle (SDLC) and data mining life cycle (CRISP-DM) are thus not suited for data science projects. The data science life cycle (DSLC) sprint is better suited to data science projects due to its more lightweight and agile nature. It consists of six phases, taking offset in the scientific method . These six phases are:

• Identify
• Question
• Research
• Results
• Insights
• Learn

Figure 4: Data Science Life Cycle (Inspired by D. Rose (2016) )

Unlike in the SDLC where each phase leads to the next, a 
team can “cycle” through 
the questioning, researching and
 results phases in each DSLC sprint. This mechanism is illustrated by arrows in figure 4. This inner cycle provides 
data science projects flexibility and
agility. In the DSLC phases, the scrum team starts by identifying
 the key roles and players in a given
 context in the identification phase. Then, in the questioning phase of the DSLC sprint, the team starts asking relevant questions about these key players in order to explore the data the best way possible. To help the team identifying all the right questions, the use of a question board is recommended. The question board allows the rest of the organization to contribute with its questions. The board should be easily accessible for all to add a post-it with a question when walking by .

During the research phase, the team’s data analysts explore the data based on the identified players and questions, also known as research topics. Here, the rest of the team will work closely with the data analysts to make sure that there is compliance with the desired outputs. When the data analysts have finished exploring data within the given research topics, a short report with the results from each research topic is produced during the results phase. These short reports can subsequently be used if a research topic and its results provide a solid basis for further research. When all research topics have been explored, the team evaluates the results and assesses if they provide any valuable and interesting insights. Based on these insights, the team will in the last phase try to create organizational knowledge, which eventually can provide actual value to the rest of the organization showing what has been learned from the research in the sprint .

Data Science Sprint Meetings

In order to run scrums efficiently, it is necessary to manage sprints carefully. A way to ensure that the sprint is on track is to hold regular meetings. For scrum teams in data science projects it is recommended to run five different types of meetings during a sprint, which is recommended to last no longer than two weeks for data science projects . These five meetings are:

• Research Planning
• Question Breakdown
• Visualization Design
• Storytelling Session
• Team Improvement

The meetings are to be held together for every sprint to make sure that the scrum team provides interesting insights at the end of the sprint. These five types of meetings have different purposes and they all together ensure that each of the six phases of a DSLC sprint is covered properly, as is illustrated in figure 5. It is essential that these five sprint meetings are all time-boxed.

A meeting is time-boxed, when the team agrees upon the duration of the meeting prior to initiating it. What will be agreed during this meeting will thus have to last for the rest of the sprint, since time-boxed meetings cannot be rescheduled or followed-up upon. This helps ensuring that a sprint is rapid and agile, focusing on exploring and experimenting rather than planning and organizing ].

Figure 5: Data Science Sprint Meetings (Inspired by D. Rose (2016)

Research Planning

Starting with the first sprint meeting, namely the Research Planning meeting, the scrum team evaluates all the questions that have been identified during the identification phase and on the question board. During this typically two-hour long meeting, the team selects the most interesting questions and the data analysts and scrum master subsequently prepares the sprint agenda. The sprint agenda is thus based on a compromise between the data analysts and scrum master to ensure minimum viable research topics exploring data but avoiding spending unnecessary time on it .

Question Breakdown

During the Question Breakdown 
meeting, the scrum team gathers to review any new questions
from the question board and try to
come up with some new ones. This
 type of meeting is also to identify potential clusters in the questions and whether any questions can be broken down into easier manageable questions. It is recommended to have at least two one-hour sessions of such meetings during a sprint. These meetings ensure a constant flow in the research topics and that the whole team is in compliance. It is also a good setting for prioritizing all questions and preparing the following sprint .

Together with the Research Planning meetings, the Question Breakdown meetings are thus ways of managing the first two phases of the DSLC sprint: the identifying and questioning phase.

Visualization Design

During the third meeting, the Visualization Design meeting, the data analysts and scrum master develop interesting visualizations together based on the results from the data analysts’ experimentations with data. The meeting should not last more than an hour and the visualizations can remain as a rough draft . These visualizations will subsequently be used in the Storytelling Session.

The Visualization Design is thus a meeting that helps managing the sprint results from the DSLC result phase and creating valuable insights during the DSLC insights phase.

Storytelling Session

Based on the visualizations done by the data analysts and scrum master, the scrum team meets at a Storytelling Session to present the story of what they have learned during the sprint. The meeting should last an hour, in which data visualizations are shown, questions from the question board are discussed and the stories behind the questions are told .

The Storytelling Session is thus a meeting during the DSLC learning phase that helps transmitting the knowledge gained through the sprint.

Team Improvement

At the end of the sprint, the scrum team meets for a two-hour improvement meeting where the team progress is discussed and evaluated. Here, potential changes can be agreed upon before the next sprint and this helps ensuring that the team is in constant progress and learning fast from its mistakes and successes .

Limitations

- Is the risk really always minimised?

- Are there other risks combined with APM?

- what are the limitations/pitfalls to running sprints?

- what benefits are there with sprints? What can it do? (summing up the benefits clearly)

Limitations: critically reflect on the tool/concept/theory and its application context. What can it do, what can it not do? Under what circumstances should it be used, and when not? How does it compare to the “status quo” of the standards – is it part of it, or does it extent them?

Refer to PMBOK

References:

Avoiding Pitfalls in Delivering in Data Science Sprints (Doug, 2016)

RISKS CHARACTERISTIC OF AGILE PROJECT MANAGEMENT METHODOLOGIES AND RESPONSES TO THEM (WALCZAK, 2013)

• Is the Wiki article credible? • Critical reflection on status quo of standards: To what degree are the core arguments of your wiki article covered by the P/P/P standards and literature? To what degree does your article extend (or maybe contradict) the status quo? • Use of reference material provided in class: Elements from the reference material were appropriately incorporated into the article. • Annotated bibliography: Does the article properly cite and acknowledge previous work (reference material provided, and appropriate other sources where necessary)? Does it briefly summarize the key references at the end of the article?

Glossary

APM - Agile Project Management

DSLC - Data Science Life Cycle

SDLC - Software Development Life Cycle

CRISP-DM - Cross Industry Standard Process for Data Mining

Annotated Bibliography

Rose D. (2016) Working in Sprints. In: Data Science. Apress, Berkeley, CA

Project Management Institute (2013) A Guide to the Project Management Body of Knowledge (PMBOK® Guide). In: Project Management Institute

H. Frank Cervone (2011) Understanding agile project management methods using Scrum.

Annotated bibliography: Provide key references (3-10), where a reader can find additional information on the subject. The article MUST make appropriate references to the and reference material provided in class – either incorporating it as a source, or critically discussing aspects that are missing from it but covered by this article. Summarize and outline the relevance of each reference to the topic (around 100 words per reference). The bibliography is not counted in the suggested 3000 word target length of the article.