Learning plans for high uncertainty projects

By Kavikrishnan Balakrishnan (s164338)

The learning plan offers a systematic and iterative learning approach that deals with the high uncertainties to reduce maturation time or the time needed to reach a decision to kill the project by as much as 50%. Using the learning plan allows a team to manage the ongoing evaluation and redirection in innovation projects, where specific objectives most likely are unclear or where the final goal is clear, but the pathway is uncertain. This article describes what the learning plan is, how to implement it and highlights the benefits and limitation of the tool.

Introduction

Re-edited version of the Learning Plan template developed by MIT Sloan School of Management.

When the uncertainties are high, project teams often need to proceed based on assumptions rather than known facts. For instance, this is the case when dealing with innovation projects where the final market is unclear, which products will gain market acceptance most quickly and fully are unknown. The significance and number of uncertainties make it difficult to define milestones and the pathways to achieving them. In such projects, it is more reasonable and useful to use the approach of the learning plan. The learning plan allows a project team to evaluate and redirect the project in a proactive way when objectives may be unclear or when the goal is clear, but the path is uncertain. There is no standardized version of the learning plan, however different versions typically consist of the following basic steps:

• Identify and categorize uncertainties into market, technical, organizational and resource uncertainties
• Identify the critical uncertainties to be resolved first
• Develop assumption for each uncertainty
• Define approaches to test the assumptions and define success criteria to approve/disapprove the
• Conduct the tests and evaluate the learning
• Iterate

The template in figure 1 shows an approach that emerged from a seven year study by MIT Sloan School of Management, where twelve innovation projects in ten large technology firms from the USA were tracked.

Figure 2 illustrates how the learning plan compares to alternative planning approaches in terms of dealing with uncertainty.

*Market, technical, organizational and resource uncertainties. Re-edited version of the one developed by MIT Sloan School of Management,

Measuring Progress by 'Learning Per Dollar Spent'

The progress in projects is measured by ‘Learning per dollar spent’, which gives management a metric to continually assess the value of cumulative learning compared to the costs incurred. Learning in this context is the extent to which an unknown has become a known. This is an effective way of measuring progress in high uncertainty projects because it acknowledges that the direction or goal of high uncertainty projects tend to change much more frequent compared to projects of low uncertainty. This gives managers a foundation to decide on an ongoing basis whether the cumulative learning is of sufficient value to warrant continuing the project.

The Four Categories of Uncertainty Used in the Learning Plan

The learning plan categorizes uncertainties into four groups to be systemically examined as shown in figure 1: technical, market, organizational and resource uncertainties.

Technical uncertainties comprise those related to the completeness of scientific knowledge regarding the problem, technical specifications, which technologies to apply to to solve technical challenges, maintability and so forth.

Market uncertainties are related to the extend to which customers' needs, customer segments and competitors' products have been understood. Uncertainties in relation to revenue models andsales and distribution channels are also a part of market uncertaines. The difficulty in understanding the customers’ needs and translating them into functional and symbolic characteristics of the product generates market uncertainty.

Organizational uncertainties are associated with the dynamics of the organization. The time horizon for innovation projects are typically 10 years or more, which creates organizational uncertainties that should be taken into consideration. These include, organizational resistance, lack of persistence, inconsistencies in expections and metrics, changes in strategies, and changes in internal and external partners. Organizational uncertainties also comes into play when an innovation project transitions to an operating entity.

Resource uncertainties include financial resources but also competencies. Project teams often lack one or more competencies critical to the success of pursuing opportunities. This may lead to project teams spending large amounts of time on acquiring the competencies needed.

Implementation of the Learning Plan

The learning plan template in figure 1 consists of three phases each containing a number of steps, which the project team must go through. The three phases will be explained in the following in a practical way that allows for implementation: conducting a learning loop, evaluating the learning, and proceeding with the next learning loop.

1. Conduct Learning Loop

The idea of the learning loop is to develop an assumption for an unknown, test it, and raise the knowledge level on the uncertainty so it becomes a known. If the success criteria are not met, the process iterates.

The first phase starts by defining what is known and unknown within the four categories of uncertainty. Next, the unknowns are assessed based on their criticality. The most critical unknowns are selected for the learning loop. Testing one set of critical assumptions is defined as a learning loop. The purpose of selecting unknowns it to increase the knowledge level on this specific set to an extend where they become facts instead of unknowns. This is done by developing assumptions based on the team’s current knowledge for each of the selected assumptions. The team then develops different approaches to test them to prove or disprove them. The idea of moving from unknown to fact is illustrated in figure 3.

The selected test approaches should be the most efficient in terms of learning per dollar spent. Success criteria for proving or disproving the assumptions are made, and tasks are defined along with a timetable for the learning loop. Testing one set of critical assumptions is defined as a learning loop. Finally, the team conducts the tests.

2. Evaluate Learning

The second phase focuses on evaluating what has been learned. When a learning loop has been completed, an evaluation occurs involving the project team and an oversight board. The purpose of the oversight board is to act as a quality control and help assess the learning produced by learning loops. It typically consists of managers experienced in dealing with high uncertainty projects. The learning from a learning loop can be discovering unexpected insights coming from testing an assumption, which sets the project back. In this case, the project can either be redirected based on the new information or closed if the issue is a showstopper. The learning of the learning loop is reviewed, new identified uncertainties are prioritized, a new set of critical assumptions are clarified, and next tests are identified. It is important that the team and the oversight board agree on the assumptions to be tested and how they should be tested in the next learning loop.

3. Proceed With Next Learning Loop

In the third phase, the project team proceeds with the next learning loop.

Benefits and Limitations

The following section lists key benefits and limitations of the learning plan as a planning approach.

• Reduced maturation time or time needed to gain sufficient information to kill the project. The systematic way of reducing uncertainty using the learning plan has proven to reduce either the maturation time or the time needed to reach a decision to kill the project by as much as 50%.

• Limited to facilitating a learning process. The learning plan facilitates a step-by-step learning process, which makes it easy to implement. However, the tool is limited to facilitating its three phases as it does not suggest the means to identify uncertainties, prioritize them, develop assumptions and so forth. Thus, Other tools may be used within the steps of each phase.

• The tool encourages the project team to identify and act on the full spectrum of uncertainties to be resolved to succeed with a project. Furthermore, it helps to get an overview of what is known and unknown, which uncertainties to prioritize, suggest different assumptions about an uncertainty to be tested, and to find ways to test these assumptions as fast and inexpensive as possible.

• Project team tends to prioritize uncertainties they are more comfortable with and to ignore those who are difficult to address. This is a pitfall for teams composed of specialists from the same professional backgrounds, who generally prefer to focus on challenges they can see a way through. Failing to recognize uncertainties of other categories increases the likelihood that one of these uncertainties will turn out to be a project killer. To counteract such tendencies, it is important for oversight boards to be staffed with veterans of high-uncertainty projects.

• Identifying success criteria for assumption testing can be difficult. It can be difficult to define the right success criteria based on which the results of an assumption test will be approved or disapproved. When does the project team have enough information to state an unknown has become a known? If the success criteria are slightly off, they may be measuring another parameter than the intended, and the project team will falsely claim to have uncovered an unknown. Thus, it is important for the project team to discuss its plan for testing assumptions and defining success criteria with the overview board before executing tests.

Glossary

• Learning loop: Testing one set of critical assumptions in the first phase of the learning plan.
• Learning per dollar spent: The metric used by management to measure the progress of projects. The metric is also used when selecting approaches to test assumptions to maximize the learning from learning loops.
• Learning plan template: No standardized template exists, however, the template shown in figure 1 has been developed by MIT Sloan School of Management, and provides a step-by-step process to implement the Learning plan.
• Oversight board: A board consisting of managers that acts as quality control and help assess the learning produced by learning loops.

Annotated bibliography

G.C. O’Connor and M.P. Rice, “A Comprehensive Model of Uncertainty Associated with Radical Innovation,” 2007. This article explains about the uncertainty matrix, which is a useful tool for the project team when prioritizing uncertainties during the first phase of the learning plan.

Strategyzer, "The Learning Card", [1]. The Learning Card by the company Strategyzer can be combined with the learning plan as it offers a practical approach to carry out the first phase of the learning plan.