RAPID Outcome Mapping Approach (ROMA)

Developed by Guillermo Altuna Faus

Introduction

Project managers find many complications when handling complex projects in which the environment within policies are made are also tangled. To just roll out a plan and measure predefined indicators could not be enough. ROMA, born from these situations, is an approach to develop policy engagement processes to influence change. It comprises a suite of tools that can be used to improve how organizations and companies diagnose the problem, understand the types of impact their work could have on policy-making, set realistic objectives for policy influence, develop a plan to achieve those objectives, monitor and learn from the progress they are making and reflect this learning back into their work. In other words, it will help to unpack complex problems, analyze stakeholder influence and understand pathways of impact in conditions of uncertainty.

Policy processes can be highly political and can involve complicated networks of actors with different values and interests. These types of environment require a collaborative approach, and ROMA has been designed specifically to facilitate collaborative engagement. ROMA is about critical thinking. It can be used in any situation and circumstance because it eases a process of thought about the context, organisations, skills, objectives, stakeholders, tactics, tools, how to use them, why, etc.

Concepts

Figure 1: Policy change

• Policy: Is the set of decisions that give rise to specific proposals for action. In other words, is the attitude of organizations or countries towards some specific issue. These decisions can vary from legislation to setting standards, allocating resources between organizations, changing the levels of subsidies, taxes, etc.

• Outcome Mapping: It is a methodology for planning and assessing development projects oriented towards change and social transformation. It provides a set of tools to design and gather information that puts people at the centre, defines outcomes as small and incremental changes in behaviour and helps measure the contribution to complex change processes. It also helps a project learn about its influence on the progression of change to be flexible enough to adapt the strategies to achieve the desired outcomes.

Background

• RAPID: The RAPID (Research and Policy in Development) team is part of the Overseas Development Institute (ODI). The main focus of RAPID team is to achieve a deeper understanding of the links between projects, of every kind of knowledge but mainly research projects, and policy.

• ODI: ODI is an independent think tank on international development and humanitarian issues. Its purpose is
  
  "to inspire and inform policy and practice which lead to the reduction of poverty, the alleviation of suffering and the achievement of sustainable livelihoods in developing countries."
  
  

ROMA began in 2004 with the aim of allowing researchers to directly engage with policy-makers from the initial phases of their projects in order to influence the direction of their decisions. It is based on the evidence of more than 100 cases where it was reflected whereas the research in question has or has not contributed to the policy change (Figure 1).

Method

Figure 2: ROMA Methodology Activities and Steps

This method is designed to be done as a group exercise in which more diversity in the team will outcome as a more detailed evaluation. ROMA is an analysis and workshop-based technique that encourages feedback from teams working on diverse issues and in very distinct contexts. It is also scalable, meaning that it can be applied to a small intervention, such as the promotion of research findings during an event, or to a large multi-year program or campaign to try to influence in a particular sector. As research-based evidence is used in this method, it cannot only rely on the quality of the evidence but also on other major factors.

These factors are the Context, Evidence and Links (CEL) framework:

• The political context – political and economic structures and processes, culture, institutional pressures, incremental vs radical change etc.
• The evidence – credibility, the degree it challenges received wisdom, research approaches and methodology, simplicity of message, how it is packaged etc
• The links between policy and research communities – networks, relationships, power, competing discourses, trust, knowledge etc.

ROMA consists of three main activities, each of which is broken down into a series of steps (Figure 2) :

Diagnose the problem

Figure 3: Stakeholder approach

1. The problem has to be identified. However, on many occasions, the pressure of the deadline to define the root cause, lead to symptoms and therefore all the work afterwards is useless because it is not treating the main problem. To avoid this situation, ROMA proposes two techniques:

• Five Whys: This technique consists of asking several times why every time a reason is found. This will allow the team to reach a deep understanding of the issue and to come up with new perspectives that otherwise will not be detected.

• Fishbone Diagram: After executing the Five Whys technique, the main problem can be broken down into different causes allowing the team to have a wider vision of the problem and a better understanding of the big picture.

2. Recognition of the stakeholders. By putting these techniques into practice, most of the actors who have some interest in the project will have appeared. In order to clarify which of them are more relevant because of their influence or power, they should be classified. ROMA suggest doing a Stakeholder mapping so as to know how to engage them properly (Figure 3). With some of them will be enough to just monitor but with others a close working relationship should be built.

3. Understanding of the context. To ensure a deep diagnosis of the policy change, it is required to examine the complexity and uncertainty around it. ROMA focuses on three main characteristics that will allow the team to learn how to respond to complexity:

• The capacity of change policies differs from simple problems, where usually there is a hierarchy and a well define decision-making structure, to complex problems. In these, the responsibilities, skills, and resources are distributed among different organizations. Facing a simple or complex problem will need different methods in order to have some influence on them.
• The goals to be achieved will be significantly distinct and even contradictory in complex projects. It will be needed a lot of negotiations and commitment to finding common goals. Whereas in a simpler one, it is easier to agree or to influence towards some direction.
• Depending on the complexity, the pathway of the making-policy will be predictable or uncertain. For simpler problems, these processes have regular routines with foreseeable opportunities. However, complex ones are much less predictable and the team should be able to adapt and to be flexible enough to overcome the emerging situations.

4. Assessment of wider aspects. It is key to consider the political and institutional environments, how they interact with each other, the power relations between them and from where could the changes or reforms come from.

Develop a strategy

Figure 4: Communication methodologies

1. Objectives The policy influence objective has to be identified and clarified. It is required to know why the proposed changes are important, who they affect, what has to be done, where the team stands in relation to other stakeholders who are also trying to influence a change. One way to reflect this information is to do a Force Field analysis where all the barriers and enablers will be reflected.

2. Outcomes It is important to understand and consider the final outcome because depending on this, the trait and relationship towards the stakeholders will be significantly different.

3. Theory of change Once the outcomes have been defined, the likelihood of happening and how they will come have to examine. ROMA suggest the creation of more hypothesis by developing two theories with the five whys technique: the theory of no-change, asking why the change is not going to happen and the theory of change, asking how changes occur.

4. Communication One of the most important activities is to develop the communication strategy (Figure 4) towards all the stakeholders. Approaches like advising, advocating, lobbying and activism, could be selected to encourage a particular policy position; however it is relevant to know the differences between them, how the stakeholders could respond, the key actions to be made, which channels are going to be used, how many and which resources are going to be invested and finally for how long, in other words, it is important to develop a schedule.

5. Resources As policy engagement should be considered as a collaborative approach, it is essential to map the competencies of all the people involved in the project. Through the engagement process different situation will come along, so it is wise to know who are the good political enablers to understand politics and identify key players, the good storytellers for the communication strategy developed, the good networkers and the good engineers. In other words, recognize the people that could be valuable at some point of the project due to their competencies. Moreover, at this point, the complexity of the project would have been understood, so it is easier to know which are the key competencies that will be required in the future.

6. Strategy By following these steps, the team will be able to develop an engaging strategy. However, there are other issues that are worthy to consider like interactions between different stakeholders affect the behaviour of the people, depending on the organization structure, the approach will be different or knowing how to influence actors indirectly could help at the time of developing the strategy.

Develop a monitoring and learning plan

Traditional M&E (monitoring and evaluation) approaches are not adequate to the complex context of policy-influencing projects because they rely on simple feedback models with indicators, data collection and assessment that are already predefined. ROMA aims to shift from M&E approach to M&L (monitoring and learning) approach where the focus is on the "sense-making" of the monitored information and also focused on extracting valuable learnings that could be used in future projects. This approach is based on the next principles:

• It has to be appropriate to the purpose, scale and context in question. As already mentioned before, small and large projects cannot be handled in the same way.
• It defines realistic results within the limitations of resources, time, scope and politics. Beyond this sphere, the team will need to rely on others to influence on these results.
• It focuses on key stakeholders and graduated change.
• It is based on a reasoned judgement as it seeks to generate evidence that will help to understand the effects of the policy change.
• As ROMA builds on systems and techniques already used and collect data with discrete studies, this approach keeps it focused on learning.
• It is built on collaboration and engagement. It is needed a wide range of expertise and perspective but also a close relationship with influential people.

Before developing the plan it is useful to understand why monitoring has to be done.

• It will help the team to become more financially accountable by knowing where, how much and the money is spent somewhere.
• It will improve the efficiency of the resources used.
• It will allow the readjustment the strategy to achieve the desired goals.
• It will strengthen the capacity of the team by improving their performance.
• It will help to understand the context under the project is working.
• It will increase the knowledge on the topic in question.
• It will help to build a trusting relationship between the stakeholders by sharing information and enable the participation.
• It will help to create a critical mass of support in order to change a policy.

There are six levels that can be monitored: the strategy that is being followed, how is the management doing, the outputs obtained, the response of the stakeholders after the release of the outputs, the outcomes and finally, the context. ROMA proposes two different methods in order to monitor, collect and manage the data. The first method relies on real-time data collection and the second can be categorized as retrospective studies. Real-time data collection tools are related to the output and their consequences and the more immediate outcomes. Some of the tools that can be found are journals, logs, surveys, after action reviews and web analysis. On the other hand, retrospective studies focus on longer-term outcome measures. Inside these methods, the techniques found are the stories of change, the episode studies, the Bellwether interviews or the relational mapping.

Once the data is collected and gathered, it needs to be processed in order to be useful. Sense-making takes place in formal or informal meetings and events following a particular rhythm. Examples include annual reports, monthly team meetings, quarterly board meetings, project reports or stakeholder consultations among others. ROMA provides a framework that allows the learning and reflection in these sense-making process operating at a macro or micro level.

• For informal sense-making, ROMA suggests the following practices: the establishment of a common language will speed up the communication between the team members allowing a better response, openness to a diversity of sources of knowledge will help to identify and understand unexpected effects and the use of visual tools will improve the ability to spot patterns and will help to have a better understanding of the big picture.

• Some processes will require a more structured way of sense-making. Some suggestions are provided by ROMA: check the timing of the outcomes by using timelines and roadmaps, investigate alternative explanations and the use of comparison with the theory using different tests: Straw-in-the-wind, Hoop, Smoking gun and Double decisive.

By following these steps, the monitoring and the treatment of the data will support the decision-making and will show the working progress towards the policy influence desirable.

Case of study

The explanation and development of the next case will show the relationship between ROMA and the learnings from project and portfolio management:

The humanitarian organization World Vision wanted to start a project in some communities in Zambia so as to improve their way of life and provide them with the indispensable resources. So, they used ROMA for the early stages of the project. To know which kind of challenge they are going to face, what needs to be done and who are the organizations and people that will have some interest in the project.

The team identified that child health was one of the main problems for these communities but in order to go deeper into the issue, they used the Five Whys technique. In addition to this, they conducted a survey in these communities regarding healthcare and children and some causes were pointed out:

• Long distance from health centre
• Few qualified health personnel
• Low-level understanding of prevention strategies
• Need of strengthening health care system at the community level
• Integration of nutrition into health service

The next thing done by the team was to recognize the key stakeholders and the outcomes that were expected from them. Mainly institutions with huge political power and influence came out from this activity. Once the goals and the actors have been identified, the team started to develop their strategy to influence in policies that will have some effects in child healthcare. They used the Force Field technique to detect the enablers and barriers of entries along with an assessment of their internal competencies and capacities in order to point out what is needed to fulfil the goal.

To summarize, World Vision followed the steps and used the tools provided by ROMA in order to achieve a better understanding of the issue that they will have to meet during the whole life of the project if they decide to go ahead with it.

Limitations

• ROMA is an excellent tool for planning policy influencing interventions but it does not suggest a method or an approach for managing structured project management. However, project management principles could be useful here combined with ROMA. A structured project management method divides the life of the project into manageable stages with their deliverables and activities. They can be planned to match with ROMA's steps. While ROMA is used to map the policy change roadmap, a project management method will help to navigate the complexity of policy influencing.

• There will be some circumstances where ROMA may not be the right tool. ROMA needs some degree of flexibility to be useful, therefore it cannot be appropriate for situations in which there is a clearly defined process or where interaction with some stakeholders needs to be highly structured.

Annotated Bibliography

ROMA Guide

This guide comprehends and clarifies the ROMA methodology. It includes all the techniques explained above with some examples of how to use them and also incorporate and illustrate the tools that ROMA method is based on.

Outcome mapping

The outcome mapping guide helps to clarify the presumed relationships among the objects of a program, project or activity. By using it, a deeper understanding of what challenges to face will be the result.