Revision as of 10:34, 22 September 2017

Introduction

Project managers find many complications when handling complex projects in which the environment within policies are made are also tangled. To just roll out a plan and measure predefined indicators could not be enough. ROMA, born from these situations, is an approach to develop policy engagement processes to influence change. It comprises a suite of tools that can be used to improve how organizations and companies diagnose the problem, understand the types of impact their work could have on policy-making, set realistic objectives for policy influence, develop a plan to achieve those objectives, monitor and learn from the progress they are making and reflect this learning back into their work. In other words, it will help to unpack complex problems, analyze stakeholder influence and understand pathways of impact in conditions of uncertainty.

Policy processes can be highly political and can involve complicated networks of actors with different values and interests. These types of environment require a collaborative approach, and ROMA has been designed specifically to facilitate collaborative engagement. https://onthinktanks.org/resources/rapid-outcome-mapping-approach-more-useful-when-used-to-plan-the-whole-policy-research-initiative/

Concepts

• Policy

Set of decisions that give rise to specific proposals for action. These decisions can vary from legislation to setting standards, allocating resources between organizations, changing the levels of subsidies or taxes...

• Outcome Mapping

It is an approach to planning, monitoring, and evaluation that puts people at the center, defines outcomes as changes in behavior and helps measure contribution to complex change processes. It is based on a series of small and incremental steps and states that sustainable change comes from people's behavior. [2]

[2] https://www.outcomemapping.ca/

Background

• RAPID

The RAPID (Research and Policy in Development) team is part of the Overseas Development Institute(ODI). Its main focus is to achieve a deeper understanding of the links between projects (mainly research) and policy.

https://www.odi.org/our-work/programmes/research-and-policy-development/our-work

• ODI

ODI is an independent think tank on international development and humanitarian issues. Its puerpose is "to inspire and inform policy and practice which lead to the reduction of poverty, the alleviation of suffering and the achievement of sustainable livelihoods in developing countries." [1]

[1] https://www.odi.org/about-odi

Method

ROMA Methodology Activities and Steps

This method is designed to be done as a group exercise in which more diversity in the team will outcome as a more detailed analysis. It is an analysis and workshop-based technique that encourages feedback from teams working on diverse issues and in very distinct contexts. It is also scalable, meaning that it can be applied to a small intervention, such as the promotion of research findings during an event, or to a large multi-year program or campaign to try to influence in a particular sector. As research-based evidence is used in this method, it cannot only rely on the quality of the evidence but also on other major factors.

These factors are the Context, Evidence and Links (CEL) framework:

• The political context – political and economic structures and processes, culture, institutional pressures, incremental vs radical change etc.
• The evidence – credibility, the degree it challenges received wisdom, research approaches and methodology, simplicity of message, how it is packaged etc
• The links between policy and research communities – networks, relationships, power, competing discourses, trust, knowledge etc.

ROMA consists of three main activities, each of which is broken down into a series of steps:

Diagnose the problem

Stakeholder

Usually, the pressure of the deadline to define the root cause lead to symptoms. To avoid this situation, ROMA proposes two techniques:

• Five Why

Basically, consists of asking several times why to the cause found. This will allow the team to reach a deep understanding of the issue.

• Fishbone Diagram

After the Five Why technique, the main cause can be broken down into different ones.

By executing these techniques, most of the actors who have some interest in the project will have appeared. In order to clarify which of them are more relevant, a Stakeholder mapping could be useful.

To ensure a deep diagnosis of the policy change, it is required to examine the complexity and uncertainty around it. ROMA focuses on three main characteristics that will allow the team to learn how to respond to complexity:

• The capacity of change policies differs from simple problems, where usually there is a hierarchy and a well define decision-making structure, to complex problems. In these, the responsibilities, skills, and resources are distributed among different organizations. Facing a simple or complex problem will need different methods in order to have some influence on them.
• The goals to be achieved will be significantly distinct and even contradictory in complex projects. It will be needed a lot of negotiations and commitment to finding common goals. Whereas in a simpler one, it is easier to agree or to influence towards some direction.
• Depending on the complexity, the pathway of the making-policy will be predictable or uncertain. For simpler problems, these processes have regular routines with foreseeable opportunities. However, complex ones are much less predictable and the team should be able to adapt and to be flexible enough to overcome the emerging situations.

Finally, to assess wider aspects, it is key to consider the political and institutional environments, how they interact with each other, the power relations between them and from where could the changes or reforms come from.

Develop a strategy

Communication methodologies

First, the policy influence objective has to be identified and clarified. It is required to know why the proposed changes are important, who they affect, what has to be done and where your team stands in relation to other stakeholders who are also trying to influence a change. One way to reflect this information is to do a force field analysis where all the barriers and enablers will be reflected. Second, it is important to understand and consider the final outcome because depending on this, the trait and relationship towards the stakeholders will be significantly different. Once the outcomes have been defined, the likelihood of happening and how they will come have to examine. ROMA suggest the creation of more hypothesis by developing two theories with the five whys technique: the theory of no-change, asking why the change is not going to happen and the theory of change, asking how changes occur. Fourth, one of the most important activities is to develop the communication strategy towards all the stakeholders. Approaches like advising, advocating, lobbying and activism, could be selected to encourage a particular policy position; however it is relevant to know the differences between them, how the stakeholders could respond, the key actions to be made, which channels are going to be used, how many and which resources are going to be invested and finally for how long, in other words, it is important to develop a schedule. Fifth, as policy engagement should be considered as a collaborative approach, it is essential to map the competencies of all the people involved in the project. Through the engagement process different situation will come along, so it is wise to know who are the good political enablers to understand politics and identify key players, the good storytellers for the communication strategy developed, the good networkers and the good engineers. In other words, recognize the people that could be valuable at some point of the project due to their competencies. Moreover, at this point, the complexity of the project would have been understood, so it is easier to know which are the key competencies that will be required in the future.

Finally, by following this steps, the team will be able to develop an engaging strategy. However, there are other issues that are worthy to consider like interactions between different stakeholders affect the behaviour of the people, depending on the organization structure, the approach will be different or knowing how to influence actors indirectly could help at the time of developing the strategy.

Develop a monitoring and learning plan

Traditional M&E (monitoring and evaluation) approaches are not adequate to the complex context of policy-influencing projects because they rely on simple feedback models with indicators, data collection and assessment that are already predefined. ROMA aims to shift from M&E approach to M&L (monitoring and learning) approach where the focus is on the "sense-making" of the monitored information. This approach is based on the next principles:

• Appropriate to the purpose, scale and context in question. As already mentioned before, small and large projects cannot be handled in the same way.
• It defines realistic results within the limitations of resources, time, scope and politics. Beyond this sphere, the team will need to rely on others to influence on these results.
• It focuses on key stakeholders and graduated change.
• It is based on a reasoned judgement as it seeks to generate evidence that will help to understand the effects of the policy change.
• As ROMA builds on systems and techniques already used and collect data with discrete studies, this approach keeps it focused on learning.
• It is built on collaboration and engagement. It is needed a wide range of expertise and perspective but also a close relationship with influential people.

Before developing the plan it is useful to understand why monitoring has to be done.

• It will help the team to become more financially accountable by knowing where, how much and the money is spent somewhere.
• It will improve the efficiency of the resources used.
• It will allow the readjustment the strategy to achieve the desired goals.
• It will strengthen the capacity of the team by improving their performance.
• It will help to understand the context under the project is working.
• It will increase the knowledge on the topic in question.
• It will help to build a trusting relationship between the stakeholders by sharing information and enable the participation.
• It will help to create a critical mass of support in order to change a policy.

There are six levels that can be monitored: the strategy that is being followed, how is the management doing, the outputs obtained, the response of the stakeholders after the release of the outputs, the outcomes and finally, the context.

ROMA proposes two different methods in order to monitor, collect and manage the data. The first method relies on real-time data collection and the second can be categorized as retrospective studies. Real-time data collection tools are related to the output and their consequences and the more immediate outcomes. Some of the tools that can be found are journals, logs, surveys, after action reviews and web analysis. On the other hand, retrospective studies focus on longer-term outcome measures. Inside this methods are the stories of change, the episode studies, the Bellwether interviews or the relational mapping.

Once the data is collected and gathered, it needs to be processed in order to be useful. Sense-making takes place in formal or informal meetings and events following a particular rhythm. Examples include annual reports, monthly team meetings, quarterly board meetings, project reports or stakeholder consultations among others. ROMA provides a framework that allows the learning and reflection in these sense-making process operating at a macro or micro level.

For informal sense-making, ROMA suggests the following practices: the establishment of a common language will speed up the communication between the team members allowing a better response, openness to a diversity of sources of knowledge will help to identify and understand unexpected effects and the use of visual tools will improve the ability to spot patterns and will help to have a better understanding of the big picture.

Some processes will require a more structured way of sense-making. Some suggestions are provided by ROMA: check the timing of the outcomes by using timelines and roadmaps, investigate alternative explanations and the use of comparison with the theory using different tests:

• Straw-in-the-wind
• Hoop
• Smoking gun
• Double decisive

By following these steps, the monitoring and the treatment of the data will support the decision-making and will show the working progress towards the policy influence desirable.

Cases

Limitations

References