Welcome to our third session! During this session we will get into the details of the “results chain.” We’ll define and link the components that make up this chain and talk about how it is used in M&E. We’ll also discuss an important point: attribution and contribution when it comes to defining, framing, and communicating the results, or impact, of our research capacity building activities.

Here are our objectives for this session. We’ll return to check in with these at the end of the session.

As we discussed in our last session, the approach that we take to M&E is based on what’s called a results framework, and a fundamental part of this framework is the results chain.

The results chain breaks our activity, project or programme out into different components. This can help us in our design process, as well as when it comes time to analyse results. The results chain organises these components into a linear flow that suggests their relationship. Of course, we know that real life can be complicated – there are many factors and influences on how our activities function and how they turn out. The results chain, however, provides a model for us to think carefully about factors we control, and factors we don’t control, to determine what our priorities are, and to look at our assumptions.

Let’s take a look at the five main components of the results chain. We’ll start with an overview and then go into detail on each component. To make it a bit less abstract, we return to our case study to illustrate the chain. Again, Maria has made some notes as she begins to sketch out her results chain.

First, we have inputs. In Maria’s case, these include the classroom she’ll reserve for the workshop, her colleagues, who she is bringing in as trainers, and the curriculum that she´s scrambling to produce. Once she gets all of that together, she’ll be able to produce her activity.

And what exactly is her activity? How can we briefly sum it up? Well, it’s a three-day, face-to-face workshop on Good Clinical Practices. We could add more details, but that gives us a sense of what kind of research capacity building activity she has planned.

This activity will hopefully lead to some direct products, or outputs. In Maria’s case, what are the outputs? Well, what she wants is to have trained staff members.

But why or what for? Why does Maria, her boss, and her institution want to train staff members? Is it just to pad their résumés? What is the change that they want to see because this activity took place?

We could say – “a higher level of staff competence to complete clinical trials.”

So, long-term – goals level – what impact are Maria and her team looking for? When we talk about the results chain in the context of health interventions, we would say that “impact” defines the change we want to see at the level of the population. Since we’re dealing strictly with research capacity building activities here, we’re going to define the population level as the institutional level. What impact – or longer-term change do Maria and her team want to see at the Institute for Research?

What do you think? Do Maria’s initial thoughts make sense to you?

Let’s take a slightly deeper look at each of the components.

Inputs are the resources that you put into your activity.

Let’s think about the activity we’re all engaged in right now – a workshop on M&E for Research Capacity Building Activities.

What do you think are some of the inputs to this activity?

The selected candidates are inputs, the workbook is an input, an online platform, pens and pencils, a pre-test – these are all inputs. There are many resources that go into any activity that we could potentially measure. Often inputs have a budget allocation - they are resources that must be paid for and acquired.

Activities are what you do. In the context of research capacity building, that could be a course or a mentoring programme, or, in our case, a workshop.

Even when you have just one activity, such as this training activity, there can be other activities embedded within it. In this activity, one would first need to create curriculum, for example. This is a sort of sub-activity, something that must be done prior to, or along with, the main activity. Our role in M&E is to make decisions about what, and how much, we are going to measure -- what we are going to monitor and evaluate.

We’ll unpack this a bit more as we continue.

Outputs. Simply stated, outputs are what gets ‘put-out’.

They are the immediate results when an activity is completed. Often, it is the activity restated in the past tense.

If our activity is a training, our output might be ‘people trained’.

Outputs are the immediate result from an activity, and they are typically easy to measure.

Outcomes are results from an activity that signal a change. They are the measure of change from an activity.

The key words to differentiate output and outcome are:

OUTPUT – The direct product of an activity, easy to measure (usually just counting), and always immediate.

OUTCOME – The key word is CHANGE. Outcomes are associated with change and with project objectives. Outcomes usually require more advanced methods to measure than just simple counting. They could be immediate, but they could also be seen after some time.

Let’s go back to the activities that you mentioned from the previous slide. We’ve thought about possible inputs and outputs, now what are some examples of outcomes?

Impacts are the final results that your programme or activity can achieve. It is the longest-term result.

An impact is a population level change. Here, as we have said, we might talk about population level as “institutional level”. When we conduct research capacity building activities, what change do we want to see at the institutional level? When we measure impact, we want to measure the role our activity has had in this change.

Now we’re going to review the components that we just discussed by putting them together into a results chain. As we’ve discussed, a results chain lays out these components in a sequence that helps us see how they should flow together to achieve the change, or the results, that we anticipate and desire. Here’s what we’re going to do.

For this exercise, we will use this M&E Workshop as our example:

We have our results chain here on the wall, with each of the components, starting from inputs, to activity, to outputs, to outcomes, and finally to impact.

We’re going to give each of you labels with different definitions – some are inputs, some are outputs, etc. [Alternatively, you can have participants copy their definitions on a whiteboard or blackboard.] Your job is to place each of the labels in the proper column. We’ll do this part of the exercise individually. And, be careful, because we have placed some tricky labels in each packet that are not, in fact, related to any of the columns!

We have 5 minutes to place the labels on the wall, and then we will come back as a group to review how we have sorted out our results chain.

Discussion

Before we wrap up, let’s look at one more important aspect of M&E.

You may have noticed that as we progress along the results chain, moving from inputs to impact, it is more difficult to determine to what degree our activity can be considered the “cause” of the results. In outputs, the results can be linked to the activity. This is what we call attribution. But how certain we are that the outcomes are the direct result of our activity?

For example, in our case study, increased capacity to implement Good Clinical Practices can be the result of participation in a well-planned and well-implemented workshop, but other things, like the incentive of a promotion, or changes in working conditions, or even a better working atmosphere can also contribute. In this scenario, we can say that the improved GCP capacity can be partially attributed to the workshop, or that the workshop has contributed to improve staff capacity.

In the case of the impact, attribution is even more difficult to establish, as increasingly more factors may influence how and to what degree we reach our objectives. Continuing with our case, completing a clinical trial according to the funder’s requirements (specific objective) does not depend exclusively on increasing the GCP capacity, but also on other elements such as the timely availability and quality of resources (equipment and reagents), staff motivation and incentives, project management conditions, and the capacity to recruit and retain participants, etc.

Even if we can guarantee all these elements and successfully complete the research project, benefiting our institution’s reputation and, as a result, being considered a potential grant-recipient for further projects, this will also be conditioned by a number of other factors. These include internal factors, like how well other departments of the institution work, and the type and quality of the institutional leadership, and external factors, like the number and quality of the competitors, and the evolving financial landscape of the funder. In this case, it is safer to say that the training activity has plausibly contributed to reaching the objectives of the institution, rather than trying to establish attribution.

Evaluation is fundamentally an analytical exercise to help decision makers understand when, how, and to what extent the activity is responsible for a particular measured change. However, relatively few programs go as far as establishing cause-and-effect between the programme or activity and the change.