- Jakob Jespersen

A director from a private business would never give an annual report to the shareholders without mentioning the size of the profit from the past year. Nobody, however, reacts when the annual report to the ‘stakeholders’ by a ‘boss’ in a development ‘business’ does not mention how big the ‘development profit’ was i.e. how many poor people benefited in which way and how much from the activities. Why is it that we do not measure the impact of our activities asks Jakob Jespersen in this article. He will in co-operation with NEPAN be assisting MS/Nepal and some of its partners in making Impact Monitoring and Evaluation a part of the regular activities of the organisations.

‘Impact’ is the technical term for the actual benefit that the ‘beneficiaries’ get from development activities. All people who work with development activities assume that their work will, in one way or the other, result in an improved quality of life for some poor people. It may be a better life in terms of better health, clothing, housing etc., higher status, more equality, more economic security, more decision power or political influence or whatever the aim of the programmes might be.

The whole justification for all the money, time and resources spent is that poor people will eventually benefit. All the cars, air-tickets, files, meetings, seminars, workshops etc. are supposed to benefit the poor in the end. We - directors, field workers, supervisors, peons, PAB members, Programme Officers, DWs, Accountants, Drivers, Receptionists etc. - are all intermediaries.

But when asked if all the efforts actually help the poor, we tend to say something like : ‘well I think so, but it is difficult to measure’ - and - ‘there are many problems, but we are working on them.’ We find it difficult to be precise about the effect of our work because impact evaluation is not a tradition in our organisations.

We should make it a habit in our organisations to measure impact. Field workers, DWs, Supervisors etc. go to the field anyway. We go there to implement or to see that programme activities are running. And when the training is completed, toilets built, crops planted, schools repaired according to plan then we tend to consider our tasks completed. But we shouldn’t. The ‘task’ of development is not a toilet but a better life. We should make it a habit to ask and investigate until we feel sure we know how people feel that their lives have improved, which aspects of their lives, how many people etc.

If we do this, people will give us ideas for improvements of the programmes. Then we will learn what really works and what doesn’t, we will ourselves generate ideas for improving our programmes, we will ourselves feel more satisfaction in our jobs, the donors will learn more about what makes development work in Nepal and the taxpayers in the North will know how the money they give benefits poor people. Yet we don’t do it.

It seems so obvious that we should evaluate the impact of our work - and yet we don’t do it. This is because there are as many reasons for not doing impact evaluations as there are reasons for doing it. Here are some of them :

1. I don’t know how to do it

2. It is not part of my job description

3. If I do it I might discover that some of the things I do don’t work - I would rather not know. Or I might discover that some of the things my colleagues do don’t work - I would rather not tell them something unpleasant since we are friends.

4. If I do, I might discover that some things don’t work and if I tell my boss s/he will just blame me or my colleagues for not having done a proper job

5. The boss doesn’t actually want to know if things don’t work because s/he is afraid the donor will cut funding if they don’t get ‘value for money’.

6. The donor really doesn’t want to know about failures because they fear that if the taxpayers in the North find out, they will not want to pay for development aid in the future.

Out of the above reasons only number one and two are ‘good’ reasons - the rest are ‘bad’ reasons. They are, however, all real and if impact evaluation is going to become a habit in development work we need to address both the ‘good’ and the ‘bad’ reasons. We need to address both the practical issue of how to find out whether toilets, chimneys, literacy courses etc. really result in a better life for poor people or not, we need to include impact monitoring in job descriptions and report guidelines AND we need to address the issue of attitude towards ‘failures’.

Much ‘partnership co-operation’ is in reality nothing more than a contract between a donor, which wants a certain programme to be carried out and an NGO which is willing to carry out the job if paid. It is a business relation : the donor pays - the NGO delivers the service. In such a relation impact evaluation is dangerous for the NGO, because any failure to reach the aims could be seen by the donor as ‘services not provided’. In a true partnership co-operation the partners must agree on aims and strategies and they must realise that doing development is always a learning process for both. When impact evaluation is done, then both partners (and the beneficiaries) will learn how appropriate the strategy that they both believed in really was. There are, of course, strategies that we are fairly sure are working, but in most cases we don’t really now - we assume. We assume that adult literacy will lead to the empowerment of women, but we don’t know because people are different. In all partnership agreements we should have a learning objective - an objective of learning about the appropriateness of the strategy chosen. An objective of developing better ways of achieving the real development objectives. Impact monitoring is the activity, which is necessary for learning about impact. This must be clearly written in our agreements.

Failure is an opportunity for learning: ‘Fail Forward’, ‘Embrace Error’. The only real failure is not to learn from them. To look at failures in this way requires a change of attitude - first of all from the managers, presidents and other leaders. When beneficiaries or field staff report that this or that doesn’t work, the ‘boss’ must say: ‘Thank you for telling me - What can we learn from this? - How do you think we could improve our activities?’. And when things work we must document it well so others can build on it. And we would all - fieldworkers, supervisors, managers, presidents, DWs, POs and directors - like to hear about successes because this is the whole reason for our work and all the resources we spend.

So let us get going with impact monitoring. As a beginning MS/N will offer support for some of its partners in developing systems for ‘learning from failures - building on successes’. One of the organisations will be CWD in Nepalgunj. Other partners, which would like to make impact monitoring a habit are most welcome to contact MS/N via Madhu or Jakob.