Measuring impact

QUALITY, QUANTITY AND DEMONSTRATING DIFFERENCE

Organisational and programme monitoring an evaluation (M&E) is increasingly being demanded by donors. Erica Emdon, ProBono.Org Advancement Director, reflects on the need for M&E that demonstrates her organisation’s transformative impact.

ProBono.Org is frequently required by its donors to report on the impact of its work. This information is critical in order for donors to assess whether or not to support the organisation. The Board of Directors also needs to assess the effectiveness of the work that is done to judge staff performance. One way to satisfy both donors and our Board is to try to measure our impact.

Our core work is to assist clients and beneficiaries by hearing their legal problems and, where appropriate, referring them to pro bono attorneys and advocates for assistance. We are able to quantify the number of clients we see and keep statistics on various important aspects of our work. This gives us a quantitative picture of our impact to report on. For example, we saw about 3 000 clients in 2011, of whom 60 percent were women, 10 percent were disabled, and 40 percent were refugees.

Every quarter we also contact the attorneys and advocates to whom we refer pro bono matters, to find out how many hours would have been spent on each case. In this way, we are able to quantify and cost the time that has been spent, pro bono, as facilitated by the organisation.

Every so often when cases are finalised, either because the assisted client has obtained relief of because the matter has been heard successfully in court, we write up these success stories for our newsletter. As a means of demonstrating impact, this method relies on case examples rather than quantifiable results as we do not have the capacity to track the final outcome of all cases taken on by our panel of attorneys.

What none of these devices measures is the qualitative impact of ProBono.Org on both our beneficiaries and the lawyers who undertake the pro bono work. By failing to measure this we ignore a very important element of our impact in this way is that we are unable to report and reflect on a significant dimension of what we do, this being our transformative impact.

The impact that doing pro bono work has on lawyers is frequently enormous, and the long-term shift in values and perceptions intrinsic to the legal profession is great. We know this, but are not measuring it. Occasionally, when we interview lawyers for our newsletters, they make comments about the impact that working for poor clients have on them and how meaningful the work is. However, we have no way of measuring if we are embedding a deep-rooted and sustainable pro bono culture within the legal profession.

In future, we have to consider ways to measure these less obvious and more qualitative, rather than quantitative, aspects of our work. This will satisfy our stakeholders and donors, and will assure them that we are in fact a valuable organisation within the legal landscape.