Evaluating outcomes and impact

As well as information on the meaning and importance of impact, this page also provides information on: Getting help with evaluating outcomes and impact Outcomes-based accountability Social accounting Social return on investment (SROI) What is impact? The term ‘impact’ is used in different ways. It can be seen as the broad, longer-term effects of your work, or the institutional changes it brings about. However, for nonprofits it is also used to mean the outcomes achieved for your immediate user group, whether they occur in the short or long term. The crucial distinction to make is between what you do (that is, your outputs, such as services offered) and what you achieve (that is, your outcomes, the changes that result from what you do). The importance of demonstrating impact It is important to communicate with your stakeholders and the wider public about your work and the difference you make. Information about outcomes has also become part of regulatory requirements. Foundations and other grantmakers are requiring evidence of measurable benefits to clients and commissioners are also focusing more on outcomes. More than that, getting information as you go along about how effective you are in achieving positive change is crucial if you are to achieve even more for your client group. Identify a realistic level of change Certain types of work have intended impacts that take many years to achieve. It may be difficult to get good information over time, so intermediate outcomes may be more important to identify in order to do justice to your work, for example in relation to improved skills and confidence and readiness for work. You may also agree with your funding bodies certain outcomes that you can report on which will contribute towards a desired area of impact. Standards of evidence Evidence of changed outcomes relies on comparing the situation before and after your activities. Even with good outcomes information, it may be difficult to demonstrate that the outcomes are a result of your intervention. To overcome this, evaluations sometimes do a comparison study by setting up a control group and some argue that this will provide a better standard of evidence. Potential funders and other stakeholders may find the results of an evaluation using a comparison group more persuasive.  But a matched comparison group should be the same in all respects – in age, gender and socio-economic factors – other than the fact that one group only receives the intervention and this can be difficult to establish. It may be even more difficult to get reliable information on the control group if you are relying on external sources for it. For most small nonprofits a control group will not be appropriate, and even for larger ones, the investment of tine and resources should be set against what might be limited gains. Measuring the impact of campaigning You may find it particularly difficult to measure the impact of your campaigns, and how far your particular activities have contributed to a change of public opinion and attitudes, and changed policy and practice. One reason for this is that there are likely to be a number of other significant contributors to any change that is seen. However, you can still demonstrate the effectiveness of your campaigns, for example in building effective partnerships, in mobilising press or public opinion, or gaining important advocates, so it will be important to identify these as planned outcomes from the start and put in place appropriate monitoring. See our Useful links below for further reading on this.  Get help with evaluating your outcomes and impact Many organisations are successfully working together with consultants over a period of time to improve the quality of outcome and impact information, and this can be a relatively cost-effective way of improving your reporting on impact. Consultants can help you to establish a monitoring and evaluation framework, to identify appropriate data collection tools or to develop your own. They can carry out interviews to supplement internal monitoring, help with data analysis and write reports. See our Useful links below for further resources on outcomes monitoring tools. What is outcomes-based accountability? This is the term used for an approach to planning and monitoring services for children, young people and families. It describes an outcomes-based approach that has gained in use in local authorities and some nonprofits. Idea have produced a brief summary of outcomes-based accountability (pdf – 225.86kb). Central to this is a process called ‘turning the curve’ which compares progress against what would have happened over time if nothing changed. What is social accounting? Social accounting is a framework for monitoring and evaluation that links a systematic reporting of organisational activities to the issue of social impact and the ethical behaviour of an organisation. The social accounting process can be quite flexible and may look like, or incorporate, other evaluation processes. However, there is greater emphasis on public accountability and a social audit panel reviews the report or social accounts and only then will the report be available to a wider public. Get more information and support from the Social Audit Network. What is Social Return on Investment (SROI)? SROI is a way to measure and account for the value you create with your work. Currently there is a lot of interest in the approach from funders and nonprofits and from both public and private sectors. It is a type of cost-benefit analysis, which is playing an increasingly important role as nonprofits are required to carry out more hard-headed analysis of the value and social impact of their work. SROI is an outcomes approach that puts monetary value on social and environmental benefits relative to a given amount of investment. For example, an organisation might have a ration of £4 of social value created for every £1 spent on its activities. SROI may not be appropriate for many organisations, but the SROI methodology could help make a good case for providing certain types of services and is especially useful if an organisation’s funders require outcomes information in financial terms. Organisations that have already done some work on identifying and measuring their outcomes will find SROI much easier. Download a copy of the new SROI guide from the SROI Network’s website (opens a PDF, 4.3MB). Recent research by Demos has found that most nonprofits are still struggling to identify and measure their outcomes and are not ready for SROI. See Useful links below for more resources on SROI. Useful links Resources on measuring the impact of campaigning Is your campaign making a difference? (NCVO) Campaigning for Success (NCVO publication) Resources on outcomes monitoring tools Off-the-shelf monitoring tools (CES) Developing monitoring tools (CES) Further resources on SROI Read New Philanthropy Capital’s position paper on when SROI is most useful

%d bloggers like this: