Project Everest

What is an impact assessment, and why are we doing it 1.0?

by
Jessica Stephanie Arvela
Jessica Stephanie Arvela | Jan 27, 2017 | in Ideas Box

An Impact Assessment is: ‘Evaluating the positive and negative, primary and secondary long-term effects on final beneficiaries that result from a development intervention; assessing the direct and indirect causal contribution claims of these interventions to such effects especially for the poor whether intended or unintended”

 

What is an Impact Assessment?

 

At Project Everest our work in country is entirely community based, thus inevitably there will be impact. Measuring our impact is to ensure it is positive, and geared towards our vision, mission, and values.

 

Defining terms:

 

An impact assessment is a measurement of outcome.

Impact is defined as a marked effect or influence.

 

The Organisation for Economic Co-operation and Development - Development Assistance Committee (OECD - DAC) defines ‘Impact Assessment’ as:

 

‘Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended’.

 

This definition adequately represents the need to consider not only an intended effect, but those potentially produced by an ‘intervention’, and focuses on the long term impact of such contact.

 

Project Everest Vision: Enable one billion people

 

Project Everest Mission: Solve the world's most complex development problems by building financially sustainable solutions known as 'social enterprise'.

 

Project Everest Values:

  1. Build Leaders: For the influence of a leader to be enduring they must build leaders; this enables impact to extend beyond themselves.

  2. Eat Last: In the way you live your life (and when it comes to eating!) you place the welfare of your team before your own.

  3. Work Hard: Put in 100%. The harder you work the luckier you get.

  4. Make the Hard Decisions - Always: You always have two choices, the easy decision or the hard decision. In almost every situation the hard decision is the right one.

  5. Enable Autonomy: Wherever possible give others the freedom to chose their own path.

  6. Be Raw: Be authentically you and embrace the ‘weird/crazy’ in others around you as it builds stronger teams and outcomes.

  7. Moonshot Thinking: Seek solutions that change the world.

 

Assessing our impact in country is not simply to ensure project and student success, while that’s critical, it is also to mitigate harm.

 

Most individuals would be familiar with the term ‘Hippocratic Oath’, where medical practitioners pledge to assist their patients by recommending treatment for which the potential benefit outweigh the risk of harm’.

 

Interestingly, the phrase ‘first, do no harm’ (‘Primum non nocere’, Latin) originates from the ancient Greek text ‘Of the Epidemics’ by physician Hippocrates, rather than the ‘Hippocratic Oath’. Regardless of its etymology, the ‘first, do no harm’ concept holds significant psychological power and influence.

 

A similar perspective can be adopted to assess the execution of projects. Questions that illustrate this position include:

 

‘What changes happened as a result of our work?’.

 

‘What unexpected or unplanned effects did our actions have on people, the environment, or the local economy?’.

 

Screen Shot 2016-11-25 at 4.43.49 pm.png

 

Why are we talking about this?

 

Project Everest operates in nations considered ‘developing’, where resources are scarce and social systems strained. This elevates the necessity of ensuring our influence is positive, and our outcomes or results measured, considered, and communicated. While we currently record activity, there is a need to define the impact of that activity.

 

“This is not data collection for data collection’s sake. Every number is a person, every case study represents someone’s life. This is knowledge that has a direct impact on community wellbeing, people’s happiness, health and welfare for this generation and those that follow.”

 

Screen Shot 2016-11-25 at 4.28.28 pm.png

 

Why this may be a difficult concept to grasp:

 

Assessing economic impact is simple, assessing social impact is difficult and is time intensive, resource expensive, requires lateral thinking, creativity, and imagination. You are not only measuring your activity, but attempting to determine what possible impact those daily and specific actions could have on the environment surrounding your team.

 

How we can implement it on Project:

 

An impact assessment should be conducted in real time (Fig. 1A), throughout the project application, and considered during enterprise strategy, design, and execution. As information is collected, perspectives and influences considered – this should feed directly into how teams complete project, develop their business model, and conduct themselves on the ground.

 

Screen Shot 2016-11-25 at 4.34.41 pm.png

 

Teams will benefit from negative responses and positive responses and should accept feedback readily and respectfully. Questions to guide your thinking: ‘does the community support it?’, ‘how will this negatively impact existing structures?’, and following that: ‘how can we use this information to build a model that is most likely to succeed for all?’.

 

Consider the concept of ‘causal attribution’: ‘did the intervention cause the outcomes and impacts that have been observed?’ or ‘causal contribution’: ‘did the intervention contribute to the outcomes and impacts that have been observed?’. Clearly, outcomes have multiple influences, and are a result of a combination of factors however, this theory may further guide your assessment.

 

These findings are invaluable for future projects, students, local partners, universities, and development in general.

 

Best practice guidelines have been created in a ‘Learning Driven Assessment’ through the McKinsey and Company Institute and may be useful to guide your application of assessment. These guidelines reflect the Design Thinking process and are as follows:

 

  1. Hear the constituent’s voice: Involve constituents at every stage of the assessment.

  2. Exercise rigor within reason: Given the context of the effort, design the most credible assessment that is feasible.

  3. Drive assessment with learning: Address gaps in existing knowledge, understand what works and why, and harvest learning opportunities from “failures.”

  4. Don’t measure everything: Focus assessment on information that will truly be used.

  5. Assessment and strategy are inextricably linked: Design assessment and strategy together.

  6. Don’t let assessment sit on a shelf: Actively use the lessons learned through assessment to drive decisions about strategy, program design, and execution.

  7. Collaborate, don’t dictate: Co-create the assessment with program leaders; make sure there are sufficient resources to execute it successfully.

  8. Build off and build up: Don’t assess what is already “known” in the sector; look beyond your own organisation for answers.

  9. Borrow, don’t reinvent: Repurpose existing assessment tools when possible; don’t create new methods unless necessary.

  10. Foster a learning culture: In addition to tools and methods, assessment needs leadership that values learning from mistakes in order to succeed.

 

There are a plethora of methodologies that may be incorporated into project that will work towards ensuring positive impact and reduced harm. Which tool you use will depend on what your discipline is, what stakeholders are involved, and what influences you are measuring.

 

The following methodologies may be useful, if you are not familiar with the terms, there are multiple sources online that walk through their operation.

 

  • Social Return on Investment (SROI)

  • GRI Guidelines, Social Accounting

  • Most Significant Change

  • Third Sector Performance Dashboard

  • SE Balanced Scorecard

  • Results Based Accountability

  • Project Logic and Developmental Evaluation



Screen Shot 2016-11-25 at 4.40.36 pm.png

 

How this translates to positive development:

 

Deciding on a tool and practicing the use of that tool throughout project will allow teams to better plan, implement, and scale while increasing accountability, the correct allocation of resources, and stakeholder awareness.

 

What are the basic requirements?

 

In each team’s final report and handover, there is a requirement for the four following questions to be answered in depth:

 

  1. ‘How will you measure your (positive and negative) impact in country?’

  2. ‘What impact assessment tool/s have you incorporated into your weekly task list?’

  3. ‘What social, economic, environmental or other influences are you measuring and how?’

  4. ‘What are teams implementing to mitigate negative impact?’

 

Ensure your actions are always relevant, ethical and informed; achieving The 2030 Sustainable Development Goals depends on this.

 

Relevant: ‘What secondary research have we done in order to assure us that this outcome is required?’.

 

Ethical: ‘What specific steps will we take to provide study subjects with opportunities to review and refute the study findings?’.

 

Informed: ‘What don’t we know?’.

 

Screen Shot 2016-11-25 at 4.43.22 pm.png

 

Useful tools:

 

  1. Evidence Map Gaps outline what is known to be effective and ineffective in a particular location based on a particular topic, such as water and sanitation. May help prioritize best practice methods: http://www.3ieimpact.org/en/evaluation/evidence-gap-maps/

  2. B Corps is a certification that is awarded following a rigorous assessment that aims to determine a business’: ‘social and environmental performance, accountability and transparency’. May be used to guide business model: https://www.bcorporation.net

  3. Social Impact Assessment Practitioners (SIA) has a ‘guides and tools’ tab which leads to multiple resources for example: ‘The Good Practice Guide for Indigenous Persons and Mining’: http://www.socialimpactassessment.com/resources-cat.asp?type=Guides%20and%20tools

  4. The Foundation Center has a ‘Tools and Resources for Measuring Impact’ online database where a specific area may be selected for measurement and an appropriate tool provided: http://trasi.foundationcenter.org/search.php

  5. The McKinsey and Company ‘Learning Driven Assessment Workbook’ aid the development of metrics, and and assessment options. You will need to complete a swift online application to use the workbook however, it is free to use: http://lda.mckinseyonsociety.com/home.php

  6. ‘Broadening the Range of Designs and Methods for Impact Evaluations’, by the Department of International Development has a complex guide to rethinking evaluation in relation to development. May be useful for a deeper understanding of the importance of assessments: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67427/design-method-impact-eval.pdf

  7. ‘Prove and Improve’ by Economics as if People and the Planet Mattered have an excellent framework for discovering metrics, starting the conversation, and evaluating findings: http://proveandimprove.org/getst/index.php

  8. The International Association of Impact Assessment provides a comprehensive ‘Social Impact Assessment Guidance Document’ we would highly recommend viewing: http://www.iaia.org/uploads/pdf/SIA_Guidance_Document_IAIA.pdf

  9. World Risk has multiple illustrations that may assist in your understanding, and could be used to interpret findings in your handover and final report, an example may be seen below: http://weltrisikobericht.de/english/

 

Screen Shot 2016-11-25 at 3.49.22 pm.png

 

Final note for Trekkers:

 

Completed successfully, teams will develop a sound and informed understanding of the needs, complexity, and potential of each community. It will allow Project Everest to illustrate the impact Project Everest is having in-country prior to an enterprise, and demonstrate a consideration of the environment, and genuine human connection critical for conducting development in a dignified and thorough manner.

 

This is by no means a comprehensive or conclusive guide. As aforementioned, assessing social impact is inherently challenging. We encourage innovation, best practice, and leadership in the effort to obtain sustainable social change through enterprise.

 

Please contact your Group Leader with questions and improvements.

 

Jessica Stephanie Arvela

Training and Development Manager

Project Everest

jess.arvela@projecteverest.ventures

Australia: +61 411952617

Timor Leste: +670 73893343

 

Explanation for Leaders:

 

Think of this as a risk assessment of the social and political environment.

 

The outcome requirement is that each team completes the four questions included in the Impact Assessment for the month. The four questions are as follows:

 

  1. ‘How will you measure your (positive and negative) impact in country?’

  2. ‘What impact assessment tool/s have you incorporated into your weekly task list?’

  3. ‘What social, economic, environmental or other influences are you measuring and how?’

  4. ‘What are teams implementing to mitigate negative impact?’

 

You will find that to answer these questions, a deep understanding of the community (developed during the empathy and test stages of the design thinking process) is required.

 

These should be considered early week one when the team decides on who they will talk to, and about what. Once you have that information, teams will use this Impact Assessment Guide, and conduct desktop research into what would be the most effective framework for measuring your positive and negative impact.

 

Ultimately, the economic success of the enterprises will illustrate the impact. However, while that is still in development, an understanding of what we are doing well and what harm we may be exacerbating is essential for ensuring we are indeed positively impacting communities - which is one of our key drivers of impact.

 

Example #1

 

When the Atmospheric Water Generator (AWG) team has developed a successful prototype and is interviewing members of the local business district to determine if there is a viable distributorer - who the team decides to interview or propose a partnership with has an impact.

 

In Dili, Timor Leste, there are multiple ‘martial art gangs’ that regularly exacerbate civil unrest, it would be beneficial for teams to understand the gangs and their influence.

 

Martial art gang influence:

1. Ethnic disputes over the control of trade routes and markets,

2. Property disputes still in questions post 1999 resettlement,

3. Clashes between security forces,

4. Endemic gang rivalries,

5. Politically driven destabilisation campaigns by oppositing political parties.

 

Gangs potential influence of Project (each number reflects above point):

1. May prevent Project Everest operation through particular trade routes or in particular markets depending on allegiances,

2. May exacerbate conflict between gangs if Project Everest works with rival family living on disputed land,

3. May experience corruption through gangs supporting particular security forces and garnering their support,

4. May exacerbate gang rivalries by inadvertently supporting one segment of the community through Project,

5. May experience more difficulty establishing a viable enterprise if it conflicts with the political status quo.

 

While this is a risk to the safety of the team, conversely the project can influence the community in which the gangs operate - thus should be considered when devising a business within these social constructs. The outcome of these assessments is not to create roadblocks to success or halt progress. The end goal is to create road block mitigation plans to establish the path of least resistance.

 

Example #2:

 

When an assessment team has discovered a fuel substitution method due to the abundance of biofuels outside of Dili, and is interviewing members of the community to determine its viability - who the team decides to interview or propose a partnership with has an impact.

 

Teams may investigate the feasibility of the business model with local business owners and distribution companies. During the empathise stage, teams would understand who their customer base is, and their interest for such a product or service.

 

As a result of thorough mapping, teams may innovate out of socially determined ideas and practices that tend to assume and define what roles are deemed appropriate for each gender, and propose distribution and manufacturing roles to women.

 

‘Project planning and implementation from a gender-based perspective can have only one ultimate goal…contribute to changing the balance of the sexual division of power and resources so as to make it more equitable’.

 

The outcome of a systematic and inclusive process to business development is in line with the United Nations Sustainable Development Goal number five: ‘Achieve gender equality and empower all women and girls’. One of the five primary targets within Goal Five is to:

 

‘Undertake reforms to give women equal rights to economic resources, as well as access to ownership and control over land and other forms of property, financial services, inheritance and natural resources, in accordance with national laws’

 

Successful development projects engage women in the planning process - for Project Everest, involving women would establish appropriate and responsive enterprises, thus encouraging resilient economic growth.

 

‘When you invest in women, they typically invest 90% back into the health, nutrition and education of their families, as opposed to 30-40% for men.’

 

The intention for this example is to challenge and change socially prescribed roles in the pursuit of gender equality, poverty reduction, empowerment, and economic diversification. For example: women who have been successfully integrated into roles such as water technicians or builders, conventionally male roles - thus enabling women to become members of mainstream economy. This process requires measures to include women as ‘decision-makers, participants and beneficiaries in all relevant development activities, irrespective of the sector or type of activity’.

 

‘Gender equality is a prerequisite for poverty reduction because of the contribution women make and the role they play in society and in the economic well-being of the family and communities. Be it in rural or urban areas, be it in micro or medium and large enterprises, women must be an integral part of development, not only as beneficiaries, but also as decision–makers and agents of change’.

 

Final note for Leaders:

 

As you would complete a risk assessment prior to step off into an environment recognised to hold potential risk - similarly, for an impact assessment - your team would consider the risks (political and social) in their actions within the community.

 

  1. Choose a metric (a metric is ‘what’ and ‘how’ will you gather the information - which is highly dependant on what the task is hence, no standard framework - multiple resources in the Guide to choose from - this kind of research will aid Trekkers understanding of need/significance/method).

  2. Ensure a thorough brainstorm and mapping of social and political environment is completed by teams. Each assumption detailed and how the team will approach the potential conflict to avoid exacerbation is detailed.

  3. Ensure teams document this process and findings to include in their report and handover document.

  4. This information will become available to universities, alumni, future Project Everest members, investors, and will feature on the website. It’s important you consider how Project Everest (you) is defining the operation of social enterprise in the developing world. Is this the best path forward for this community? Are we considering all factors in development when designing the business (gender equality, corruption, child protection/labour, sustainability)? Are we doing more good than harm? How do you know that?

 

Completed successfully, teams will develop a sound and informed understanding of the needs, complexity, and potential of each community. It will allow Project Everest to illustrate the impact Project Everest is having in country prior to an enterprise, and demonstrate a consideration of the environment and genuine human connection critical for conducting development work in a dignified and thorough manner.

 

 

edited on 27th January 2017, 12:01 by Jessica Stephanie Arvela
No comments yet, be the first to post one!