Evaluation: An outside view or internal task?

MonitoringThe systematic and continuous collection of data during the course of the project. Its aim is to obtain up-to-date information in order to be able to steer and control the project. Monitoring should be carried out internally. However, an evaluation can be conducted either internally or by external experts.

Whether you or someone external evaluates the data depends on:

  • If you have the necessary skills within the project.
  • The complexity of the question you want to examine.
  • What kind of budget is available to you.
Captain

An internal evaluation has the advantage of being less cheaper to carry out than an external evaluation. Moreover, social-impact analysis is primarily about learning from the findings, and this is a task you can outsource only to a certain degree.

Another plus is that internal staff can locate relevant information more quickly, and possess more expertise regarding the project being assessed. However, this can also become problematic if this expertise degenerates into "operational blindness” or if personal relationships lead to bias. All of this also of course assumes that your project’s staff has the necessary skills to be able to carry out a competent evaluation.

By contrast, an evaluation by an external expert has several advantages. It will generally be technically and methodologically more precise; will be implemented in a more structured way (and also not as a sideline to project staffers’ regular activities); and will contribute an objective, nonpartisan view.

  • Qualifications

    Evaluators should have the following qualifications:

    • Experience with evaluations in the project’s area of focus.
    • Good methodological knowledge and high quality standards.
    • Good communication skills and a trustworthy personality.
    • Sensitivity in dealing with the target group.

A disadvantage will be the higher costs incurred by using an external evaluation. It may also be the case that the evaluator fails to access all relevant information, or cannot integrate it effectively, and thus fails to do full justice to the project.

All advantages and disadvantages compared

 

 

Advantages

Disadvantages

Internal evaluation
  • Familiarity with the project
  • Access to information
  • Little training period needed
  • Saves on costs
  • Lack of distance and neutrality
  • Potentially conflicting roles
  • Lack of needed skills
External evaluation
  • Know-how
  • Methodological knowledge
  • Acceptance by stakeholder
  • Neutrality
  • Unfamiliar with project
  • Costs
  • Possibly complex process

 

One solution to this dilemma could take the form of a mixed internal and external evaluation. Under this model, project staff work side by side with an external consultant who contributes expertise and a valuable outside view. This allows internal staff members to build up their own skills, while the external evaluation ensures a neutral viewpoint. In this way, the advantages of both approaches are combined – provided that roles and responsibilities are clearly defined.

Regardless of whether an internal or external evaluation is taking place, responsibilities within the project team should be clarified to determine who is responsible for coordinating the impact analysis. All lines of responsibility on the issue should lead to this person.

  • How YEA solved this dilemma

    In YEA’s case, each mentor-group supervisor collects all relevant information from mentors, students and teachers. The project’s management team prepares and reviews the data. A researcher from the local university’s education department is hired to oversee the evaluation with the help of masters students.

Who should be involved

Impact-oriented project management is fundamentally destined to fail if it is not actively supported by the organization’s executive or project-leadership team.

In addition to resources, defined processes and responsibilities are necessary. Ideally, individual lines of responsibility will be lead to a single unit or person with decision-making authority. However, this should not create a situation in which all others are no longer involved in the impact analysis. The purpose is to learn from the results together – thus, all staff members should be involved!

During the project-planning phase, you should already have identified the project’s stakeholders (remember who your stakeholders are). These should also be regularly involved in the impact-analysis process. This will assure the quality of the results, in part by preempting possible opposition.

  • Clarify the role of stakeholders in the impact analysis
    • What role do stakeholders play in implementation?Illustration Steuerrad
    • What information from the impact analysis is relevant to them?
    • Can they support, influence, slow or hinder the process? Do they have particular hopes, interests or fears with regard to data collection or particular questions?
    • Does this have consequences for the planned survey? For example, should other forms of participation be found?

    In order to answer these questions you might find the results of your stakeholder analysis helpful. Stakeholder analyses are conducted as part of a context analysis (Download PDF).

previous Timing
next Costs