Four steps are required to convert the evaluation into recommendations for action that will help you make your project more impact-oriented.
Step 1: Prepare and evaluate the data
First, the data are systematized and consolidated.
Quantitative data can be prepared in tables. For qualitative data collected through interviews or through questionnaires with open questions, the core statements are extracted and summarized.
Step 2: Check for plausibility
In the next step, the data should be reviewed for plausibility. The plausibility check is a small but very important step, because even small mistakes can badly distort an analysis.
While the data evaluation (step 1) should be performed by only a few people, it is useful to have significantly more people participate in the plausibility check. Those whose expertise will be useful in assessing the results should be involved. This includes project managers, project staffers, and external stakeholders with relevant experience.
Step 3: Analyze the data
The data evaluation in step 1 is purely descriptive. This means the results are presented on a 1:1 basis, as in "30 percent of the youths have an apprenticeship".
Step three is based on this description, and reflects the findings. The results are assessed and interpreted on a comparative basis (read more on that in the next chapter). Assessment in this case does not simply mean distinguishing between "good" and "bad"; rather, the results are placed in context so you can determine whether the project is still on the originally planned path.
Step 4: Draw conclusions, derive recommendations for action
Even the most extensive data analysis is of no benefit if the findings remain unused. For this reason, the findings should be widely communicated and discussed. As in the other portions of the impact-oriented management cycle, participation is here, too, the key to success.
The findings should be discussed with the relevant stakeholders and, when appropriate, also with subject-area experts. At the end, the findings are channeled into practical recommendations that have direct impact on project activities.
In the end, the findings flow into practical recommendations, which are used to adapt the project.
For YEA, a table depicting the status quo after the fourth step would look as follows:
| Step 1 |
Step 2 | Step 3 | Step 4 | |
|
How many youths found a vocational training position? How does their school performance develop? |
Were the data subject to a quality check? | Are objectives and plans congruent? | How can we explain these findings? Can we detect systematic interrelations? |
How can we change that? How can we adjust the project to improve? |
|
50% of the participants were successful in finding a training position. Overall, the youths achieve a C- average. |
The data were checked on the basis of the 2 eye principle; results were discussed. |
The project did not achieve its objectives in full (help 70% of the youths in finding a vocational training position). In addition, participants’ academic performance improved only slightly. |
Youths, who performed below-average in school were not able to find a vocational training position. One can assume that good educational performance is a decisive factor in finding a job after school. |
The youths need additional support. The offering of training modules needs to be extended. YEA offers a new training module. |
What to do in the event of unsatisfactory results?
It may be that your results don’t meet your expectations. But maybe they’re not really so bad? Closely examine the data underlying the evaluation for deviations, as there may have been inaccuracies in the evaluation process. Systematic monitoring during the project should prevent you from being surprised by bad results.
If this doesn’t help, place the result in the project’s overall context – the problem tree might help you structure the problemhttp://www.social-impact-navigator.org/planning-impact/needs/social-problem/. maybe the initially disappointing results don’t seem quite as bad when placed in the context of the big picture?
If this doesn’t ease your concerns, inform all the stakeholders what’s going on. Make it clear to them why results are falling short of expectations, and what countermeasures you’re planning. Ideally, your funders will support you in the subsequent learning and improvement process.
-
YEA – and how they evaluate
YEA’s evaluation shows that the youths’ lack of social skills is a serious obstacle in their transition into vocational training. As a result, the project’s managers and funding foundation decide to design an additional project module focusing on training social skills.