If information is to be used to its full potential, it must be collected at the right time. Of course, this depends on when data can be collected at all. Moreover, the questions posed and the focus of analytical interest also make a difference.
- When is the right point in time to collect data? – Is it a continuous process or will you only collect data when having reached a milestone or after having finished a certain activity? These questions must be answered for every project individually. Of course, the definition of an adequate point of time to collect the data often depends on exogenous factors, as there are only certain windows of opportunity that allow for (valid) data collection.
- Also, you should always be certain about what you seek to find out and which data will help you retrieve this information (and which will not).
At the level of processes and OutputOutput describes the countable offerings and products of a project as well as their utilization by the target group. Outputs form the basis for a project to have a desired result. Yet, they do not describe results per se. outputs (products and services), the question for project managers is above all whether the project’s offerings are being carried out according to the predetermined schedule and financial plan. The data should be regularly collected (weekly, monthly or quarterly, depending on needs) as a part of the monitoring process. This enables swift interventions and countermeasures to be implemented as required.
At the level of results ( OutcomeOutcomes describe the effects of your project at the level of the target group. They are a core part of the logic model and are subdivided into three stages: changes in knowledge, attitudes and abilities (level 4); in behavior (level 5); or in the living situation / status of the target group (level 6) outcomes and ImpactImpact (as a level of result / effect / change) covers the social or economic changes at the societal level (level 7). Since reference to society as a whole is neither reasonable nor possible, the changes usually refer to a defined part of society or a region. impacts), information on medium- and long-term results is gathered instead. The data collected here is generally more complex than is true of the outputs. Thus, it is analyzed less frequently. However, even at this level, some information is relatively easy to collect on a regular basis, such as changes in students’ grades.
-
A continuous process ...
Data collection shouldn’t take place only at the end of a project. This is because it’s important to be able to learn from the findings in order to enable adjustments during the project. For this reason, social-impact analysis should be a continuous process.
When you evaluate is totally dependent on your individual case.
The evaluation at the beginning of the project serves the needs assessment and context analysis. This enables the needs of the target group to be compared with the project objectives and the planned approach.
Through continuous monitoring and interim evaluations, you can determine whether the project is on the right track. This offers the opportunity to make adjustments as necessary.
Of course it makes sense to take stock once a project has been completed. However, aside from its utility for final reports to funders, an evaluation at the end of a project is often of little use.
By the time the results are available, project staff members are often already busy with other projects. Moreover, afterward it is no longer possible to respond to the findings. Thus, if you have the opportunity, you should try to make an interim evaluation.
At the end of a project comes the final evaluation. Usually, the whole project is assessed here. Of course some projects have no planned end point. In this case, certain points should be identified at which a comprehensive evaluation can be carried out – for example, when a new project component is to be developed, or the project is to be transferred to other locations.
An ex post evaluation takes place some time after the project’s completion. Its goal is to find out what results have been produced by a project even after its end.
An example of an ex post evaluation would be a follow-up study that seeks to determine what has become of the young people that completed the project some time ago.
Independently of the issue of timing, you should think about an evaluation if the monitoring data shows significant deviations from your plan (for more on this, see "Develop indicators").
-
The YEA impact analysis
YEA collects data systematically. Most importantly, this involves documenting the services provided, such as the number of tutoring hours carried out.
At the level of results, changes in the participants’ school grades are recorded.
To determine how project participation affects youths’ social behavior, an evaluation is carried out after two-and-a-half years. The results help further develop the ongoing project, and enable the successor project to be designed optimally from the beginning.