Four thematic focus groups were organised online as part of T3.2 Tool to extract contextual specifications from various settings and initiatives and explore their potential impact on the evidence-based evaluation model. The input from the workshop was meant to feed the development of the Tool. The major principles and components of the model were put in the core of the discussion to understand the current state of the art in planning and conducting evaluations in different sectors.
Each thematic group focused on a common type of PVE/CVE and deradicalisation initiatives: law enforcement ad-hoc actions, policies and strategies, Exit programs and educational initiatives. Through these thematic groups, each level of PVE/CVE work was considered: primary, secondary and tertiary. The initiatives and programmes that were considered also touched upon specific target groups (women, children, as well as former and present prisoners) and focused on religion-motivated, right-wing and left-wing extremism.
1. Thematic workshop for evaluation of LEA ad-hoc actions (interventions)
The workshop took place online on 19 April 2023 and it was organized by the University of Helsinki and Hellenic Police. The INDEED LEA partners (Polish Gdansk Police, Belgian Local Police Rupel, Spanish Madrid Municipal Police and Latvian Riga Police) took part in the workshop to discuss the practical aspects in relation to the evaluation of police action. For this workshop, the case (scenario) was developed to facilitate the guided discussion. The case included information on radicalisation-based violence. It primarily described the response provided by the police, including its cooperation with other actors. The case helped discuss the evaluation of ad-hoc interventions by law enforcement in detail and see how the evidence-based evaluation model fits to this context. The thematic group covered the organisational and operational contexts in different EU MS: Belgium, Greece, Latvia, Spain and Poland to underpin similarities and differences in legal frameworks and organisational procedures that might impact evaluation processes.
The main takeaways are:
- In LEA settings evidence-based evaluations are still quite rare, although some of its elements are present. In the majority of cases, LEA actors are involved in ad-hoc internal evaluations due to data-related restrictions. They rarely participate in external evaluations conducted by third parties. Therefore, on the one hand, there is interest and a need in developing a unified methodology for internal evaluations. On the other hand, there is a need in involving external actors to promote and safeguard impartiality in the evaluation outcomes. For this, there is a clear demand in increasing multi-agency partnerships to allow for smooth cooperation between different agencies, also for the purpose of evaluation.
- Both formal and informal communication and ties with external stakeholders and partnerships in LEA settings vary from country to country. However, even when the multi-agency partnerships are formalised, the issues of multi-agency cooperation remain one of the main priorities in the domain of evaluation.
- The lack of funding for evaluation is one of the main obstacles to planning and conducting evaluations in LEA. Another obstacle is related to the lack of time. This is also due to the fact that many actions usually require rapid response. The decisions need to be taken fast, as part of the formal procedures that LEA actors are involved in.
- LEA actors might seek support in planning and conducting evaluations, or at least in the organisation of some related activities. In other words, some help might be needed with a few methodological aspects regarding the data collection process based on the Research and Ethics requirements, as well as the data analysis itself.
- The issues of data protection and Ethics remain the main challenge in designing evaluations and these are resolved differently in different countries based on national legislations and organizational protocols. Taking Ethics and GELSA into account in the evaluation of ad-hoc actions is the way to reconsider organisational traditions, protocols and practices. A gender-sensitive approach needs more attention to lead such practices.
2. Thematic workshop for evaluation of policies and strategies
The thematic workshop discussing the evaluation of policies and strategies took place online on 22 May 2023. It was organised in close cooperation between EFUS (France) and the University of Helsinki. The participants included policy-makers and decision-makers from different governmental institutions in the EU Member States, such as Germany, Portugal, Bulgaria, Spain, and Finland as well as EU actors, such as European Commission. The discussion touched upon primary and secondary prevention policies and strategies. For this workshop, a policy example case was developed by the University of Helsinki to facilitate the discussion. The case presented a scenario for the evaluation of the national programme featuring a multi-agency approach.
The main takeaways are:
- Evidence-based evaluation of national policies and strategies in P/CVE is still rare all over the EU, although the necessity for this is discussed and the need for evaluation is recognised.
- The EU institutional level neither evaluates P/CVE policies, although monitoring activities that may potentially contribute to evaluation do take place.
- The mechanisms for the involvement of various stakeholders are often already created, however, in some cases in some countries, certain partnerships still need to be developed.
- The collection and interpterion of incoming data might be a challenge in the sector.
- Although (mostly) policy- and decision-makers have internal expertise for conducting evaluations, the evaluations still do not take place for many reasons, such as, for instance, a lack of resources, scarce evaluation culture and complexities of the filed.
- Working along the Ethical standards is challenging from the point of view of time-consuming procedures, as different authorities have their own regulations and practices.
- Understanding of the context and circumstances where the initiative is implemented also requires time and resources.
3. Thematic group on the evaluation of NGO-led Exit programs
The hybrid thematic workshop on evaluation of mentoring and Exit programs took place on 15 May 2023. It was organized by TRANSFORM in cooperation with the University of Helsinki. Practitioners and evaluators coming from Sweden, Finland, Germany, Italy, Slovakia and Romania took part in the workshop. The participants represented mainly NGO experts involved in secondary or tertiary prevention as well as evaluators of related programmes.
A case of an Exit program was developed for this workshop to set up the scenes for the discussion. The case focused on the example of the programme focused on disengagement from right-wing extremism.
The main takeaways are:
- There is a clear need for evaluation of Exit programs in PVE/CVE sector that is stemming from the fact that many of those are sponsored by different agencies. The sponsoring bodies take their decisions on further funding based on the outcomes of those programmes. In some cases, pressure to evaluate might also serve as a demotivating factor.
- The challenge for evaluation lies in the lack of resources, such as finding and time. Evaluation, if not built in the program design, is often unfunded.
- The programs are structured in a variety of ways, which makes evaluation complex.
- These programs are also not always results-oriented but process-oriented, what might create a challenge for the evaluation field.
- The programs are mainly centered around individual cases, where sensitive approach is needed also in relation to evaluation.
- Data collection, data storage and data use issues might create a challenge from the point of view of data privacy and Ethics. And mitigating these challenges might take a lot of resources.
4. Thematic Workshop on evaluation of educational long-term programmes
The takeaways from this workshop can be found here.