Year: 2022

A multidimensional and cross-disciplinary resource for local AE policy making

Mapping the Impact, Validation and Evaluation of Adult Education Policies has identified that EU wide AE Evaluation is highly fragmented and lacks common definitions and standards. This makes it very difficult for policy makers to control and assess the effectiveness of current AE policies. Plain coherent and straightforward policy development is needed. Any AE evaluation framework should ensure that cross organisational policy coherence occurs in vertical and horizontal terms. Vertical coherence requires that the different levels of government – national, regional, and local – follow common policy objectives and align systems of funding, accreditation and quality assurance. Horizontal coherence implies that there is understanding and coordination across policy areas within national or regional or local government.
The challenge for the wider adult learning community is to articulate targets and indicators for adult learning that capture the humanistic conceptions and practices of nonformal adult learning. The articulation of such a metric would ensure that this domain of adult learning can no longer be considered as ‘invisible’ by the state.
With reference to findings and evidences provided by partners, the CREATE 2 Evaluate ToolBox is conceived to ensure that local and regional policy makers from across Europe will be able to use the policy tools to better plan, design, implement and monitor Adult Education policies with a clear vision of sustainability of public funding in AE.
The CREATE and CREATE 2 EvaluateToolBox comprises a series of practical tools for policy makers to ensure evidence-based policy making and better align AE policies and programmes with local socio-economic development. Resources elaborated and published by partners are available in free, open and multilanguage version so as to ease and sustain the visibility and dissemination of training and education opportunities stemming from the model.
The ToolBox is structured in six different areas, each area includes specific resources and references that sustain local policy makers in better strategizing the alignment, consistent and coherence of local lifelong learning plans to EU long-term objectives and horizons.
Consistency of the objectives and outcomes
• Checklist to assess consistency of the objectives and outcomes of the AE trainings and programmes with the overarching strategies and agendas adopted by the EU

Programme creation at the policy/public administration level
• Checklist for providers: Evaluation of provider inclusion in Adult Education Policy Making
• Evaluation of Creativity in Adult Education Policy Making Matrix
• Focus Group Questions
• Survey for providers
• Survey for Public Administration Staff

Inclusivity of AE policies and availability of AE programmes
• Checklist to assess: Inclusivity of AE policies and availability of AE programmes in geographically excluded areas
• Dimensions of Diversity in Adult Learning

AE trainings and programmes delivery
• Individual Interview with participant: Methods, equipment and personnel in Adult Education and adjustment to the target group.

Value added stemming from the participation in AE
• Measuring the Wider Benefits of Adult Learning – Focus Group Guidelines for Learners and Training Providers
• Value added stemming from the participation in AE.

Continuity of programme evaluation and use of its results to improve AE policies
• Valorisation and Follow-Up to Evaluation. Some Success Factors
• Critical Success Factors: A Case Study
• Tips for policy makers seeking stakeholder engagement

All resources developed by the CREATE2Evaluate partnership are available via the official OPEN EDUCATION RESOURCE Platform of the project: www.create2evaluate.eu

Findings from the transnational analysis conducted by CREATE2’s partners

Evaluation and Valorisation are crucial steps in creating viable strategies with long-term benefits. Yet many AE policies on the local and regional levels are not systematically evaluated. This is partly due to policy makers lacking the appropriate tools. CREATE2Evaluate addresses this gap by designing a set of useful tools and templates to help AE policy makers and agents active in the AE policy processes evaluate existing strategies, draw actionable conclusions and design better strategies in the future.

The rationale for our project CREATE2Evaluate stems from the thorough investigation of the state of the art and the needs analysis that builds a multi-pronged approach:

1. robust investigation of the field through the analysis of relevant public policy documents at both national and EU levels that confirm how at present there are no effective means and tools to monitor and assess AE policies and programmes; and where those exist, they are used in a very fragmented way.

• direct experience of the partners: the proposal brings together AE providers (both formal and non-formal, both public and private), umbrella AE organisations, AE practitioners from research & academia, the third sector and the public sector (represented by local authorities tasked with policy formulation and monitoring & evaluation).

• the results of another Erasmus+ Project KA2 funded in Germany that was recently (2019) and successfully completed (final evaluation score from the National Agency of 85), the CREATE Project that developed tools for appropriate policy formulation in the domain of AE – in the 24 months of implementation and constant involvement of local policy makers, the clear need for monitoring and evaluation tools and mechanisms emerged.

During the first half of 2021, the CREATE2 Consortium finalised a very detailed cross-dimensional and transnational mapping activity aimed at identifying current methodologies, frameworks and/or models applied at local level for policy evaluation.

The final report is available for free by consulting the section of the CREATE2 Open Educational Resource Platform of the project.

Results, takeaways and findings suggested that the primary focus of current evaluation policy is quantitative outputs and student specific learning outcomes. Within any quantitative outcome evaluation partners identified a range of gaps and inconsistencies in target group statistical data, and in private and non-formal evaluation provision. Whilst significant qualitative evaluation is conducted in adult education centres according to project partners, the valuable data emerging remains inaccessible due to lack of transparent systematic overarching analysis and learner protection requirements. This makes it particularly difficult for policy makers to assess the effectiveness of their adult education policies.