topics of the Austrian CAP Strategic Plan 2023-2027 and the Rural Development Program 2014-2020
As in the previous funding periods, evaluation will play an important role in the implementation period of the CAP Strategic Plan (GSP) 2023-2027. The legal framework for the monitoring and evaluation system is essentially provided by the CAP Strategic Plan Regulation (EU) 2021/2115 and its Implementing Regulation (EU) 2021/2289 (content of the GSP) and Implementing Regulation (EU) 2021/2290 (calculation of common output and result indicators). In accordance with the legal requirements, the evaluation focuses on the impact of the GSP and its relevance (i.e. fitness for purpose), effectiveness (i.e. extent to which objectives are achieved), efficiency (i.e. cost-effectiveness), coherence and Union added value (i.e. the added value of the GSP achieved at Union level).
In the evaluation plan for the GSP Austria 2023-2027 (BML, 2023), the 45 national needs of the Austrian GSP 23-27 along the nine specific objectives and the cross-cutting objective (QZ10 - qualification and capacity building) were combined with the 17 evaluation elements (e.g. ‘competitiveness of agricultural enterprises’, ‘climate protection’, ‘gender equality and social inclusion’) and 5 evaluation criteria (relevance, effectiveness, efficiency, coherence, Union added value) to identify key substantive issues (evaluation topics). The 36 evaluation topics identified as the basis for the overall evaluation of the GSP 23-27 (evaluation during the implementation period and ex-post evaluation) were worked out in detail. An evaluation question was defined for each evaluation topic, which was further specified in terms of content using success factors. All relevant interventions and associated indicators (output, result and impact indicators) as well as national indicators are linked to this. In connection with the nine specific objectives and the cross-cutting objective, there are 44 outcome indicators (each specific intervention of the GSP must be assigned to at least one outcome indicator) and 29 impact indicators.
The evaluation of the Austrian GSP 23-27 requires that each of the three general objectives (Objective A - promoting the agricultural sector and food security; Objective B - promoting environmental protection, biodiversity and climate protection, contributing to the achievement of EU environmental and climate-related objectives; Objective C - strengthening the socio-economic fabric in rural areas) of the Common Agricultural Policy (CAP) and each specific objective, including the cross-cutting objective (QZ10) and each specific topic (e.g. environmental and climate architecture, AKIS) is evaluated at least once during the implementation period. (e.g. environmental and climate architecture, AKIS) is evaluated at least once during the implementation period, whereby the impact of the interventions on the individual specific objectives must be assessed. According to the evaluation plan, the period for this evaluation can therefore be limited to the years 2025-2028; the ex-post evaluation of the Austrian GSP 23-27 is planned for the period 2029-2031.
In terms of timing, the evaluation of the GSP 23-27 will overlap with the ex-post evaluation of the Rural Development Programme 2014-2020 (RDP 14-20) and with the programming of the GSP 2028-2035.
Objective
The objectives of the long-term project are i) the scientific monitoring of the overlapping evaluations by providing support and scientific advice to the Federal Ministry of Agriculture, Forestry, Environment and Water Management (Department II/1 and intervention managers) in the preparation of the evaluation, by identifying necessary evaluation studies, etc.; ii) the specification of necessary and review of existing data, which - together with suitable methods - are necessary to answer the evaluation questions, as well as the identification of data gaps, taking into account experiences from previous evaluations; and iii) the evaluation of selected evaluation areas.
The project consists of several work packages in which the evaluators of the Federal Agency will contribute to selected evaluation topics. The work package leaders are each responsible for the coordination and implementation of the evaluation activities relating to their work packages. The main tasks of the management of the overall project at the Federal Agency are, for example, the coordination of the evaluation activities across the work packages, the continuous exchange with the BML regarding the organisation of the evaluation activities and the exchange with the project team of the project BAB 073/24 ‘GAP Data Pool - Data Modelling and Data Management’.
Status of the overall project
Together with Department II/1, the evaluation topics and areas in which the Federal Agency will be involved in the evaluation of GSP 23-27 were defined in 2024: The Federal Agency will coordinate the evaluation of General CAP Objective B (‘Promoting environmental protection, biodiversity and climate change mitigation, contributing to the achievement of EU environmental and climate-related objectives’). In addition, the Federal Agency will evaluate the following areas: Specific Objective 02 (Increased market orientation and competitiveness of farms), Specific Objective 07 (Increasing the attractiveness for young farmers; facilitating business development), the evaluation topic Q2 ‘Qualification and competence building, AKIS’, the evaluation topic ‘Added value of LEADER’ and the evaluation topic ‘Gender equality’.
In connection with these areas and beyond, the BML was continuously supported in the preparation of the evaluation of the GSP 23-27, e.g. by identifying data gaps, determining the need for evaluation studies, providing content/scientific advice - including in the further development of the existing forms on the evaluation topics or by participating in workshops.
Planned work 2025
The activities required in connection with the evaluation of the GSP 23-27 during the implementation period will be coordinated, planned and carried out in cooperation with the BML.Examples include finalising the evaluation topics (forms), monitoring evaluation studies, participating in coordination meetings at the BML on the relevant evaluation topics, specifying the required data, etc.In addition, the Federal Agency will carry out the evaluation as part of the work packages, e.g. answering the evaluation questions using various analytical methods.
For the evaluation of GSP 23-27, there will be a continuous exchange with the project team of the project BAB 073/24 ‘GAP Data Pool - Data Modelling and Data Management’ in order to coordinate the development of the database and the data requirements on the part of the evaluators.
In 2025, activities will begin in connection with the ex-post evaluation of the RDP 14-20. The focus here will be on updating the reports on the mid-term evaluation of the RDP 14-20 from 2019 on the basis of current data, our own evaluations and analyses, results from the literature, etc. This work will be carried out after a pending agreement with the BML on details of activities and scope within the respective work packages.
Further details can be found below in the descriptions of the individual work packages:
Work package ‘Education and counselling, AKIS’
(Work package lead: Sarah Lappöhn; collaboration: Sophie Pfusterschmid)
In the evaluation plan for the GSP 23-27, the evaluation topic ‘ET-Q2 Qualification and competence building, AKIS’ was defined, which concerns the cross-cutting objective QZ10, AKIS (Agricultural Knowledge and Innovation Systems) and the digital strategy as an integrated topic. The evaluation question is: ‘To what extent has the GSP contributed to improving the national AKIS on the basis of a strategic approach (Ch 8 GSP) and supported the transfer and application of knowledge, innovations and digital solutions in the agricultural sector and for actors in rural regions, and how can these impacts be assessed in terms of effectiveness, efficiency, relevance/purposefulness, coherence and EU added value?’
To answer this question, a robust data set is required for the directly assigned interventions (77-03, 77-06, 78-01, 78-02, 78-03) as well as for other relevant interventions (CAP network, LEADER, etc.). Experience from previous evaluations showed that there were insufficient data sets available for the evaluation, particularly for the educational measures.
In order to improve the data situation for the evaluation, data requirements and data gaps should be clarified at an early stage and appropriate precautions taken to ensure data availability.
Education and counselling in particular are a very complex evaluation project due to their contribution to the various objectives, some of which interact with each other. Data relevant to the evaluation should be identified in advance and data gaps closed so that the funded activities can be mapped and evaluated as part of the ex-post evaluation of the LE 14-20 (priority areas 1A, 1B, 1C) and the improvement, further development, capacity building of the AKIS and the achievement of the impact objectives can be presented in the evaluation of the GSP 23-27 on this basis.
The first step will be data research and a review of relevant literature on AKIS as well as the identification of data gaps based on experiences from the mid-term evaluation of the LE 14-20. Further work steps include research work at funded educational institutions regarding the availability of data on participants in educational measures in connection with specific interventions. In a further step, this data from the 2014-2020 RDP period will be linked to INVEKOS data via the establishment number, which will serve as preparation for the ex-post evaluation of RDP 14-20. Depending on the availability of the data, attempts will be made to analyse selected issues such as the contribution of knowledge transfer measures to improving farmers' knowledge of climate change mitigation and adaptation or awareness raising in the areas of biodiversity, soil and water conservation and entrepreneurial skills.
Work package ‘Competitiveness and farm succession’
(Work package management: Christoph Tribl, Josef Hambrusch; collaboration: Dieter Kömle, Martin Weigl, Julian Zeilinger)
The aim of the work package in relation to GSP 23-27 is to evaluate Specific Objective 02 (‘Increased market orientation and competitiveness of farms’) and Specific Objective 07 (‘Increased attractiveness for young farmers; facilitation of business development’).
Specific Objective 02 is assigned to General Objective A (‘Promotion of the agricultural sector and food security’). In the evaluation plan of the GSP 23-27, two evaluation topics are assigned to Specific Objective 02, each with its own evaluation questions:
- ET_A01_3: ‘Competitiveness of agricultural enterprises and the sector’ with the evaluation question: ’To what extent has the GSP contributed to increasing the competitiveness of agricultural and forestry enterprises and how can these impacts be assessed in terms of effectiveness, efficiency, relevance/utility and coherence?’ (Evaluation element ‘Competitiveness of agricultural enterprises’).
- ET_A0203_4: ‘Market orientation’ with the evaluation question ‘To what extent has the GSP contributed to increasing the market orientation of agricultural and forestry enterprises through the production of processed quality products and improved vertical cooperation along the value chain, and how can these impacts be assessed in terms of effectiveness, efficiency, relevance/suitability for purpose and coherence?’ (evaluation element ‘Increased orientation’). (Evaluation element ‘Increased focus on the market’). Specific objective 07 is assigned to general objective C (‘Strengthening the socio-economic fabric in rural areas’). An evaluation topic with two evaluation questions is assigned to this specific objective in the GSP 23-27 evaluation plan:
- ET_C07_6: ‘Farm succession and farm development’ with the evaluation questions
‘To what extent has the GSP contributed to increasing the attractiveness of agriculture for young farmers and to facilitating intra-family and extra-family farm succession and how can these effects be assessed in terms of effectiveness, efficiency, relevance/suitability for purpose and coherence?’ (Evaluation element ‘Generational transfer of farms’)
‘To what extent has the GSP contributed to supporting the diversification of farms? ‘To what extent has the GSP contributed to supporting farm diversification and how can this impact be assessed in terms of effectiveness, efficiency, relevance/utility and coherence?’ (Evaluation element ‘Business development’)
With regard to the ex-post evaluation of the RDP 14-20, this work package is thematically linked to Priority 2 ‘Improving the viability of farms and the competitiveness of all types of agriculture in all regions and promoting innovative agricultural techniques and sustainable forest management’ with the two priority areas (SPB):
- SPB 2A ‘Improving the economic performance of all farms, supporting farm restructuring and modernisation, in particular with a view to increasing market participation and orientation as well as agricultural diversification’;
- SPB 2B ‘Facilitating access of appropriately qualified farmers to the agricultural sector and in particular generational renewal’.
Details on the activities and scope of the evaluation of the GSP 23-27 and the ex-post evaluation of the RDP 14-20 as part of this work package will be agreed with the BML in the course of 2025.
Work package ‘Environment and climate’
(Work package leader: Julia Niedermayr; collaboration: Christoph Tribl)
With regard to the GSP 23-27, the objective of this work package is the overarching evaluation of the General CAP Objective B (‘Support and enhance environmental protection, including biodiversity, climate change mitigation and contribute to the achievement of the Union's environmental and climate-related objectives, including its commitments under the Paris Agreement’).
According to the evaluation plan of the GSP 23-27, the following two evaluation questions are to be answered for the evaluation of the evaluation topic ‘Overarching assessment of general CAP objective B’ (ET_B):
- ‘To what extent has the GSP contributed to increasing environmental and climate impacts and contributing to the achievement of set environmental and climate objectives and how should these impacts be assessed in terms of effectiveness, efficiency, relevance/utility, coherence and EU added value?’
- ‘What environmental and climate-related interactions exist between the interventions of the GSP and how are possible negative interactions for the environment and climate mitigated or offset?’
According to the evaluation plan, the following priority objectives (PAs) are assigned to general CAP objective B and are relevant for the overarching evaluation:
- SC04: Promotion of climate protection, adaptation to climate change and sustainable energy
- SD05: Sustainable management of natural resources (water, soil, air) and reducing dependence on chemicals
- SC06: Contributing to halting and reversing biodiversity loss, enhancing ecosystem services and conserving habitats and landscapes
The evaluation of General CAP Objective B is therefore based on the evaluation results of the evaluators of these SCs and can only take place once the respective evaluation reports are available. The evaluation team coordinates with the evaluators of the SCs as required and supports the BML in the formulation and monitoring of evaluation studies.
With regard to the ex-post evaluation of the RDP 14-20, the work of the evaluation team ties in with the activities for the preparation of the 2019 mid-term evaluation report. This includes coordinating the evaluation of the following priority areas (SPB) for selected project types, in particular those of the ÖPUL programme:
- SPB 4B ‘Improving water management, including the use of fertilisers and pesticides’
- SPB 4C ‘Prevention of soil erosion and improvement of soil management’
Equality’ work package
(Work package lead: Theresia Oedl-Wieser; collaboration: Erika Quendler, Katrin Hofer, N.N.)
The evaluation question on the evaluation topic ‘Equality’ (ET_Q1) in relation to the GSP 23-27 is: ‘To what extent has the GSP contributed to improving equality between women and men and the reconciliation of work and family life, to supporting social and political participation and social diversity, and how can these effects be assessed in terms of effectiveness, efficiency (administrative implementation), relevance/suitability for purpose, coherence and EU added value?’ In addition to this general question, specific equality objectives are formulated for selected measures (currently 13), which are to be answered by the evaluators in the course of the accompanying evaluation. The evaluation results can then be incorporated into the answer to the gender equality question. Furthermore, the evaluation team will support the BML in the formulation of accompanying research studies.
With regard to the ex-post evaluation of the RDP 14-20, the extent to which a gender equality evaluation will be carried out will be clarified at the beginning of 2025.
LEADER/CLLD’ work package
(Work package leader: Ingrid Machold; Collaboration: Theresia Oedl-Wieser, Katrin Hofer, N.N.)
The evaluation question on the evaluation topic ‘Added value of LEADER’ (ET_C08_1) in relation to GSP 23-27 is: ‘To what extent has the LEADER/CLLD [Community-Led Local Development] implementation model under the GSP contributed to local development work, achieved an improvement in social capital in the local environment as well as local governance and contributed to positive change in socio-economic dynamics in local areas through projects and initiatives with high acceptance and local/regional benefits and how can these impacts be assessed in terms of effectiveness, efficiency (administrative implementation), relevance/utility, coherence and EU added value? ’ In addition, the evaluation team will support the BML in the formulation of accompanying research studies.
As part of the ex-post evaluation of the RDP 14-20, it is planned to analyse the LEADER/CLLD measure based on the procedure used for the mid-term evaluation of the RDP 14-20 in 2019.
Work package ‘Mobility and broadband infrastructure’
(Work package leader: Oliver Tamme; Collaboration: N.N.)
For the ex-post evaluation of the RDP 14-20, it is planned to analyse certain measures in connection with mobility and broadband infrastructure based on the procedure for the mid-term evaluation of the RDP 14-20 in 2019. Details on this and possible evaluation activities as part of the evaluation of the GSP 23-27 will be clarified together with the BML in the course of 2025.
Timetable
Project start: 01/2024
Project duration: CAP Strategic Plan 2023-2027 accompanying long-term project