Reflection reinforces effectiveness of analyses of potential

Quality standards for analyses of potential

Concepts funded within the scope of the Vocational Orientation Programme (BOP) must conform to quality standards set out by the Federal Ministry of Education and Research (BMBF) for conducting analyses of potential. These standards originate from the context of vocational integration support and have been slightly adjusted for the purposes of the BOP, i.e. for execution in lower years at general schools. The stipulations relating to BOP analyses of potential underwent only small-scale amendment in the period from 2010 to 2022. In a parallel process between 2011 and 2017, modes of implementation based on the BMBF quality standards but with some variations were developed as part of agreements made between the Federal Government and the federal states. These variations include a shortening of the analysis of potential from its original length of two days down to one day, execution by teaching staff rather than by education and training providers, and the expansion of computer-aided test procedures. The federal state which has moved furthest away from the largely action-oriented and diagnostic nature¹ of the BOP analysis of potential is Mecklenburg-Western Pomerania, where a cross-year group cohort model featuring a reflection-oriented approach has been developed (cf. Kalisch/Pilz/Prill in this issue).

ISPA – design of the study and key results

In 2018 and in light of these developments, the BMBF decided to conduct a research investigation into the BOP quality standards for analyses of potential via the ISPA Intervention Study (cf. Information Box). The study’s conceptual design has its foundations in findings from VET research, according to which the main focus of an analysis of potential performed in the early stages of vocational orientation should be on familiarising young people with the topic and on encouraging initial and reflective engagement with their own abilities and interests. For this reason, the investigation centred on effects of three dimensions of self-reflection and on five dimensions of the motivation to deal with career choice questions (cf. Figure 1).

Securing the sustainability of developments initiated

In each of the four groups, all dimensions of self-reflection and of the motivation to address career choice exhibit a higher level directly after the intervention. The brief period of time which elapses between the pre-test and follow-up test means that third-party influences may be disregarded. This before-and-after comparison therefore shows the general effectiveness of the analyses of potential and of the reflective discussions. The effects measured in this way correspond with the specialists’ and pupils’ assessments with regard to the impacts of the appraisals, e.g. gaining a new self-view or being encouraged to become actively involved in vocational orientation. Nevertheless, many of these impetuses thus instigated peter out in the subsequent weeks leading up to the third follow-up questionnaire (see Figure 2 for the example of the dimension of self-knowledge).

This result proves the importance of addressing initiated developments again later in order to secure their sustainability.

Offering space for reflection

All three analyses of potential concluded with individual reflective discussions, which were deemed successful if

• they had been implemented in a way which adhered to the procedure;
• they had been successful in terms of communication (e.g. it had been possible to establish trust) and
• they had encompassed certain content results (e.g. further reflection had taken place on occupational preferences or strengths and consideration had been given to possibilities for developing these).

Success of the discussion correlates significantly with the desired effects of the analyses of potential. Whether pupils come to find that vocational orientation is relevant to them is strongly dependent on whether the discussion has produced certain specific outcomes for them. If a discussion is conducted in a communicatively successful manner, this will in turn facilitate the achievement of such results. In overall terms and in various places, the study confirms that reflection processes are highly significant within the context of vocational orientation provision.

The study further indicates how important it is for occupational interests to be addressed and illustrates the role played in later working life by the skills and strengths being discussed, even though making a career decision is not an objective of the analyses of potential. The more intensive the consideration of occupational preferences in the reflective discussion, the stronger the development of motivation to deal with the topic will be. Self-knowledge will also increase.²

Differing effects depending on the type of analysis of potential

A pattern is revealed across the eight aspects of self-reflection which formed the object of investigation. “In the case of the action-oriented analysis of potential, the intended developments were below average. They tended to be strongest in the instance of the longer reflective discussion without an analysis of potential” (Sommer/Rennert 2020, p. 40). It must then be concluded that the form in which the action-oriented analysis of potential was investigated is not the best possible option for fostering motivation and self-reflection. It should be noted here that the action-oriented analysis of potential differs from other forms of intervention both with regard to the type of tasks and in respect of a lower scope of phases of reflection and of occupational references important to effectiveness. Nevertheless, it may be noted that self-reflection developed more positively than in the control group. It also proved possible to ascertain gender-specific effects. The action-oriented analysis of potential showed a significantly stronger change amongst male pupils in the dimensions investigated than amongst female pupils.

New standards for analyses of potential

The final report of the Intervention Study was published in the spring of 2020. The results were then discussed and evaluated by a group of experts from the fields of policy making and practice. This process progressed to the formulation of recommendations for structuring analyses of potential. Initially, the main finding and important policy making message of the Intervention Study is that the combination of analysis of potential and reflective discussion contributes effectively towards stimulating career choice-related self-reflection and motivation, at least in the short term. The differences between types of intervention investigated are also not very large. They exhibit various strengths and weaknesses, and their impact is heavily influenced by pupils’ individual characteristics. As a consequence, it is technically justifiable for federal states to adopt different structural routes under the proviso that phases of reflection, and 1-to-1 discussions in particular, are afforded a high degree of importance.

However, the results also show that improvement is needed with regard to the BOP analysis of potential because not all pupil groups benefit from action-oriented tasks in the same measure. In fact, they tend to suggest that the strengths of the different types of intervention should be combined. For this reason, the BMBF’s quality standards for conducting analyses of potential have been rigorously revised within the scope of the BOP.³

The new standards have been in force since 1 January 2023 and require far-reaching adjustment of the existing analysis of potential concepts in the framework of the BOP. The strengthening of the pedagogical objective necessitates a significant rethink by the specialists delivering the provision. The aim in future is for a balanced mix of tasks to be deployed. Employment-oriented tasks have been reduced. At the same time, biography-related tasks have been included and proportions of reflection have been expanded both in terms of time and quality.

Transfer of results to practice has already also been initiated in other places. Vehicles for this include

• embedding reflection modules in “Profil AC” analysis of potential procedures,
• further development of “Mission ICH” in Mecklenburg-Western Pomerania and
• implementation of the pilot project “Reflexionsgespräch” in Schleswig-Holstein.

There will need to be scrutiny as to what further consequences will arise from the results or from the new quality standards, e.g. including for federal-state specific modes of implementation of the analysis of potential.

Carolin Kunert
Academic researcher at BIBB 

Dr. Jörn Sommer
Divisional Head at InterVal GmbH Berlin
 

Translation from the German original (published in BWP 2/2023): Martin Kelsey, GlobalSprachTeam, Berlin