Training pathways and wishes of examiners in the dual system

There are over 300,000 voluntary examiners in Germany¹. Their duties include setting examination tasks, conducting examinations and evaluating examination performances. This means that their training and good preparation for the role of examiner make an important contribution to quality assurance. This article presents the results of an online survey, in which BIBB questioned examiners on their qualifications, on the preparation received for exercising their function and on their support wishes.

Who are the examiners surveyed

Despite their significance, very few investigations have been conducted into examiners in the dual system and reliable data material is scarce. There has only been a single qualitative and quantitative study conducted by the Ministry of Education and Research (BMBF) into the "Acquisition of voluntary examiners in vocational education and training" that surveys examiners with regard to their routes into taking on the role, the amount of time required to exercise the function and their motives (cf. EKERT et al. 2011). The objective of the BIBB survey is to analyse further aspects, especially the current examination practice from the point of view of the examiners.

The Vocational Training Act (BBiG) regulates the appointment, composition and tasks of examination boards ². Of the 1,821 active examiners who responded to the BIBB online survey (cf. box), 42 percent state that they are representatives of the employer side only, 32 percent represent employee organisations and 16 percent are teaching staff at vocational schools. Ten percent have varying functions due to reasons such as belonging to more than one examination board.

Due to the fact that the access to the target survey group was obtained via an open online portal, no equal distribution of respondents across federal states and training areas could be achieved. 60 percent are examiners in chamber districts in Baden-Württemberg, 13 percent are in Bavaria and 7 percent from North Rhine-Westphalia and the Rhineland Palatinate respectively. The other federal states are represented by fewer than 100 respondents. This imbalance is also continued with regard to training areas. Over 80 percent of survey participants are examiners in occupations that come under the remit of "Trade and industry" ³. Examiners from the training area of "Housekeeping" have an appreciable representation of ten percent. Male examiners predominate, making up a proportion of 66 percent of respondents. The youngest examiner is 22, the oldest 76. Average age is 48.4 years.

The degree of experience of respondents varies widely. Both newly appointed examiners without experience took part alongside some who have been in the role for as long as 45 years. Average duration of examiner activity at the time when the survey took place is 12.8 years. This means that examiners are in their third period of appointment on average. The BMBF study achieves similar results with regard to gender distribution, age structure and duration of examiner activity (cf. EKERT et al. 2011, pp. 30 ff.).

What vocational qualifications do the examiners have?

Little is known about the vocational qualifications of the examiners. In this regard, the BBiG prescribes that the members of an examination board must be competent within the examination area and suitable for participation in the examination system (cf. BBiG § 40 Paragraph 1). "All persons who are in possession of relevant occupational knowledge and experience are deemed to be competent for the examination areas. [.] Persons who are able to demonstrate that they themselves have passed a final examination in the training occupation and have relevant occupational experience or have undertaken qualified continuing training should normally be assumed to be competent and suitable." (cf. GEDON/HURLEBAUS 2014, § 40 margin No. 11).

In order to identify competence in the various status groups, examiners were asked about the qualifications they held in connection with the occupation being examined. Multiple qualifications could be stated. The qualification most frequently stated is a vocational education and training qualification in the occupation being examined (998 mentions = 55%). This was followed by an advanced training qualification (823 = 45%) and a degree from an institute of higher education or University of Applied Sciences (727 = 40%). A vocational education and training qualification in a related occupation was stated by significantly fewer respondents (385 = 21%).

There is scarcely any difference between the qualifications of the groups of employer and employee representatives. More than half state that they hold a vocational education and training qualification in the occupation being examined, and 50% have a relevant advanced training qualification (cf. Figure 1). Significantly fewer respondents are in possession of a vocational education and training qualification in a related occupation or a degree from an institute of higher education/University of Applied Sciences, whereby the proportion of employer representatives is slightly above the proportion of employee representatives in this regard.

The only statistically significant differences between the status groups occurs in the case of the vocational school teaching staff. As expected, the institute of higher education/University of Applied Sciences degree is the most frequent qualification here and is stated by 84 percent of respondents. Only just over a third of teachers have a vocational education and training qualification in the occupation being examined. The proportion of advanced training qualifications is even lower.

This means that the requirement regarding competence is fulfilled via various training pathways. The examiners representing the employer or employee side draw their professional expertise from initial and advanced training, whereas the teachers predominantly emerge from higher education and are significantly less likely to demonstrate a vocational education and training qualification or participation in advanced training courses. The extent to which the course of higher education study relates to the occupation being examined was not surveyed.

The quality of examinations crucially depends on professional suitability as well as suitability for participation in the examination system (cf. BBiG § 40 Paragraph 1). "Suitability for participation in the examination system requires every member to be in possession of full personal and vocational teaching competence in order to enable preparation, execution and evaluation of the examination to be conducted in a way that is appropriate to the target group." (cf. GEDON/HURLEBAUS 2014, § 40 margin No. 12). Within the scope of this quantitatively aligned survey, it was not possible to investigate the relevant personality aspects of the examiners identified by GEDON/HURLEBAUS (communication skills, maturity, life experience, judgement, empathy). Academic research into personal suitability could provide important indications for the appointment of examination board members and for the design of training provision.

How do the examiners prepare?

In the BMBF study mentioned at the start of the article, the aim was for examiners to describe their experiences with new examination forms, how they adjust to changes emerging from updated regulations, how sufficiently they feel they have been trained and in which regard they see a need for improvement. The results indicate that the greatest importance is attached to networking with experienced examiners, reading specialist literature and attending an institute of continuing training (cf. EKERT et al. 2011 p. 38).

The BIBB survey adopted a broader focus. The aim was not only to identify how examiners adapt to new developments but also how they prepare for their activity in general terms and which different sources of information they use (regulatory instruments, observations, literature etc.) and at what time. The BIBB survey arrives at similar results to the BMBF study with regard to the significance of networking. 81 percent of respondents state that they regularly exchange views within the examination board and at a cross-cutting level (cf. Figure 2). Examination and training regulations also play an important role. Apart from a small number of exceptions, all respondents refer to them for preparatory purposes. Just under half use the regulatory instruments regularly, and approximately 40 percent of each status group do so if amendments to examination regulations take place, for example as a result of updated regulations.

The BIBB Board's recommendation for the training of examination staff called for the development of support materials. "Guides and training material should be provided to examiners, e.g. a basic examiner's folder containing information on the main legal and pedagogical principles." (BIBB Board 1990, p. 5). This requirement was addressed and implemented in various ways. The service sector union Ver.di, for example, has published "The examiner's handbook - a guide to examination practice in vocational training" (cf. REETZ/HEWLETT 2008). 36 percent of the examiners surveyed use specialist literature in order to stay up to date. Just under a third of each status group seek guidance from specialist literature when changes take place. Just under a quarter do so at the beginning of their activity as an examiner.

Examiner seminars have a particular significance for examiner training. They represent an opportunity to provide comprehensive information on rights, duties and contents relating to the topic of examinations and enable a large number of persons to be trained at the same time. They play a role for respondents, particularly at the beginning of their period as an examiner and when changes take place. Only a small number of respondents make regular use of this provision, and as many as 19 percent have never taken part in an examiner seminar. The reasons for this may be problems gaining release from work, financing or a lack of suitable provision.

Comparatively little use is made of the Internet. Respondents are most likely to turn to it for regular research or when changes have occurred. Nevertheless, 32 percent of respondents state that they do not use the Internet at all.

The aim of observations is to provide newly appointed examiners with insights into examination practice. As expected, they mainly have a part to play at the beginning of examiner activity, although only for 37 percent of respondents. The proportion of those stating that they have never observed at an examination is surprisingly high at just under a third.

To sum up, we may say that examiner preparation is diverse. Half of respondents use at least five of the sources of information to choose from. Only marginal differences can be ascertained between the various status groups in most cases. The only significant correlation is between regular use of regulatory instruments and status Group ⁴. Vocational school teachers are most likely to use training regulations and framework curricula. Examiners expressed a high degree of satisfaction with their preparation in overall terms. 23 percent assessed this preparation as "very good", whilst 49 percent felt it was "good". Only about four percent of respondents feel that their preparation is "bad" or "very bad⁵. Despite this high level of satisfaction with preparation, further information and training are stated to be needed.

What support do examiners wish to have?

In the BMBF study, examiners expressed a wish for more examiner seminars to achieve a standardised status of knowledge and for more opportunities to network with other examiners, boards and chambers. Areas where deficiencies were identified included the high amount of time required and the absence of computer-aided assessment programmes. In order to obtain better comparability of examinations, respondents wished to have uniform or standardised assessment criteria (cf. EKERT et al. 2011 p. 40).

The examiners who took part in the BIBB survey make approximately 650 open remarks regarding their training and support wishes that state very similar thematic areas. Some examiners are satisfied with provision and feel that they receive sufficient support and preparation. Others express a desire for compulsory introductory seminars, specialism-related training courses, refresher seminars and blended learning provision via the Internet. The examiners expect the competent bodies to pass on information regarding such issues as changes to the occupational profile or training provision on a regular basis. They also feel that cross-chamber networking with other examination boards should be organised and that specialist advice and literature should be made available. Evaluation and networking following the completion of examinations are also considered to be important for quality assurance. With regard to the assessment of examination performances, particularly in the case of practical examinations tasks and specialist oral examinations, the respondents call for standardised assessment forms with a clearly defined scope of expectations.

Examiners who alongside their examiner activity are also involved in the setting of examinations tasks perceive an additional requirement for supporting materials and training provision on the setting of such tasks. In order to be able to develop qualitatively comparable examination tasks, more materials and sample tasks to illustrate design criteria are needed. Particularly in the case of decentralised task setting, collections of tasks which examination boards could exchange with one another would be of assistance.

Further expansion of training provision and informal networking

The results presented here regarding the qualification and preparation of examiners focus on an extract from the examiner survey only. They show that professional suitability of the examiners is established via various kinds of training. Alongside professional suitability, however, personal suitability should also be systematically investigated and training provision aligned accordingly.

Although the examiners themselves state that they pursue diverse forms of preparation and feel that they are well prepared in overall terms, requirements for further development may be derived from the results.
An introduction to the multifarious tasks of an examiner and the general prevailing legal conditions should be standard procedure at the beginning of examiner activity. All examiners should be given the opportunity to take part in a seminar or training course or to undertake observations in order to prepare themselves for their work as an examiner.

In the event of changes to examination regulations, transfer to examination practice should be secured and examiners should receive occupationally related training on new forms of examination and new examination instruments. Considerable significance is attached to networking between stakeholders.
This can take place both locally and nationally. The Internet also offers an opportunity to access information flexibly and as required and to exchange experiences with other examiners. The BIBB "Examiners' Portal" has also been established for this purpose. "It acts as an information and communication platform for every aspect of the examination system and a chance to network with other examiners (cf. HENSGE/FRIEDLÄNDER/SCHNEIDER 2008)".

Further results from the survey are published via the portal.

MIRIAM MPANGARA
Research associate in the “Structural issues of regulatory work, auditing and implementation concepts” Division at BIBB

Translation from the German original (published in BWP 3/2014): Martin Stuart Kelsey, Global SprachTeam, Berlin