AI-powered language generators in vocational final examinations – information and discussion urgently needed!

Barbara Schürger

Since its release in November 2022, the use of ChatGPT has been gaining popularity. The general consensus is that AI-powered language generators are also expected to impact work and education. Experts discuss their intentional use and/or misuse in exams primarily in the general education and university sector. But what about examinations in vocational education and training? This article analyses the points of contact between the software and VET examination instruments and discusses possible future developments.

Artificial intelligence (AI) for text generation

AI-based language generators such as ChatGPT act as virtual conversation partners. Based on their training with extensive text databases, they generate answers and dialogues in natural language. By using AI and machine learning, they recognise patterns in large amounts of data and use probabilities to generate the best possible answer based on context and learned knowledge (cf. Ouyang et al. 2022; Heaven 20231).

Chatbots such as ChatGPT (Generative Pre-Trained Transformer) can not only efficiently create, edit, summarise and evaluate texts, but also write program codes, for example. With the upgrade of the underlying language model GPT 3.5 to 4.0 in March 2023, performance and accuracy have been improved and, in addition to text, handwritten documents and images can now also be used as input (cf. Hahn 20232).

However, language generators also have weaknesses: problems with longer conversations, logic, factuality and timeliness, as well as risks due to a lack of transparency, security and confidentiality of data (cf. Albrecht 2023; BSI 2023).

According to a survey commissioned by the digital association Bitkom, 70 percent of companies surveyed expect that AI for text generation will be part of everyday working life in the future and that more knowledge about AI must therefore be taught in schools and training.3 In April 2023, the Education Committee of the German Bundestag also called for the acquisition of appropriate skills for dealing with AI.4 In the consultations on the EU’s Artificial Intelligence Act (AI Act), AI systems used in general or vocational education are expected to be classified as high-risk, among other things (cf. European Commission 2021). It requires AI-generated texts to be labelled as such. Digital watermarks to identify these texts are under development (cf. Glauner 20235; Weßels 20236).

AI-based language generators in education and examinations

According to experts, the impact of AI-based language generators on the education system will be immense, but difficult to estimate. AI offers great opportunities to support text creation and editing. Texts can be structured, summarised, simplified or corrected, saving time and energy. In addition to linguistic tasks, AI can perform other creative tasks such as generating IT programs, spatial concepts or pieces of music. Thus, on the one hand, AI can support teachers and examiners in creating teaching materials and tasks as well as in evaluating and documenting examinations. On the other hand, it can be used as a personal learning companion, for exam preparation, for first drafts and for overcoming writer’s block. It can also facilitate access to exams for people with impairments or learning difficulties.

However, learners should avoid “delegating” their own assignments to AI and limiting their own learning process. Instead, they should be sensitised to the critical use of AI-generated texts. The educational mandate for responsible and safe use of digital media also applies to text-generating AI applications (cf. Weßels 2023; MSB NRW 2023; Schepers 2023; Spannagel 2023).

In examinations, there is a danger that examinees may pass off AI-based texts as their own. It is reported from the USA that ChatGPT has already mastered exams in law and medicine (cf. Choi et al. 2023; Kung et al. 2023). In Hamburg, the first high school graduate has already been caught cheating with ChatGPT.7 Lively discussions are taking place in the general education and higher education sectors. While some want to completely ban ChatGPT and the like in exams and assignments, others consider such a ban unrealistic and uncontrollable. They advocate teaching a critical approach and the targeted use of AI in homework and exams (cf. Pinkwart/Paaßen/Burchardt 20238).

In order to obtain fair and bias-free results in examinations with AI support, it is recommended to design tasks that are as complex as possible and individually tailored to the examination group. Ideally, the entire work and examination process should be accompanied and not only the result evaluated. For privacy reasons, examinees should not use their own devices and accounts. It is also advised to combine different examination formats and instruments and to increase the use of examination interviews and presentations. Some universities have already revised their examination regulations accordingly and issued recommendations to teaching and examination staff, as have some federal states (cf. Weßels 20236; MSB NRW 2023; Spannagel 2023).

Vocational final examinations

One advantage of final examinations in recognised training occupations is the variety of examination instruments used (cf. BIBB Board 2013). Written assignments in final examinations are partly conducted digitally, but without internet access and not on personal devices (cf. Hollmann et al. 2023). Therefore, with the exception of wilful attempts at deception, the use of AI-assisted language generators by examinees is just as impossible here as in oral and practical tasks that are carried out under the supervision of examiners. In practical tasks carried out in the presence of examiners, they can prevent the misuse of AI. The situation is different for work tasks carried out in the company, such as company assignments or company project work in the IT occupations. Here, the training companies must certify to the examination board that the examinees have carried out the tasks independently. Whether a training company can reliably monitor the independence of the task preparation and certify it to the examination board must be discussed. A closer look at this issue and legal clarification seem necessary.

It should be noted that company tasks must be documented with practice-related documents that are intended to support the assessment but do not receive their own weighting for the examination result. For example, industrial clerks, office management clerks and wholesale and foreign trade management clerks prepare a three- to five-page report on the performance of a specialist task. This report itself does not contribute to the evaluation of the examination. However, based on the report, a case-related professional discussion is conducted during the examination, which is assessed. The task of the examination board is to assess whether the examinees have mastered the topic and meet the examination requirements (cf. BIBB Board 2013).


The use of AI-controlled language generators is still in its early stages, but development is progressing fast. It is therefore important to have a timely and comprehensive discussion about their use in final VET examinations. In what form can and may AI support the creation and evaluation of examination tasks in the future? What strategies can be used to avoid misuse in examination situations? Do examination instruments, examination regulations and/or examination requirements need to be adapted? VET stakeholders should proactively address these questions and find common answers. If this technological development – like calculators and computers years ago – finds its way into everyday working life, then its responsible use should be taught in training and thus also become part of the examination content. Trainers and examiners must be informed and trained in time, and social partners, policymakers and competent bodies should create the necessary framework conditions.


Albrecht, S.: ChatGPT und andere Computermodelle zur Sprachverarbeitung – Grundlagen, Anwendungspotenziale und mögliche Auswirkungen. Berlin 2023. URL: publikationen.bibliothek.kit.edu/1000158070/150614893

Board of BIBB [Hauptausschuss des BIBB]: Struktur und Gestaltung von Ausbildungsordnungen – Prüfungsanforderungen. Empfehlung Nr. 158 vom 12. Dezember 2013. URL: www.bibb.de/dokumente/pdf/HA158.pdf

Bundesamt für Sicherheit in der Informationstechnik – BSI: Große KI-Sprachmodelle. Chancen und Risiken für Industrie und Behörden. Bonn 2023. URL: www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/KI/Grosse_KI_ Sprachmodelle.pdf

Choi, J. H.; Hickman, K. E.; Monahan, A.; Schwarcz, D. B.: ChatGPT Goes to Law School. In: Journal of Legal Education 72 (2023) 1, pp. 1–16. URL: https://ssrn.com/abstract= 4335905

European Commission [Europäische Kommission]: Vorschlag für eine Verordnung des Europäischen Parlaments und des Rates zur Festlegung harmonisierender Vorschriften für künstliche Intelligenz (Gesetz über künstliche Intelligenz) und zur Änderung bestimmter Rechtsakte der Union. COM(2021) 206 final of 21 April 2021. URL: https://eur-lex.europa.eu/legal-content/DE/TXT/?uri= CELEX%3A52021PC0206

Hollmann, C.; Kiedrowski von, M.; Lorig, B.; Schürger, B.: Das Prüfungswesen in der digitalen Transformation: Status quo und Entwicklungsperspektiven. Bonn 2023. URL: www.bibb.de/dienst/publikationen/de/18914

Kung, T. u. a.: Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. In: PLOS Digital Health 2 (2023) 2. URL: https://doi.org/10.1371/journal.pdig.0000198

Ministerium für Schule und Bildung des Landes Nordrhein-Westfalen – MSB NRW: Umgang mit textgenerierenden KI-Systemen. Ein Handlungsleitfaden. Düsseldorf 2023. URL: www.schulministerium.nrw/system/files/ media/document/file/handlungsleitfaden_ki_ msb_nrw_230223.pdf

Ouyang, L. u. a.: Training language models to follow instructions with human feedback. Online published 2022. URL: www. semanticscholar.org/reader/d766bffc357127e 0dc86dd69561d5aeb520d6f4c

Spannagel, C.: ChatGPT und die Zukunft des Lernens: Evolution statt Revolution. In: Hochschulforum Digitalisierung (Ed.): Dossier GENERATIVE KI. Welche Rolle spielen ChatGPT und weitere, generative KI-Tools für die Zukunft der Hochschullehre? Was bedeutet die rasante Entwicklung für Kompetenzerwerb und Prüfungen? Online published 2023. URL: hochschulforumdigitalisierung.de/de/blog/chatgpt-und-die-zukunft-des-lernens-evolution-statt-revolution

(All links: as of 11/09/2023)

Barbara Schürger
Academic researcher at BIBB

Translation from the German original (published in BWP 3/2023): Martin Kelsey, GlobalSprachTeam, Berlin