When Words Fail: Why AI-Driven Context Beats Word Error Rate for Safety-Critical Air Traffic Control
Date Approved
6-30-2025
Embargo Period
6-30-2025
Document Type
Thesis
Degree Name
M.S. Electrical and Computer Engineering
Department
Electrical and Computer Engineering
College
Henry M. Rowan College of Engineering
Advisor
Nidhal Bouaynaya, Ph.D.
Committee Member 1
Ying Tang, Ph.D.
Committee Member 2
Tom Tessitore
Abstract
Accurate speech-to-text transcription is critical for Air Traffic Control (ATC) to en- sure safety and operational efficiency. However, existing Automatic Speech Recognition (ASR) systems struggle with domain-specific phraseology, high noise levels, and real-time constraints. This work explores domain adaptation strategies of OpenAI’s Whisper ASR model, applying Transfer Learning and Low-Rank Adaptation (LoRA) on ATC benchmark and a Federal Aviation Administration (FAA) corpus. Experimental results show that fine- tuning significantly reduces Word Error Rate (WER) from 55.2% to 6.8%, yet WER alone fails to capture safety-critical transcription errors. To address WER’s limitations, we introduce the Contextual Error Rate (CER), a novel evaluation metric powered by a Large Language Model (LLM) that prioritizes errors with operational impact rather than surface-level differences. Our experiments showed that while Transfer Learning improved domain-specific accuracy, LoRA preserved better gen- eralization to conversational English. This research demonstrates the importance of combining domain adapted ASR models with context-aware evaluation metrics. By integrating LLM-assisted CER scoring into the evaluation process, we offer a more reliable and context aware method for evaluating tran- scription quality of domain adapted ASR models in aviation, paving the way for scalable, automated speech recognition in air traffic management. The approach proposed is extensible to other context-critical communication systems where transcription fidelity is paramount. These contributions advance the intersection of speech recognition, machine learning, and aviation safety.
Recommended Citation
Maaz, Muhammad, "When Words Fail: Why AI-Driven Context Beats Word Error Rate for Safety-Critical Air Traffic Control" (2025). Theses and Dissertations. 3414.
https://rdw.rowan.edu/etd/3414