Primary focus area: Safeguards and trust in AI
Secondary focus areas: Adaptation of foundation models
Abstract:
Seq2seq models used in translation and speech recognition often produce errors like repetition or irrelevance. These are difficult to manage due to a lack of reliable uncertainty estimation. This RBO aims to distinguish and quantify two types of uncertainty: content uncertainty (what to say) and delivery uncertainty (how to say it). We propose formalizing this distinction, adapting model architectures, and evaluating our methods across domains, with the goal of enhancing the trustworthiness and applicability of AI-generated outputs.
Research Gap:
While uncertainty in AI outputs has been studied, current models conflate different types of uncertainty. Token-level probabilities offer limited interpretability and fail to separate content from delivery ambiguity. No existing approach provides meaningful estimates of these distinct uncertainties, especially over longer sequences. This limits the ability to build safeguards, calibrate confidence, or involve human reviewers effectively.
Objective:
To design methods that annotate seq2seq outputs with separate estimates of content and delivery uncertainty—numerical or distributional—tailored for translation and speech recognition tasks. These will enable better risk management and user trust in AI systems.
Impact:
This work aims to improve model transparency and reduce AI overconfidence, enabling safer deployment in language technologies.