Trustworthy AI Feedback - A Hybrid Approach to Automated Feedback: Combining Rule-Based Reasoning with Generative AI in Ask-Elle

Abstract

Ask-Elle is an intelligent tutoring system for learning functional programming in Haskell, providing feedback through strategy-based model tracing and property based testing. However, the system fails to offer meaningful guidance in cases where a student’s program deviates from model solutions. This thesis investigates integrating generative AI into Ask-Elle to address this limitation. The proposed hybrid architecture uses a large language model to complete the student’s partial code, which is then validated by Ask-Elle’s existing compiler and property-based tests before any hint is shown to the student. Three models were evaluated, Claude Sonnet 4.6, GPT-5.3-Codex, and GPT-4.1-nano, with the two larger models achieving success rates of 88–97%. GPT-5.3-Codex was selected as the final model based on both technical performance and a blind pedagogical evaluation of hint quality. A user study indicated that AI-generated hints provided higher educational value than the system’s existing feedback. While code correctness is reliably enforced by the chosen architecture, occasional hallucinations in natural language hints remain an open challenge