Publication Date

Spring 2026

Degree Type

Thesis

Degree Name

Master of Science (MS)

Department

Computer Engineering

Advisor

Magdalini Eirinaki; Bernardo Flores; Wencen Wu

Abstract

LLM-based web navigation agents fail on the majority of tasks while consuming substantial computational resources before failure becomes apparent. This thesis investigates whether closed sequential pattern mining on the first K steps of agent execution traces can predict task failure early enough to enable meaningful computational savings with interpretable justification. We develop a two-phase system: an offline pipeline that symbolizes agent traces, extracts K-step prefixes, mines closed patterns via BIDE+, and ranks them by failure precision; and an online detector that matches live executions against the resulting pattern library. We evaluate on 1,544 MiniWoB++ traces across three open-weight language models (Llama-3.2-3B, Qwen-2.5-7B, Mistral-7B) using five evaluation configurations of increasing difficulty, with cross-benchmark transfer analysis on 577 WebArena shopping traces. Eight detection methods are compared, including neural baselines (Bi-LSTM, DeepLog), feature-engineering methods, and targeted pattern mining. BIDE Coverage achieves the highest failure detection F1 at K=3 (0.717) across all configurations, outperforming neural methods that require 8–10 steps to reach comparable accuracy. On the hardest evaluation—natural failures versus multi-step successes only—BIDE leads all methods from K=5 through K=10. At the recommended operating point (K=5, 92% precision), the system achieves 26.3% overall token savings with 3.2% impact on successful tasks. The mined pattern library of 32 closed patterns organizes into four interpretable failure categories—recovery, validation, navigation, and context failures—each with concrete repair recommendations. Cross-benchmark transfer is model-dependent, with an inverse correlation between models’ self-monitoring capability and pattern transferability.

Download

Share

COinS