Publication Date

Spring 2026

Degree Type

Master's Project

Degree Name

Master of Science in Computer Science (MSCS)

Department

Computer Science

First Advisor

Robert Chun

Second Advisor

Fabio Di Troia

Third Advisor

Rayees Shamsuddin

Keywords

Vision-Language Models, Model Routing, Multimodal Embeddings, Inference Efficiency, Cost Optimization

Abstract

Vision-language models (VLMs) are increasingly used for multimodal tasks, but their inference costs vary widely across model tiers. This work presents a predictive routing framework that assigns each query to the most cost-effective VLM by using multimodal embeddings, clustering, and per-cluster model error estimates. Evaluated on six commercial VLMs, the selected router reduces average cost by about 49 percent relative to the quality-first setting on both validation and test. This comes with accuracy losses of 0.4 percentage points on validation and 0.9 on held-out test data. It also outperforms K-NN and ZeroRouter routing baselines while providing lowercost operating points than the strongest fixed model. New VLMs can be added by computing their validation error vectors, without rebuilding the embedding space or clusters. These results indicate that per-query routing is a practical strategy for reducing VLM inference cost while preserving accuracy.

Download

Share

COinS