arXiv 2026 · Preprint
MiCo
Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference
Yichen Guo · Co-first author
A project overview on Yichen Guo's research homepage, based on the linked paper.
Overview
MiCo reduces the cost of multimodal language model inference without additional training. Through a semantic erasure model, it derives a mutual information coverage objective from task log-loss. Rather than ranking visual tokens independently, it selects a subset that covers useful visual information: first from visual features, then with task-aware signals inside the language model.
How it works
Build a representative visual candidate pool
Before the language model, Stage 1 combines visual feature norms with vision-encoder CLS attention. It retains a larger candidate pool to preserve visual coverage before the tokens interact with the question.
Refine the subset using the task
At a selected decoder layer, Stage 2 combines contextualized feature norms with text-to-visual attention. It selects a smaller subset that covers information relevant to the current question.
Optimize weighted coverage greedily
Both stages use nonnegative cosine similarity to measure how well retained tokens represent the source positions. Tokens are added by their marginal weighted coverage gain until the budget is reached. The observable coverage objective is monotone and submodular, giving greedy selection a 1 − 1/e approximation under a cardinality constraint.
Evaluation
The paper evaluates eight image models and one video model across visual reasoning, OCR, grounding, hallucination detection, and long-video understanding. On LLaVA-NeXT-13B, it reports retaining 97.5% of average baseline performance with 5.6% of visual tokens and about 3.8× one-token inference speedup. The NeXT timing protocol uses one A800 80GB GPU, batch size one, and excludes loading, preprocessing, and host-to-device transfer. On LLaVA-Video-7B at 16 tokens per frame, MiCo retains 93.0% of average baseline performance with a 2.31× complete-answer speedup. Full benchmark results, ablations, and limitations are in the paper.
Paper and resources
MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference
arXiv:2609.34330
Figures and technical descriptions are based on the linked paper. The original author list and publication status are shown above.