Spatial Grounding after a Model-Merging Intervention
Abstract
Merging perception and reasoning can improve one ability while changing another. Evaluation should retain image-region and relation checks alongside answer correctness. Spatial attention can diagnose focus but cannot prove the causal source of success. Independent task perturbations show whether the merged system responds to the evidence needed for the actual spatial conclusion. This focused synthesis develops an empirical comparison that preserves the terminal action, observation process, and total verification cost. Its proposed tests are prospective; no new simulation run, model-training result, or intervention effect is reported.
References
S. Chen, J. Zhang, T. Zhu, W. Liu, S. Gao, M. Xiong, M. Li, and J. He, “Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging,” Proceedings of the 42nd International Conference on Machine Learning, 2025.
S. Chen, T. Zhu, R. Zhou, J. Zhang, S. Gao, J. Niebles, M. Geva, J. He, J. Wu, and M. Li, “Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas,” Proceedings of the 42nd International Conference on Machine Learning, 2025.
Zhou, Yongxi, Junwei Yao, Yuanzhe Liu, Zihan Dong, Wenbo Ye, Jiaxi Wen, and Lai Yun Choi. “Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity.” arXiv, 2026, arXiv:2608.02665. https://arxiv.org/abs/2608.02665.
Geifman, Yonatan, and Ran El-Yaniv. “Selective Classification for Deep Neural Networks.” Advances in Neural Information Processing Systems, vol. 30, 2017. https://arxiv.org/abs/1705.08500.
Ovadia, Yaniv, et al. “Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift.” Advances in Neural Information Processing Systems, vol. 32, 2019. https://arxiv.org/abs/1906.02530.
Bertsimas, Dimitris, and Melvyn Sim. “The Price of Robustness.” Operations Research, vol. 52, no. 1, 2004, pp. 35–53. https://doi.org/10.1287/opre.1030.0065.
Rasmussen, Carl Edward, and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006. https://gaussianprocess.org/gpml/.
Hu, Edward J., et al. “LoRA: Low-Rank Adaptation of Large Language Models.” International Conference on Learning Representations, 2022. https://openreview.net/forum?id=nZeVKeeFYf9.
Guo, Chuan, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. “On Calibration of Modern Neural Networks.” Proceedings of the 34th International Conference on Machine Learning, PMLR, vol. 70, 2017, pp. 1321–1330. https://proceedings.mlr.press/v70/guo17a.html.
Sutton, Richard S., and Andrew G. Barto. Reinforcement Learning: An Introduction. 2nd ed., MIT Press, 2018. http://incompleteideas.net/book/the-book-2nd.html.
Levine, Sergey, Aviral Kumar, George Tucker, and Justin Fu. “Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems.” arXiv, 2020, arXiv:2005.01643. https://arxiv.org/abs/2005.01643.
Published
Versions
- 2026-10-15 (2)
- 2025-04-02 (1)