Estimated Variance at a Finite-Time Selection Deadline
Abstract
Finite-time knowledge-gradient evidence and unknown-variance ranking need to distinguish observed precision from known precision. Evaluation varies pilot sizes and compares complete independently checked runs. It retains the reward assumptions of any finite-time theorem. A small variance estimate in a dependent or rare-event sample cannot automatically inherit the theorem's protection. This focused synthesis develops an empirical comparison that preserves the terminal action, observation process, and total verification cost. Its proposed tests are prospective; no new simulation run, model-training result, or intervention effect is reported.
References
J. Du, S. Gao, and I. Ryzhov, “Revisiting an Open Question in Ranking and Selection under Unknown Variances,” Proceedings of the 2025 Winter Simulation Conference, pp. 378–389, 2025.
Y. Li and S. Gao, “On the Finite-Time Performance of the Knowledge Gradient Algorithm,” Proceedings of the 39th International Conference on Machine Learning, pp. 12741–12764, 2022.
Zhou, Yongxi, Lai Yun Choi, Jiaxi Wen, Wenbo Ye, and Junwei Yao. “How Much Can Repeated Running Buy? A Measured Variance Ceiling for LLM Code Evaluation.” arXiv, 2026, arXiv:2606.00920. https://arxiv.org/abs/2606.00920.
Ankenman, Bruce, Barry L. Nelson, and Jeremy Staum. “Stochastic Kriging for Simulation Metamodeling.” Operations Research, vol. 58, no. 2, 2010, pp. 371–382. https://doi.org/10.1287/opre.1090.0754.
Kim, Seong-Hee, and Barry L. Nelson. “A Fully Sequential Procedure for Indifference-Zone Selection in Simulation.” ACM Transactions on Modeling and Computer Simulation, vol. 11, no. 3, 2001, pp. 251–273. https://doi.org/10.1145/502109.502111.
Chick, Stephen E., and Koichiro Inoue. “New Two-Stage and Sequential Procedures for Selecting the Best Simulated System.” Operations Research, vol. 49, no. 5, 2001, pp. 732–743. https://doi.org/10.1287/opre.49.5.732.10615.
Frazier, Peter I., Warren B. Powell, and Savas Dayanik. “A Knowledge-Gradient Policy for Sequential Information Collection.” SIAM Journal on Control and Optimization, vol. 47, no. 5, 2008, pp. 2410–2439. https://doi.org/10.1137/070693424.
Rasmussen, Carl Edward, and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006. https://gaussianprocess.org/gpml/.
Sacks, Jerome, William J. Welch, Toby J. Mitchell, and Henry P. Wynn. “Design and Analysis of Computer Experiments.” Statistical Science, vol. 4, no. 4, 1989, pp. 409–423. https://doi.org/10.1214/ss/1177012413.
Bertsimas, Dimitris, and Melvyn Sim. “The Price of Robustness.” Operations Research, vol. 52, no. 1, 2004, pp. 35–53. https://doi.org/10.1287/opre.1030.0065.
Campi, Marco C., and Simone Garatti. “The Exact Feasibility of Randomized Solutions of Uncertain Convex Programs.” SIAM Journal on Optimization, vol. 19, no. 3, 2008, pp. 1211–1230. https://doi.org/10.1137/07069821X.
Published
Versions
- 2026-10-15 (2)
- 2025-04-02 (1)