16 upvotes
BestRand Cuts Benchmark Costs with Tiny Random Subsets
What is Random Is Almost Enough? Stable and Low-Cost Benchmark Compression via BestRand?
The method selects a small, carefully chosen subset of benchmark tests that still predicts the full benchmark scores. It provides a mathematical formula to estimate the error you’ll see when using this compressed benchmark.
Who it is for
ML researchers and engineers can speed up model evaluation and cut cloud bills by using BestRand to run only a fraction of the tests.