What to Measure When Evaluating Model Refresh, Benchmarks, and Production Switching
https://www.mediafire.com/file/bj2qr0zejl3ytp4/pdf-8880-1899.pdf/file
Between Jan 10 and Feb 28, 2024 I ran a focused evaluation across 40 publicly available and vendor-hosted model endpoints