Sample real traffic#
Select a representative week and group requests by workflow. Preserve token counts and timings without keeping sensitive content in analytics.
Run a bounded evaluation#
Send a small approved test set to the candidate model. Compare task success, output length, retries and tools. Tokenization differs between providers.
Budget with uncertainty#
Estimate low, expected and high volumes. Add expected retries and output growth. Set per-key budgets before routing production traffic.