A $3,900 overnight bill from our LLM eval suite: the incident, and the spend guard I shipped after

​ TL;DR. Our LLM-judge eval suite had no cost ceiling. It ran the full judge over 1,200 cases on every CI trigger. On 08/07 a dependency bot…Continue reading on Medium »   Read More LLM on Medium 

#AI