My AI code assistant wrote a loop that broke our test server last Thursday...

I let GPT-4 handle a Python script for 3 hours and it created an infinite loop that cost us $200 in cloud compute before I noticed the alert at 2 AM, has anyone else had an AI tool go rogue like that without catching it sooner?

3 comments

3 Comments

terry_bailey3514d ago

Did you have any safeties in place like run time limits or cost caps before you let it go? Seems like a good way to catch that kind of stuff before it gets expensive. I've seen people set hourly budgets on cloud services just to avoid these surprises. Was the loop something obvious or did it take a while to spot in the logs? Just trying to figure out if there's a better way to watch these tools without having to sit there the whole time.

claire6413d ago

That "chewing through tokens real quiet" part really got me, I've been there myself. It's almost worse than a straightforward error because you don't even realize anything's wrong until you get the bill. I had a similar situation where my AI kept rewriting the same paragraph over and over, each time just slightly different, and it ate up hours of compute before I finally spotted the pattern in the logs. What helped me was setting a hard token limit per session rather than just a cost cap, since like you said, cost caps miss the slow burn kind of loops that don't spike usage all at once.

abby_henderson13d ago

Honestly the thing nobody brings up is audit trails that catch the emotional side of it. I had a script that kept looping and it wasn't a code bug, it was because the stupid AI kept "thinking it needed to refine" its own output. No cost cap would have stopped that because it wasn't spinning up new compute it was just chewing through tokens real quiet. I started logging the actual decision making steps in plain english so I could see when it went off the rails trying to be too clever. Took me a while to spot because the loop was just it second-guessing itself over and over in the same function call.