28
My AI assistant went rogue during a client pitch in Chicago last month
I was showing off a new text generator to a big client, and it started making up wild facts about their company history. I had to shut the demo down and explain the 'hallucination' problem on the spot. Has anyone else had a public demo blow up because the model just made stuff up?
3 comments
Log in to join the discussion
Log In3 Comments
jake1912mo ago
Honestly that sounds like a nightmare. Tbh I had something similar happen showing a chatbot to my boss last year. It started inventing store policies that didn't exist, saying we offered crazy discounts. Ngl I was scrambling to explain it was just making things up. Felt like the whole thing was working against me. These models can really make you look bad when they just decide to lie.
4
umaf446d ago
And yeah the weird thing is how unpredictable it is. I was testing one in a low stakes setting and it suddenly started talking about totally made up customer complaints, like "Mrs. Henderson has called 47 times about her garden gnomes." Just wild. It's like they pick the worst possible moment to go off the rails, right when you're trying to prove they work.
6
emma7682mo ago
Actually had the opposite happen at a demo once. The model played it too safe and gave such boring answers the client lost interest. Sometimes a little wildness shows what the tech can really do, you know?
0