Fallacies of AI-driven product research
The customer is always right. But what if the customer was lied to?
If a sales person over promises, and the product under delivers, the customer won't be happy. This often leads to a cancellation and refund request.
But what if the customer didn't do any research outside of asking a robot a few questions? I've already touched on this with regard to tech support, and was blessed with a customer recently that was disappointed that my product didn't do all of the things they thought it did.
Sadly, I knew before the customer had put in their credit card they were going to be difficult. My radar for problem users has gotten pretty good over the years. They come in with a certain urgency, even on the weekends. They ask obvious questions, and usually are looking for phone support.
They usually will rush into purchasing, rather than spending any time with the free tier of a product. This is contrary to my direct suggestion to do so.
GenAI hallucinations
This so-called "artificial intelligence" is compliant to a fault. Ask a question, it gives an answer. Show any sort of bias, and it will agree with it. Oh, and sometimes it just makes shit up. Call it out, and you're "absolutely right".
I use a bunch of different tools, LLMs, and the like, and I couldn't imagine not doing some sort of fact checking. Yet, so many people ask the robot something, and take the answer as the gospel, rather than a starting point.
Greed in the machine
There's a lot of money changing hands right now. Valuations are absolutely bonkers. AI companies are being sued, and losing. There are non-profit "entities", but at the end of the day, everybody is charging for access.
These generative AI models are still very much a black box. They are fed a multitude of data. To assume they have zero bias is idiotic.
Companies get creative to generate revenue. It's not unrealistic to think that at some point in future, if not already, that contracts will be penned, to guarantee LLM bias as a form of advertising.
Do your own research
If a robot told you a certain car was the perfect one for you, would you walk into a car dealership and take out a loan without taking a test drive first?
Probably not.
The same holds true for every product suggestion a robot makes. Use it as a way to cut through the initial research phase, but not as a substitute for real decision making.
And for the love of God, please don't chew out a support person because you didn't do enough research.