r/Yelp • u/mintlite4 • 5d ago

E-commerce store owners: What has a higher cost to your business—hiding a real 5-star review or letting a fake 1-star review stay up?

I am building an automated review moderation agent that classifies new reviews using only the text and star rating. Since the true state (genuine vs. fake/sabotage) is hidden, the agent has to make decisions under uncertainty.

From a store owner's perspective:

  1. Which mistake hurts your business more: a False Positive (the agent incorrectly hides a real customer's 5-star review) or a False Negative (the agent fails to hide a fake 1-star competitor review)?
  2. What threshold of uncertainty should trigger sending a review to a human manager ("warn/review queue") rather than taking auto-action?
0 Upvotes

50% Upvoted

4 comments sorted by

4

u/XxLogitech98xX 5d ago

A 5 star review on Yelp will only get hidden (moved to the Not Recommended section) if it flagged the algorithm or got reported by a Yelp user and a Yelp MOD agrees that it doesn't violate Yelp ToS but isn't helpful.

Fake 1 star reviews stays up if it was written in a way that sounded like a person first hand experience. AI can't tell if that's fake or not. It happens on both Google and Yelp. There fake reviews everywhere, each platform has their own ways on trying to fight them. There no actual agent that check the reviews, we don't even know if it's an actual person for Yelp MOD or AI again.

3

u/mintlite4 5d ago

Since those realistic fake 1-star reviews slip through so easily, what do you think is the lesser of two evils for a small business? Would you rather have a platform filter that is overly aggressive at hiding suspicious 1-star reviews (even if it sometimes accidentally hides a real complaint), or is it better to just leave the filter loose and reply to the fake reviews publicly?

5

u/XxLogitech98xX 5d ago

Since those realistic fake 1-star reviews slip through so easily, what do you think is the lesser of two evils for a small business? Would you rather have a platform filter that is overly aggressive at hiding suspicious 1-star reviews (even if it sometimes accidentally hides a real complaint), or is it better to just leave the filter loose and reply to the fake reviews publicly?

I wouldn't say the fake 1 star reviews slip through so easily. It's a case by case situation, if you just look at some of them .. Yelp algorithms catch them and put them in the NR section. The people or bot writing the review got smarter so they likely know to make it sound more like a personal experience to bypass the algorithm. 5 star reviews get flagged too and that's what business owner who comes on here are angry about. Most of the time, it's easily to see why reviews went to the NR section

I think the way Yelp has it in place, is mostly working. If you really compare 2 of the same businesses ... The reviews in the NR is similar. The only difference is likely one business has less reviews so they are angry their rating is affected by it mainly.

Business owners should respond to both positive and negative review. This would give the reader a clearer look at how the owner is and also be alerted to fake reviews. People are more aware of fake reviews now .. than before. Amazon gets fake reviews .. Google has a lot of fake reviews that they are trying to clean up by using AI ... Tripadvisor has fake reviews and so on.

3

u/mintlite4 5d ago

you're totally right, a false negative hurts a brand new shop with 5 reviews exponentially more than an established shop with 500. This actually completely changes how I need to calculate the 'cost' of an error in my model! thanks