Retail · AI in production
Unwanted reviews flagged before a person reads them
A fine-tuned BERT model reads incoming reviews for spam, off-topic and negative content, including irony and sarcasm, and tells the team which ones to remove.
- Client
- SaaS brand management platform for rank tracking, social media and reviews
- Segment
- Retail, reputation management
- Data
- 150,000 reviews provided, 80,000 for training, 8,000 for testing
- Timeline
- 2 months, 334 person-hours
- BERT
- PyTorch
- Kubernetes
- PHP
Before
What was true before
The client’s team read large volumes of reviews by hand to find the ones that should come down: spam, off-topic and negative comments. Irony and sarcasm made the task slower, because the words alone did not show the intent.
After
What we did
A BERT model fine-tuned on labelled examples of reviews to remove, with a decision threshold, integrated so the client is notified of each review that should go.
Build log
How it was built
- 01
Compared NLP models for negative tone, irony and sarcasm, and chose BERT
- 02
Built a training pipeline on labelled examples of reviews that should be removed
- 03
Tuned learning rate, optimiser and epochs for the balance of accuracy and speed
- 04
Set a decision threshold for removal
- 05
Trained on 80,000 and tested on 8,000 of the 150,000 reviews provided
- 06
Integrated the model so the client is notified of reviews to remove
Measured
Results
As reported in the published case.
- accuracy predicting unwanted reviews
- 80%
- person-hours over 2 months
- 334
Engagements
The services behind it
Engagement
Where this work sits
Next step
Yours would be the next one.
Tell us where it breaks, and you get a written answer on what breaks first and what it costs.