Project
LLM Assessment of E-commerce Review Value
A three-axis comparison of LLM and human ratings for authenticity, information value, and relevance, with explicit attention to evaluation limits.
Python · DeepSeek · G-EVAL · Data Quality · Human Evaluation
Product Snapshot
- Role
- Data analysis and evaluation design
- Users
- Research and commerce teams filtering useful reviews or assessing training-data quality.
- Stage
- Rubric, DeepSeek/G-EVAL scoring, human comparison, and error analysis completed in 2025.02-2025.07.
- Focus
- Authenticity, information value, relevance, and agreement between LLM scores and human judgment.
- Validation
- Built a three-axis MSE/MAE comparison workflow; project versions record 2,559 raw and 1,589 processed samples.
- Public Proof
- Datasets, scoring scripts, and report versions; the two sample counts represent different processing stages.
Next Step
The problem
Review volume is not the same as data value. Duplicate, false, irrelevant, or low-information reviews can weaken both insight and model training.
Method
I separated quality into authenticity, information value, and relevance, applied DeepSeek and G-EVAL scoring, and compared the results with human ratings through MSE, MAE, and disagreement analysis.
Sample definitions
Different report versions record 2,559 raw samples and 1,589 processed samples. They belong to different cleaning stages and should not be presented as one consistent evaluation cohort.
Conclusion
LLMs can scale initial screening, but they still require an explicit rubric, human anchors, and error review. For high-risk or ambiguous examples, an automated score is a review signal rather than final truth.
Configure the public Giscus environment variables to open discussion on the public site.