VEHI DEFECT: Vehicle Defect Transparency Tool using Hybrid LLM approach

Abstract

"Analysing online product reviews for both aspect sentiment and relationships between entities presents a challenge. Existing models often excel in one area but struggle in the other, limiting their comprehensive understanding of user opinions. This limitation can hinder accurate sentiment analysis and potentially mislead users. This work proposes MUTA (Mistral-T5 Unified Analysis): a novel hybrid large language model (LLM) approach that addresses this challenge. MUTA leverages the strengths of two models: 1. T5 small (fine-tuned): Summarizes product reviews, capturing the essence of user opinions. 2. Mistral-7b: Analyzes the summarized text to identify crucial components, extract relationships between them (particularly ""Customer-Manufacturer"") and assign severity scores. MUTA's effectiveness is demonstrated through a multi-faceted evaluation. T5 summaries provide concise overviews of user opinions in reviews, leading to improved understanding of overall sentiment. Mistral, utilizing these summaries, excels at extracting vehicle components and uncovering relationships between them, particularly focusing on the crucial ""CustomerManufacturer"" dynamic. Additionally, MUTA achieves a high Rogue score, a metric specifically designed for evaluating summarization tasks"