الأقل هو الأكثر: قياس نماذج اللغة الكبيرة في تحليل المشاعر العربي

This research paper presents a rigorous benchmark comparing generalist large language models with specialized encoder-based models for the task of Arabic sentiment analysis on noisy, real-world data.

Models Evaluated

• Llama-3 (Generalist LLM)
• Qwen-2.5 (Generalist LLM)
• CAMeLBERT (Specialized Arabic Encoder — Baseline)

Dataset

• 3,822 samples of Arabic company reviews
• Noisy, colloquial Arabic from real-world sources
• Binary sentiment classification (positive/negative)

Key Results

| Model | Accuracy | F1-Score | Avg Latency |
|------------|----------|----------|-------------|
| Qwen-2.5 | 91.52% | 91.54% | 0.405s |
| CAMeLBERT | 89.32% | 89.42% | N/A (base) |
| Llama-3 | 87.86% | 87.92% | 0.290s |

Findings

• Qwen-2.5 achieves the highest accuracy (91.5%) but is 28.3% slower than Llama-3
• Llama-3 offers the best accuracy-speed trade-off for production systems
• Statistical significance confirmed (McNemar test, p < 0.00001)
• Specialized encoders remain competitive for resource-constrained deployments

Practical Recommendation: For Arabic sentiment pipelines, Llama-3 provides the optimal balance of speed and accuracy.