Multi-Modal Opinion Integration for Financial Sentiment Analysis using Cross-Modal Attention
Multi-Modal Opinion Integration for Financial Sentiment Analysis using Cross-Modal Attention
This paper presents a deep learning framework for financial sentiment analysis that integrates two distinct textual opinion modalities: recency modality (timely news, analyst reports) and popularity modality (trending social media, forum discussions). The core innovation is the Financial Multi-Head Cross-Attention (FMHCA) mechanism, which models asymmetric information flow between these modalities through a two-stage attention process. Features are extracted using BERT (Chinese-wwm-ext), refined via transformer layers, and fused using Multimodal Factorized Bilinear Pooling (MFB) for sentiment classification into negative, neutral, or positive.
Experiments on a dataset covering 837 companies demonstrate state-of-the-art performance with 83.5% accuracy, significantly outperforming baselines such as BERT+Transformer (62.5%) and BERT+Transformer+Fusion (77.0%). Ablation studies confirm the critical contributions of both cross-modal attention and fusion components, each improving accuracy by over 6%. The model also shows robustness across different BERT variants, indicating that performance gains stem from architectural innovations rather than specific pre-trained models.
The work has practical implications for real-time market sentiment monitoring, risk assessment, and investment decision support. Limitations include focus on Chinese market data and lack of source credibility or temporal dynamics modeling. Future directions include multilingual extensions, source weighting, and temporal sentiment modeling.
Highlights
- 1Proposes a novel multi-modal framework integrating recency (timely) and popularity (trending) opinion modalities for financial sentiment analysis.
- 2Introduces Financial Multi-Head Cross-Attention (FMHCA) to capture asymmetric cross-modal relationships between factual updates and collective sentiment.
- 3Achieves 83.5% accuracy on a dataset of 837 companies, outperforming baselines by up to 21%.
- 4Demonstrates robustness across different BERT variants, confirming architectural innovation over model choice.
Methods
- MBERT (Chinese-wwm-ext) for feature extraction from textual opinions.
- MFinancial Multi-Head Cross-Attention (FMHCA) with two-stage attention (trending-to-timely and attended-to-trending).
- MTransformer layers for feature refinement after cross-modal attention.
- MMultimodal Factorized Bilinear Pooling (MFB) for fusion of processed representations.
Results
- ROur model achieves 83.5% accuracy, 82.0% weighted F1, 82.0% weighted precision, and 81.0% weighted recall.
- ROutperforms best baseline (BERT+Transformer+Fusion) by 6.5% accuracy.
- RAblation shows removing cross-modal attention reduces accuracy by 6.5%; removing fusion reduces by 7.0%.
- RPerformance varies by only 1% across different Chinese BERT variants, indicating robustness.
Analyze Paper
Generate insights from "Multi-Modal Opinion Integration for Financial Sentiment Anal...".