FinDPO introduces the first finance-specific large language model (LLM) framework that leverages Direct Preference Optimization (DPO) for financial sentiment analysis, moving beyond traditional supervised fine-tuning (SFT). By aligning the Llama-3-8B-Instruct model with human preferences on a curated dataset of financial news, FinDPO achieves state-of-the-art performance on standard sentiment classification benchmarks, outperforming existing SFT models by an average of 11% in weighted F1 score. The use of Low-Rank Adaptation (LoRA) ensures parameter-efficient training, requiring only 0.52% of the model's parameters to be updated, which enables training on a single GPU in under 5 hours.

A key methodological contribution is the 'logit-to-score' conversion, which transforms the discrete sentiment predictions of causal LLMs into continuous, rankable sentiment scores via softmax over logits and temperature scaling. This innovation allows FinDPO to be integrated into realistic long-short portfolio strategies, where assets are ranked daily by sentiment strength. In simulations using over 200,000 financial news articles and S&P 500 data from 2015 to 2021, the FinDPO-driven portfolio achieves an annualized return of 67% and a Sharpe ratio of 2.0, even under realistic transaction costs of 5 basis points. These results demonstrate that human preference alignment not only improves classification accuracy but also yields substantial and robust financial performance, establishing a new paradigm for sentiment-driven algorithmic trading.