The paper introduces Open-FinLLMs, a pioneering suite of open-source multimodal large language models specifically designed for financial applications. This work addresses key limitations in existing financial LLMs, such as scarce corpora, weak multimodal capabilities, and narrow evaluations, by developing models that can process text, tabular, time-series, and chart data. The suite includes three main components: FinLLaMA, pre-trained on a 52-billion-token corpus; FinLLaMA-Instruct, fine-tuned with 573K financial instructions; and FinLLaVA, enhanced with 1.43M multimodal tuning pairs for robust cross-modal reasoning. These models are evaluated comprehensively across 14 financial tasks, 30 datasets, and 4 multimodal tasks, with the introduction of two new multimodal evaluation datasets to ensure thorough assessment.

Results demonstrate that Open-FinLLMs outperform advanced models like GPT-4 in financial NLP, decision-making, and multimodal tasks, highlighting their potential for real-world applications. The models' effectiveness in zero-shot, few-shot, and fine-tuning settings underscores their adaptability and robustness. By releasing all codes and models under OSI-approved licenses, the authors aim to promote innovation and collaboration across academia and industry, making these tools accessible for further development and application in the financial domain. This contribution not only advances the state of financial AI but also sets a benchmark for open-source multimodal LLMs in specialized fields.