Masters Thesis: GRPO for Multilingual Math and Logic Reasoning
AI Sweden
Summary
Post-training approaches using Reinforcement Learning (RL) are critical for eliciting deep reasoning capabilities in LLMs. Group Relative Policy Optimization (GRPO) removes the need for a separate value model, drastically reducing compute overhead. However, the efficacy of GRPO on non-English reasoning tasks remains largely unexplored.
Job description
Introduction
Post-training approaches using Reinforcement Learning (RL) are critical for eliciting deep reasoning capabilities in LLMs. Group Relative Policy Optimization (GRPO) removes the need for a separate value model, drastically reducing compute overhead. However, the efficacy of GRPO on non-English reasoning tasks remains largely unexplored.
Project Background and Problem Statement
OpenEuroLLM is integrating the Dolci-Think-SFT datasets into its post-training pipeline. We need to validate whether GRPO can effectively utilize translated and native math/logic rollouts to enhance Prelude 9B's reasoning capabilities in Swedish and other EU languages. The thesis asks:
- Does GRPO on translated Swedish math problems yield compute-optimal reasoning improvements compared to standard SFT?
Outline
The goal is to apply compute-efficient RL to improve Prelude 9B's reasoning capabilities.
- Literature study: Analyze RLHF, PPO, GRPO methodologies, and reasoning benchmarks (e.g., GSM8k, MATH).
- Implementation: Adapt the oellm-rlvr framework to perform GRPO on Prelude 9B using Swedish reasoning datasets (e.g., translated Dolci-Think).
- Evaluation: Benchmark the resulting model on multilingual reasoning tasks, explicitly measuring the trade-off between training TFLOPs/s, memory constraints, and final accuracy.
Who we're looking for
We are seeking curious, self-driven MSc students eager to work at the frontier of open-weight European AI research (LLMs). You thrive on empirical discovery, design rigorous experiments, and let data challenge your assumptions.
- Ongoing Master's studies in Computer Science, Data Science, Machine Learning, Engineering Physics, or a related quantitative field.
- Proficiency in Python and hands-on experience with modern deep learning frameworks (PyTorch, Hugging Face ecosystem).
- Familiarity with LLM post-training alignment (e.g., SFT, DPO, RLHF/RLVR) or context-extension, alongside comfort running distributed GPU training in Linux/HPC environments.
Requirements
- Ongoing Master's studies in Computer Science, Data Science, Machine Learning, Engineering Physics, or a related quantitative field.
- Proficiency in Python.
- Hands-on experience with modern deep learning frameworks (PyTorch, Hugging Face ecosystem).
- Familiarity with LLM post-training alignment (e.g., SFT, DPO, RLHF/RLVR) or context-extension.
- Comfort running distributed GPU training in Linux/HPC environments.
Company overview
AI Sweden is Sweden's national center for applied artificial intelligence, established in 2019 with a mission to accelerate the adoption and implementation of AI technologies for societal benefit, economic competitiveness, and national advancement. As a collaborative research institute, it brings together over 170 partners from diverse sectors including government agencies, universities, research institutions, startups, and global corporations. The organization operates as a catalyst for AI innovation, focusing on practical applications across healthcare, energy systems, municipalities, and various industries. With a core team of over 80 professionals comprising AI experts, data scientists, research engineers, linguists, mathematicians, policy experts, and change leaders, AI Sweden facilitates knowledge sharing, talent development, and infrastructure building. The institute operates on a not-for-profit basis, funded by Sweden's innovation agency Vinnova, the Swedish Agency for Economic and Regional Growth, European Regional Development Fund, Västra Götalandsregionen, and partner contributions. Through its AI Labs, talent programs, workshops, and collaborative projects, AI Sweden addresses critical societal challenges while bridging the gap between cutting-edge research and real-world implementation, positioning Sweden as a leader in responsible and impactful AI development.