Kaustubh Ponkshe
PhD Student, Machine Learning & Optimization Lab, EPFL.
INJ Building, Station 14
EPFL, 1015 Lausanne
Switzerland
I am a PhD student in Computer Science at EPFL, where I am part of the Machine Learning and Optimization Lab (MLO), advised by Prof. Martin Jaggi. I am supported by the EDIC Fellowship, and I contribute to Apertus — the largest (70B) fully open and compliant LLM training run to date.
I want to understand how language models learn. My research rethinks the training pipeline of foundation models — pretraining, midtraining, and post-training:
- Knowledge-free & retrieval-grounded pretraining — training models that lean on external context instead of memorizing the world, to reduce parametric memorization and externalize memory.
- Agentic pretraining — teaching models tool use and on-policy decision-making beyond passive next-token prediction.
- Data & training dynamics — how midtraining and post-training data mixtures interact to shape reasoning and generalization.
Previously, I was a researcher at MBZUAI with Prof. Praneeth Vepakomma, where I worked on efficient and federated fine-tuning of large models, and on the safety and memorization behavior of LLMs. Before that, I completed my B.Tech in Electrical Engineering and M.Tech in Artificial Intelligence at IIT Bombay, where I also led the path-planning and controls teams of IIT Bombay Racing, building India’s first driverless racecar.
Outside research, I enjoy traveling (24 countries so far 🌍) and playing tennis 🎾.
news
| Jan 22, 2026 | Two papers — ABBA and Safety Subspaces are Not Distinct — accepted at ICLR 2026  |
|---|---|
| Sep 18, 2025 | TokenSwap selected as a Spotlight at NeurIPS 2025 🎉 |
| Sep 02, 2025 | Apertus is out — the largest (70B) fully open and compliant LLM to date. Proud to have contributed! [paper] [models] |
| Sep 01, 2025 | Started my PhD in Computer Science at EPFL, joining the Machine Learning and Optimization Lab with Prof. Martin Jaggi 🇨🇭 |
| May 16, 2025 | FedEx-LoRA accepted as an Oral at ACL 2025 (Main Conference) |
