Publications

publications by categories in reversed chronological order. generated by jekyll-scholar.

2025

  1. Publication Preview
    Initialization Using Update Approximation for Low-Rank Fine-Tuning
    Kaustubh Ponkshe*, Raghav Singhal*, Eduard Gorbunov, and 3 more authors
    2025
    arXiv; SCOPE @ ICLR 2025; Under Review @ NeurIPS 2025
    PDF Code
  2. Publication Preview
    ABBA: Highly Expressive Hadamard Product Adaptation for Large Language Models
    Raghav Singhal*, Kaustubh Ponkshe*, Rohit Vartak*, and 1 more author
    2025
    arXiv; ES-FOMO @ ICML 2025; Under Review @ NeurIPS 2025
    PDF Code
  3. Publication Preview
    Safety Subspaces are Not Distinct: A Fine-Tuning Case Study
    Kaustubh Ponkshe*, Shaan Shah*, Raghav Singhal*, and 1 more author
    2025
    arXiv; Under Review @ NeurIPS 2025
    PDF Code
  4. Publication Preview
    TokenSwap: A Lightweight Method to DisruptMemorized Sequences in LLMs
    Parjanya Prashant, Kaustubh Ponkshe*, and Babak Salimi
    2025
    arXiv; Under Review @ NeurIPS 2025
    PDF
  5. Publication Preview
    Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning
    Raghav Singhal*, Kaustubh Ponkshe*, Rohit Vartak, and 2 more authors
    2025
    arXiv; ES-FOMO @ ICML 2025; Under Review @ NeurIPS 2025
    PDF Code

2024

  1. Publication Preview
    FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Foundation Models
    Raghav Singhal*, Kaustubh Ponkshe*, and Praneeth Vepakomma
    Oct 2024
    Oral, ACL 2025 (Top 8.1%), Poster, FITML Workshop @ NeurIPS 2024, Under Review @ ICLR 2025
    DOI Code
  2. Publication Preview
    Power Learning: Privacy-preserving Embeddings
    Kaustubh Ponkshe, Praneeth Vepakomma, and Peter Kairouz
    Oct 2024
    Oral, Trustworthy ML Workshop @ CIKM 2024
    PDF Code
  3. Publication Preview
    StructFormer: Document Structure-Based Pre-training
    Kaustubh Ponkshe, Venkatapathy Subramanian, Ganesh Ramakrishnan, and 1 more author
    Oct 2024
    Poster, DocUI Workshop @ AAAI 2025, Under Review @ TMLR
    PDF Code
  4. Publication Preview
    GUIDEQ: Framework for Guided Questioning
    Priya Mishra, Suraj Racha, Kaustubh Ponkshe, and 2 more authors
    Oct 2024
    Poster @ NAACL Findings
    PDF Code

2021

  1. Publication Preview
    Crop Yield Prediction of Indian Districts Using Deep Learning
    Parjanya Prashant, Kaustubh Ponkshe, Chirag Garg, and 2 more authors
    Oct 2021
    ICIIP 2021
    PDF Code