Woomin Song
Graduate student at KAIST
KAIST AI
woominsong0 [AT] gmail.com / woomin.song [AT] kaist.ac.kr
About
I am a Ph.D. student at KAIST advised by Prof. Jinwoo Shin, and a Research Scientist Intern at Amazon AGI. I received my B.S. in Electrical Engineering / Computer Science (double major) / Mathematics (minor) at KAIST in 2022.
My research aims to build efficient machine learning systems, with a primary focus on large language models (LLMs). I am particularly interested in reducing the inference cost of LLMs. Currently, I am working on architectural modification techniques to adapt existing pre-trained models for improved computational efficiency.
Publications
Sparsified State-Space Models are Efficient Highway Networks
Woomin Song, Jihoon Tack, Sangwoo Mo, Seunghyuk Oh, Jinwoo Shin
NeurIPS Workshop on Efficient Natural Language and Speech Processing (ENLSP), 2024, Oral Presentation
Tabular Transfer Learning via Prompting LLMs
Jaehyun Nam, Woomin Song, Seong Hyeon Park, Jihoon Tack, Sukmin Yun, Jaehyung Kim, Kyu Hwan Oh Jinwoo Shin
Conference on Language Modeling (COLM), 2024
ICML Workshop on Efficient Systems for Foundation Models (ICMLW-ES-FoMo), 2023
Hierarchical Context Merging: Better Long Context Understanding for Pretrained LLMs
Woomin Song*, Seunghyuk Oh*, Sangwoo Mo, Jaehyung Kim, Sukmin Yun, Jung-Woo Ha, Jinwoo Shin
International Conference on Learning Representations (ICLR), 2024
Sparsified State-Space Models are Efficient Highway Networks
Woomin Song, Jihoon Tack, Sangwoo Mo, Seunghyuk Oh, Jinwoo Shin
NeurIPS Workshop on Efficient Natural Language and Speech Processing (ENLSP), 2024, Oral Presentation
Tabular Transfer Learning via Prompting LLMs
Jaehyun Nam, Woomin Song, Seong Hyeon Park, Jihoon Tack, Sukmin Yun, Jaehyung Kim, Kyu Hwan Oh Jinwoo Shin
Conference on Language Modeling (COLM), 2024
ICML Workshop on Efficient Systems for Foundation Models (ICMLW-ES-FoMo), 2023
Hierarchical Context Merging: Better Long Context Understanding for Pretrained LLMs
Woomin Song*, Seunghyuk Oh*, Sangwoo Mo, Jaehyung Kim, Sukmin Yun, Jung-Woo Ha, Jinwoo Shin
International Conference on Learning Representations (ICLR), 2024
Education
M.S./Ph.D. in Artificial IntelligenceSep. 2022 - Present
Korea Advanced Institute of Science and Technology (KAIST)
B.S. in Electrical Engineering, Computer Science and Mathematics (minor)Mar. 2018 - Aug. 2022
Korea Advanced Institute of Science and Technology (KAIST)
GPA: 4.20/4.3 (Summa Cum Laude)
Experience
Laon PeopleAug. 2021 - Feb. 2022
AI Research Intern
Honors & Awards
Recipient, National Presidental Scholarship for Science
Dean's List, KAIST College of Engineering (2019F, 2020S, 2020F)
Recipient, Korea Foundation for Advanced Studies (KFAS) Undergrad Scholarship
Recipient, KAIST Alumni Association Scholarship
Recipient, KAIST Presidental Fellowship
Recipient, National Science and Engineering Undergraduate Scholarship