Shaoyi Huang

Shaoyi is a 4-th year Ph.D. student in the School of Computing at University of Connecticut, working with Prof. Caiwen Ding and Prof. Omer Khan as a member of Intelligent & Efficient Systems Laboratory. Her research agenda is grounded in advancing AI systems, including algorithm-system co-design for AI acceleration, emerging deep learning models inference acceleration (i.e., Transformer and LLM), privacy preserving machine learning, and machine learning for EDA. Shaoyi’s work has been published in high-impact conferences such as HPCA, ASPLOS, SC, DAC, ICCAD, ACL, ICCV, NeurIPS, IJCAI, etc.

Her work on FPGA-based language models acceleration through sparse attention and dynamic pipelining won DAC'22 Publicity Paper Award. She is the recipient of Cigna Graduate Fellowship in 2021, Eversource Energy Graduate Fellowship in 2022, and Synchrony Fellowship in 2022. She was awarded both Predoctoral Prize for Research Excellence and GE Fellowship for Excellence from UConn in 2022 and 2023. She was invited to serve as a student panelist to share experience with the female researchers and K-12 students at the 6th Workshop for Women in Hardware and Systems Security (WISE).

Shaoyi is on the 2024 academic job market, please do not hesitate to reach out for any relevant job openings.

Email  /  CV  /  Google Scholar  /  Github

profile photo
Publications Teaching Internship Talks Awards Services Misc
Research Interests

News

  • 10/2023: One paper was accepted to ASPLOS 2024.
  • 10/2023: One paper was accepted to HPCA 2024.
  • 10/2023: Received travel grant from CACC. Thanks, CACC!
  • 09/2023: Honored to be invited to be the student panelist and received student travel grant from WISE 2023.
  • 09/2023: One proposal on Efficient Multi-party Computation-based Private Inference was accepted to WISE 2023 (The 6th Workshop for Women in Hardware and Systems Security).
  • 09/2023: One paper was accepted to NeurIPS 2023.
  • 09/2023: Honored to be invited to give a guest lecture at the University of Rochester.
  • 09/2023: Invited to serve as a PC member for SDM 2024.
  • 09/2023: Honored to present at TechCon organized by Semiconductor Research Corporation (SRC). See u in Austin.
  • 07/2023: One paper was accepted to ICCAD 2023.
  • 07/2023: One paper was accepted to ICCV 2023.
  • 07/2023: Invited to serve as a PC member for AAAI 2024.
  • 06/2023: Received the GE Fellowship for Excellence from UConn School of Engineering.
  • 05/2023: Honored to give a talk and a poster presentation at Semiconductor Research Corporation (SRC) AIHW & CADT Annual Review in IBM Research. See u in San Jose.
  • 05/2023: Received the Predoctoral Prize for Research Excellence from UConn CSE.
  • 03/2023: Honored to be invited to give a talk at MLNLP.
  • 03/2023: One paper was accepted to IJCAI 2023.
  • 03/2023: Two papers on sparse training were accepted to DAC 2023. See u in San Francisco.
  • 01/2023: Invited to serve as a reviewer for the First Workshop on DL-Hardware Co-Design for AI Acceleration at AAAI 2023.
  • 12/2022: Invited to serve as a PC member for KDD 2023.
  • 08/2022: Received the GE Fellowship for Excellence from the School of Engineering. Thanks for the support!
  • 07/2022: Invited to serve as a PC member for AAAI 2023.
  • 07/2022: Received the Synchrony Fellowship from Connecticut Advanced Computing Center. Thanks, CACC!
  • 05/2022: Started to work at TikTok (ByteDance) as a research intern.
  • 05/2022: Received the Predoctoral Prize for Research Excellence from UConn CSE.
  • 05/2022: Received the Eversource Energy Graduate Fellowship from Eversource Energy Center. Thanks, Eversource!
  • 04/2022: Our DAC 22 paper “A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining” was recognized as a Publicity Paper.
  • 02/2022: One paper was accepted to ACL 2022.
  • 02/2022: One paper was accepted to DAC 2022.
  • 02/2022: One paper was accepted to ISQED 2022.
  • 08/2021: Received the Cigna Graduate Fellowship from UConn CSE. Thanks, Cigna!
  • 07/2021: One paper was accepted to ICCAD 2021.
  • 06/2021: One paper was accepted to SC 2021.
  • 04/2021: Three papers were accepted to GLSVLSI 2021.
  • 02/2021: One paper was accepted to ISQED 2021.
Selected Publications

Full publication list

2024
3DSP MaxK-GNN: Towards Theoretical Speed Limits for Accelerating Graph Neural Networks Training
H. Peng, X. Xie, K. Shivdikar, M. A. Hasan, J. Zhao, S. Huang, O. Khan, D. Kaeli, C. Ding
[ASPLOS 2024] Paper
3DSP PruneGNN: An Optimized Algorithm-Hardware Framework for Graph Neural Network Pruning
D. Gurevin, M. Shan, S. Huang, M. A. Hasan, C. Ding, O. Khan
[HPCA 2024] Paper
(Acceptance rate: 75/410=18.3%)

2023
3DSP Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off
S. Huang, B. Lei, D. Xu, H. Peng, Y. Sun, M. Xie, C. Ding
[DAC 2023] Paper
(Acceptance rate: 263/1156=22.7%)
3DSP Neurogenesis Dynamics-inspired Spiking Neural Network Training Acceleration
S. Huang, H. Fang, K. Mahmood, B. Lei, N. Xu, B. Lei, Y. Sun, D. Xu, W. Wen, C. Ding
[DAC 2023] Paper
(Acceptance rate: 263/1156=22.7%)
3DSP AutoReP: Automatic ReLU Replacement for Fast Private Network Inference
S. Huang*, H. Peng*, T. Zhou*, W. Wen, X. Xu, C. Ding, et al.
[ICCV 2023] Paper / Code
(Acceptance rate: 26.8%)
3DSP Towards Lossless Head Pruning through Automatic Peer Distillation for Large Language Models
B. Li, Z. Wang, S. Huang, M. Bragin, J. Li, C. Ding
[IJCAI 2023] Paper
(Acceptance rate: 15%)
3DSP Accel-GCN: High-Performance GPU Accelerator Design for Graph Convolution Networks
X. Xie, H. Peng, A. Hasan, S. Huang, J. Zhao, H. Fang, W. Zhang, T. Geng, O. Khan, C. Ding
[ICCAD 2023] Paper / Code
3DSP LinGCN: Structural Linearized Graph Convolutional Network for Homomorphically Encrypted Inference
H. Peng, R. Ran, Y. Luo, J. Zhao, S. Huang, K. Thorat, T. Geng, C. Wang, X.lin Xu, W. Wen, C. Ding
[NeurIPS 2023] Paper / Code
(Acceptance rate: 26.1%)

2022
3DSP Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
S. Huang, D. Xu, I. E. Yen, S. Chang, B. Li, C. Ding, et al.
[ACL 2022] Paper / Code
(Acceptance rate: 714/3350=21.3%)
In collaboration with Moffett AI
3DSP A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining
S. Huang*, H. Peng*, S. Chen, B. Li, T. Geng, A, Li, W. Jiang, W, Wen, J, Bi, H. Liu, C. Ding
[DAC 2022] Paper
Publicity paper award
3DSP Towards Sparsification of Graph Neural Networks
H. Peng, D. Gurevin, S. Huang, T. Geng, W. Jiang, O. Khan, C. Ding
[ICCD 2022] Paper / Code
3DSP Analyzing and Defending against Membership Inference Attacks in Natural Language Processing Classification
Y. Wang, N. Xu, S. Huang, K. Mahmood, D. Guo, C. Ding, W. Wen, S. Rajasekaran
[IEEE BigData 2022] Paper
3DSP CoDG-ReRAM: An Algorithm-Hardware Co-design to Accelerate Semi-Structured GNNs on ReRAM
Y. Luo, P. Behnam, K. Thorat, Z. Liu, H. Peng, S. Huang, S. Zhou, O. Khan, A. Tumanov, C. Ding, T. Geng
[ICCD 2022] Paper

2021
3DSP E.T.: Re-Thinking Self-Attention for Transformer Models on GPU
S. Huang*, S. Chen*, S. Pandey, B. Li, G. Gao, C. Ding, H. Liu
[SC 2021] Paper / Code
(Acceptance rate: 86/365=23.6%)
3DSP Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization
P. Qi, E. Sha, Q. Zhuge, H. Peng, S. Huang, Z. Kong, Y. Song, B. Li
[ICCAD 2021] Paper
3DSP HMC-Tran: A Tensor-core Inspired Hierarchical Model Compression for Transformer-based DNNs on GPU
S. Huang, S. Chen, H. Peng, D. Manu, Z. Kong, G. Yuan, L. Yang, S. Wang, H. Liu, C. Ding
[GLSVLSI 2021] Paper
3DSP Accelerating Transformer-based Deep Learning Models on FPGAs Using Column Balanced Block Pruning
H. Peng, S. Huang, T. Geng, A. Li, W. Jiang, H. Liu, W. Wang, C. Ding
[ISQED 2021] Paper
Teaching
  • Guest Lecturer

    • ECE 403-1: Advanced Computer Architecture for Machine Learning, Sep 2023, University of Rochester

  • Teaching Assistant at University of Connecticut

    • CSE 4502/5717 - BigData Analytics, Spring 2023
      Instructor: Prof. Suining He   

    • CSE5819 - Introduction to Machine Learning, Fall 2022
      Instructor: Prof. Fei Miao   
      As a co-instructor, I designed final project, developed and delivered coding tutorials, led final project presentation courses, supervised students on course projects, held office hours and graded assignments.

  • Teaching Assistant at University of Rochester

    • Microcontroller, Spring 2018
      Instructor: Prof. Qiang Lin   

    • Circuits & Signals LAB, Fall 2017
      Instructor: Prof. Jack G. Mottley   
      I led the laboratory sessions and provided hands-on instruction to around 100 undergraduate students.

  • Mentering

    • Amit Hasan, Ph.D. Student at University of Connecticut
      Topic: LLM Inference Acceleration

    • Yifan Shan, Undergraduate at University of Connecticut (Now master at Cornell Tech)
      Project: Evaluating the Impact of Preferential Trade Agreements on Agricultural and Food Trade: New Insights from Natural Language Processing and Machine Learning

    • Jiwon Kim, Undergraduate at University of Connecticut
      Project (NSF-REU): Utilization of DeepShift for Privacy Based Machine Learning

    • Alison Menezes, Undergraduate at Clemson University
      Project (NSF-REU): Deep Leakage from Gradients on GNNs

    • Maryam Abuissa, Undergraduate at Amherst College
      Project (NSF-REU): Sequestered Encryption for GPU

Internship

  • May 2022 - Aug 2022, Research Intern, TikTok, Austin, TX

  • May 2021 - Aug 2021, Research Intern, Moffett AI, Los Altos, CA

Talks

  • Student panelist - The 6th Workshop for Women in Hardware and Systems Security
    Oct 2023, California State University, Fullerton, Fullerton, CA

  • Towards Efficient Model Inference and Training
    Sep 2023, University of Rochester
    Guest Lecture of ECE 403-1: Advanced Computer Architecture for Machine Learning.
    Host: Prof. Tony Geng

  • Towards Efficient Training and Inference Under Pretrain-and-Finetune Paradigm
    Sep 2023, TechCon - Semiconductor Research Corporation (SRC), Austin, TX

  • Exploring Extreme Sparsity in Training and Inference for Graph Neural Networks to Achieve High Performance Scaling on Large Core Count Machines
    May 2023, Semiconductor Research Corporation (SRC) AIHW \& CADT Annual Review, IBM Research, San Jose, CA

  • Towards Efficient Model Inference and Training
    Apr 2023, Machine Learning and Natural Language Processing Community (MLNLP)
    MLNLP Outstanding Speaker

Honors and Awards
  • NeurIPS Travel Grant from CACC, 2023
  • WISE Student Travel Award, 2023
  • GE Fellowship of Excellence, 2023
  • Predoctoral Prize for Research Excellence, 2023
  • MLNLP Outstanding Speaker, 2023
  • GE Fellowship for Excellence, 2022
  • Synchrony Fellowship, 2022
  • DAC Publicity Paper Award, 2022
  • Predoctoral Prize for Research Excellence, 2022
  • Eversource Energy Graduate Fellowship, 2022
  • DAC Young Fellow, 2021
  • Cigna Graduate Fellowship, 2021
Professional Services
  • Panelist
    • The 6th Workshop for Women in Hardware and Systems Security (WISE)
  • Program Committee Member
    • ACL'24
    • SDM'24
    • AAAI'24
    • NeurIPS'23 Datasets and Benchmarks
    • KDD'24, 23
    • AAAI'23
    • DCAA'23
  • Conference External Reviewer
    • NeurIPS'23
    • DAC'23, 22, 21
    • ICCAD'23, 22, 21
    • AAAI'23, 22, 21
    • EMNLP'23, 22
    • IJCAI'23, 22, 21
    • ICCD'23, 22
    • EACL'22
    • FPGA'21
    • NAACL'21
  • Journal Reviewer
    • Neurocomputing
    • Pattern Recognition
    • Engineering Applications of Artificial Intelligence
    • Neural Networks
    • Journal of Systems Architecture
    • IEEE Transactions on Neural Networks and Learning Systems
  • Journal External Reviewer
    • ACM Transactions on Architecture and Code Optimization
    • IEEE Transactions on Neural Networks and Learning Systems
    • IEEE Network Magazine
    • Journal on Emerging Technologies in Computing Systems
Misc

  • Home aquarium:

  • Playing with Nobel:

    nobel

  • Painting:

    watermelon
    watermelon

  • Outdoor activities: mushroom hunting (Boletus edulis, Honey mushroom, Chicken of the Woods, Chanterelles I found in Connecticut); sea foraging (crab, oyster, clam and mussel I found in Connecticut, Long island, Boston, New Jersey):

    mushroom mushroom mushroom mushroom
    crab oyster shell mussel

Last updated on 12/2023 EDT.
Webpage template