I am a final-year PhD student in the School of ECE at Purdue University, advised by Dr. Stanley Chan. I obtained my BASc in Computer Engineering and Mathematics at the University of Toronto.

I am interested in both the theoretical and applied sides of deep learning. My research interests primarily revolve around developing a mechanistic understanding of deep neural networks, towards the goal of improving the generalization and interpretability of these models.

In particular, my recent efforts have been focused on studying how large language models (LLMs) reason. In a recent work, we studied how tiny transformers and LLMs (Mistral-7B and Gemma-2-9B) solve simple propositional logic problems. We were able to characterize circuits employed by the models for performing latent multi-hop reasoning. The surprisingly localized and human-interpretable nature of the circuits point to the possibilities of debugging and steering the LLMs’ reasoning actions in the future.

I have also developed mathematical theories which connect generalization performance of DNNs with their feature-learning process, in the settings of transfer learning, and feature-based distillation.

News

(June 2024) Started student researcher position at Google Research with Dr. Rina Panigrahy.

(Oct 2022) Started student researcher position at Google Research with Dr. Yin Cui and Dr. Enming Luo.