My research focuses on artificial intelligence, computer vision, multimodality, medical AI and embodied AI. I work with Profs. Daniel Khashabi, Tianmin Shu, and Rama Chellappa in DSAI and CLSP.
I am on the job market for 2025! Would love to chat more if you are interested. I am also happy to give talks on my research in related seminars.
Research Themes
My research is driven by a desire to understand the computational foundations of human-level and expert-level intelligence. I strive to replicate human intelligence by developing general-purpose spatial and embodied intelligence, and to build expert-level intelligence to address challenges like health inequity and climate change.
Spatial Intelligence:
my vision is grounded in interpreting 3D spatial configurations and advancing spatial reasoning capabilities.
(1) develop vision models capable of predicting 3D orientation and geometry of objects from the monocular image and video.
(2) benchmark and bridge the (significant) gap in 3D spatial reasoning capabilities between humans and GPT models.
Embodied Intelligence:
🧠 I am developing human-like multi-sensory embodied agents and working towards addressing critical challengs in embodied AI.
(1) develop embodied large multimodal models through the vision-focused and spatial-aware redesign.
(2) build generative world explorer to address the challenge of planning with partial observations.
(3) equip embodied agents with 3D spatial intelligence in the dynamic physical world.
Medical Intelligence:
my vision is grounded in foundational algorithms and scalable, clinically relevant applications to address health inequity 🙏.
(1) algorithms: I developed the TransUNet, pioneering the Transformer era in medical image analysis.
(2) clinical applications: I developed the world's first AI model capable of detecting and diagnosing eight major cancers.
Once detected, I monitored the cancer's prognosis using multi-scale spatial analysis in CT and IHC.
(3) Spatial and embodied intelligence holds immense potential to revolutionize patient care through homecare agents.
News
I will be offering a brand new course 'Machine Imagination' for students of all levels at JHU in Winter 2025.
selected as a Siebel Scholar, recognizing exceptional students from the world’s leading graduate schools.
Medical Image Analysis (MedIA), 2024
ICML-W 2021 |
Journal |
Code |
Top ScienceDirect downloaded articles in 2024, published all time
Top 15 cited 2021 paper in all AI fields (cited 5K times as of 2024) for arXiv version[Source]
Teaching & Service
Instructor: He is the instructor for a course in Winter 2025 at JHU.
CSE290D: Neural Computation, at UC Santa Cruz, 2024.
Serving: He is on the invited reviewers and program committees for major conference and journals, such as CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, AAAI, TPAMI, TMI and MICCAI. He provided mentor hours for PhD students and underrepresented students at JHU.
and open_clip 
.
