Jieneng Chen

Jieneng is a fourth-year Ph.D. candidate in Computer Science at Johns Hopkins University, advised by Distinguished Professor Alan Yuille.

His research primarily focuses on designing intelligent encoders that convert highly complex visual signals into semantic and structured representations, which are more easily decoded for intelligent tasks such as perception, generation, and action. This transformation is crucial for making complex raw signals more comprehensible by machines, marking a significant step toward achieving human-level artificial intelligence.

During his PhD journey, he co-develops a few highly-cited algorithms in computer vision and medical applications, which are among the top 15 cited 2021 paper in all AI fields (TransUNet), top 3 most influential AAAI 2023 papers (TransFG), and top 3 most cited ECCV papers in five years in Google Metrics (SwinUNet).


profile photo
  • [July 2024] Check out LLaVolta, an efficient large multi-modal model.
  • [June 2024] Our multi-modal foundation model, the ViTamin model, has been incorporated into two leading machine learning codebases: timm GitHub Stars Badge and open_clip GitHub Stars Badge. The model is downloaded 1K+ times per month on HuggingFace.
Selected Publications

Full list on Google Scholar Profile. His publications have over 8,000 citations (as of Jul. 2024) with an increase of over 5,000 per year.

Designing Scalable Vision Models in the Vision-Language Era
Jieneng Chen, Qihang Yu, Xiaohui Shen, Alan Yuille, Liang-Chieh Chen

In Conference on Computer Vision and Pattern Recognition (CVPR), 2024
Paper | Code | 🤗 HuggingFace | Timm | OpenCLIP

LLaVolta: Efficient Multi-modal Models via Stage-wise Visual Context Compression
Jieneng Chen, Luoxin Ye, Ju He, Zhao-Yang Wang, Daniel Khashabi, Alan Yuille

Technical Report, 2024
Paper | Code | Project

Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers
Jieneng Chen, Yingda Xia, Jiawen Yao, Ke Yan, Jianpeng Zhang, Le Lu, .., Jingren Zhou, Alan Yuille, Zaiyi Liu, Ling Zhang

In International Conference on Computer Vision (ICCV), 2023

Compositor: Bottom-up Clustering and Compositing for Robust Part and Object Segmentation
Ju He *, Jieneng Chen *, Mingxian Lin, Qihang Yu, Alan Yuille

In Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Paper | * Equal contributed

TransMix: Attend to Mix for Vision Transformers
Jieneng Chen, Shuyang Sun, Ju He, Philip Torr, Alan Yuille, Song Bai

In Conference on Computer Vision and Pattern Recognition (CVPR), 2022
Paper | Code

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan Yuille, Yuyin Zhou

International Conference on Marchine Learning (ICML) workshop, arXiv 2021

3D-TransUNet is published in Medical Image Analysis, 2024 (IF>10)
Paper | Code | GitHub Stars Badge

Top 15 Cited 2021 Paper in All AI Fields (Cited 3800 times as of Jul. 2024) [Source]

TransFG: A Transformer Architecture for Fine-grained Recognition
Ju He, Jieneng Chen, Shuai Liu, Adam Kortylewski, Cheng Yang, Yutong Bai, Changhu Wang, Alan Yuille

In AAAI Conference on Artificial Intelligence (AAAI), 2022
Paper | Code

Top 3 Most Influential AAAI 2023 Papers [Source]

Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
Hu Cao, Yueyue Wang, Jieneng Chen, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, Manning Wang

In European Conference on Computer Vision (ECCV), 2022
Paper | Code | GitHub Stars Badge

Top 3 Most Cited ECCV Papers in Five Years According to Google Metrics [Source]

Semi-supervised Medical Image Segmentation Through Dual-task Consistency
Xiangde Luo, Jieneng Chen, Tao Song, Guotai Wang

In AAAI Conference on Artificial Intelligence (AAAI), 2021
Paper | Code

Top 15 Most Influential AAAI 2021 Papers [Source]

Sequential Learning on Liver Tumor Boundary Semantics and Prognostic Biomarker Mining
Jieneng Chen, Ke Yan, Yu-Dong Zhang, Youbao Tang, Xun Xu, Shuwen Sun, Qiuping Liu, Lingyun Huang, Jing Xiao, Alan L Yuille, Ya Zhang, Le Lu

In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2021
Paper | Early Accept | Travel Award (top 10%)

Service & Teaching
  • Serving: He is invited reviewers and program committees for major conference and journals, such as TPAMI, CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, TMI and MICCAI. He is on the workshop organizing committee for MICCAI and ISBI.