Weijian Deng
I am a Research Fellow at the Australian National University, collaborating with
Prof. Stephen Gould. My research primarily focuses on
3D Content Modeling and Machine Learning Safety. I am also fortunate to work alongside
Dr. Dylan Campbell.
Previously, I pursued my PhD at the same institution, where I explored model generalization prediction. My work involved close collaboration with
Prof. Stephen Gould,
Dr. Yumin Suh, and
Dr. Liang Zheng.
Robust Foundational Vision-Language Models: My research centers on improving the robustness and reliability of AI systems, with a particular emphasis on vision-language models.
I am currently exploring model generalization prediction, developing methods to estimate model accuracy on unlabeled test sets.
Additionally, I investigate model robustness, focusing on the geometric understanding within CLIP models and assessing their sensitivity to variations in visual factors.
Developing Advanced Simulation and 3D Modeling Techniques: I am also interested in
creating advanced simulation frameworks using neural rendering and large language models
(LLMs). These simulations are designed to enhance AI adaptability and reliability, especially
in autonomous systems, by accurately reflecting complex, real-world scenarios and incorpo-
rating physics-based principles into 3D modeling.
Email  / 
CV  / 
Google
Scholar
 / 
Github
 / 
Linkedin
 / 
PhD Thesis
|
|
News
[Sep 2024] One paper on model generalization prediction is accepted to NeurIPS 2024 [paper]
[May 2024] One paper on unsupervised model ranking is accepted to TMLR 2024 [paper]
[May 2024] One paper on calibration analysis of VLM is accepted to ICML 2024 [paper]
[Mar 2024]
I'm honored to serve as an Action Editor for [Transactions on Machine Learning Research],
a new open-review journal in machine learning.
I warmly invite you to join us in this innovative publication journey.
[Feb 2024] One paper on 3D reconstruction is accepted to CVPR 2024
[Jan 2024] Excited for my Journey with HEX International Singapore's Youth Entrepreneurship Programs
[My Reflection]
[Dec 2023] Enthusiastically teaching Introduction to Computer Science at
Beloved SDUW
[Nov 2023] My PhD thesis is available on the [ANU Open Research]
[Nov 2023] Honoured to be recognized by NeurIPS 2023 as a top reviewer
[Oct 2023] One paper on novel view synthesis of refrative objects is accepted to WACV
2024 [paper]
[Sep 2023] One paper on robustness of visual foundation model is accepted to NeurIPS
2023 [Paper]
[Jul 2023] One paper on (out-of-distribution) predictive calibration is accepted to
ICCV 2023 [Paper]
[Apr 2023] One paper on model (out-of-distribution) generalization prediction is
accepted to ICML 2023 [Paper]
[Feb 2023] One paper on dataset-level analysis is accepted to CVPR 2023 [Paper]
[Jan 2023] Started the Research Fellow position
[Jan 2023] Submitted PhD thesis
[Dec 2022] Completed PhD oral presentation
[Oct 2022] One paper on multi-task learning is accepted to WACV 2023 [Paper]
[Oct 2022] Honoured to receive NeurIPS 2022 Scholar Award (Travel Award)
[Sep 2022] One paper on model invariance and generalization is accepted to NeurIPS
2022
[Paper]
[Jul 2022] Honoured to be recognized by ICML 2022 as a top 10% reviewer
[Jun 2022] Organized the CVPR 2022 Tutorial on ["Evaluating
Models Beyond the Textbook: Out-of-distribution and
Without Labels"]
[May 2022] One paper on fine-grained clasification is accepted to IEEE TIP [Paper]
[Dec 2021] One paper on model decision understanding is accepted to IEEE TPAMI
[Project]
[May 2021] One paper on generalization prediction is accepted to ICML 2021 [Paper,
Project]
[Mar 2021] One paper on generalization prediction is accepted to CVPR 2021 [Paper,
Project]
[Jul 2020 - Sep 2020] Summer intern in NEC Labs America. Wonderful experience
with my talented mentors Yumin Suh, Xiang Yu, Masoud Faraki, and Manmohan Chandraker.
[Aug 2020] VisDA-2020 challenge successfully ends. Congratulations to the [final
teams]!
Big thanks to committee members Kate Saenko, Liang Zheng, and Xingchao Peng!
[Jun 2020] Honoured to be recognized by ECCV2020 as a top reviewer
[Jan 2020] One paper is accepted to IEEE TCSVT [Paper]
[Jul 2019] Study PhD program at The Australian National University. My reseach is
supported by Australian Governmen Scholarship [AGRTP].
[Jun 2019] Receive M.Eng. from the University of the Chinese Academy of Sciences
[Jun 2019] Win 3rd place out of 84 participants in vehicle re-identification in
CVPR 2019 AI-City Challenge
[ Paper,
Code]
[Jul 2018 - Nov 2018] Reserch assistant in Singapore University of Technology and
Design (SUTD)
[Mar 2018] One paper is accepted to CVPR 2018 [Paper,
Code]
[Jul 2017] One paper is accepted to ICCV 2017 [Paper,
Code]
|
Research
My research works focus on
Predicting Model Generalization,
Monitoring Model Reliability,
Enhancing Visual Recognition, and
3D Modeling & Generation.
|
I. Predicting Model Generalization
PhD Research Topic.
At the core of the research lies the quest to unravel how deep neural networks interact with
data. This exploration has led to innovative criteria designed to predict model resilience
without human annotations.
It provides insights to identify and address failure cases, guiding enhancements in future
model training.
Here are [some slides] to introduce my research.
|
MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts
Renchunzi Xie, Ambroise Odonnat, Vasilii Feofanov, Weijian Deng, Jianfeng Zhang, Bo An
NeurIPS 2024
[Paper,
BibTex]
|
|
What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?
Weijie Tu, Weijian Deng, Liang Zheng, Tom Gedeon
TMLR 2024
[Paper,
BibTex]
|
|
A Bag-of-Prototypes Representation for Dataset-Level Applications
Weijie Tu, Weijian Deng, Tom Gedeon, Liang Zheng
CVPR 2023
[Paper,
Code,
BibTex,
Poster]
|
|
Confidence and Dispersity Speak: Characterising Prediction Matrix for
Unsupervised Accuracy Estimation
Weijian Deng, Yumin Suh, Stephen Gould, Liang Zheng
ICML 2023
[Paper,
Slides,
Poster,
BibTex,
Codes
]
|
|
AutoEval: Are Labels Always Necessary for Classifier Accuracy
Evaluation?
Weijian Deng, Liang Zheng
IEEE TPAMI 2022 [Project,
BibTex,
Paper]
|
|
What Does Rotation Prediction Tell Us about Classifier Accuracy under
Varying Testing Environments?
Weijian Deng, Stephen Gould, Liang Zheng
ICML 2021 (Spotlight) [Paper,
BibTex, Project,
Slides,
Poster]
|
|
Are Labels Always Necessary for Classifier Accuracy Evaluation?
Weijian Deng, Liang Zheng
CVPR 2021 [Paper,
Project,
BibTex,
Poster,
Slides]
|
II. Monitoring Model Reliability
|
An Empirical Study into What Matters for Calibrating Vision-Language Models
Weijie Tu, Weijian Deng, Dylan Campbell, Stephen Gould, Tom Gedeon
ICML 2024
[Paper,
BibTex,
]
|
|
A Closer Look at the Robustness of Contrastive Language-Image
Pre-Training (CLIP)
Weijie Tu, Weijian Deng, Tom Gedeon
NeurIPS 2023
[Paper,
BibTex,
OpenReview
]
|
|
Adaptive Calibrator Ensemble: Navigating Test Set Difficulty in
Out-of-Distribution Scenarios
Yuli Zou*, Weijian Deng* (equal contribution), Liang Zheng
ICCV 2023 [Paper,
Poster,
Code,
BibTex
]
|
|
On the Strong Correlation Between Model Invariance and
Generalization
Weijian Deng, Stephen Gould, Liang Zheng
NeurIPS 2022 (Spotlight) [Paper,
OpenReview,
Slides,
Poster,
BibTex]
|
|
Ranking Models in Unlabeled New Environments
Xiaoxiao Sun, Yunzhong Hou, Weijian Deng, Hongdong Li, Liang
Zheng
ICCV 2021 [Paper,
Code,
BibTex
]
|
III. 3D Modeling & Generation
|
Ray Deformation Networks for Novel View Synthesis of Refractive Objects
Weijian Deng, Dylan Campbell, Chunyi Sun, Shubham Kanitkar, Matthew Shaffer, Stephen Gould.
WACV 2024 [
Project,
Paper,
Poster,
Slides,
BibTex
]
|
|
Differentiable Neural Surface Refinement for Transparent Objects
Weijian Deng, Dylan Campbell, Chunyi Sun, Shubham Kanitkar, Matthew Shaffer, Stephen Gould.
CVPR 2024
[
Project,
Poster,
Slides,
BibTex
]
|
IV. Enhancing Visual Recognition
|
Image-Image Domain Adaptation with Preserved Self-Similarity and
Domain-Dissimilarity for Person Re-identification
Weijian Deng, Liang Zheng, Qixiang Ye, Guoliang Kang, Yi Yang,
Jianbin
Jiao
CVPR 2018 [Paper,
Code,
BibTex,
Poster]
|
|
Similarity-preserving Image-image Domain Adaptation for Person
Re-identification
Weijian Deng, Liang Zheng, Qixiang Ye, Yi Yang, Jianbin Jiao
Arxiv 2019 [Paper]
|
|
Domain Alignment with Triplets
Weijian Deng, Liang Zheng, Jianbin Jiao
Arxiv 2019 [Paper]
|
|
Rethinking Triplet Loss for Domain Adaptation
Weijian Deng, Liang Zheng, Yifan Sun, Jianbin Jiao
IEEE TCSVT 2020 [Paper,
BibTex]
(Journal version of "Domain alignment with
triplets")
|
|
Fine-grained Classification via Categorical Memory Networks
Weijian Deng, Joshua Marsh, Stephen Gould, Liang Zheng
IEEE TIP 2022 [
BibTex,
Paper
]
|
|
Split to Learn: Gradient Split for Multi-Task Human Image
Analysis
Weijian Deng, Yumin Suh, Xiang Yu, Masoud Faraki, Liang Zheng,
Manmohan
Chandraker
WACV 2023 [
Paper,
US Patent,
Poster,
Slides,
BibTex
]
|
|
SVDNet for Pedestrian Retrieval
Yifan Sun, Liang Zheng, Weijian Deng, Shengjin Wang
ICCV 2017 (Spotlight) [Paper,
Code,
BibTex,
Poster]
|
Academic Activity
Action Editor, Transactions on Machine Learning Research
Lecturer, "Introduction to Computer Science", SDUW (Joint ANU-SDUW Program, Winter
Semester 2023)
ACM Multimedia 2024 Area Chair
Reviewer: NeurIPS 2022-2023; ICML 2022-2024; ICLR 2022-2023; ICCV 2021, 2023; CVPR
2021-2024; ECCV 2020; ACM MM 2020-2023; IEEE-TPAMI; IEEE-TIP; IJCV
Co-organizer: ECCV 2020 Workshop on "Visual Domain Adaptation Challenge"
Co-organizer: CVPR 2022 Tutorial on "Evaluating Models Beyond the Textbook:
Out-of-distribution and Without Labels"
Guest speaker: SUTD 2018/12 (image-image translation); ANU 2019/09 (SVDNet)
|
Awards & Honors
NeurIPS 2023 Top Reviewer, 2023
ICML 2022 Top 10% Reviewer, 2022
ECCV 2020 Outstanding Reviewer, 2022
Australian Government Research Training Program (AGRTP) Scholarship, 2019-2023
The Third Place in Vehicle Re-identification track of CVPR 2019 AI-City Challenge, 2019
China National Scholarship (Master), 2018
China National Scholarship (Bachelor), 2014, 2015
|
Grants & Funds
Academic research grant from the Google PaliGemma Academic Program, 2024 - 2025
Academic research grant from the Google Cloud Research Credits Program, 2024 - 2025
ICML Early-Career Travel Fund, 2024
ANU Early-Career Travel Fund, 2024
ANU-SDUW Teaching Fellowship, 2023
NeurIPS 2022 Scholar Award, 2022
|
|