Weijian Deng

I am a Research Fellow at the Australian National University, working with Prof Stephen Gould. My current research focuses are 3D Content Modeling and Machine Learning Safety.
Previously, I was a PhD student at the Australian National University, where I worked on model generalization prediction, working with Prof Stephen Gould, Dr Yumin Suh, and Dr Liang Zheng.

Machine Learning Safety: Concentrated on the safety of large language models and multimodal models, with a focus on improving resilience and reliability. Emphasis is placed on developing robust models for varied environments and creating monitoring mechanisms to detect misuse and analyze failure patterns.

3D Content Modeling & Generation: Focused on the advancement of 3D modeling and generation, specifically for refractive objects. By applying optical principles, it aims to significantly enhance the realism and accuracy of 3D objects and scenes.

Email  /  CV  /  Google Scholar  /  Github  /  Linkedin  /  PhD Thesis

News

  • [Mar 2024] I'm honored to serve as an Action Editor for [Transactions on Machine Learning Research], a new open-review journal in machine learning. I warmly invite you to join us in this innovative publication journey.
  • [Feb 2024] One paper on 3D reconstruction is accepted to CVPR 2024
  • [Jan 2024] Excited for my Journey with HEX International Singapore's Youth Entrepreneurship Programs [My Reflection]
  • [Dec 2023] Enthusiastically teaching Introduction to Computer Science at Beloved SDUW
  • [Nov 2023] My PhD thesis is available on the [ANU Open Research]
  • [Nov 2023] Honoured to be recognized by NeurIPS 2023 as a top reviewer
  • [Oct 2023] One paper on novel view synthesis of refrative objects is accepted to WACV 2024 [paper]
  • [Sep 2023] One paper on robustness of visual foundation model is accepted to NeurIPS 2023 [Paper]
  • [Jul 2023] One paper on (out-of-distribution) predictive calibration is accepted to ICCV 2023 [Paper]
  • [Apr 2023] One paper on model (out-of-distribution) generalization prediction is accepted to ICML 2023 [Paper]
  • [Feb 2023] One paper on dataset-level analysis is accepted to CVPR 2023 [Paper]
  • [Jan 2023] Started the Research Fellow position
  • [Jan 2023] Submitted PhD thesis
  • [Dec 2022] Completed PhD oral presentation
  • [Oct 2022] One paper on multi-task learning is accepted to WACV 2023 [Paper]
  • [Oct 2022] Honoured to receive NeurIPS 2022 Scholar Award (Travel Award)
  • [Sep 2022] One paper on model invariance and generalization is accepted to NeurIPS 2022 [Paper]
  • [Jul 2022] Honoured to be recognized by ICML 2022 as a top 10% reviewer
  • --- click to read more ---

  • [Jun 2022] Organized the CVPR 2022 Tutorial on ["Evaluating Models Beyond the Textbook: Out-of-distribution and Without Labels"]
  • [May 2022] One paper on fine-grained clasification is accepted to IEEE TIP [Paper]
  • [Dec 2021] One paper on model decision understanding is accepted to IEEE TPAMI [Project]
  • [May 2021] One paper on generalization prediction is accepted to ICML 2021 [Paper, Project]
  • [Mar 2021] One paper on generalization prediction is accepted to CVPR 2021 [Paper, Project]
  • [Jul 2020 - Sep 2020] Summer intern in NEC Labs America. Wonderful experience with my talented mentors Yumin Suh, Xiang Yu, Masoud Faraki, and Manmohan Chandraker.
  • [Aug 2020] VisDA-2020 challenge successfully ends. Congratulations to the [final teams]!
  • Big thanks to committee members Kate Saenko, Liang Zheng, and Xingchao Peng!
  • [Jun 2020] Honoured to be recognized by ECCV2020 as a top reviewer
  • [Jan 2020] One paper is accepted to IEEE TCSVT [Paper]
  • [Jul 2019] Study PhD program at The Australian National University. My reseach is supported by Australian Governmen Scholarship [AGRTP].
  • [Jun 2019] Receive M.Eng. from the University of the Chinese Academy of Sciences
  • [Jun 2019] Win 3rd place out of 84 participants in vehicle re-identification in CVPR 2019 AI-City Challenge
  • [Paper, Code]
  • [Jul 2018 - Nov 2018] Reserch assistant in Singapore University of Technology and Design (SUTD)
  • [Mar 2018] One paper is accepted to CVPR 2018 [Paper, Code]
  • [Jul 2017] One paper is accepted to ICCV 2017 [Paper, Code]
  • Research

    My research works focus on Predicting Model Generalization, Monitoring Model Reliability, Enhancing Visual Recognition, and 3D Modeling & Generation.

    Predicting Model Generalization

    PhD Research Topic. At the core of the research lies the quest to unravel how deep neural networks interact with data. This exploration has led to innovative criteria designed to predict model resilience without human annotations. It provides insights to identify and address failure cases, guiding enhancements in future model training. Here are [some slides] to introduce my research.

    A Bag-of-Prototypes Representation for Dataset-Level Applications

    Weijie Tu, Weijian Deng, Tom Gedeon, Liang Zheng

    CVPR 2023 [Paper, Code, BibTex, Poster]

    Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation

    Weijian Deng, Yumin Suh, Stephen Gould, Liang Zheng

    ICML 2023 [Paper, Slides, Poster, BibTex, Codes ]

    AutoEval: Are Labels Always Necessary for Classifier Accuracy Evaluation?

    Weijian Deng, Liang Zheng

    IEEE TPAMI 2022 [Project, BibTex, Paper]

    What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?

    Weijian Deng, Stephen Gould, Liang Zheng

    ICML 2021 [Paper, BibTex, Project, Slides, Poster]

    Are Labels Always Necessary for Classifier Accuracy Evaluation?

    Weijian Deng, Liang Zheng

    CVPR 2021 [Paper, Project, BibTex, Poster, Slides]

    Monitoring Model Reliability

    A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)

    Weijie Tu, Weijian Deng, Tom Gedeon

    NeurIPS 2023 [Paper, BibTex, OpenReview ]

    Adaptive Calibrator Ensemble: Navigating Test Set Difficulty in Out-of-Distribution Scenarios

    Yuli Zou*, Weijian Deng* (equal contribution), Liang Zheng

    ICCV 2023 [Paper, Poster, Code, BibTex ]

    On the Strong Correlation Between Model Invariance and Generalization

    Weijian Deng, Stephen Gould, Liang Zheng

    NeurIPS 2022 (Spotlight) [Paper, OpenReview, Slides, Poster, BibTex]

    Ranking Models in Unlabeled New Environments

    Xiaoxiao Sun, Yunzhong Hou, Weijian Deng, Hongdong Li, Liang Zheng

    ICCV 2021 [Paper, Code, BibTex ]

    3D Modeling & Generation

    Ray Deformation Networks for Novel View Synthesis of Refractive Objects

    Weijian Deng, Dylan Campbell, Chunyi Sun, Shubham Kanitkar, Matthew Shaffer, Stephen Gould.

    WACV 2024 [ Paper, Poster, Slides, BibTex ]

    Differentiable Neural Surface Refinement for Transparent Objects

    Weijian Deng, Dylan Campbell, Chunyi Sun, Shubham Kanitkar, Matthew Shaffer, Stephen Gould.

    CVPR 2024 [ BibTex ]

    Enhancing Visual Recognition

    Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification

    Weijian Deng, Liang Zheng, Qixiang Ye, Guoliang Kang, Yi Yang, Jianbin Jiao

    CVPR 2018 [Paper, Code, BibTex, Poster]

    Similarity-preserving Image-image Domain Adaptation for Person Re-identification

    Weijian Deng, Liang Zheng, Qixiang Ye, Yi Yang, Jianbin Jiao

    Arxiv 2019 [Paper]

    Domain Alignment with Triplets

    Weijian Deng, Liang Zheng, Jianbin Jiao

    Arxiv 2019 [Paper]

    Rethinking Triplet Loss for Domain Adaptation

    Weijian Deng, Liang Zheng, Yifan Sun, Jianbin Jiao

    IEEE TCSVT 2020 [Paper, BibTex]

    (Journal version of "Domain alignment with triplets")

    Fine-grained Classification via Categorical Memory Networks

    Weijian Deng, Joshua Marsh, Stephen Gould, Liang Zheng

    IEEE TIP 2022 [ BibTex, Paper ]

    Split to Learn: Gradient Split for Multi-Task Human Image Analysis

    Weijian Deng, Yumin Suh, Xiang Yu, Masoud Faraki, Liang Zheng, Manmohan Chandraker

    WACV 2023 [ Paper, US Patent, Poster, Slides, BibTex ]

    SVDNet for Pedestrian Retrieval

    Yifan Sun, Liang Zheng, Weijian Deng, Shengjin Wang

    ICCV 2017 (Spotlight) [Paper, Code, BibTex, Poster]

    Academic Activity

    Lecturer, Introduction to Computer Science, SDUW (Joint ANU-SDUW Program, Winter Semester 2023)

    ACM Multimedia 2024 Area Chair

    Reviewer: NeurIPS 2022-2023; ICML 2022-2024; ICLR 2022-2023; ICCV 2021, 2023; CVPR 2021-2024; ECCV 2020; ACM MM 2020-2023; IEEE-TPAMI; IEEE-TIP; IJCV

    Co-organizer: ECCV 2020 Workshop on "Visual Domain Adaptation Challenge"

    Co-organizer: CVPR 2022 Tutorial on "Evaluating Models Beyond the Textbook: Out-of-distribution and Without Labels"

    Guest speaker: SUTD 2018/12 (image-image translation); ANU 2019/09 (SVDNet)

    Awards & Honors

    NeurIPS 2023 Top Reviewer, 2023

    NeurIPS 2022 Scholar Award, 2022

    ICML 2022 Top 10% Reviewer, 2022

    ECCV 2020 Outstanding Reviewer, 2022

    Australian Government Research Training Program (AGRTP) Scholarship, 2019-2023

    The Third Place in Vehicle Re-identification track of CVPR 2019 AI-City Challenge, 2019

    China National Scholarship (Master), 2018

    China National Scholarship (Bachelor), 2014, 2015


    Website design from Jon Barron.