Tao Wang
I am a fourth-year Ph.D. student at National University of Singapore, generously supported by the Institue of Data Science scholarship, advised by Dr. Feng Jiashi, and Dr. Wang Xinchao. I was externally supervised by Prof. Yan Shuicheng during 2020-2021. Prior to Ph.D., I obtained my bachelor degree from Yingcai Honors College (elite school, candidates selected from top 5% undergraduates), University of Electronic Science and Technology of China.
Email address: Â twangnh[dot]ai[at]gmail[dot]com
Other links:Â Google Scholar, GitHub
Research Statement
I'm developing Machine Learning approaches for visual perception in the open world. The major focus is designing effective methods to handle practical challenges in real-world non-ideal environments. Specifically, I'm taking two perspectives: 1) Handle open-world data that may be long-tailed, biased, unlabeled, or contains unknown, anomaly concepts. 2) Develop efficient learning and inference methods to gain superior training data efficiency and runtime efficiency, these include knowledge distillation, X-shot & Y-supervised learning, domain adaptation, and network architecture design.
Research Interests
My current interest spans object and human recognition, detection, and segmentation, with the following topics:
Open-world Data:Â
long-tailed/imbalanced data, biased data, unlabeled data, high-resolution data, multi-modal data
Efficient Learning Methods:
knowledge distillation, open-vocabulary/set recognition, domain adaptation, self/weakly-supervised learning
Architecture Design:
     transformers for instance-level understanding, network design for mobile applications
I'm always open to research collaboration and discussion, please feel free to contact me!
News
(2023/02) Our Object Detector Distillation technique is implemented in Yolov5!Â
(2022/12) CRAT is available on arxiv!
(2022/11) CondHead is available on arxiv!
(2022/09) MvP is integrated into XRMoCap, a new open-source PyTorch-based codebase for the use of multi-view motion capture, from OpenXRLab
(2022/03) One paper is accepted by CVPR'22 Oral
(2022/03) T2T-ViT is included in Most Influential ICCV Papers by Paper Digest (rank 3rd in ICCV 2021)
(2022/02) Offered Research Scientist Internship at Facebook AI Research (FAIR).
(2022/01) One paper is accepted by TIP'22.
(2021/09) One paper is accepted by NeurIPS'21.
(2021/07)Â Two papers accepted by ICCV'21.
(2020/03) Internship at Sea AI Lab
(2020/10) We are best grand challenge winner at ACM MM 2020!
(2020/07) One paper accepted by ECCV'20.
(2020/06) We win 1st place at the ACM MM grand challenge Human in Events Track4.
(2020/06) We win 2st place at the ACM MM grand challenge Human in Events Track2.
(2020/02) Three papers accepted by CVPR'20, two as Oral
(2020/01) Internship at Yitu Tech
(2019/11) Invited talk at ICCV 2019 to present our winner solution on LVIS, glad to meet Ross Girshick!
(2019/10) We win 1st place in the LVIS challenge!
(2019/08) Distilling object detection technology is integrated into product developement at Huawei SG.
(2019/04) Two papers accepted by CVPR'19
Featured Works [Full List]
Open-world Data: Long-tailed Distribution
Learning Box Regression and Mask Segmentation under Long-tailed Distribution with
Gradient Transfusing (CRAT)
Tao Wang, Li Yuan, Jiashi Feng and Xinchao Wang,
Preprint 2022, Under review at IJCV Special Issue for Robust Vision, Code will come soon
We study how box regression and mask segmentation are affected by long-tailed distribution and propose CRAT, which is guided by Fisher to augment tail class training during back-propagation.
Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax
Yu Li, Tao Wang, Bingyi Kang, Sheng Tang, Chunfeng Wang, Jintao Li, Jiashi Feng
CVPR 2020 Oral, [Paper][Code][Video]
Widely adopted by LVIS challenge 2020 and 2021 top-entries
We propose a specifically re-designed softmax classification module which further improves over SimCal on long-tail object detection and instance segmentation.
The Devil is in Classification: A Simple Framework for Long-tail Instance Segmentation (SimCal)
Tao Wang, Yu Li, Bingyi Kang, Junnan Li, Junhao Liew, Sheng Tang, Steven Hoi, and Jiashi Feng
ECCV 2020, [Paper][Code][Video]
Based on our LVIS winner solution, we further extend it and improve the performance by discovering a better initialization strategy.
Joint COCO and Mapillary Workshop at ICCV 2019: LVIS Challenge Track: Classification Calibration for Long-tail Instance Segmentation
Tao Wang, Yu Li, Bingyi Kang, Junnan Li, Junhao Liew, Sheng Tang, Steven Hoi, Jiashi Feng
Winner solution for the 1st LVIS challenge at ICCV 2019 [Tech Report]
Efficient Learning Methods
Learning to Detect and Segment for Open Vocabulary Object Detection (CondHead)Â
Tao Wang, Nan Li
Preprint 2022, Under review at CVPR 2023, Code will come soon
CondHead conditions the bounding box regression and mask segmentation on the text embeddings, to facilitate open vocabulary object detection.
PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision
Kehong Gong*, Bingbing Li*, Jianfeng Zhang*, Tao Wang*, Jing Huang, Michael Bi Mi, Jiashi Feng, Xinchao Wang (* equal contribution)
CVPR 2022 Oral, [Paper][Code][Video]
We construct an effective self-supervised framework for 3D human pose estimation, it is self-improving by generating physically plausible 2D-3D training pose data.
Revisiting Knowledge Distillation via Label Smoothing Regularization
Li Yuan, Francis EH Tay, Guilin Li, Tao Wang, Jiashi Feng
CVPR 2020 Oral, [Paper][Code][Video]
We reveal that knowledge distillation (KD) works as a learned label smoothing regularization, and further propose a novel Teacher-free Knowledge Distillation (Tf-KD) framework.
Distilling Object Detectors with Fine-grained Feature Imitation
Tao Wang, Li Yuan, Xiaopeng Zhang, Jiashi Feng
Highly cited work for knowledge distillation of object detection model.
We develop a knowledge distillation (KD) framework for object detection, based on feature-level imitation of the estimated foreground object regions.
Few-shot Adaptive Faster R-CNN
Tao Wang, Xiaopeng Zhang, Li Yuan, Jiashi Feng
CVPR 2019, [Paper][Code]
We reveal that knowledge distillation (KD) works as a learned label smoothing regularization, and further propose a novel Teacher-free Knowledge Distillation (Tf-KD) framework.
Network Architecture Design
SODAR: Segmenting Objects by Dynamically Aggregating Neighboring Mask Representations
Tao Wang, Jun Hao Liew, Yu Li, Yunpeng Chen, Jiashi Feng
We reveal the usefulness of neighboring mask predictions and introduce a simple and efficient neighbor aggregation method to improve dense instance segmentation models.
Detect Multi-person with 3D Pose Directly from Multi-view images (Multi-view Pose Transformer, MvP)
Tao Wang, Jianfeng Zhang, Yujun Cai, Shuicheng Yan, Jiashi Feng
NeurIPS 2021, [Paper][Code][Industrial Recognition by XRMoCap][Video][Slides]
We develop a simple transformer algorithm that directly detects multi-person and predicts their 3D pose from multi-view images.
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
Li Yuan*, Yunpeng Chen, Tao Wang*, Weihao Yu, Yujun Shi, Zihang Jiang, Francis E.H. Tay, Jiashi Feng, Shuicheng Yan (Work done during internship at Yitu)
ICCV 2021, [Paper][Code][Video][Most Influential ICCV Papers]
We introduce a Tokens-to-Token (T2T) transformation scheme to progressively structurize the image to tokens by recursively aggregating neighboring Tokens.
Academic Activities
Competitions:
best grand challenge winner at ACM Multimedia 2020
winner at the ACM Multimedia Human in Events challenge Track4.
2st place at the ACM Multimedia Human in Events challenge Track2.
winner in the LVIS challenge!
Academic Services:
Programme Committee/Conference Reviewer:Â
NeurIPS [2020, 2021, 2022], ICML[2023], ICLR [2020, 2022]
CVPR [2020,2021,2022,2023], ICCV[2021, 2023], ECCV[2022], WACV[2021, 2022]
Journal Reviewer: IJCV, TIP, TCSVT, TNNLSÂ
Talks:
Transformers for Multi-person Pose Detection @ Sea AI Lab, 2021
Efficient Object Detection Methods @ NUS ECE department, 2020
Efficient Object Detection Methods @ Huawei SG R&D Center, 2020
Long-tailed Object Detection @ ICCV Joint COCO and Mapillary Workshop 2019
Work Experience
2017.11-2019.10 Research Engineer, National University of Singapore, with Feng Jiashi
2019.10-2020.01 Research Collaboration, Salesforce Research Asia, with ‪Steven C.H. Hoi‬ and Junnan Li
2020.01-2020.08 Research Intern, Yitu Tech., with Yan Shuicheng and Chen Yunpeng
2021.03-2021.07 Research Intern, Sea AI Lab, with Yan Shuicheng
Teaching
NUS EE5907: Pattern Recognition, 2021/2022 semester1
NUS EE5934/EE6934: Deep Learning, 2020/2021 semester2
NUS EE3801: Data Engineering Principles, 2020/2021 semester1