Research

The long-term research goal is to build understand, evaluate, and enhance modern AI models, such as pre-trained and large foundation models. We create new theory, algorithms, applications, and open-sourced library to achieve our goal.

• Machine learning: I’m generally interested in designing algorithms and applications to make machine learning systems more robust, trustworthy, and responsible. Related topics include: machine learning with foundation models, robust machine learning, OOD / domain generalization, transfer learning, semi-supervised learning, federated learning, and related applications.
• Large language models: We mainly focus on understanding the potential and limitation of large foundation models. Related topics: LLM evaluation and enhancement.
• AI for social sciences: How to measure the impact of generative AI on different domains? How to assist interdisciplinary domains using powerful AI models? How to use existing social science knowledge to help us better understand AI behaviors?

Our research consists of the following topics with selected publications: [View by year] [Google scholar]

Foundation model understanding

• [ICML’24] Dynamic Evaluation of Large Language Models by Meta Probing Agents. Kaijie Zhu, Jindong Wang, Qinlin Zhao, Ruochen Xu, Xing Xie.
• [ICML’24] Trustllm: Trustworthiness in large language models. Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, and others.
• [ACL’24 main] KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models. Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Wei Ye, Jindong Wang , Xing Xie, Yue Zhang, and Shikun Zhang.
• [ICLR’24 spotlight] DyVal: Graph-informed Dynamic Evaluation of Large Language Models. Kaijie Zhu, Jiaao Chen, Jindong Wang, Neil Zhenqiang Gong, Diyi Yang, Xing Xie.
• [ICLR’24] PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization. Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, Wei Ye, Shikun Zhang, Yue Zhang. [code]
• [ICML’24] CompeteAI: Understanding the Competition Behaviors in Large Language Model-based Agents. Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, Xing Xie.
• [arXiv’23] Meta Semantic Template for Evaluation of Large Language Models. Yachuan Liu, Liang Chen, Jindong Wang, Qiaozhu Mei, Xing Xie.
• [ACM TIST’24] A Survey on Evaluation of Large Language Models. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, Xing Xie. [code]
• [arXiv’23] PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts. Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Neil Zhenqiang Gong, Yue Zhang, Xing Xie. [code]
• [ACL’23 findings] GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective. Linyi Yang, Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang, Xing Xie, Yue Zhang.
• [ICLR’23 large model workshop] On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective. Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang, Wei Ye, Xiubo Geng, Binxin Jiao, Yue Zhang, and Xing Xie.

Machine learning and large foundation models

• [arXiv’24] Slight Corruption in Pre-training Data Makes Better Diffusion Models. Hao Chen, Yujin Han, Diganta Misra, Xiang Li, Kai Hu, Difan Zou, Masashi Sugiyama, Jindong Wang, Bhiksha Raj.
• [arXiv’24] On Catastrophic Inheritance of Large Foundation Models. Hao Chen, Bhiksha Raj, Xing Xie, Jindong Wang.
• [ICML’24] Open-Vocabulary Calibration for Vision-Language Models. Shuoyuan Wang, Jindong Wang , Guoqing Wang, Bob Zhang, Kaiyang Zhou, and Hongxin Wei.
• [ACL’24 main] Detoxifying Large Language Models via Knowledge Editing. Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang , and Huajun Chen.
• [ICLR’24 spotlight] Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks. Hao Chen, Jindong Wang, Ankit Shah, Ran Tao, Hongxin Wei, Xing Xie, Masashi Sugiyama, Bhiksha Raj.
• [ICLR’24] Supervised Knowledge Makes Large Language Models Better In-context Learners. Linyi Yang, Shuibai Zhang, Zhuohao Yu, Guangsheng Bao, Yidong Wang, Jindong Wang, Ruochen Xu, Wei Ye, Xing Xie, Weizhu Chen, Yue Zhang.
• [arXiv’23] ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning. Wang Lu, Hao Yu, Jindong Wang, Damien Teney, Haohan Wang, Yiqiang Chen, Qiang Yang, Xing Xie, Xiangyang Ji.
• [IJCV’23] Exploring Vision-Language Models for Imbalanced Learning. Yidong Wang, Zhuohao Yu, Jindong Wang, Qiang Heng, Hao Chen, Wei Ye, Rui Xie, Xing Xie, Shikun Zhang. [code]
• [IEEE Data Engineering Bulletin’23] FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning. Wang Lu, Xixu Hu, Jindong Wang, Xing Xie. [arxiv]

Open source:

• Project SearchAnything: semantic local search.

AI for social sciences:

• [arXiv’24] CulturePark: Boosting Cross-cultural Understanding in Large Language Models. Cheng Li, Damien Teney, Linyi Yang, Qingsong Wen, Xing Xie, Jindong Wang.
• [ICML’24 Oral] Competeai: Understanding the competition behaviors in large language model-based agents. Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, and Xing Xie.
• [ICML’24] The good, the bad, and why: Unveiling emotions in generative ai. Cheng Li, Jindong Wang, Yixuan Zhang, Kaijie Zhu, Xinyi Wang, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang, and Xing Xie.
• [IJCAI’24] NegativePrompt: Leveraging Psychology for Large Language Models Enhancement via Negative Emotional Stimuli. Xu Wang, Cheng Li, Yi Chang, Jindong Wang , and Yuan Wu.
• [ACL’24] MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms. Yiqiao Jin, Minje Choi, Gaurav Verma, Jindong Wang , and Srijan Kumar.
• [arXiv’24] CultureLLM: Incorporating Cultural Differences into Large Language Models. Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, Xing Xie.
• [arXiv’24] AgentReview: Exploring Peer Review Dynamics with LLM Agents. Yiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kaijie Zhu, Yijia Xiao, Jindong Wang.
• [arXiv’23] EmotionPrompt: Leveraging Psychology for Large Language Models Enhancement via Emotional Stimulus. Cheng Li, Jindong Wang, Kaijie Zhu, Yixuan Zhang, Wenxin Hou, Jianxun Lian, Xing Xie.

Out-of-distribution (Domain) generalization and general machine learning

• [ICML’24] Selective mixup helps with distribution shifts, but not (only) because of mixup. Damien Teney, Jindong Wang, and Ehsan Abbasnejad.
• [IEEE TPAMI’24] DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization. Wang Lu, Jindong Wang, Xinwei Sun, Yiqiang Chen, Xiangyang Ji, Qiang Yang, Xing Xie.
• [IEEE VR 24] Generating Virtual Reality Interaction Data from Out-of-Distribution Desktop Data: An Exploration Using Stroke Gestures. Linping Yuan, Boyu Li, Jindong Wang, Huaming Qu, Wei Zeng.
• [UbiComp’24] Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition. Shuoyuan Wang, Jindong Wang, HuaJun Xi, Bob Zhang, Lei Zhang, Hongxin Wei.
• [CPAL’24] Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition. Shuoyuan Wang, Jindong Wang , Huajun Xi, Bob Zhang, Lei Zhang, and Hongxin Wei.
• [TNNLS’24] UP-Net: An Uncertainty-Driven Prototypical Network for Few-Shot Fault Diagnosis. Ge Yu, Jindong Wang, Jinhai Liu, Xi Zhang, Yiqiang Chen, and Xing Xie.
• [NeurIPS’23] Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models. Andy Zhou, Jindong Wang, Yu-Xiong Wang, Haohan Wang.
• [ICCV’23] Improving Generalization of Adversarial Training via Robust Critical Fine-Tuning. Kaijie Zhu, Xixu Hu, Jindong Wang, Xing Xie, Ge Yang.
• [ICLR’23] Out-of-distribution Representation Learning for Time Series Classification. Wang Lu, Jindong Wang, Xinwei Sun, Yiqiang Chen, and Xing Xie.
• [KDD’23] Domain-Specific Risk Minimization for Out-of-Distribution Generalization. YiFan Zhang, Jindong Wang, Jian Liang, Zhang Zhang, Baosheng Yu, Liang Wang, Xing Xie, and Dacheng Tao.
• [KDD’23] Generalizable Low-Resource Activity Recognition with Diverse and Discriminative Representation Learning. Xin Qin, Jindong Wang, Shuo Ma, Wang Lu, Yongchun Zhu, Xing Xie, Yiqiang Chen.
• [ACL’23 findings] GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective. Linyi Yang, Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang, Xing Xie, Yue Zhang.
• [KDD’23 workshop] Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solution. Wang Lu, Jindong Wang, Yidong Wang, Kan Ren, Yiqiang Chen, Xing Xie.
• [TKDE’22] Generalizing to Unseen Domains: A Survey on Domain Generalization. Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Philip S. Yu.
• [TMLR’22] Domain-invariant Feature Exploration for Domain Generalization. Wang Lu, Jindong Wang, Haoliang Li, Yiqiang Chen, and Xing Xie.
• [UbiComp’22] Semantic-Discriminative Mixup for Generalizable Sensor-based Cross-domain Activity Recognition. Wang Lu, Jindong Wang, Yiqiang Chen, Sinno Pan, Chunyu Hu, and Xin Qin.
• [NeurIPS’21] Learning causal semantic representation for out-of-distribution prediction. Chang Liu, Xinwei Sun, Jindong Wang , Haoyue Tang, Tao Li, Tao Qin, Wei Chen, and Tie-Yan Liu.
• [CIKM’21] Adarnn: Adaptive learning and forecasting of time series. Yuntao Du, Jindong Wang, Wenjie Feng, Sinno Pan, Tao Qin, Renjun Xu, and Chongjun Wang.
• [TNNLS’20, 400 + citations] Deep subdomain adaptation network for image classification. Yongchun Zhu, Fuzhen Zhuang, Jindong Wang, Guolin Ke, Jingwu Chen, Jiang Bian, Hui Xiong, and Qing He.
• [ACMMM’18, 500+ citations] Visual domain adaptation with manifold embedded distribution alignment. Jindong Wang, Wenjie Feng, Yiqiang Chen, Han Yu, Meiyu Huang, and Philip S Yu.
• [ICDM’17, 500+ citations] Balanced distribution adaptation for transfer learning. Jindong Wang, Yiqiang Chen, Shuji Hao, Wenjie Feng, and Zhiqi Shen.
• Open-source:
  • Transfer learning
  • robustlearn: A unified repo for robust machine learning, such as OOD and adversarial robustness: robustlearn
  • PandaLM: PandaLM

Semi-supervised learning for low-resource learning

• [ICML’24] A General Framework for Learning from Weak Supervision. Hao Chen, Jindong Wang, Lei Feng, Xiang Li, Yidong Wang, Xing Xie, Masashi Sugiyama, Rita Singh, and Bhiksha Raj.
• [ICLR’23] FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning. Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Yue Fan, Zhen Wu, Jindong Wang, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj, Bernt Schiele, and Xing Xie.
• [ICLR’23] SoftMatch: Addressing the Quantity-Quality Tradeoff in Semi-supervised Learning. Hao Chen, Ran Tao, Yue Fan, Yidong Wang, Jindong Wang, Bernt Schiele, Xing Xie, Bhiksha Raj, and Marios Savvides.
• [NeurIPS’22] USB: A Unified Semi-supervised Learning Benchmark. Yidong Wang, Hao Chen, Yue Fan, Wang Sun, Ran Tao, Wenxin Hou, Renjie Wang, Linyi Yang, Zhi Zhou, Lan-Zhe Guo, Heli Qi, Zhen Wu, Yu-Feng Li, Satoshi Nakamura, Wei Ye, Marios Savvides, Bhiksha Raj, Takahiro Shinozaki, Bernt Schiele, Jindong Wang, Xing Xie, and Yue Zhang.
• [TASLP’22] Exploiting Adapters for Cross-lingual Low-resource Speech Recognition. Wenxin Hou, Han Zhu, Yidong Wang, Jindong Wang, Tao Qin, Renjun Xu, and Takahiro Shinozaki.
• [NeurIPS’21, 700+ citations] Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki.
• Open-source:
  • USB: A unified semi-supervised learning toolbox for CV, NLP, and Audio: USB
  • A unified Pytorch-based semi-supervised learning library: **TorchSSL

Safe transfer learning for security

• [ECCV’24] SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalization. Xixu Hu, Runkai Zheng, Jindong Wang, Cheuk Hang Leung, Qi Wu, Xing Xie.
• [TNNLS’23] MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcare. Yiqiang Chen, Wang Lu, Xin Qin, Jindong Wang , and Xing Xie.
• [ICSE’22] ReMoS: Reducing Defect Inheritance in Transfer Learning via Relevant Model Slicing. Ziqi Zhang, Yuanchun Li, Jindong Wang, Bingyan Liu, Ding Li, Xiangqun Chen, Yao Guo, and Yunxin Liu.
• [IEEE Data Engineering Bulletin’23] FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning. Wang Lu, Xixu Hu, Jindong Wang, Xing Xie. [arxiv]
• [IEEE TBD’22] Personalized Federated Learning with Adaptive Batchnorm for Healthcare. Wang Lu, Jindong Wang, Yiqiang Chen, Xin Qin, Renjun Xu, Dimitrios Dimitriadis, and Tao Qin.
• [TKDE’22] Unsupervised deep anomaly detection for multi-sensor time-series signals. Yuxin Zhang, Yiqiang Chen, Jindong Wang, and Zhiwen Pan.
• [IntSys’22, 800+ citations] Fedhealth: A federated transfer learning framework for wearable healthcare. Yiqiang Chen, Xin Qin, Jindong Wang, Chaohui Yu, and Wen Gao.
• Open-source:
  • PersonalizedFL: a personalized federated learning libraty: PersonalizedFL
  • robustlearn: A unified repo for robust machine learning, such as OOD and adversarial robustness: robustlearn

Imbalanced learning for long-tailed tasks

• [arXiv’23] Exploring Vision-Language Models for Imbalanced Learning. Yidong Wang, Zhuohao Yu, Jindong Wang, Qiang Heng, Hao Chen, Wei Ye, Rui Xie, Xing Xie, Shikun Zhang.
• [ACML’22] Margin Calibration for Long-Tailed Visual Recognition. Yidong Wang, Bowen Zhang, Wenxin Hou, Zhen Wu, Jindong Wang, and Takahiro Shinozaki.
• Open-source:
  • Imbalance-VLM: a library for imbalanced learning in vision-language models. [Imbalance-VLM]

Miscellaneous

1. An easy-to-use speech recognition toolkit based on Espnet: EasyESPNet
2. Leading the transfer learning tutorial (迁移学习简明手册) on Github: Tutorial
3. I’m also leading other popular research projects: Machine learning, Activity recognition
4. I started a software studio Pivot Studio and made many applications in 2010-2014: Our applications