[1] Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features | [code]

Xinyi Shang, Yang Lu^†, Gang Huang, Hanzi Wang.

• We first find that the biased classifier is the primary factor behind the poor performance of the global model, then propose CReFF to optimize a small set of learnable features for classifier re-training.

[2] FEDIC: Federated Learning on Non-IID and Long-Tailed Data via Calibrated Distillation | [code]

Xinyi Shang, Yang Lu^†, Yiu-ming Cheung, Hanzi Wang.

• A new distillation method with logit adjustment and calibration gating network to solve the joint problem of heterogeneous and long-tailed data.

[3] No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier | [code]

Zexi Li, Xinyi Shang, Rui He, Tao Lin^†, Chao Wu^†.

• A neural-collapse-inspired method that mitigates classifier biases in federated learning, achieving high global-model generalization together with strong local-model personalization.

[4] Understanding the Training Dynamics in Federated Deep Learning via Aggregation Weight Optimization | [code]

Zexi Li, Tao Lin^†, Xinyi Shang, Chao Wu^†.

• We analyze FL training dynamics through client coherence and global weight shrinking, and design an aggregation algorithm that measurably improves generalization.

[5] Mind the Gap: Confidence Discrepancy Can Guide Federated Semi-Supervised Learning Across Pseudo-Mismatch

Yijie Liu, Xinyi Shang, Yiqun Zhang, Yang Lu^†, Chen Gong, Jing-Hao Xue, Hanzi Wang.

• We show that (1) data heterogeneity intensifies pseudo-label mismatches, and (2) local- and global-model predictive tendencies diverge with heterogeneity. We propose a simple yet effective method to correct pseudo-labels by exploiting confidence discrepancies.

[6] Federated Semi-Supervised Learning with Annotation Heterogeneity

Xinyi Shang, Gang Huang, Yang Lu^†, Jian Lou, Bo Han, Yiu-ming Cheung, Hanzi Wang.

• We formalize Federated Semi-Supervised Learning with annotation heterogeneity and propose a new framework with a mutual-learning strategy.

[7] GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Cost | [code]

Xinyi Shang^*, Peng Sun^*, Tao Lin^†.

• Models trained on distilled datasets are highly sensitive to the soft-label loss. Building on this insight, we introduce a plug-and-play approach that efficiently leverages full label information at near-zero cost.

[8] Information Compensation: A Fix for Any-scale Dataset Distillation

Peng Sun, Bei Shi, Xinyi Shang, Tao Lin^†.

• A near-lossless information-compression approach that distills the key information of original datasets with minimal loss, surpassing existing methods in both efficiency and effectiveness across dataset scales.

[9] Early-Stage Training with Distilled Data Helps Prevalently

Xinyi Shang^†, Jing-Hao Xue.

• We uncover a key advantage of distilled datasets: in early training phases, models trained on distilled data show substantially higher efficiency than those trained on original data. We provide explanations and validate the practical implications.

[10] Rethinking the Use of Distilled Data: Embrace, Don’t Abandon, the Original Data

Xinyi Shang^†, Jing-Hao Xue.

• We revisit the use of distilled data and propose a dynamic framework that integrates distilled and original data by analyzing their distinct properties. Efficacy is demonstrated both theoretically and empirically.

[11] Privacy as a Free Lunch: Crafting Initial Distilled Datasets through the Kaleidoscope

Shuo Shi^*, Peng Sun^*, Xinyi Shang^*, Tianyu Du^†, Xuhong Zhang, Jianwei Yin, Tao Lin.

• We identify explicit privacy leakage in distilled datasets and show theoretically that it stems from initializing distilled images with real data. A plug-and-play module applies strong perturbations to real data during initialization to address this.

[12] Dataset Distillation in the Era of Large-Scale Data: Methods, Analysis, and Future Directions

Xinyi Shang, Peng Sun, Zhiqiang Shen, Tao Lin, Jing-Hao Xue.

• We identify four significant shifts in the field of dataset distillation and provide the first comprehensive, stage-wise review through the dataset-distillation pipeline.

[13] Equally Critical: Samples, Targets, and Their Mappings in Datasets

Runkang Yang^*, Peng Sun^*, Xinyi Shang^*, Yi Tang, Tao Lin^†.

• Recent work mostly optimizes samples X while overlooking targets Y. We review the sample-target relationship and comprehensively analyze how variations in target and sample types, quantities, and qualities influence training efficiency and efficacy.

[14] Collaborative Unlabeled Data Optimization

Xinyi Shang^*, Peng Sun^*, Fengyuan Liu^*, Tao Lin^†.

• We pioneer a data-centric paradigm for collaborative unlabeled data optimization, demonstrating effectiveness and efficiency across diverse datasets and architectures.