Proxy model-guided reinforcement learning for client selection in federated recommendation
Abstract
Federated recommender systems (FedRSs) have emerged as a promising privacy-preserving paradigm, enabling personalized recommendation services without exposing users' raw data. By keeping data local and relying on a central server to coordinate training across distributed clients, FedRSs protect user privacy while collaboratively learning global models. However, most existing FedRS frameworks adopt fully random client selection strategy in each training round, overlooking the statistical heterogeneity of user data arising from diverse preferences and behavior patterns, thereby resulting in suboptimal model performance. While some client selection strategies have been proposed in the broader federated learning (FL) literature, these methods are typically designed for generic tasks and fail to address the unique challenges of recommendation scenarios, such as expensive contribution evaluation due to the large number of clients, and sparse updates resulting from long-tail item distributions. To bridge this gap, we propose ProxyRL-FRS, a proxy model-guided reinforcement learning framework tailored for client selection in federated recommendation. Specifically, we first introduce ProxyNCF, a dual-branch model deployed on each client, which augments standard Neural Collaborative Filtering with an additional proxy model branch that provides lightweight contribution estimation, thus eliminating the need for expensive per-round local training traditionally required to evaluate a client's contribution. Furthermore, we design a staleness-aware (SA) reinforcement learning agent that selects clients based on the proxy-estimated contribution, and is guided by a reward function balancing recommendation accuracy and embedding staleness, thereby enriching the update coverage of item embeddings. Experiments conducted on three public recommendation datasets demonstrate that ProxyRL-FRS outperforms state-of-the-art baselines, achieving faster convergence and improved recommendation accuracy.
Keywords
client selection, federated recommender systems, proxy model, reinforcement learning
Document Type
Journal Article
Date of Publication
1-1-2026
E-ISSN
15582191
ISSN
10414347
Publication Title
IEEE Transactions on Knowledge and Data Engineering
Publisher
IEEE
School
School of Business and Law
Copyright
subscription content
Recommended Citation
Qu, L., Li, J., Yuan, W., Ruan, P., Shi, Y., & Yin, H. (2026). Proxy model-guided reinforcement learning for client selection in federated recommendation. IEEE Transactions on Knowledge and Data Engineering. Advance online publication. https://doi.org/10.1109/TKDE.2026.3732645