Robust Federated Opportunistic Learning in the Presence of Label Quality Disparity

In this article, the problem of federated learning (FL) in the presence of label quality disparity is considered. To address this problem, the federated opportunistic computing for ubiquitous system (FOCUS) has been proposed very recently. In FOCUS, the central server utilizes its accurately labeled benchmark samples to quantify the credibility of different clients by computing the cross-entropy (CE) loss of the locally updated models on the benchmark data set and the CE loss of the global model on the local data sets. However, FOCUS assumes the availability of the accurate labels of the benchmark data set, which is difficult to guarantee under many practical scenarios. To overcome this limitation of FOCUS, we propose a new algorithm named robust federated opportunistic learning (RFOL), which does not require the benchmark samples at the central server to be labeled. In RFOL, the client credibility is evaluated by computing the Kullback-Leibler (KL) divergence among the soft predictions on the benchmark samples of different locally updated models and the CE loss of the global model on the local data sets. The experimental results on several popular data sets reveal that: 1) with an unlabeled benchmark data set at the server, the proposed RFOL algorithm attains almost the same learning performance as FOCUS, which requires an accurately labeled benchmark data set at the server; 2) with an inaccurately labeled benchmark data set, RFOL outperforms FOCUS, which shows that the former is more robust to the inaccurate labels of the benchmark samples; and 3) RFOL outperforms FedAvg, which assigns equal credibility to all the clients.