Re-examining Contribution Fairness in Federated Learning

Abstract

Federated Learning (FL) allows multiple data owners to train a shared model collaboratively without exposing their local datasets. Because participation imposes real computation, communication, and data-collection costs, sustaining long-term collaboration requires reward mechanisms that satisfy contribution fairness, the principle that each client should be rewarded commensurately with its contribution to the training.

A substantial body of research pursues this goal through the distribution of contribution-based Top-K sparsified gradient rewards, such that higher-contributing clients receive denser, and, therefore, more useful, gradients than lower-contributing ones. Top-K sparsification, however, was originally devised as a gradient-compression technique intended to preserve convergence, which raises the question of whether it can actually produce the client-level model differentiation that contribution fairness demands.

Furthermore, existing frameworks often combine distinct contribution estimation algorithms, reward generation rules, and client model update methods, making it unclear which component is responsible for the observed fairness behavior.

This thesis investigates these questions by implementing two representative state-of-the-art frameworks, namely ACGSV and CFFL, and evaluating them across four benchmark datasets under distinct training-data partition settings. Building on this comparative evaluation, a series of controlled ablation studies is conducted to isolate the effects of three key components: the gradient used for reward generation, Top-K sparsification, and the retention of each client's locally accumulated gradient.

The analysis yields three main findings. First, contribution fairness is commonly evaluated using Pearson’s r between clients’ standalone and federated test accuracies, with standalone accuracy serving as the reference contribution value and forming a set of contribution ranks. However, Pearson's r can remain high even when most clients converge to nearly identical federated accuracies and the reference contribution ranking is not preserved. Thus, Pearson’s r alone may provide an incomplete fairness assessment. To address this limitation, this thesis proposes a novel protocol that uses Kendall’s τ to measure the reference contribution ranks preservation and the Gini Mean Difference to quantify differentiation among the final client models.

Second, Top-K sparsification induces significant model differentiation primarily when reward sparsity is high, corresponding to at least 70% in the studied settings. At lower sparsity levels, clients tend to converge to models with nearly identical performance. These findings indicate that sparsifying the aggregated gradient through Top-K is not, by itself, an effective and calibrated mechanism for estimated-contribution-based reward allocation.

Third, local gradient retention, whereby clients retain their locally accumulated gradients generated during model training alongside the received estimated-contribution-based reward, is commonly treated as a secondary detail in framework design because it is not governed by the FL framework's contribution estimation algorithm and the contribution-score-to-reward mapping pipeline. However, it is an under-acknowledged confounding factor that drives much of the model differentiation, preservation of the reference contribution ranking, and, consequently, the high fairness scores reported by these frameworks.