On the optimality of the Hedge algorithm in the stochastic regime

Jaouad Mourtada (1) , Stéphane Gaïffas (2, 3)

1 CREST - Centre de Recherche en Économie et Statistique
2 LPSM (UMR_8001) - Laboratoire de Probabilités, Statistique et Modélisation
3 DMA - Département de Mathématiques et Applications - ENS Paris

In this paper, we study the behavior of the Hedge algorithm in the online stochastic setting. We prove that anytime Hedge with decreasing learning rate, which is one of the simplest algorithm for the problem of prediction with expert advice, is remarkably both worst-case optimal and adaptive to the easier stochastic and adversarial with a gap problems. This shows that, in spite of its small, non-adaptive learning rate, Hedge possesses the same optimal regret guarantee in the stochastic case as recently introduced adaptive algorithms. Moreover, our analysis exhibits qualitative differences with other versions of the Hedge algorithm, such as the fixed-horizon variant (with constant learning rate) and the one based on the so-called "doubling trick", both of which fail to adapt to the easier stochastic setting. Finally, we determine the intrinsic limitations of anytime Hedge in the stochastic case, and discuss the improvements provided by more adaptive algorithms.

Mots clés: Online learning; prediction with expert advice; Hedge; adaptive algorithms

Domaines: Apprentissage [cs.LG]; Machine Learning [stat.ML]; Statistiques [math.ST]

Jaouad Mourtada, Stéphane Gaïffas. On the optimality of the Hedge algorithm in the stochastic regime. Journal of Machine Learning Research, 2019, 20, pp.1 - 28. ⟨hal-02976455⟩