Non Stationary Bandits with Periodic Variation

In numerous real-world scenarios, we encounter periodic patterns in the dynamics of non-stationary data. Unfortunately, current approaches to addressing non-stationary bandit problems overlook the valuable potential offered by the presence of periodicity. In response, we introduce SW-PUCB, a novel sliding window algorithm explicitly designed to exploit periodicity in bandit arms, surpassing the performance of the conventional UCB approach when dealing with perfectly periodic bandit environments. Recognizing that perfect periodicity is seldom encountered in real-world setting, we further present SW-NPUCB, another sliding window algorithm tailored to data exhibiting near-periodic characteristics. Lastly, we demonstrate the practical efficacy of our algorithms through comprehensive experimentation, on synthetically generated data. By bench-marking against existing non-stationary bandit techniques, we emphasize the superiority of our approaches.