SEMINAR: Optimal Allocation Strategies in a Discrete-Time Two-Armed Bandit Problem

Event details

Speaker: Audrey Hu (City University of Hong Kong)

Co-authored: Liang Zou

Abstract:  This study addresses a theoretic-bandit problem involving a "safe" and a "risky" arm
across countable periods. The agent, with one time unit per period, strategically allocates time
between these two arms aiming at achieving a "breakthrough." The risky arm's type is unknown,
which can be "good" or "bad," and breakthrough depends on proving it to be good.
Breakthrough probability is an exponential function of the allocated time, given the risky arm
is good. Departing from the "either-or" binary choices in previous studies, we explore smooth
allocation strategies in the [0,1] range. A methodological contribution of this study lies in a
problem transformation that enhances tractability, going beyond the standard Bellman-equation
approach for bandit problems. The re-formulation of the problem allows us to obtain clear
analytical solutions and comparative statics results. In general, we find that the optimal
allocation plan significantly differs from binary strategies, and stopping after any finite periods
of unsuccessful trials is suboptimal.

Host: Zaifu Yang (York)