Provable distributed adaptive temporal-difference learning over time-varying networks
Publication Name
Expert Systems with Applications
Abstract
Multi-agent reinforcement learning (MARL) has been successfully applied in many fields. In MARL, the policy evaluation problem is one of crucial problems. In order to solve this problem, distributed Temporal-Difference (TD) learning algorithm is one of the most popular methods in a cooperative manner. Despite its empirical success, however, the theory of the adaptive variant of distributed TD learning still remain limited. To fill this gap, we propose an adaptive distributed temporal-difference algorithm (referred to as MS-ADTD) under Markovian sampling over time-varying networks. Furthermore, we rigorously analyze the convergence of MS-ADTD, the theoretical results show that the local estimation can converge linearly to the optimal neighborhood. Meanwhile, the theoretical results are verified by simulation experiments.
Open Access Status
This publication is not available as open access
Volume
228
Article Number
120406
Funding Number
22HASTIT014
Funding Sponsor
National Natural Science Foundation of China