Q-learning Channel Access Methods for Wireless Powered Internet of Things Networks

The Internet of Things (IoT) is becoming critical in our daily life. A key technology of interest in this thesis is Radio Frequency (RF) charging. The ability to charge devices wirelessly creates so called RF-energy harvesting IoT networks. In particular, there is a hybrid access point (HAP) that provides energy in an on-demand manner to RF-energy harvesting devices. These devices then collect data and transmit it to the HAP. In this respect, a key issue is ensuring devices have a high number of successful transmissions. There are a number of issues to consider when scheduling the transmissions of devices in the said network. First, the channel gain to/from devices varies over time. This means the efficiency to deliver energy to devices and to transmit the same amount of data is different over time. Second, during channel access, devices are not aware of the energy level of other devices nor whether they will transmit data. Third, devices have non-causal knowledge of their energy arrivals and channel gain information. Consequently, they do not know whether they should delay their transmissions in hope of better channel conditions or less contention in future time slots or doing so would result in energy overflow.