Human-machine Authority Allocation in Indirect Cooperative Shared Steering Control with TD3 Reinforcement Learning
Publication Name
IEEE Transactions on Vehicular Technology
Abstract
In the man-machine co-driving, most of the existing indirect cooperative shared steering control(ICSSC) strategies adopt fixed driver models and are designed based on rules. However, the fixed driver model is difficult to match with the actual situation, and the rule-based strategy is hard to be designed under the multi-dimensional feature input and the multi-objective conditions and require complicated parameters adjustment. A driver model that conforms to the driving characteristics of drivers with actual driving data is established, and an ICSSC strategy is proposed based on reinforcement learning in this paper, so as to realize the dynamic allocation of human-machine steering driving weight. Firstly, the vehicle dynamics model is established according to the vehicle longitudinal, lateral and yaw dynamics, the driver driving data is collected, and then the trajectory tracking MPC (Model Predictive Control) steering controller is designed. Secondly, DQN (Deep Q-Network), DDPG (Deep Deterministic Policy Gradient) and TD3 (Twin Delayed Deep Deterministic Policy Gradient) reinforcement learning schemes that can adapt to the complex state variables are selected to design ICSSC strategies, where TD3 obtains the best iterative convergence effect under the same reward function and input state conditions. Compared with the rule-based strategy, the tracking accuracy, driving comfort, man-machine conflict, driver and controller load indexes are designed to evaluate the ICSSC strategy. Finally, simulation and hardware in the loop experiment results show that the ICSSC strategy based on TD3 can dynamically allocate the steering weights of the driver and controller under multi-objective conditions more effectively.
Open Access Status
This publication is not available as open access