Multiple unmanned surface vehicles pathfinding in dynamic environment

In the burgeoning field of autonomous maritime operations, efficiently coordinating and navigating multiple unmanned surface vehicles (USVs) in dynamic environments is a significant challenge. This study presents an enhanced Q-learning algorithm designed to improve pathfinding for multiple USVs in such settings. The algorithm innovates on the traditional Q-learning framework by adjusting the learning rate, Epsilon-greedy strategy, and penalty and reward functions, integrating a collision avoidance mechanism specifically tailored for complex maritime navigation. Extensive simulations across six diverse scenarios – ranging from single to multiple USVs operations in both static and dynamic obstacle environments – demonstrate the algorithm's superior adaptability and efficiency compared to existing methods. Notably, in single USV scenarios, the improved Q-learning algorithm not only plots more direct paths but also reduces computational demands significantly over traditional path planning methods such as the A^∗ and APF algorithms. In multi-USV scenarios, it demonstrates robust performance, reducing calculation times by an average of 55.51% compared to SARSA, 49.14% compared to the original Q-learning, and 45.26% compared to the Speedy Q-learning approach. These advancements underscore the algorithm's potential to enhance autonomous maritime navigation, laying a strong foundation for future improvements in the safety and efficiency of USV operations.