Metamorphic testing of navigation software: A pilot study with Google Maps

Millions of people use navigation software every day to commute and travel. In addition, many systems rely upon the correctness of navigation software to function, ranging from directions applications to self-driving machinery. Navigation software is difficult to test because it is hard or very expensive to evaluate its output. This difficulty is generally known as the oracle problem, a fundamental challenge in software testing. In this study, we propose a metamorphic testing strategy to alleviate the oracle problem in testing navigation software, and conduct a case study by testing the Google Maps mobile app, its web service API, and its graphical user interface. The results show that our strategy is effective with the detection of several real-life bugs in Google Maps. This study is the first work on automated testing of navigation software with the detection of real-life bugs.