Preparing Future SQA Professionals: An Experience Report of Metamorphic Exploration of an Autonomous Driving System

Computing systems are becoming increasingly complex and sophisticated. Technologies such as artificial intelligence, big data, and autonomous vehicles are pushing the boundaries of system size, complexity, and comprehensibility beyond anything seen before. These advances, however, have left the associated software quality assurance (SQA) tools and processes behind. This is compounded by many training and education programs also not attempting to address this inadequacy in the preparation of future software engineering professionals. We face a situation of extensively-deployed advanced computing systems, many of which lack sufficient SQA support. Metamorphic Testing (MT) and Metamorphic Exploration (ME) are SQA approaches that have a record of being able to alleviate some of the challenges associated with the advanced computer systems. This paper reports on an MT/ME experience with the Baidu Apollo autonomous driving system (ADS). The experience included identifying an apparent problem in Apollo, which was later confirmed to be a misunderstanding, but which illustrated the potential for ME to scaffold learning how to perform SQA on such complex systems. The report will be of benefit not only to other ADS developers and testers, but also to other SQA professionals, and especially to SQA trainers and educators.