Degree Name

Master of Philosophy


School of Computing and Information Technology


Metamorphic testing (MT) is recognized as an important quality assurance paradigm for complex systems. MT’s growing popularity is due not only to its effectiveness in alleviating the test oracle and test case generation problems but also to its unique perspective on software testing. The identification of effective metamorphic relations is vital to metamorphic testing. In addition, a testing platform that allows researchers and testers to automate the process of implementing metamorphic relations can broaden the audience for practical application of metamorphic testing.

By adopting the concept of metamorphic relation patterns, we can effectively implement concrete metamorphic relations. This thesis presents MTKeras, a generic and extensible metamorphic testing platform that implements the concept of metamorphic relation patterns. We demonstrate the platform’s applicability and fault-detection effectiveness by conducting case studies on the testing of image classification and sentiment analysis models, search engines, and a database management system. This study not only proves the usefulness of metamorphic relation patterns but also confirms that the composition of metamorphic relations can greatly improve the fault-detection effectiveness of individual metamorphic relations. It also proves that metamorphic relations can help to enhance a user’s understanding of the underlying system. We have open-sourced MTKeras.

FoR codes (2020)

4611 Machine learning, 4612 Software engineering



Unless otherwise indicated, the views expressed in this thesis are those of the author and do not necessarily represent the views of the University of Wollongong.