University of Wollongong
Browse

On automatic testing of web search engines

Download (2.94 MB)
thesis
posted on 2024-11-11, 18:35 authored by Shaowen Xiang
Web search engines are very important because they are the means by which people retrieve information from the World Wide Web. However, testing these web search engines is difficult because there are no test oracles, so this research proposes seven new metrics based on the idea of metamorphic relations to alleviate the oracle problem in search engine testing. Using these metrics, our method can test search engines automatically in the absence of an ideal oracle. Using this method, we further conduct large-scale empirical studies to investigate and compare the qualities of four major search engines, namely, Google (www.google.com), Baidu (www.baidu.com), Bing (www.bing.com), and Chinese Bing (www.bing.com.cn). Our empirical studies involve more than 50 million queries sent to the search engines across 9 months, and about 300 GB data collected from the search engine responses. It is found that different search engines have significantly different performance and that the nature of the query terms can have a significant impact on the performance of the search engines. These empirical study results demonstrate that our method can effectively alleviate the oracle problem in search engine testing, and can help both developers and users to obtain a better understanding of the search engine behaviour under different operational profiles.

History

Year

2015

Thesis type

  • Masters thesis

Faculty/School

School of Computer Science and Software Engineering

Language

English

Disclaimer

Unless otherwise indicated, the views expressed in this thesis are those of the author and do not necessarily represent the views of the University of Wollongong.

Usage metrics

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC