MDA: Masked Language Modeling Meets Data Augmentation for Multiple Choice Question Answering
Publication Name
Proceedings - 2023 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Autonomous and Trusted Vehicles, Scalable Computing and Communications, Digital Twin, Privacy Computing and Data Security, Metaverse, SmartWorld/UIC/ATC/ScalCom/DigitalTwin/PCDS/Metaverse 2023
Abstract
Multiple Choice Question Answering (MCQA) is a well-established task in the field of Machine Reading Comprehension (MRC). Its objective is to identify the correct answer from a given set of options, based on the provided background passage and question. Recent advancements in large-scale Pre-trained Language Models (PLMs) have yielded impressive performance in MCQA. However, achieving such performance requires a significant number of training samples, leading to time-consuming and labor-intensive sample acquisition and annotation processes. To overcome the limitation posed by the availability of training samples, this paper explores the potential of leveraging the [Mask] token, which is commonly used in Masked Language Modeling (MLM) during the self-supervised training of PLMs. Specifically, the paper introduces a straightforward yet effective approach called [Mask] based Data Augmentation (MDA). The proposed method involves injecting [Mask] tokens into background passages to create masked versions of the original data. Moreover, a self-evaluator is introduced to regulate the process of masking production, with the objective of minimizing negative impact caused by argumentation noise. The effectiveness of the proposed method is empirically validated using various benchmark MCQA datasets. Experimental results demonstrate considerable improvements over state-of-the-arts.
Open Access Status
This publication is not available as open access