Multi-document reading comprehension model based on electra and document sliding window

Zheyu Yang; Benkui Zhang; Nan Jiang

doi:10.1117/12.3014619

8 December 2023 Multi-document reading comprehension model based on electra and document sliding window

Zheyu Yang, Benkui Zhang, Nan Jiang

Proceedings Volume 12943, International Workshop on Signal Processing and Machine Learning (WSPML 2023); 129430X (2023) https://doi.org/10.1117/12.3014619
Event: International Workshop on Signal Processing and Machine Learning (WSPML 2023), 2023, Hangzhou, ZJ, China

Abstract

Multi-document reading comprehension is an important and difficult task in natural language processing. To address the issue that ELECTRA pre-training model has length limitation and cannot be directly adapt to multi-document reading comprehension task, this paper proposes a novel model based on ELECTRA and document sliding windows. In the model multiple documents are split and merged through document sliding windows, new segmentation embedding is introduced, answer position in documents is modelled as a learning target, and ELECTRA is used for joint training in each window. After obtaining all prediction outcomes of each window, the results are comprehensively sorted to achieve the optimal answer. The experiments show that Rouge-L of this model reaches 51.28% on the multi-document reading comprehension dataset MS-MARCO, ranking the current best result.

Citation Download Citation

Zheyu Yang, Benkui Zhang, and Nan Jiang "Multi-document reading comprehension model based on electra and document sliding window", Proc. SPIE 12943, International Workshop on Signal Processing and Machine Learning (WSPML 2023), 129430X (8 December 2023); https://doi.org/10.1117/12.3014619

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available