Content

• Survey papers
• Slides
• Evaluation papers
• Models (for single-hop)
• Knowledge-based Machine Reading Comprehension
• Open-domain Question Answering
• Unanswerable Questions
• Multi-Passage Machine Reading Comprehension
• Conversational Question Answering
• Datasets
• Datasets with Explanations
• QA over KG
• Question answering systems
• Knowledge bases/Knowledge sources
• Papers misc

Survey/Overview papers/documents should read on Machine Reading Comprehension

• Fengbin Zhu et al., Retrieving and Reading : A Comprehensive Survey on Open-domain Question Answering, arXiv, 2021, paper
• Mokanarangan Thayaparan, Marco Valentino, and André Freitas, A Survey on Explainability in Machine Reading Comprehension, arXiv, 2020, paper
• Viktor Schlegel et al., Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models, arXiv, 2020, paper
• Viktor Schlegel et al., A Framework for Evaluation of Machine Reading Comprehension Gold Standards, arXiv, 2020, paper
• Chengchang Zeng et al., A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics, and Benchmark Datasets, arXiv, 2020, paper.
• Razieh Baradaran, Razieh Ghiasi, and Hossein Amirkhani, A Survey on Machine Reading Comprehension Systems, arXiv, 6 Jan 2020, paper.
• Matthew Gardner et al., On Making Reading Comprehension More Comprehensive., aclweb, 2019, paper.
• Shanshan Liu et al., Neural Machine Reading Comprehension: Methods and Trends, arXiv, 2019, paper.
• Xin Zhang et al., Machine Reading Comprehension: a Literature Review, arXiv, 2019, paper.
• Boyu Qiu et al., A Survey on Neural Machine Reading Comprehension, arXiv, 2019, paper.
• Danqi Chen: Neural Reading Comprehension and Beyond. PhD thesis, Stanford University, 2018, paper.

Slides

• Sebastian Riedel, Reading and Reasoning with Neural Program Interpreters, slides, MRQA 2018.
• Phil Blunsom, Data driven reading comprehension: successes and limitations, slides, MRQA 2018.
• Jianfeng Gao, Multi-step reasoning neural networks for question answering, slides, MRQA 2018.
• Sameer Singh, Questioning Question Answering Answers, slides, MRQA 2018.

Evaluation papers

• Diana Galvan, Active Reading Comprehension: A dataset for learning the Question-Answer Relationship strategy, ACL 2019, paper.
• Divyansh Kaushik and Zachary C. Lipton, How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks, EMNLP 2018, paper.
• Saku Sugawara et al., What Makes Reading Comprehension Questions Easier?, EMNLP 2018, paper.
• Pramod K. Mudrakarta et al., Did the Model Understand the Question?, ACL 2018, paper.
• Robin Jia and Percy Liang, Adversarial Examples for Evaluating Reading Comprehension Systems, EMNLP 2017, paper.
• Saku Sugawara et al., Evaluation Metrics for Machine Reading Comprehension: Prerequisite Skills and Readability, ACL 2017, paper.
• Saku Sugawara et al., Prerequisite Skills for Reading Comprehension: Multi-perspective Analysis of MCTest Datasets and Systems, AAAI 2017, paper.
• Danqi Chen et al., A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task, ACL 2016, paper.

Basic Papers/Models

Year	Title	Model	Datasets	Misc	Paper, Source Code
2019	XLNet: Generalized Autoregressive Pretraining for Language Understanding	XLNet	Race, SQuAD 1.1, SQuAD 2.0	pretrained LM	paper, code
2019	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	BERT	GLUE, SQuAD 1.1, SQuAD 2.0, SWAG	pretrained LM	paper, code
2018	S-NET: From Answer Extraction to Answer Generation for Machine Reading Comprehension	S-NET	MS-MARCO	multiple passages	paper, [code]
2018	QANET: Combining local Convolution with global Self-Attention for Reading Comprehension	QANet	SQuAD 1.1		paper, code
2017	ReasoNet: Learning to Stop Reading in Machine Comprehension	ReasoNet	CNN and Daily Mail, SQuAD 1.1		paper, [code]
2017	Reading Wikipedia to Answer Open-Domain Questions	DrQA	Wikipedia, SQuAD 1.1, CuratedTREC, WebQuestions, WikiMovies	OPQA, Multi-Passage MRC	paper, code
2017	R-Net: Machine Reading Comprehension with Self-Matching Networks	R-Net	SQuAD 1.1, MS-MARCO		paper, code
2017	Machine Comprehension Using Match-LSTM and Answer Pointer	Match-LSTM + Pointer Network	SQuAD 1.1		paper, code
2017	Gated-Attention Readers for Text Comprehension	Gated-attention Reader	CNN and Daily Mail, Children’s Book Test, Who Did What		paper, code
2017	Gated Self-Matching Networks for Reading Comprehension and Question Answering	Gated Self-Matching	SQuAD 1.1		paper, [code]
2017	Dynamic CoAttention Networks for Question Answering	Dynamic coattention networks	SQuAD 1.1		paper, code
2017	DCN+: Mixed Objective and Deep Residual CoAttention for Question Answering	DCN+	SQuAD 1.1		paper, code
2017	Bi-directional Attention Flow for Machine Comprehension	BiDAF	SQuAD 1.1		paper, code
2017	Attention-over-Attention Neural Networks for Reading Comprehension	Attention-over-Attention Reader	Children’s Book Test, CNN and Daily Mail		paper, code
2016	Text Understanding with the Attention Sum Reader Network	Attention Sum Reader	Children’s Book Test, CNN and Daily Mail		paper, code
2016	Multi-Perspective Context Matching for Machine Comprehension	Multi-Perspective Context Matching	SQuAD 1.1		paper, [code]
2016	Key-Value Memory Networks for Directly Reading Documents	Key-Value Memory Networks	WikiMovies, WikiQA		paper, code
2016	Iterative Alternating Neural Attention for Machine Reading	Iterative Attention Reader	Children’s Book Test, CNN and Daily Mail		paper, [code]
2016	A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task		CNN and Daily Mail		paper, [code]
2015	Teaching Machines to Read and Comprehend	Attentive Reader	CNN and Daily Mail		paper, code

KBMRC: Knowledge-based Machine Reading Comprehension

• Details: https://github.com/xanhho/Reading-Comprehension-Question-Answering-Papers/wiki/Knowledge-based-MRC

OPQA: Open-domain Question Answering

• Details: https://github.com/xanhho/Reading-Comprehension-Question-Answering-Papers/wiki/Open-domain-Question-Answering

UQ: Unanswerable Questions

• Details: https://github.com/xanhho/Reading-Comprehension-Question-Answering-Papers/wiki/Unanswerable-Questions

Multi-Passage MRC: Multi-Passage Machine Reading Comprehension

• Details: https://github.com/xanhho/Reading-Comprehension-Question-Answering-Papers/wiki/Multi-Passage-MRC

CQA: Conversational Question Answering

• Details: https://github.com/xanhho/Reading-Comprehension-Question-Answering-Papers/wiki/CQA

Datasets

• Following Danqi Chen, we have four answer types:

  • Cloze test
  • Multiple choice
  • Span extraction
  • Free answering

• Details: https://github.com/xanhho/Reading-Comprehension-Question-Answering-Papers/wiki/MRC-Datasets

Year	Dataset	Task	Size	Source	Web/Paper	Answer type	Misc	Similar datasets
2019	ROPES	RC	14k	Wikipedia + science textbooks	web, paper	Span extraction	background passage + situation	ShARC
2019	RC-QED	RC	12k	Wikipedia	web, paper	Multiple choice	multi-passage	HotpotQA
2019	QUOREF	RC	24k+	Wikipedia	web, paper	Span extraction	coreference resolution
2019	COSMOS QA		35,600	narrative	web, paper	Multiple choice
2019	DROP	RC	96k	Wikipedia	web, paper	Span extraction + numerical reasoning	multi-span answers
2019	Natural Questions	RC	323k	Wikipedia	paper	Span extraction
2018	SQuAD 2.0	RC	150k	Wikipedia	paper	Span extraction	no answer: 50k	NewsQA
2018	MultiRC	RC	6k+ questions	various articles	web, paper	Multiple choice	multiple sentence reasoning	MCTest
2018	CSQA	QA	200k dialogs, 1.6M turns		paper
2018	QuAC	RC	100k	Wikipedia	web, paper	Span extraction	conversational questions	CoQA
2018	QAngaroo (Wikihop + Medhop)	RC		Wikipedia + Medline	web, paper	Multiple choice	multi-passage	HotpotQA
2018	HotpotQA	RC	113k	Wikipedia	web, paper	Span extraction	multi-passage	QAngaroo
2018	CoQA	RC	127k	various articles	paper	Free answering	conversational questions	QuAC
2018	ComplexWebQuestions	RC	34,689	WebQuestionsSP	web, paper	Span extraction?	multi-passage
2018	SWAG	QA	113k	video caption		Multiple choice	situational commonsense reasoning
2018	RecipeQA	RC	36k	various			multimodal comprehension
2018	ProPara	RC	2k	procedural text				bAbI, SCoNE
2018	OpenBookQA	QA	6k	science facts		Multiple choice	external knowledge	ARC
2018	FEVER
2018	DuReader					Free answering
2018	DuoRC	RC	186k	movie plot		Span extraction		NarrativeQA
2018	CLOTH	RC	99k	English exams		Cloze test		RACE
2018	CliCR	RC	100k	clinical case text		Cloze test
2018	ARC	RC	8k	science exam			easy 5197, challenge 2590
2017	WikiSuggest				paper
2017	TriviaQA	RC	96k question-answer pairs	Web + Wikipedia	web, paper	Span extraction		SQuAD
2017	SQA				paper
2017	SearchQA				paper	Free answering
2017	RACE				paper	Multiple choice
2017	NarrativeQA				paper	Free answering
2016	Who-did-What				paper	Cloze test
2016	SQuAD 1.1	RC	87k training + 10k development	Wikipedia	paper	Span extraction		NewsQA
2016	NewsQA				paper	Span extraction
2016	MS MARCO				web	Free answering
2016	LAMBADA				paper	Cloze test
2016	WikiMovies	QA
2015	CuratedTREC	QA
2015	CNN and Daily Mail	RC	93k + 220k articles	CNN + Daily Mail	paper web	Cloze test
2015	Children's Book Test	RC	108 children's books		web	Cloze test
2015	bAbI	RC		classic text adventure game	web	Free answering	20 tasks
2013	WebQuestions	QA
2013	QA4MRE	RC		various articles	paper	Multiple choice
2013	MCTest	RC	500 stories + 2k questions	fictional stories	paper	Multiple choice	open-domain
1999	DeepRead	RC	60 development and 60 test?	news stories	paper	Free answering

Datasets with Explanations

• Details: https://github.com/xanhho/Reading-Comprehension-Question-Answering-Papers/wiki/Datasets-with-Explanations

QA over KG

• Details: https://github.com/xanhho/Reading-Comprehension-Question-Answering-Papers/wiki/QA-over-KG

Knowledge Bases/Knowledge Sources

• Wikidata, web
• Freebase, paper
• DBPedia

Question Answering Systems

• IBM's DeepQA
• QuASE
• Microsoft's AskMSR
• YodaQA
• DrQA

Others (Misc: Model, transfer learning, data augmentation, domain adaption, cross lingual ...)

• Minghao Hu, Yuxing Peng, Zhen Huang and Dongsheng Li, A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning, EMNLP 2019, paper.
• Huazheng Wang, Zhe Gan, Xiaodong Liu, Jingjing Liu, Jianfeng Gao and Hongning Wang, Adversarial Domain Adaptation for Machine Reading Comprehension, EMNLP 2019, paper.
• Yimin Jing, Deyi Xiong and Zhen Yan, BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels, EMNLP 2019, paper.
• Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang and Guoping Hu, Cross-Lingual Machine Reading Comprehension, EMNLP 2019, paper.
• Todor Mihaylov and Anette Frank, Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehension, EMNLP 2019, paper.
• Kyungjae Lee, Sunghyun Park, Hojae Han, Jinyoung Yeo, Seung-won Hwang and Juho Lee, Learning with Limited Data for Multilingual Reading Comprehension, EMNLP 2019, paper.
• Qiu Ran, Yankai Lin, Peng Li, Jie Zhou and Zhiyuan Liu, NumNet: Machine Reading Comprehension with Numerical Reasoning, EMNLP 2019, paper.
• Yiming Cui, Ting Liu, Wanxiang Che, Li Xiao, Zhipeng Chen, Wentao Ma, Shijin Wang and Guoping Hu, A Span-Extraction Dataset for Chinese Machine Reading Comprehension, EMNLP 2019, paper.
• Daniel Andor, Luheng He, Kenton Lee and Emily Pitler, Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension, EMNLP 2019, paper.
• Tsung-Yuan Hsu, Chi-Liang Liu and Hung-yi Lee, Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model, EMNLP 2019, paper.
• Kyosuke Nishida et al., Multi-style Generative Reading Comprehension, ACL 2019, paper.
• Alon Talmor and Jonathan Berant, MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension, ACL 2019, paper.
• Yi Tay et al., Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives, ACL 2019, paper.
• Haichao Zhu et al., Learning to Ask Unanswerable Questions for Machine Reading Comprehension, ACL 2019, paper.
• Patrick Lewis et al., Unsupervised Question Answering by Cloze Translation, ACL 2019, paper.
• Michael Hahn and Frank Keller, Modeling Human Reading with Neural Attention, EMNLP 2016, paper.
• Jianpeng Cheng et al., Long Short-Term Memory-Networks for Machine Reading, EMNLP 2016, paper.

Thanks to these repositories:

• https://github.com/penzant/nlu_datasets_2018
• https://github.com/seriousmac/awesome-qa