A dataset for question answering and reading comprehension from a set of Wikipedia articles

The Stanford Question Answering Dataset (SQuAD) consists of questions posed by crowd workers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage.

The "ContentElements" field contains eight options; "Dataset", "TrainingData", "ValidationData", "TrainingMetadata", "ValidationMetadata", "Data", "ColumnNames" and "ColumnDescriptions". "Dataset" contains the full dataset. Please note that data marked "Validation" in the ValidationRole field can have multiple possible answers for each question. "TrainingData" and "ValidationData" are formatted for standard question answering usage; for every question, only the first answer of the full dataset is selected. "TrainingMetadata" and "ValidationMetadata" contain the title of the Wikipedia article to which each question ID corresponds. "Data" contains the full dataset structured as an association. "ColumnNames" and "ColumnDescriptions" provide more information about the columns of the dataset.

Modifications from the original dataset: Data marked "Training" in the ValidationRole field corresponds to the Training Set v1.1 subset of the original dataset. Data marked "Validation" in the ValidationRole field corresponds to the Dev Set v1.1 subset of the original dataset. The original dataset is 0-indexed, in order be accurate in the Wolfram Language, 1 was added to the value of "AnswerPosition", as the Wolfram Language is 1-indexed.

Examples

Example Notebook

• Open in Cloud
• Download Notebook

Basic Examples

Retrieve the resource:

In[1]:=

Out[1]=

Retrieve a sample of the dataset:

In[2]:=

Out[2]=

Working with Training and Validation data:

Obtain the first example from the training dataset:

In[3]:=

Out[3]=

View the question and answer associated with the passage:

In[4]:=

Out[4]=

In[5]:=

Out[5]=

"AnswerPosition" shows where the first character of the answer is located in the passage. The position of the last character can be computed:

In[6]:=

Out[6]=

In[7]:=

Out[7]=

Test that the span between the starting position and the ending position is equivalent to the answer:

In[8]:=

Out[8]=

In[9]:=

Out[9]=

Add the training metadata to the training data:

In[10]:=

There is a one-to-one correspondence between the data and the metadata, therefore all lists will have the same length:

In[11]:=

Out[11]=

View an example of the joined data:

In[12]:=

Out[12]=

Aggregation

Set the types of questions:

In[13]:=

Define the patterns corresponding to these questions:

In[14]:=

Classify the questions from the SQuAD dataset:

In[15]:=

Print the statistics about the question types

In[16]:=

Out[16]=

In[17]:=

Out[17]=

Display some examples for each type of question:

In[18]:=

In[19]:=

Out[19]=

In[20]:=

Out[20]=

In[21]:=

Out[21]=

In[22]:=

Out[22]=

Dataset size

Display the number of Wikipedia pages:

In[23]:=

Out[23]=

Display the number of Wikipedia paragraphs:

In[24]:=

Out[24]=

Display the number of question/answer pairs in the training and validation datasets:

In[25]:=

Out[25]=

In[26]:=

Out[26]=

Bibliographic Citation

Wolfram Research, "SQuAD v1.1" from the Wolfram Data Repository (2018) 

License Information

Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)

Data Resource History

• Date Created: 2 January 2018

Source Metadata

• Title: SQuAD: 100, 000+ Questions for Machine Comprehension of Text
• Creator: Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev and Percy Liang
• Publisher: Empirical Methods in Natural Language Processing (EMNLP)
• Date: 2016
• Language: English
• Source: https://rajpurkar.github.io/SQuAD-explorer/

Data Downloads

• CSV
• JSON
• TSV
• WL

Publisher Information

• Publisher of Record: Wolfram Research