A parallel corpus for machine translation systems, information extraction and other language processing techniques
The Japanese-English Subtitle Corpus (JESC) is the product of a collaboration among Stanford University, Google Brain and Rakuten Institute of Technology. It was created by crawling the internet for movie and TV subtitles and aligning their captions. It is one of the largest freely available English-Japanese corpora (3.2M parallel sentences), and covers the poorly represented domain of colloquial language.
The "ContentElements" field contains six options: "TrainingData", "TestData", "ValidationData", "TrainingDataset", "TestDataset" and "ValidationDataset". "TrainingData", "TestData" and "ValidationData" are structured as associations. "TrainingDataset", "TestDataset" and "ValidationDataset" are structured as datasets.
Retrieve the resource:
Obtain the first three training examples:
Obtain the last three test examples:
Obtain the one random validation example:
Obtain five random pairs from the training set in Dataset form:
Obtain five random pairs from the test set in Dataset form:
Obtain five random pairs from the validation set in Dataset form:
Obtain a character-level histogram of test example lengths:
"Japanese-English Subtitle Corpus"
from the Wolfram Data Repository
Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
Data Resource History