[data request] <syneval>

• Name of dataset: syneval
• URL of dataset: https://github.com/BeckyMarvin/LM_syneval
• License of dataset: EMNLP 2018
• Short description of dataset and use case(s): a collection of tasks that evaluate language models along three different structure-sensitive linguistic phenomena: subject-verb agreement, reflexive anaphora and negative polarity items. Given a large number of minimally different pairs of English sentences, each consisting of a grammatical and an ungrammatical sentence (where the first sentence of each pair has the correct grammar), a language model should assign a higher probability to a grammatical sentence than an ungrammatical one.

Folks who would also like to see this dataset in tensorflow/datasets, please thumbs-up so the developers can know which requests to prioritize.

And if you'd like to contribute the dataset (thank you!), see our guide to adding a dataset.

Start with docs/add_dataset.md, then inspect the syneval dataset at the linked LM_syneval repository to understand its files and format. Add the dataset to tensorflow/datasets with the three described linguistic evaluation tasks, and confirm that the dataset can be generated and used as described.