Create dataset multilingual_knowledge_questions_answers

- uid: multilingual_knowledge_questions_answers
- type: processed
- description:
- name: Multilingual Knowledge Questions & Answers
- description: MKQA is an open-domain question answering evaluation set comprising question-answer pairs aligned across typologically diverse languages.
- homepage: https://github.com/apple/ml-mkqa
- validated: True
- languages:
- language_names:
- Arabic
- English
- Spanish
- French
- Vietnamese
- Chinese
- Portuguese
- Danish
- German
- Finnish
- Hebrew
- Hungarian
- Italian
- Japanese
- Korean
- Norwegian
- Polish
- Russian
- Swedish
- Thai
- Turkish
- language_comments:
- language_locations:
- validated: False
- custodian:
- name: Shayne Longpre
- in_catalogue:
- type: A private individual
- location:
- contact_name:
- contact_email: slongpre@mit.edu
- contact_submitter: True
- additional: https://www.shaynelongpre.com/
- validated: False
- availability:
- procurement:
- for_download: Yes - it has a direct download link or links
- download_url: https://github.com/apple/ml-mkqa/raw/master/dataset/mkqa.jsonl.gz
- download_email:
- licensing:
- has_licenses: Yes
- license_text: https://github.com/apple/ml-mkqa/blob/master/LICENSE
- license_properties:
- open license
- license_list:
- cc-by-sa-3.0: Creative Commons Attribution Share Alike 3.0 Unported
- pii:
- has_pii: Unclear
- generic_pii_likely:
- generic_pii_list:
- numeric_pii_likely:
- numeric_pii_list:
- sensitive_pii_likely:
- sensitive_pii_list:
- no_pii_justification_class: other
- no_pii_justification_text: In the paper (https://arxiv.org/pdf/1911.02116.pdf), the author states that it follows Wenzek et al. (https://arxiv.org/abs/1911.00359) to build a clean CommonCrawl Corpus. After further investigation in Wenzek et al., descriptions regarding PII is not found.
- validated: False
- processed_from_primary:
- from_primary: Taken from primary source
- primary_availability: Yes - they are fully available
- primary_license: Unclear / I don't know
- primary_types:
- web | other
- validated: False
- from_primary_entries:
- media:
- category:
- text
- text_format:
- .TXT
- audiovisual_format:
- image_format:
- database_format:
- .JSON
- text_is_transcribed: No
- instance_type: An original English query, and then queries and answers in 26 languages.
- instance_count: 10K