google / google/corpuscrawler

Use available sentences corpora for Wikipedia (290+ languages)

Open
#92 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
218
Forks
53
PR merge metrics
No merged PRs in 30d

Description

There are ready-to-download open licence Wikipedia corpora available.

| Project introduction | Type | Languages (2024) | Portal all | Language specific | Download link | Comments
|---|---|---|---|---|---|---
| Wortschatz by Leipzig | Sentences
Monolingual | 290+ | - | [bre](https://wortschatz.uni-leipzig.de/en/download/bre) | [bre 100k sentences (2021)](https://downloads.wortschatz-leipzig.de/corpora/bre_wikipedia_2021_100K.tar.gz) | List of sentences corpora : [API reference](https://api.wortschatz-leipzig.de/ws/swagger-ui/index.html#/Corpora/getAvailableCorpora) > https://api.wortschatz-leipzig.de/ws/corpora

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.