Language definitions used by Weblate
Enriched Estonian word list: 160K words with POS tags, CEFR proficiency levels, 5.4M inflected forms, and frequency rankings. Data from Ekilex (EKI) and OpenSubtitles.
a High Agreement Multi-lingual Outlier Detection dataset
A curated catalog of resources for human languages, from books and courses to corpora, communities, and digital tools — with a focus on lesser-resourced and hard-to-find materials.
A collection of language information tracked by the linguist project.
Embeddable submodule of parallel/monolingual text data, for use in testing code and sanity checks
Mexican Spanish slang dataset for AI evaluation, NLP research, language annotation, and conversational AI applications.
k-sncacs dataset for Universal Depdencies
Zimbabwe Language Data Commons — public site: governed AI-ready Shona/Ndebele speech & text corpus
several flex databases to be merged
Languages Sql Table - Diller Sql Tablosu
Academic texts and linguistic corpora for German language research.
This interactive Python tool enables the recording of bilingual audio samples using PyAudio and ipywidgets. Designed for data collection tasks such as speech datasets, it provides a user-friendly interface to record, save, label, and manage audio files directly within a Jupyter Notebook.
Collection of approximately 20K German texts from the 2010s: User written texts in form of personal stories, poems, poetry, articles and opinion pieces pulled from archives of the Stern NEON Community website.
Language identification with as few characters as possible
To associate your repository with the language-data topic, visit your repo's landing page and select "manage topics."