Ilim ha’m ja’miyet Issue 4-1 (2026) · pp. 46-48
COLLECTING TEXTUAL DATA FROM WEBSITES FOR LOW-RESOURCE LANGUAGES
Uteuliev, Nietbay Uteulievich, Abdalieva, Gulistan Rafikovna, Kalmuratov, Bekbos Kushkinbaevich, Asenbaeva, Dalloris Adilbayevna, Утеулиев, Ниетбай Утеулиевич, Абдалиева, Гулистан Рафиковна, Калмуратов, Бекбос Кушкинбаевич, Асенбаева, Даллорис Адилбаевна, Uteuliev, Nietbay Uteulievich, Abdalieva, Gulistan Rafikovna, Kalmuratov, Bekbos Kushkinbaevich, Asenbaeva, Dalloris Adilbay qizi
Abstract
This article focuses on the automatic collection of textual data from websites for low-resource languages, using the Karakalpak language as a case study. The study developed a methodology for collecting article texts from kknews.uz, yuz.uz, and joqargikenes.uz using Python, Requests, and BeautifulSoup. As a result, 1 386 navigation pages were analyzed, and a primary raw text database containing 18 784 articles and 6 090 553 words was formed. The collected raw corpus may serve as an initial resource for further preprocessing and for future development of NLP systems, machine translation tools, and Transformer-based models for the Karakalpak language.
qoraqalpoq tiliweb scrapingelektron korpuskam resursli tillarNLPкаракалпакский языквеб-скрейпингэлектронный корпусязыки с ограниченными ресурсамиNLP
Metadata source: the journal's OAI-PMH archive · Sindex does not store the full text; it links to the source.