Рақамли технологияларнинг назарий ва амалий масалалари 7-том 3-сан (2024) · 112-116-беттер
Lora effectiveness in text generalization tasks: comparison of T5 and uzT5 models
Адилова, Фатима, Давронов, Рифкат, Кушмуратов, Самариддин
Аннотация
This paper presents an analysis of the application of the Low-Rank Adaptation method (LoRA) for the task of monolingual text generation in Uzbek. We used the T5-base, T5-Large and uzT5 models to determine which one shows the best results when using LoRA, and also compared their performance with traditional fine tuning. The text from 5,000 news items from the Kun.uz platform was used as a dataset of which 4,000 were used for training and 1,000 for testing. The performance of the models was evaluated using the metrics BLEU, ROUGE-1, ROUGE-2, ROUGE-L and ROUGE-LSUM. Our results showed that the uzT5-base model with LoRA r=256 and α=512 parameters demonstrate the highest performance among all the considered models, providing the best values of the ROUGE and BLEU metrics with a moderate number of training parameters, which makes it more computationally efficient compared to mT5-Large.
низкоранговые адаптацииT5-baseT5-LargeuzT5сжатые модели
Метадайындар булагы: журналдын OAI-PMH архиви · Sindex толук текстти сактабайт, булакка шилтеме берет.