About me

I am an NLP researcher with a PhD in Computer Science from the University of Tartu, where I worked in the Natural Language Processing lab under the supervision of Prof. Mark Fishel.

My research focused on learning under data scarcity, with an emphasis on cross-lingual knowledge transfer for low-resource languages. I studied how synthetic and native data can be leveraged at different stages of model training, through continued pre-training, fine-tuning and prompting, to improve language understanding in settings where high-quality data is limited. I contributed to the development of Llammas 🐑 and maintain Estonian Chatbot Arena.

In addition that, my work also addressed practical challenges of model evaluation in low-resource settings, including the use of translated and synthetic datasets, and resulted in the development of a new evaluation benchmark for low-resource Finno-Ugric languages.

Before doing my PhD, I worked as a data scientist at [Texta], where I solved practical NLP problems such as text classification, hate speech detection, and information extraction. During my Master’s studies, I completed an internship at STACC, where I worked on detecting adverse drug reactions from clinical text. This later became the basis of my Master’s thesis. Earlier in my career, I have also worked as a software developer.

My current research interests include language model evaluation, with a particular focus on using item response theory to support model assessment and produce more reliable model rankings.

I am open to new opportunities, collaborations, and adventures.

News

13/02/2026	Estonian WinoGrande Dataset was accepted at LREC 2026. [link]
06/05/2025	Launched Estonian Chatbot Arena @ baromeeter.ai.
03/05/2025	Presented at NAACL 2025. [poster]
03/03/2025	Presented at NoDaLiDa/Baltic-HLT 2025. [slides]
23/01/2025	LLMs for Extremely Low-Resource Finno-Ugric Languages was accepted at NAACL 2025 Findings. [link]

Publications

Marii Ojastu, Hele-Andra Kuulmets, Aleksei Dorkin, Marika Borovikova, Dage Särg, Kairit Sirts. 2026. Estonian WinoGrande Dataset: Comparative Analysis of LLM Performance on Human and Machine Translation. In Proceedings of LREC 2026. [link]
Hele-Andra Kuulmets, Taido Purason, Mark Fišel. 2025. How Well do LLMs know Finno-Ugric Languages? A Systematic Assessment. In Proceedings of NoDaLiDa/Baltic-HLT 2025. [link]
Taido Purason^†, Hele-Andra Kuulmets^†, Mark Fišel. 2024. LLMs for Extremely Low-Resource Finno-Ugric Languages. In Findings of NAACL 2025. [link]
Hele-Andra Kuulmets^†, Taido Purason^†, Agnes Luhtaru, Mark Fišel. 2024. Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer. In Findings of NAACL 2024. [link]
Hele-Andra Kuulmets and Mark Fišel. 2023. Translated Benchmarks Can Be Misleading: the Case of Estonian Question Answering. In Proceedings of NoDaLiDa 2023. [link]
Hele-Andra Kuulmets, Andre Tättar, Mark Fišel. 2022. Estonian Language Understanding: a Case Study on the COPA Task. In Baltic Journal of Modern Computing. [link]

_{^† Equal contribution}