Iranian AI experts develop evaluation system for Persian language models

Tehran, IRNA -- A group of artificial intelligence (AI) experts at Iran’s Amirkabir University of Technology has developed an evaluation system for Persian language models, which is regarded as the most comprehensive model of its type.

The system is aimed at equipping Iran's AI arena with an accurate and integrated metric for assessing large language models.

Saeedeh Momtazi, member of the Academic Board at the university’s Faculty of Computer Engineering said that experts from the Part Artificial Intelligence Research Center and the Natural Language Processing Laboratory at Amirkabir University have developed the most comprehensive and powerful evaluation system for Persian language models, known as Open Persian LLM Leaderboard, according to an IRNA report on Saturday.

She explained that the newly-developed evaluation system includes over 40,000 samples, some of which have been translated from several reputable global benchmarks into Persian, while others have been created and labeled domestically from scratch.

She also noted that the evaluation dataset for this system is continuously increasing and being updated to improve the accuracy of the assessments.

According to Momtazi, the model presented in this project is as valuable as the top global projects, and a segment of the dataset for this evaluation system is currently available to the public as open-source.