A comparative study on the quality of large language models’ translation in literary text: Taking excerpts from Distant Sunflower Fields as an example
主流大语言模型文学翻译质量比较研究——基于《遥远的向日葵地》节选的实证分析

摘要

以李娟的散文集《遥远的向日葵地》中的连续性两节《外婆的世界》与《外婆的葬礼》为文本，分别用Kimi、Deepseek 3.2、Gemini 3.0 Flash、通义千问 3.5 Plus、通义千问 3 Max、GPT 5.4和文心一言七个大语言模型进行翻译，使用不同的翻译评估指标将各系统译文与已出版的人工译本进行比较评估，并采用豪斯量表和MQM量表对译文的准确性和文学性进行评价。结果显示，大语言模型进行文学翻译时已有良好表现，Kimi和Deepseek 3.2在各项评估中表现突出，而其他语言模型对特色文化内涵的理解传达和文学性表达上仍存在较大优化空间。

Abstract

This study extracted two consecutive sections, “Grandma’s World” and “Grandma’s Funeral,” from the essay collection Distant Sunflower Fields. These texts were translated using seven large language models (Kimi, Deepseek 3.2, Gemini 3.0 Flash, Qwen-3.5 Plus, Qwen-3 Max, GPT 5.4, and ERNIE Bot). Various translation evaluation metrics (TTR, BLEU, METEOR) were employed to evaluate and compare each translation with the published human translation. Additionally, the House’s Scale and MQM Scale were utilized to evaluate the accuracy and literariness of the translations. The results indicate that Large Language Models (LLMs) have displayed robust capabilities in literary translation, with Kimi and Deepseek 3.2 notably distinguishing themselves in multi-dimensional assessments. Nevertheless, other peer models continue to face challenges in interpreting culture-specific nuances and mastering stylistic literary expression, highlighting a considerable gap that necessitates further optimization.