The role of automated evaluation techniques in online professional translator training

Area of Science:

• Computational linguistics
• Translation studies
• Online education

Background:

• Technological advancements are shifting translation towards machine translation and post-editing.
• The COVID-19 pandemic accelerated the move to virtual environments for business and education.
• Online translator training and assessment require novel pedagogical and evaluation approaches.

Purpose of the Study:

• To introduce a novel online system for translator training.
• To develop a new method for assessing translator competence in online settings.
• To leverage automatic evaluation metrics and their residuals for detailed performance analysis.

Main Methods:

• Development of a specialized online teaching and learning system for translators.
• Application of automatic evaluation metrics (e.g., BLEU_n, PER, WER, TER, CDER, HTER) for quality assessment.
• Utilizing the residuals of these metrics to identify discrepancies between machine translation and post-edited versions.

Main Results:

• Residuals of accuracy and error rate metrics are effective for assessing post-editing performance.
• Analysis of residuals allows for the identification of critical, major, and minor post-editing errors.
• This method enables a more focused textual analysis of segments requiring attention.

Conclusions:

• The proposed approach provides a valid method for assessing translator competences in online environments.
• Residual analysis of automatic metrics offers detailed linguistic insights into post-editing quality.
• This contributes to optimizing online translator training and quality assurance processes.