Statistical Machine Translation

The epic The Knight in the Panther's Skin by Shota Rustaveli is a well-known literary work inside and outside of Georgia, which has been translated into more than 50 languages. It comes as no surprise that the epic offers many topics for intradisciplinary as well as interdisciplinary researches, one of which will be discussed here, namely aphorisms found in the translations of the epic. The translations in question are in Abkhazian, of which two exist: one translated by Dimitri Gulia (published in 1941) and the other by Mushni Lasuria (published in 1978). For a methodological analysis, the parallel corpus of the Abkhazian translations GeAbCo (https://geabco.com/) was created. This paper will explain the planning and conception of the corpus as well as show a first attempt of an analysis considering the aphorisms found, whereat even the question about the actual number of aphorisms in the epic is a research question itself.

English phrasal verbs pose complex semantic interpretations and cause significant problems in their translation into the target language. The paper selects some frequent and highly polysemous English phrasal verbs listed with 10 or more senses in the English WordNet, collects sample English sentences from the British National Corpus and examines their Hindi translation equivalents in terms of the morpho-syntactic structure of Hindi verbs and the representation of semantic information. This study attempts to present the semantic equivalence in rendering the identical semantics in the verbal system of both languages under consideration. The study found that the most frequent and common way of realising the concepts represented by English phrasal verbs in Hindi is by incorporating the compound verb constructions in the Hindi language. Besides that, we observe multiple mapping patterns of English phrasal verbs in Hindi, including the simple Hindi verb (v), compound verb (v1-v2), and conjunct verb (noun/adjective/adverb-verb) constructions.

Grammars are core elements of many NLP applications. In this paper, we present a system that automatically extracts lexicalized grammars from annotated corpora. The data produced by this system have been used in several tasks, such as training NLP tools such as Supertaggers and estimating the coverage of hand-crafted grammars. We report experimental results on two of those tasks and compare our approaches with related work.

In this paper we report on the recent advancements and current status of the XTAG Project, housed at the University of Pennsylvania. We discuss the current coverage of the system, as evaluated on the TSNLP English sentences, hierarchical organization of the grammar, and the new and more portable implementation of the X-interface to the grammar and all of the supporting tools in CLISP, which is freely available. We also present a methodology for specializing our grammar to a particular domain, and give some results on this effort.

De nombreux travaux en Traduction Automatique Statistique (TAS) pour des langues d'entrée morphologiquement riches montrent que la ségmentation morphologique et la normalisation orthographique améliorent la qualité des traductions en diminuant la sparsité des données. Dans cet article, nous étudions l'impact de ce prétraitement pour la TAS vers une langue de sortie riche morphologiquement, comme l'Arabe. Nous explorons l'espace des schémas de segmentation et des options de normalisation possibles. Nous évaluons seulement la sortie sous une forme désegmentée et enrichie orthographiquement. Nos résultats montrent d'une part que le meilleur schéma pour la ségmentation est celui de la Penn Arabic Treebank. D'autre part, la meilleure procédure de prétraitement consiste à entraîner le système sur des données normalisées orthographiquement, puis à enrichir et désegmenter les traductions en sortie.

Statistical machine translation is quite robust when it comes to the choice of input representation. It only requires consistency between training and testing. As a result, there is a wide range of possible preprocessing choices for data used in statistical machine translation. This is even more so for morphologically rich languages such as Arabic. In this paper, we study the effect of different word-level preprocessing schemes for Arabic on the quality of phrase-based statistical machine translation. We also present and evaluate different methods for combining preprocessing schemes resulting in improved translation quality.

In this paper, we study the effect of different word-level preprocessing decisions for Arabic on SMT quality. Our results show that given large amounts of training data, splitting off only proclitics performs best. However, for small amounts of training data, it is best to apply English-like tokenization using part-of-speech tags, and sophisticated morphological analysis and disambiguation. Moreover, choosing the appropriate preprocessing produces a significant increase in BLEU score if there is a change in genre between training and test data.

Statistical machine translation is quite robust when it comes to the choice of input representation. It only requires consistency between training and testing. As a result, there is a wide range of possible preprocessing choices for data used in statistical machine translation. This is even more so for morphologically rich languages such as Arabic. In this paper, we study the effect of different word-level preprocessing schemes for Arabic on the quality of phrase-based statistical machine translation. We also present and evaluate different methods for combining preprocessing schemes resulting in improved translation quality.

De nombreux travaux en Traduction Automatique Statistique (TAS) pour des langues d'entrée morphologiquement riches montrent que la ségmentation morphologique et la normalisation orthographique améliorent la qualité des traductions en diminuant la sparsité des données. Dans cet article, nous étudions l'impact de ce prétraitement pour la TAS vers une langue de sortie riche morphologiquement, comme l'Arabe. Nous explorons l'espace des schémas de segmentation et des options de normalisation possibles. Nous évaluons seulement la sortie sous une forme désegmentée et enrichie orthographiquement. Nos résultats montrent d'une part que le meilleur schéma pour la ségmentation est celui de la Penn Arabic Treebank. D'autre part, la meilleure procédure de prétraitement consiste à entraîner le système sur des données normalisées orthographiquement, puis à enrichir et désegmenter les traductions en sortie.

We describe an approach to automatic source-language syntactic preprocessing in the context of Arabic-English phrase-based machine translation. Source-language labeled dependencies, that are word aligned with target language words in a parallel corpus, are used to automatically extract syntactic reordering rules in the same spirit of and . The extracted rules are used to reorder the source-language side of the training and test data. Our results show that when using monotonic decoding and translations for unigram source-language phrases only, source-language reordering gives very significant gains over no reordering (25% relative increase in BLEU score). With decoder distortion turned on and with access to all phrase translations, the differences in BLEU scores are diminished. However, an analysis of sentence-level BLEU scores shows reordering outperforms no-reordering in over 40% of the sentences. These results suggest that the approach holds big promise but much more work on Arabic parsing may be needed.

En la actualidad, la traducción automática (TA) ha venido cobrando gran relevancia en la comunicación intercultural y en la transferencia de conocimiento multilingüe gracias a los avances en inteligencia artificial y el procesamiento del lenguaje natural. Sin embargo, a pesar de los grandes avances en la última década, seguimos encontrándonos con retos significativos, especialmente en la calidad de las traducciones, lo que subraya la necesidad de la intervención humana especializada. Esta comunicación examina los desafíos actuales que supone la TA, la evolución de sus tecnologías y la relevancia del trabajo del traductor humano en el proceso de posedición.

Novel innovations in large language models (LLMs) have demonstrated their ability to generate and analyze literary texts. As a result of intricate semantic layers, metaphors, polyphony, and nonlinear narrative structures present in literary works, their analysis by LLMs demands a deep cognitive and semantic understanding. Hence, it is essential to investigate the present abilities of LLMs to understand complex literary narratives, assess their performance, and spot their shortcomings to deliver a consistent view regarding the upcoming development of technologies in computational literary studies. This study, through a systematic scoping review of data from 48 peer-reviewed articles from five major scientific databases, assesses the current state of research on LLM performance in the interpretation and production of literary texts. We analyzed the selected studies according to two principal axes: (1) the fields of literary applications of LLMs and their performance evaluation within each domain and (2) current theoretical and technical challenges and limitations. The findings revealed that LLMs have been applied to eight major literary tasks, including comprehensive literary analysis and interpretation, understanding and extracting character relationships and traits, stylistic analysis and authorship attribution, interpretation of metaphors and rhetorical features, evaluation and generation of literary content by LLMs, quotation attribution to fictional characters, literary text summarization, and literary translation. Key challenges and limitations were also identified, including data bias and dependency, human intervention and evaluation, constraints related to text and narrative length, and limitations in deep understanding and reasoning. In addition, we formulated six recommendations for future studies on developing and implementing LLMs in literary studies. This review provides a comprehensive roadmap for future researchers to identify current strengths and weaknesses, address existing gaps, and leverage the strengths in practical applications, such as literary translation.

The acceleration in telecommunication needs leads to many groups of research, especially in communication facilitating and Machine Translation fields. While people contact with others having different languages and cultures, they need to have instant translations. However, the available instant translators are still providing somewhat bad Arabic-English Translations, for instance when translating books or articles, the meaning is not totally accurate. Therefore, using the semantic web techniques to deal with the homographs and homonyms semantically, the aim of this research is to extend a model for the ontology-based Arabic-English Machine Translation, named NAN, which simulate the human way in translation. The experimental results show that NAN translation is approximately more similar to the Human Translation than the other instant translators. The resulted translation will help getting the translated texts in the target language somewhat correctly and semantically more similar to human translations for the Non-Arabic Natives and the Non-English natives.

Many Natural Language Processing (NLP) applications involve Named Entity Recognition (NER) as an important task, where it leads to improve the overall performance of NLP applications. In this paper the Deep learning techniques are used to perform NER task on Hindi text data as it found that as compared to English NER, Hindi language NER is not sufficiently done. This is a barrier for resource-scarce languages as many resources are not readily available. Many researchers use various techniques such as rule based, machine learning based and hybrid approaches to solve this problem. Deep learning based algorithms are being developed in large scale as an innovative approach now a days for the advanced NER models which will give the best results out of it. In this paper we devise a Novel architecture based on residual network architecture for preferably Bidirectional Long Short Term Memory (BiLSTM) with fasttext word embedding layers. For this purpose we use pre-trained word embedding to represent the words in the corpus where the NER tags of the words are defined as the used annotated corpora. BiLSTM Development of an NER system for Indian languages is a comparatively difficult task. In this paper, we have done the various experiments to compare the results of NER with normal embedding and fasttext embedding layers to analyse the performance of word embedding with different batch sizes to train the deep learning models. Here we present a state-of-the-art results with said approach F1 Score measures.

India's linguistic diversity has been widely documented.Yet existing debates on language loss do not examine conceptual repertoires encoded in Indian languages. Drawing on postcolonial analysis, philosophical accounts of concept loss, and empirical work on education and mental health, this paper argues that institutional vocabularies increasingly fail to register predicate-distinctions central to the Indian life-world. Using Narahari Rao's "culture as learnable" and the notion of predicate-distinctions, it proposes an action-approach for reconstructing these repertoires and for understanding Indian languages as repositories of learnable criteria, rather than as mere vehicles of vocabulary.

Automatic Term Recognition (ATR) is an important method for the summarization and analysis of large corpora, and normally requires a significant amount of linguistic input, in particular the use of part-of-speech taggers. For an under-resourced language such as Irish, the resources necessary for this may be scarce or entirely absent. We evaluate two methods for the automatic extraction of terms, based on the small part-of-speech-tagged corpora that are available for Irish and on a large terminology list, and show that both methods can produce viable term extractors. We evaluate this with a newly constructed corpus that is the first available corpus for term extraction in Irish. Our results shine some light on the challenge of adapting natural language processing systems to under-resourced scenarios.

Despite being the seventh most widely spoken language in the world, Bengali has received much less attention in machine translation literature due to being low in resources. Most publicly available parallel corpora for Bengali are not large enough; and have rather poor quality, mostly because of incorrect sentence alignments resulting from erroneous sentence segmentation, and also because of a high volume of noise present in them. In this work, we build a customized sentence segmenter for Bengali and propose two novel methods for parallel corpus creation on low-resource setups: aligner ensembling and batch filtering. With the segmenter and the two methods combined, we compile a high-quality Bengali-English parallel corpus comprising of 2.75 million sentence pairs, more than 2 million of which were not available before. Training on neural models, we achieve an improvement of more than 9 BLEU score over previous approaches to Bengali-English machine translation. We also evaluate on a ...

Samāsa or compounds are a regular feature of Indian Languages. They are also found in other languages like German, Italian, French, Russian, Spanish, etc. Compound word is constructed from two or more words to form a single word. The meaning of this word is derived from each of the individual words of the compound. To develop a system to generate, identify and interpret compounds, is an important task in Natural Language Processing. This paper introduces a web based tool – Samāsa-Kartā for producing compound words. Here, the focus is on Sanskrit language due to its richness in usage of compounds; however, this approach can be applied to any Indian language as well as other languages. IndoWordNet is used as a resource for words to be compounded. The motivation behind creating compound words is to create, to improve the vocabulary, to reduce sense ambiguity, etc. in order to enrich the WordNet. The Samāsa-Kartā can be used for various applications viz., compound categorization, sandhi...

This study investigates present-tense verbal constructions in Classical Arabic with special focus on the discrepancy between syntactic optimality and communicative optimality. Specifically, this study challenges the traditional view that Arabic is completely rule-governed, drawing on the observation that many documented linguistic usages deviate from prescriptive syntactic constraints. Ibn Hisham al-Anṣari's Mughni al-Labib formed the primary corpus for this study. The focus was on Qur'anic recitations and Prophetic ḥadith; poetic data was excluded to avoid rhythmic and metrical justifications. Adopting Optimality Theory (OT), four types of syntactic deviations were highlighted: substitution of final long inflectional vowels with short vowels in non-jussive contexts, retention of long inflectional vowels in jussive constructions, deletion of indicative/subjunctive inflectional suffixes, and non-canonical subjunctive marking in jussive contexts. Although early Arab syntacticians consider such deviations anomalous, dialectal, or metrical-poetic exceptions, and often reinterpret them to fit established rules, this study argues that these deviant forms, despite violating rigid syntactic rules, still achieve communicative optimality and reflect dialectal or historical variations. Such non-prescriptive usages represent authentic linguistic remainders that can enrich the Arabic syntactic system. It was concluded that Arabic syntactic rules are not inviolable and Arabic syntax is flexible enough to accommodate such usages.

Machine translation (MT) has undergone a major transformation over the past decades, evolving from rule-based and statistical models into neural machine translation (NMT), which relies on deep learning architectures trained on large parallel corpora. The introduction of attention mechanisms and later the Transformer architecture marked a paradigm shift in MT research, allowing systems to capture long-range dependencies and generate more fluent output (Bahdanau et al., 2015; Vaswani et al., 2017). As a result, MT has become widely accessible to non-specialist users through platforms such as Google Translate and Microsoft Translator. Nevertheless, fluency improvements have not eliminated deeper semantic, pragmatic, and cultural challenges, particularly for morphologically rich languages such as Arabic (Ameur et al., 2020).

Japan Patent Information Organization (JAPIO) participates in scientific paper subtask (ASPEC-EJ/CJ) and patent subtask (JPC-EJ/CJ/KJ) with phrase-based SMT systems which are trained with its own patent corpora. Using larger corpora than those prepared by the workshop organizer, we achieved higher BLEU scores than most participants in EJ and CJ translations of patent subtask, but in crowdsourcing evaluation, our EJ translation, which is best in all automatic evaluations, received a very poor score. In scientific paper subtask, our translations are given lower scores than most translations that are produced by translation engines trained with the indomain corpora. But our scores are higher than those of general-purpose RBMTs and online services. Considering the result of crowdsourcing evaluation, it shows a possibility that CJ SMT system trained with a large patent corpus translates non-patent technical documents at a practical level.

System architecture, experimental settings and evaluation results of EHR group in the en-ja, zh-ja, JPCzh-ja and JPCko-ja tasks are described. Our system concept is combination of a rule based method and a statistical method. System combination of rule-based machine translation (RBMT), RBMT plus statistical post-editing (SPE) and preordering plus statistical machine translation (SMT) is conducted. From the multiple outputs of three systems, candidate selection part selects the best output by language model score. For JPCzh-ja task devtest data translation, SPE improves BLEU score by 17.81, preordering improves BLEU score by 1.89 and system combination improves BLEU score by 0.26.

The present work describes a multilingual corpus of online content in the educational domain, i.e. Massive Open Online Course material, ranging from course forum text to subtitles of online video lectures, that has been developed via large-scale crowdsourcing. The English source text is manually translated into 11 European and BRIC languages using the CrowdFlower platform. During the process several challenges arose which mainly involved the in-domain text genre, the large text volume, the idiosyncrasies of each target language, the limitations of the crowdsourcing platform, as well as the quality assurance and workflow issues of the crowdsourcing process. The corpus constitutes a product of the EU-funded TraMOOC project and is utilised in the project in order to train, tune and test machine translation engines.

Chimera is a machine translation system that combines the TectoMT deep-linguistic core with phrase-based MT system Moses. For English-Czech pair it also uses the Depfix postcorrection system. All the components run on Unix/Linux platform and are open source (available from Perl repository CPAN and the LINDAT/CLARIN repository). The main website is . The development is currently supported by the QTLeap 7 th FP project (). TectoMT (the deep-linguistic core of Chimera) is an open-source MT system based on the Treex platform for general natural-language processing. TectoMT uses a combination of rule-based and statistical (trained) modules ("blocks" in Treex terminology), with a statistical transfer based on HMTM (Hidden Markov Tree Model) at the level of a deep, so-called tectogrammatical representation of sentence structure. In the Chimera combination, TectoMT is complemented by a Moses PB-SMT system (factored setup with additional language models over morphological tags) and optionally also by an automatic postprocessing (correction) component called Depfix. Chimera can be thus characterized as a hybrid system that combines statistical MT with deep linguistic analysis and automatic post-correction system, which is useful especially for translation into inflectionally rich languages. The three systems are combined serially: TectoMT runs first, then an additional Moses phrase table is extracted from TectoMT's input and output. The additional table is then used in a weighted combination with a large Moses translation table to produce pre-final output. Depfix then re-parses the output (as well as input) and generates the final output based on rules reflecting morphosyntactic properties of the target language. Chimera was transferred from English-Czech to additional three language pairs (English to Dutch, Portuguese and Spanish) within the QTLeap 7 th EU project.

This work presents parallel corpora automatically annotated with several NLP tools, including lemma and part-of-speech tagging, named-entity recognition and classification, named-entity disambiguation, word-sense disambiguation, and coreference. The corpora comprise both the well-known Europarl corpus and a domain-specific question-answer troubleshooting corpus on the IT domain. English is common in all parallel corpora, with translations in five languages, namely, Basque, Bulgarian, Czech, Portuguese and Spanish. We describe the annotated corpora and the tools used for annotation, as well as annotation statistics for each language. These new resources are freely available and will help research on semantic processing for machine translation and cross-lingual transfer.

In this poster we present QTLeap (qtleap.eu), an ongoing project whose goal is to research on and deliver an articulated methodology for machine translation that explores deep language engineering approaches, which handle the representation of the meaning of utterances, in view of breaking the way to translations of higher quality.

Language users in multilingual environments who are trying to make sense of the linguistic challenges they face may well regard the advent of online machine translation (MT) applications as a welcome intervention. Such applications have made it possible for virtually anyone to try their hand at translation -with minimum effort, at that. However, the usefulness of the output of these translation applications varies. The empirical research described in this article is a continuation of an investigation into the usefulness of MT in a higher education context. In 2010, Afrikaans and English translations generated by Google Translate and two human translators, based on the same set of source texts, were evaluated by a panel of raters by means of a holistic assessment tool. In 2011 and 2012, the same set of source texts was translated again with Google Translate, and those translations have since been evaluated in exactly the same manner. The results show that the quality of Google Translate's output has improved over the three years. Subsequently, an error analysis was performed on the translation set of one text type by means of a second assessment tool. Despite an overall improvement in quality, we found that the 2012 translation contained unexpected new errors. In addition, the error analysis showed that mistranslation posed the largest risk when using this MT application. Users of MT should, therefore, understand the risks of their choice and that some text types and contexts are better suited to MT than others. Armed with this knowledge, translators and multilingual communities can make informed decisions regarding MT and translation technology in general.

Statistical Machine Translation (SMT) systems often make mistake in translating a multi-word term (MWT). Building a bilingual MWT lexicon is one of the important steps to improve the translation result on sentence level. This thesis proposes a novel method to produce a bilingual lexicon of English-Japanese (and vice versa) translation from comparable corpora. First we extract the candidate MWTs using a predefined linguistic filter separately on Japanese documents and English documents. Next we score the extracted MWTs using six statistical filters and prune some of the MWTs whose scores are less than the threshold. Finally we use a topic model to analyze the distribution of MWTs according to the topic information, and extract a bilingual lexicon by treating each MWT as a word. Manual evaluation on top-100 MWTs shows that a precision of about 70% can be achieved on two different comparable corpora.

RUOKONEN, M. ; MARICEL, B. ; KEMPPANEN, H. ; RUDVIN, M. ; SILVA-REIS, DENNYS ; TAKEDA, K. . Translators - A Century of Tensions and Transformations. In: Gambier, Yves; Wakabayashi, Judy. (Org.). A Cultural History of Translation: Volume 6: In the Modern and Contemporary World. 1ed.London: Bloomsbury Publishing, 2025, v. 6, p. 17-36.

Artificial Intelligence (AI) has emerged as a transformative force in research translation within higher education, shifting the paradigm from basic automation to intelligent systems capable of semantic understanding and contextual adaptation. This study explores the applied potentials of AI-driven tools, such as natural language processing (NLP) and deep learning, in facilitating research translation for university students. Unlike traditional machine translation, contemporary AI models understand cognitive structures and cultural nuances, generating outputs consistent with the target language's linguistic context. This paper analyzes how these technologies enhance cross-border academic communication and facilitate access to global knowledge by accelerating multilingual content production and reducing costs. Furthermore, it examines the collaborative human-machine review mechanism as a key factor in improving translation quality. The findings suggest that integrating AI into higher education research practices not only optimizes technical efficiency but also broadens the horizons for studying translation as a complex cognitive process. This study provides insights into how university students can leverage these advancements to bridge linguistic gaps in global academic discourse.