A Comparative Analysis of Koreans’ English Writings and Google Translations Using Coh-Metrix 3.0

Abstract

Kim, Joonkoo and Lim, Kyuyun. 2019. A comparative analysis of Koreans’ English writings and Google Translations using Coh-Metrix 3.0. Korean Journal of English Language and Linguistics 19-3, 452-474. The purpose of the present study is to identify the corpus-based differences between Koreans’ English writings and their corresponding Google translations. For this purpose, the present study utilized Coh-Metrix 3.0 and conducted comparative analyses on two types of writings in terms of 12 benchmarks of text analysis. Coh-Metrix 3.0 provided numeric values for the following selected categories of text analysis: (a) basic counts (i.e., DESSC, DESWC, and DESSL), (b) lexical aspects (i.e., WRDFRQc and LDTTRc), (c) readability (i.e., RDFRE and RDFKGL) (d) syntactic complexity (i.e., SYNLE, SYNNP, and SYNSTRUTa), and (e) cohesion (i.e., CRAFAOa and LSASS1). Each output for 5 categories computed by Coh-Metrix 3.0 was then statistically processed in order to find statistically significant differences. The quantitative findings, given the small sample size associated with lower statistical power and non-normality of some data sets, were interpreted together with results from a robust technique of bootstrapped independent t-tests since the employment of bootstrapping has been empirically justified in the field of applied linguistics (Plonsky 2013, 2014). The overall findings indicated that Google translations tend to produce significantly more words before main verbs and longer sentences compared to human writings. Furthermore, it was also found that Google translations were significantly less readable, but more cohesive. However, there were no significant differences observed in lexical aspects.