Effectiveness of textually-enhanced captions on Chinese high-school EFL learners’ incidental vocabulary learning

Huizhen Wu; Xiaohu YANG

doi:10.30827/portalin.vi38.23511

Authors

Huizhen Wu Tongji University/Shanghai Business School https://orcid.org/0000-0003-0291-5346
Dr. Xiaohu YANG Tongji University

DOI:

https://doi.org/10.30827/portalin.vi38.23511

Keywords:

Captioning types, video viewing, foreign language learning, vocabulary learning, L1 gloss

Agencies: This research was supported by a Grant (No. C20135) from Shanghai Municipal Commission of Education and a Grant (No. AG-35606-0003) from Shanghai Business School in China Mainland.

Abstract

This study employed mixed-methods approach to investigate the impact of textually-enhanced captions on EFL learners’ incidental vocabulary gains and learners’ perceptions of the captioning usefulness in a multi-modal learning environment. 133 Chinese EFL high school learners of the low-intermediate level were randomly assigned to English captions with highlighted target words and L1 gloss (ECL1), Chinese and English captions (CEC), Chinese and English captions with highlighted target words (CECGW), and no captions (NC). Our quasi-experimental findings did not detect any significant differences among the caption types on vocabulary form recognition while ECL1 was found the most effective in meaning recall and recognition. Captioning types and learners’ language proficiency exerted medium-to-large effects on meaning recall and meaning recognition. Our qualitative data suggested the participants generally viewed captioned videos positively, with variability in perceptions of concurrent presentation of information. The saliency of L1 gloss could direct the viewers’ attention to the semantic features of a word and reinforce sound-form-meaning connections. Videos lacking L1 glosses of target words had relatively little effect on learners’ vocabulary learning while more textual inputs might not necessarily result in vocabulary gains. Pedagogical implications are proposed for teachers’ adoption of L1 in captioned videos to enhance learners’ learning effectiveness.

Downloads

Download data is not yet available.

Author Biographies

Huizhen Wu, Tongji University/Shanghai Business School

Huizhen Wu is an Associate Professor at the School of Foreign Languages, Shanghai Business School in China. She is currently pursuing her Ph. D studies in Applied Linguistics in Tongji University. Her research interests lie in mobile-assisted language learning and teaching, affect and language learning, and L2 listening and vocabulary. She has published in both international and local journals, including Porta Linguarum, International Journal of Computer Assisted Language Learning and Teaching, Education Journal, Shanghai Journal of Translators.

Dr. Xiaohu YANG, Tongji University

Xiaohu Yang is a Professor in the School of Foreign languages, Tongji University in Shanghai, China. He is currently director of Speech-Language Processing Research Center. His research interests include psycholinguistics, phonetics, and speech perception. He has published prolifically in international flagship journals, such as Applied Psycholinguistics, Journal of Speech, Language, & Hearing Research, Journal of the Acoustical Society of America and Frontiers in Psychology.

References

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Academic Press.

Dai, J. (2005). Captioned video and foreign language learning. Technology Enhanced Foreign Language Education, 103, 18-22.

Danan, M. (1992). Reversed subtitling and dual coding theory: New directions for foreign language instruction. Language Learning, 42(4), 497–527.https://doi.org/10.1111/j.1467-1770.1992.tb01042.x

De Guerrero, M., & Commander, M. (2013). Shadow-reading: Affordances for imitation in the language classroom, Language Teaching Research, 2013, 17(4), 433-453.https://doi.org/10.1177/1362168813494125

Fernández-Costale, A. (2021). Subtitling and Dubbing as Teaching Resources in CLIL in Primary Education: The Teachers’ Perspective, Porta Linguarum, 36, 175-192. https://doi.org/10.30827/portalin.v0i36.16228

Frick, T.W., Chadha, R., Watson, C., Wang, Y., & Green, P. (2009). College student perceptions of teaching and learning quality. Educational Technology Research and Development, 57(5), 705–720. https://doi.org/10.1007/s11423-007-9079-9

Gass, S. M. (1999). Discussion: Incidental vocabulary learning. Studies in Second Language Acquisition, 21(2), 319–333.

Gerbier, E., Bailly, G., & Bosse, M. (2018). Audio-visual synchronization in reading while listening to texts: Effects on visual behavior and verbal learning. Computer Speech and Language, 47 (1):79-92. https://doi.org/ff10.1016/j.csl.2017.07.003f

Hsieh, H.F., & Shannon, S. E. (2005) Three approaches to qualitative content analysis. Qualitative Health Research, 15(9): 1277–1288. https://doi.org/10.1177/1049732305276687

Hsieh, Y. (2020). Effects of video captioning on EFL vocabulary learning and listening comprehension. Computer Assisted Language Learning, 33(5-6), 567-589. https://doi.org/10.1080/09588221.2019.1577898

Hsu, C.K., Hwang, G.J., Chang, Y.T., & Chang, C.K. (2013). Effects of video caption modes on English listening comprehension and vocabulary acquisition using handheld devices. Educational Technology & Society, 16(1), 403-414.

Kam, E.F., Liu, Y.T., & Tseng, W.T. (2020). Effects of modality preference and working memory capacity on captioned videos in enhancing L2 listening outcomes. ReCALL, 32(2), 213–230. https://doi.org/10.1017/S0958344020000014

Kang, H., Kweon, S. & Choi, S. (2020). Using eye-tracking to examine the role of first and second language glosses. Language Teaching Research, 6, 1-22. https://doi.org/10.1177/1362168820928567

Leonhardt, J. (2020). Using film and media in the language classroom: reflections on research-led teaching. ELT Journal, 74(2), 229–231. https://doi.org/10.1093/elt/ccaa007

Leveridge, A.N., & Yang, J.C. (2014). Learner perceptions of reliance on captions in EFL multimedia listening comprehension, Computer Assisted Language Learning, 27(6), 545-559. https://doi.org/10.1080/09588221.2013.776968

Li, M. (2016). An Investigation into the Differential Effects of Subtitles (First Language, Second Language, and Bilingual) on Second Language Vocabulary Acquisition. PhD thesis. Edinburgh: University of Edinburgh.

Liao, S., Kruger, J., & Doherty, S. (2020). The impact of monolingual and bilingual subtitles on visual attention, cognitive load, and comprehension. The Journal of Specialised Translation, 33, 70-98. https://www.jostrans.org/issue33/art_liao.pdf

Mayer, R.E., & Pilegard, C. (2014). Principles for managing essential processing in multimedia learning: segmenting, pre-training, and modality principles. In R. E. Mayer (Ed.), Cambridge handbook of multimedia learning (pp.307-337). New York: Cambridge.

Mayer, R.E. (2014). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), Cambridge handbook of multimedia learning (pp.55-81). New York: Cambridge.

Montero Perez, M., Peters, E., Clarebout, G., & Desmet, P. (2014). Effects of captioning on video comprehension and incidental vocabulary learning. Language Learning and Technology, 18(1), 118-141. http://llt.msu.edu/issues/february2014/monteroperezetal.pdf

Montero Perez, M., Peters, E., & Desmet, P. (2018). Vocabulary learning through viewing video: The effect of two enhancement techniques. Computer Assisted Language Learning, 31(1), 1-26. https://doi.org/10.1080/09588221.2017.1375960

Montero Perez, M. (2020). Incidental vocabulary learning through viewing video: the role of vocabulary knowledge and working memory. Studies in Second Language Acquisition, 42(4), 1-25. https://doi.org/10.1017/S0272263120000145

Moreno, R., & Abercrombie, S. (2010). Promoting awareness of learner diversity in prospective teachers: Signaling individual and group differences within classroom cases. Journal of Technology and Teacher Education, 18(1), 111–130. https://www.learntechlib.org/primary/p/29271/.

Neuman, S.B., & Koskinen, P. (1992). Captioned television as comprehensible input: Effects of incidental word learning from context for language minority students. Reading Research Quarterly, 27(1), 95–106.

Schmidt, R.W. (1990). The role of consciousness in second language learning. Applied Linguistics, 11(2), 129-158. https://doi.org/10.1093/applin/11.2.129

Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York: Springer.

Sydorenko, T. (2010). Modality of input and vocabulary acquisition. Language Learning & Technology, 14(2), 50–73. http://llt.msu.edu/vol14num2/sydorenko.pdf

Teng, F. (2019). Maximizing the potential of captions for primary school ESL students’ comprehension of English-language videos. Computer Assisted Language Learning, 32(7), 665-691. https://doi.org/10.1080/09588221.2018.1532912

van Gog, T. (2014). The signaling (or cueing) principle in multimedia learning. In R. E. Mayer (Ed.), Cambridge handbook of multimedia learning (pp. 255-269). New York: Cambridge.

Webb, S., & Chang, A. (2012). Vocabulary learning through assisted and unassisted repeated reading. The Canadian Modern Language Review, 68(3), 267-290.

Webb, S., & Nation, P. (2017). How vocabulary is learned. Oxford: Oxford University Press.

Yeldham, M. (2018). Viewing L2 captioned videos: What’s in it for the listener? Computer Assisted Language Learning, 31(4), 367-389. https://doi.org/10.1080/09588221.2017.1406956

Yun, J. (2011). The effects of hypertext glosses on L2 vocabulary acquisition: A meta-analysis. Computer Assisted Language Learning, 24(1), 39–58. https://doi.org/10.1080/09588221.2010.523285