Development of an instrument for measuring Spanish vocabulary knowledge

Hou Keat Khong; Muhammad Kamarul Kabilan

doi:10.30827/portalin.vi41.25406

Authors

Hou Keat Khong Universiti Kuala Lumpur
Muhammad Kamarul Kabilan Universiti Sains Malaysia & Universitas Negeri Malang

DOI:

https://doi.org/10.30827/portalin.vi41.25406

Keywords:

instrument, vocabulary test, vocabulary knowledge, validity, reliability

Abstract

Reliable estimates of vocabulary knowledge offer useful information for planning and improving language learning. However, problems arise when existing vocabulary measures signal a lack of validation evidence, and they are predominantly used for instructional purposes. Therefore, this study addresses the gap by providing a description of the development of a valid and reliable instrument for research and evaluation. Adhering to Read and Chapelle’s (2001) framework for vocabulary assessment, the instrument was created using resources which were closely related to the research subject and context to enhance the validity and reliability. Results indicated that the instrument was valid, reliable and sensitive to different vocabulary instructions. This study not only presents validation evidence which is consistent with current educational measurement theory, but also encourages language teachers and researchers to become vocabulary assessment literate to assure the psychometric qualities of the instrument they develop.

Downloads

Download data is not yet available.

Author Biographies

Hou Keat Khong, Universiti Kuala Lumpur

Hou Keat KHONG is a Spanish Language lecturer in the Universiti Kuala Lumpur Malaysian Spanish Institute. Currently he is pursuing his PhD under the supervision of Professor Muhammad Kamarul Kabilan. His research interests focus on instructed second language acquisition (ISLA) including computer-assisted language learning including pedagogical innovations, pragmatics, and cognitive processes in ISLA.

Muhammad Kamarul Kabilan, Universiti Sains Malaysia & Universitas Negeri Malang

Muhammad Kamarul KABILAN is a professor at the School of Educational Studies, Universiti Sains Malaysia, Penang. His research interests include ICT and English Language Education and, professional development and critical practices of teachers. He has published widely in his area of research in reputable journals both locally and internationally.

References

Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221. https://doi.org/10.1016/j.sapharm.2018.03.066

Aoiz Pinillos, M. (2022). Creation and validation of a bilingual test to estimate aural and written vocabulary size. Porta Linguarum Revista Interuniversitaria de Didáctica de Las Lenguas Extranjeras, 38, 247–263. https://doi.org/10.30827/portalin.vi38.23606

Brown, J. D. (1996). Testing in language programs. Prentice Hall Regents.

Creswell, J. W. (2012). Educational research: Planning, conducting and evaluating quantitative and qualitative research (4th ed.). Pearson Education Inc.

Cruse, D. A. (1986). Lexical semantics. Cambridge University Press.

Davies, M. (2006). A frequency dictionary of Spanish: Core vocabulary for learners. Routledge.

Edmonds, A., Clenton, J., & Elmetaher, H. (2022). Exploring the construct validity of tests used to assess L2 productive vocabulary knowledge. System, 108, 102855. https://doi.org/10.1016/j.system.2022.102855

Gyllstad, H. (2013). Looking at L2 vocabulary knowledge dimensions from an assessment perspective – challenges and potential solutions. L2 Vocabulary Adquisition, Knowledgement, and Use: New Perspectives on on Assessment and Corpus Analysis, 11–28.

Hofmeyer, A., Sheingold, B. H., & Taylor, R. (2015). Do you understand what I mean? How cognitive interviewing can strengthen valid, reliable study instruments and dissemination products. Journal of International Education Research, 11(4), 261–268.

Hughes, K. A. (2004). Comparing pretesting methods: Cognitive interviews, respondent debriefing, and behavior coding. Survey Methodology, 2, 1–20.

Kremmel, B. (2018). Development and initial validation of a diagnostic computer-adaptive profiler of vocabulary knowledge [PhD Thesis]. University of Nottingham.

Kremmel, B., & Pellicer-Sánchez, A. (2021). Measuring vocabulary development. In P. Winke & T. Brunfaut (Eds.), The Routledge Handbook of Second Language Acquisition and Language Testing (pp. 211–222). Routledge. https://www.routledge.com/Second-Language-

Laufer, B., Elder, C., Hill, K., & Congdon, P. (2004). Size and strength: Do we need both to measure vocabulary knowledge? Language Testing, 21(2), 202–226. https://doi.org/10.1191/0265532204lt277oa

Leong, C.H.A., Abidin, M.J.Z., & Saibon, J.S. (2019). The benefits and drawbacks of using tablet-based digital storytelling in vocabulary learning among Malaysian young English as a Second Language (ESL) learners. Asia Pacific Journal of Educators and Education, 34, 17–47. https://doi. org/10.21315/apjee2019.34.2

Ma, Q., & Lee, H. Y. (2019). Measuring the vocabulary knowledge of Hong Kong primary school second language learners through word associations: Implications for reading literacy. In B. L. Reynolds & M. F. Teng (Eds.), English Literacy Instruction for Chinese Speakers (pp. 35–56). Palgrave Macmillan. https://doi.org/10.1007/978-981-13-6653-6

Memon, A., Mumtaz, Hiram, Ting, T., Ramayah, Francis, Chuah, Jun-Hwa, & Cheah. (2017). A review of the methodological misconceptions and guidelines related to the application of structural equation modeling: A Malaysian scenario. Journal of Applied Structural Equation Modeling, 1(1), i–xiii.

Mendívil Giró, J. L. (2021). Inflection, derivation and compounding: Issues of delimitation. In A. Fábregas, V. Acedo-Matellán, G. Armstrong, M. C. Cuervo, & I. Pujol Payet (Eds.), The Routledge Handbook of Spanish Morphology (1st ed.). Routledge.

Messick, S. (1995). Standards of validity and the validity of standards in performance asessment. Educational Measurement: Issues and Practice, 14(4), 5–8. https://doi.org/10.1111/j.1745-3992.1995.tb00881.x

Milton, J. (2009). Measuring second language vocabulary acquisition. Multilingual Matters. https://doi.org/10.21832/9781847692092

Nakata, T., & Webb, S. (2016). Does studying vocabulary in smaller sets increase learning? The effects of part and whole learning on second language vocabulary acquisition. Studies in Second Language Acquisition, 38(3), 523–552. https://doi.org/10.1017/S0272263115000236

Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge University Press.

Nation, I. S. P. (2020). The different aspects of vocabulary knowledge. In S. A. Webb (Ed.), The Routledge Handbook of Vocabulary Studies (1st ed., pp. 15–29). Routledge. https://doi.org/10.4324/9780429291586.2

Nation, I. S. P., & Hunston, S. (2013). Learning vocabulary in another language (2nd ed.). Cambridge University Press. https://doi.org/10.1017/CBO9781139858656

Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12(2), 281–316.

Peterson, C. H., Peterson, N. A., & Powell, K. G. (2017). Development cognitive interviewing for item development: Validity evidence based on content and response processes. Measurement and Evaluation in Counseling and Development, 50(4), 217–223. https://doi.org/10.1080/07481756.2017.1339564

Polit, D. F., Beck, C. T., & Owen, S. v. (2007). Focus on research methods: Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing and Health, 30(4), 459–467. https://doi.org/10.1002/nur.20199

Read, J. (2000). Assessing vocabulary. Cambridge University Press. https://doi.org/10.1017/CBO9780511732942

Read, J., & Chapelle, C. A. (2001). A framework for second language vocabulary assessment. Language Testing, 18(1), 1–32. https://doi.org/10.1177/026553220101800101

Reddy, A. J., Khera, M. K., McLaughlin, J., & Szabo, C. Z. (2021). The role of motivation in incidental vocabulary learning through academic videos. Asia Pacific Journal of Educators and Education, 36(1), 135–153. https://doi.org/10.21315/apjee2021.36.1.8

Sánchez Gutiérrez, C. H. (2021). Morphology and language teaching. In A. Fábregas, V. Acedo-Matellán, G. Armstrong, M. C. Cuervo, & I. Pujol Payet (Eds.), The Routledge Handbook of Spanish Morphology (1st ed.). Routledge. https://doi.org/10.4324/9780429318191.42

Schmitt, N. (2010). Researching vocabulary: A vocabulary research manual. Palgrave Macmillan.

Sinclair, J., & Renouf, A. (1988). A lexical syllabus for language learning. In R. Carter & M. Mccarthy (Eds.), Vocabulary and Language Teaching (pp. 140–158). Routledge.

Thornton, A. M. (2018). Troubles with flexemes. In O. Olivier Bonami, G. Boyé, G. Dal, H. Giraudo, & F. Namer (Eds.), The lexeme in descriptive and theoretical morphology (pp. 303–321). Language Science Press. https://doi.org/10.5281/zenodo.1407011

Verspoor, M., & Lowie, W. (2003). Making sense of polysemous words. Language Learning, 53(3), 547–586.

Wan Mahzan, M., Alias, N., & Ismail, I. (2020). Unboxing the design of English as a second language (ESL) learning video game for indigenous learners: An empathic design-based approach. Asia Pacific Journal of Educators and Education, 35(2), 39–56. https://doi.org/10.21315/apjee2020.35.2.3

Webb, S. (2005). Receptive and productive vocabulary learning: The effects of reading and writing on word knowledge. Studies in Second Language Acquisition, 27, 33–52. https://doi.org/10.1017/S0272263105050023

Webb, S. (2007). Learning word pairs and glossed sentences: The effects of a single context on vocabulary knowledge. Language Teaching Research, 11(1), 63–81. https://doi.org/10.1177/1362168806072463

Yanagisawa, A., & Webb, S. (2020). Measuring depth of vocabulary knowledge. In S. Webb (Ed.), The Routledge Handbook of Vocabulary Studies (1st ed., pp. 371–386). Routledge. https://doi.org/10.4324/9780429291586.24