Creation and validation of a bilingual test to estimate aural and written vocabulary size
DOI:
https://doi.org/10.30827/portalin.vi38.23606Keywords:
L2 vocabulary, aural vocabulary size, written vocabulary size, vocabulary test, vocabulary teaching.Abstract
Language learners’ vocabulary size is a reliable predictor of their success in a second language as it clearly correlates with better performances in the target language. Being precise in those estimations is paramount to plan language teaching. However, the instruments employed by previous studies for those estimations might present validity and reliability issues that affect their research sensitivity and accuracy.
This paper presents a step-by-step account of the creation of an aural and a written version of a bilingual vocabulary test. Test was delivered to 73 adult L1-Spanish students attending English classes. Their answers were analysed with Rasch model to determine the best performing items in the test so that the overall reliability of the instrument was enhanced.
The final version of the test presents high levels of reliability: .89 for the listening vocabulary test and .82 for the written vocabulary test. Furthermore, descriptive statistics confirm that recognizing the words in their aural form is more challenging than in their written form: participants got 10.80% fewer correct answers in the listening vocabulary test. This finding confirms the claim that aural and written vocabulary are two separate dimensions, and impacts on how vocabulary should be taught in L2 classrooms.
Downloads
References
Abbuhl, R., & Mackey, A. (2017). Second language acquisition research methods. In King, K. A., Lai, Y. J., & May, S. (Eds.). Research methods in language and education (3rd edition). (pp. 183-193). https://doi.org/10.1007/978-3-319-02249-9
Andringa, S., Olsthoorn, N., van Beuningen, C., Schoonen, R., & Hulstijn, J. (2012). Determinants of success in native and non‐native listening comprehension: An individual differences approach. Language Learning, 62(Suppl. 2), 49–78. https://doi.org/10.1111/j.1467- 9922.2012.00706.x
Beglar, D., & Nation, P. (2007). A vocabulary size test. The Language Teacher, 31(7), 9-13.
Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: fundamental measurement in the human sciences (3rd ed.). Routledge.
Cheng, J., & Matthews, J. (2018). The relationship between three measures of L2 vocabulary knowledge and L2 listening and reading. Language Testing, 35(1), 3-25. https://doi.org/10.1177/0265532216676851
Cobb, T. (2013). Frequency 2.0: Incorporating homoforms and multiword units in pedagogical frequency lists. In Bardel, C., Lindqvist, C., & Laufer, B. (Eds.) L2 vocabulary acquisition, knowledge and use: New perspectives on assessment and corpus analysis, (pp. 79-108). EUROSLA-the European Second Language Association. Retrieved from https://www.eurosla.org/
Cobb, T. (2019). Compleat Web VP v.2 [computer program]. Accessed on 16 Jan 2019 at https://www.lextutor.ca/cgi-bin/range/texts/index.pl
Crossley, S. A., Cobb, T., & McNamara, D. S. (2013). Comparing count-based and band-based indices of word frequency: Implications for active vocabulary research and pedagogical applications. System, 41(4), 965-981. https://doi.org/10.1016/j.system.2013.08.002
Hatch, E. M., & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguistics. Heinle & Heinle Publishers.
Hazenberg, S., & Hulstijn, J. H. (1996). Defining a minimal receptive second language vocabulary for non-native university students: An empirical investigation. Applied Linguistics, 17(2), 145-163. https://doi.org/10.1093/applin/17.2.145
Huang, H. T. (2010). How does second language vocabulary grow over time? A multi-methodological study of incremental vocabulary knowledge development (Doctoral dissertation, University of Hawai’i).
Karami, H. (2012). The development and validation of a bilingual version of the Vocabulary Size Test. RELC Journal, 43(1), 53-67. https://doi.org/10.1177/0033688212439359
Levitzky-Aviad, T., & Laufer, B. (2013). Lexical properties in the writing of foreign language learners over eight years of study: Single words and collocations. In Bardel, C., Lindqvist, C., & Laufer, B. (Eds.) L2 vocabulary acquisition, knowledge and use: New perspectives on assessment and corpus analysis, (pp. 127-148). EUROSLA-the European Second Language Association. Retrieved from https://www.eurosla.org/
Linacre, J. M. (1997). KR-20 / Cronbach alpha or Rasch person reliability: Which tells the "truth"? Rasch Measurement Transactions, 11(3), 580-581
Linacre, J. M. (2012). A user’s guide to Winsteps Ministeps Rasch-model computer programs [version 3.74.0]. Retrieved from http://www.winsteps.com/index.htm
Linacre, J. M. (2012, 2019). Winsteps® Rasch Measurement, version 4.4.3. [Computer software] Downloaded from http://www.winsteps.com
Martinez, R., & Schmitt, N. (2012). A phrasal expressions list. Applied Linguistics, 33(3), 299-320. https://doi.org/10.1093/applin/ams010
Masrai, A. (2020). Exploring the impact of individual differences in aural vocabulary knowledge, written vocabulary knowledge and working memory capacity on explaining L2 learners’ listening comprehension. Applied Linguistics Review, 11(3), 423-447. https://doi.org/10.1515/applirev-2018-0106
Mathias, C.W. (2010). Sensitivity. In Salkind, N. J. (Ed.). (2010). Encyclopedia of research design (Vol. 3), (pp. 1337-1338). Sage.
Matthews, J. (2018). Vocabulary for listening: Emerging evidence for high and mid-frequency vocabulary knowledge. System, 72, 23-36. https://doi.org/10.1016/j.system.2017.10.005
McLean, S., Kramer, B., & Beglar, D. (2015). The creation and validation of a listening vocabulary levels test. Language Teaching Research, 19(6), 741- 760. https://doi.org/10.1177/1362168814567889
Meara, P. M., & Miralpeix, I. (2006). Y_Lex: The Swansea advanced vocabulary levels test. v2. 05. Lognostics.
Nation, I. S. P. (2001) Learning vocabulary in another language. Cambridge University Press.
Nation, I. S. P. (2012, 2019). The BNC/COCA word family lists (17 September 2012). Unpublished paper. [online] Retrieved from http://www.victoria.ac.nz/lals/about/staff/paul-nation
Nation, I. S. P. (2016). Making and using word lists for language learning and testing. John Benjamins. https://doi.org/10.1093/applin/amx052
Nation, I. S. P., & Webb, S. A. (2011). Researching and analyzing vocabulary. Heinle, Cengage Learning.
Nguyen, L.T.C., & Nation, P. (2011). A bilingual vocabulary size test of English for Vietnamese learners. RELC journal, 42(1), 86-99. https://doi.org/10.1177/0033688210390264
Ockey, G. J., & Green, B. A. (Eds.). (2020). Another generation of fundamental considerations in language assessment: A festschrift in honor of Lyle F. Bachman. Springer Nature.
Ovtcharov, V., Cobb, T., & Halter, R. (2006). La richesse lexicale des productions orales: mesure fiable du niveau de compétence langagière. Canadian Modern Language Review, 63(1), 107-125.
Schmitt, N. (2008). Instructed second language vocabulary learning. Language Teaching Research, 12(3), 329-363. https://doi.org/10.1177/1362168808089921
Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007). Language Teaching, 50(2), 212-226. https://doi.org/10.1017/S0261444815000075
Schmitt, N., Jiang, X., & Grabe, W. (2011). The percentage of words known in a text and reading comprehension. The Modern Language Journal, 95(1), 26-43. https://doi.org/10.1111/j.1540-4781.2011.01146.x
Schmitt, N., Schmitt, D. (2012). A reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language Teaching, 47(4), 484-503. https://doi.org/10.1017/S0261444812000018
Schmuckler, M. A. (2001). What is ecological validity? A dimensional analysis. Infancy, 2(4), 419-436. https://doi.org/10.1207/s15327078in0204_02
Silva, B. B., & Otwinowska, A. (2019). VST as a reliable academic placement tool despite cognate inflation effects. English for Specific Purposes, 54, 35-49. https://doi.org/10.1016/j.esp.2018.12.001
UCLES (2012). Cambridge English: Preliminary Wordlist. Retrieved from https://www.cambridgeenglish.org/Images/84669-pet-vocabulary-list.pdf
UCLES (2021). B1 Preliminary and B1 Preliminary for Schools Vocabulary List. Retrieved from https://www.cambridgeenglish.org/images/506887-b1-Preliminary-2020-vocabulary-list.pdf
van Zeeland, H. (2014). Second language vocabulary knowledge in and from listening (Doctoral dissertation, University of Nottingham).
van Zeeland, H., & Schmitt, N. (2013). Lexical coverage in L1 and L2 listening comprehension: The same or different from reading comprehension? Applied Linguistics, 34(4), 457-479. https://doi.org/10.1093/applin/ams074
Zhao, P., & Ji, X. (2018). Validation of the Mandarin version of the Vocabulary Size Test. RELC Journal, 49(3), 308-321. https://doi.org/10.1177/0033688216639761
Downloads
Published
How to Cite
Issue
Section
License
The authors who publish in this journal agree to the following terms:
- The authors retain copyright and guarantee to the journal the right to be the first to publish the work as well as licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the authorship of the work and the initial publication in this journal.
- Authors may separately enter into additional agreements for non-exclusive distribution of the version of the work published in the journal (e.g., placing it in an institutional repository or publishing it in a book), with acknowledgement of its initial publication in this journal.
- Authors are allowed and encouraged to disseminate their work electronically (e.g., in institutional repositories or on their own website) before and during the submission process, as this can lead to productive exchanges as well as earlier and greater citation of published work (see The Effect of Open Access).
