Development and validation of a questionnaire to assess students’ perceptions of EMI degree programmes

Anna Szczesniak; Elvira Barrios

doi:10.30827/portalin.vi45.32331

Authors

Anna Szczesniak University of Málaga (Corresponding author) https://orcid.org/0000-0002-0504-3355
Elvira Barrios University of Málaga https://orcid.org/0000-0002-9419-4031

DOI:

https://doi.org/10.30827/portalin.vi45.32331

Keywords:

EMI, English Medium Instruction, questionnaire, students' perceptions, validation

Abstract

Given the growing popularity of English-taught degree programs in higher education and the need to evaluate their effectiveness, this study aimed to develop and validate a questionnaire to assess students’ perceptions of such programmes. Specifically, four scales were constructed to measure perceptions of students’ language development, academic achievement, use of English in class, and teaching quality. The questionnaire was administered to a sample of 256 students enrolled in different English-taught degree programmes at Spanish universities. Confirmatory factor analysis (CFA) was employed to assess the psychometric properties of the scales. After refinement, the four scales demonstrated strong fit indices and solid discriminant validity. Unlike previous studies that often lack explicit reporting of instrument validity, our research provides a reliable and valid tool for researchers to explore students' perceptions of EMI, for lecturers to improve their teaching practices, and for policymakers to inform and evaluate educational reforms

Downloads

Download data is not yet available.

Author Biographies

Anna Szczesniak, University of Málaga (Corresponding author)

Anna Szczesniak holds dual BA degrees in English Philology from the College of Modern Languages, Poznan, Poland, and the University of Seville. She earned an MA in Compulsory Secondary Education, Vocational Training, and Language Teaching from the University of Malaga, along with a Ph.D. in Applied Linguistics. Currently pursuing a second Ph.D. in Education, she focuses on English Medium Instruction. Active in the Department of English Didactics at the University of Malaga, Anna's research centres on teaching English to children and enhancing teacher training. With numerous publications, she is dedicated to advancing the academic field through continuous scholarly contributions.

Elvira Barrios, University of Málaga

Elvira Barrios Espinosa is Senior Lecturer at the Language and Literature Teaching Department at the Faculty of Education (University of Málaga). She holds a Bachelor’s Degree in English Philology (University of Málaga, Spain), a Master’s Degree in Teaching English as a Foreign Language (University of Reading, UK) and a Ph.D. Degree in English Philology (University of Granada, Spain).

References

Al-Hoorie, A. H., Hiver, P., & In Nami, Y. (2024). The validation crisis in the L2 motivational self system tradition. Studies in Second Language Acquisition, 46(2), 307–329. https://doi.org/10.1017/S027226312300048

Barrios, E., López-Gutiérrez, A., & Lopez-Agudo, L. A. (2022). Language-related perceptions: How do they predict student satisfaction with a partial English Medium Instruction in Higher Education? Journal of English for Academic Purposes, 57(April). https://doi.org/10.1016/j.jeap.2022.101121

Belhiah, H., & Elhami, M. (2015). English as a medium of instruction in the Gulf: When students and teachers speak. Language Policy, 14(1), 3-23.

Byrne, B. M. (2013). Structural equation modeling with AMOS: basic concepts, applications and programming. Routledge.

Curle, S. (2018). Developing and validating a Japanese English medium of instruction attitude scale (JEMIAS). The Journal of Asia TEFL, 15(4), 1195-1221. Downloadable from: https://www.iris-database.org/iris/app/home/detail?id=york:936114&ref=search

Curle, S. M., & Derakhshan, A. (2021). Trends in using questionnaires for EMI research: Suggestions for future improvements. Research Methods in English Medium Instruction, July 2021, 32–45. https://doi.org/10.4324/9781003025115-3

Dornyei, Z. (2003). Questionnaires in second language research; Construction, administration, and processing. Mahwah, N.J.: Lawrence Erlbaum.

Dornyei, Z. (2007). Research methods in applied linguistics. New York, NY: Oxford University Press.

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equations models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. https://doi.org/10.2307/3151312

Gelabert, E., García-Esteve, L., Martín-Santos, R., Gutiérrez, F., Torres, A., & Subirà, S. (2011). Psychometric properties of the Spanish version of the Frost Multidimensional Perfectionism Scale in women. Psicothema, 23(1), 133–139.

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

Iwaniec, J., & Wang, W. (2023). Motivations to enrol in EMI programmes in China: an exploratory study. Applied Linguistics Review, 14(6), 1563–1587. https://doi.org/10.1515/applirev-2021-0180

Khan, H. I. (2013). An investigation of two universities’ postgraduate students and their teachers’perceptions of policy and practice of English medium of instruction (EMI) in Pakistani universities. PhD thesis. University of Glasgow. http://theses.gla.ac.uk/4451/

Kırkgoz, Y. (2009). Students’ and lecturers’ perceptions of the effectiveness of foreign language instruction in an English-medium university in Turkey. Teaching in Higher Education, 14(1), 81-93.

Kline, R. B. (2011). Principles and practice of structural equation modelling. Guilford Press

Kelley, T.L. (1927). Interpretation of educational measurements. World Book Company,

Kym, I., & Kym, M. H. (2014). Students’ perceptions of EMI in higher education in Korea. The Journal of Asia TEFL, 11(2), 35-61

Macaro, E. (2019). Exploring the role of language in English medium instruction. International Journal of Bilingual Education and Bilingualism, 23(3), 263-267. https://doi.org/10.1080/13670050.2019.1620678

Macaro, E., Curle, S., Pun, J., An, J., & Dearden, J. (2018). A systematic review of

English medium instruction in higher education. Language Teaching, 51(1),

–76. https://doi.org/10.1017/S0261444817000350

Mackey, A., & Gass, S. (2005). Second language research: Methodology and design.

Routledge.

Mira, S., San José Martínez, F., Hontoria, C., Sanz-Cobena, A., Rodríguez Sinobas, L., & Cuadrado, G. (2021). Attitudes of academics and students towards English-medium instruction in Engineering Studies. European Journal of Engineering Education, 46(6), 1043–1057. https://doi.org/10.1080/03043797.2021.1987392

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. McGraw-Hill.

Revelle, W., & Zinbarg, R. E. (2009). Coefficients Alpha, Beta, Omega and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. doi:10.1007/s11336-008-9102-z

Rial, A., Varela, J., Abalo, J., & Lvy, J. (2006). El análisis factorial confirmatorio [Confirmatory factor analysis]. In En J. Lévy Mangin (Ed.), Modelización con Estructuras de Covarianzas en Ciencias Sociales (pp. 119–154), Netbiblo

Rodríguez M., & Ruiz M. (2008). The reduction of skewness and kurtosis of observed variables by data transformation: Effect on factor structure. Psicológica, 29(2), 205-227.

Ruiz, M.A., Pardo, A., & San Martín, R. (2010). Modelos de ecuaciones estructurales [Structural Equation Models]. Papeles del Psicólogo, 31(1), 34-35

Satayev, M., Barrios, E., Fernandez-Costales, A., Agaidarova, S., Izbassarova, R., & Balta, N. (2022). The effect of CLIL combined with language instruction on language learning and the role of individual and institutional factors in students’ perspectives: Empirical evidence from Kazakhstan. Eurasia Journal of Mathematics, Science and Technology Education, 18(10). https://doi.org/10.29333/ejmste/12425

Sugimoto, S. (2021). Students’ experience of English medium instruction classes in medical school in Japan. JALT Journal, 43(1), 33–60.

Thompson, B. (2004). Exploratory and confirmatory factor analysis. Understanding concepts and applications. American Psychological Association

Yang, M., O’Sullivan, P. S., Irby, D. M., Chen, Z., Lin, C., & Lin, C. (2019). Challenges and adaptations in implementing an English-medium medical program: a case study in China. BMC Medical Education, 19(15), 1-8.