Underrepresentation of the construct in Spanish standardized foreign language exams: a computer assisted exam proposal
DOI:
https://doi.org/10.30827/portalin.vi.21393Keywords:
language competence, language use, multimodality, standardized tests, University Entrance examsAbstract
Standardized tests are a reality in a globalized world and, consequently, in the Spanish system. Language test developers have already claimed that the current model of assessment has little to do with real language use, as factors such as the shift from paper to computer, information search and other uses of language or multimodal communication are not being considered (García Laborda & Fernández Álvarez, 2021). This theoretical paper proposes (1) a novel construct for testing that incorporates new functionalities which have been shown able to test appropriately language performance in academic settings; and (2) the proposal and design of an online multimodal exam that adapts to the new needs for language assessment in the University Entrance Examination. It begins with an analysis of current research to show the need for a change in the construct, and concludes by proposing a model for this high-stakes exam, based on a new construct of language proficiency and multimodal interaction for computer-based exams. The paper concludes that this model can change the way we understand technology-based exams and could be applicable to most current exams.
Downloads
Download data is not yet available.
References
Amengual Pizarro, M. (2006). Análisis de la prueba de inglés de selectividad de la Universitat de les Illes Balears. Ibérica, 11, 29-59.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.
Bachman, L. F. & Palmer, A. S. (1996). Language Testing in Practice. Oxford University Press.
Bachman, L. F. & Palmer, A. S. (2010). Language Assessment in Practice. Oxford University Press.
Bruner, J. S. (1961). The act of discovery. Harvard Educational Review, 31, 21–32.
Boals, T., Kenyon, D. M., Blair, A., Cranley, M. E., Wilmes, C., & Wright, L. J. (2015). Transformation in K-12 English language proficiency assessment: Changing contexts, changing constructs. Review of Research in Education, 39(1), 122-164. http://dx.doi.org/10.3102/0091732X14556072.
Campione, J. C., Brown, A. L. (1987) Linking dynamic assessment with school achievement. In Lidz, C. S. (Ed.) Dynamic Assessment: An interactional approach to evaluating learning potential (pp. 82-115). Guilford Press.
Canale, M. (1983). From communicative competence to communicative language pedagogy. In J. C. Richards & R. W. Schmidt (Eds.), Language and communication (pp. 2-27). Longman.
Canale, M. & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1): 1-47.
Celce-Murcia, M., Dornyei, Z. & Thurrell, S. (1995). Communicative Competence: A Pedagogically Model with Content Specifications. Issues in Applied Linguistics, 6(2), 5-35.
Chapelle, C. A., Enright, M. K., Jamieson, J. M. (Eds.) (2008). Building a validity argument for the Test of English as a Foreign Language. Routledge.
Chapelle, C., Grabe, W. & Berns, M. (1997). Communicative Language Proficiency: Definition and Implications for TOEFL 2000. Educational Testing Service.
Choi, J., & Yi, Y. (2016). Teachers’ Integration of Multimodality into classroom practices for English Language Learners. TESOL Journal, 7(2), 304-327.
Cinganotto, L. (2019). Online Interaction in Teaching and Learning a Foreign Language: an Italian Pilot Project on the Companion Volume to the CEFR. Journal of e-Learning and Knowledge Society, 15(1), 135-151.
Chomsky, N. (1965). Aspects of the theory of syntax. MIT Press.
Council of Europe (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment – Companion volume. Council of Europe Publishing.
Díez Bedmar, M. B. (2011). The English exam in the University Entrance Examination: An Overview of Studies. Revista Canaria de Estudios Ingleses, 63, 101-112.
Fatemipour, H., & Jafari, F. (2015). The Effect of Dynamic-Assessment on the Development of Passive Vocabulary of Intermediate EFL Learners. Journal of Educational and Management Studies, 5(1), pp. 41-51.
Fernández Álvarez, M. & García-Hernández, S. (en Prensa). Teachers’ perceptions of linguistic mediation in the Curriculum for Advanced English in Madrid Secondary Schools. Language Teaching Research.
García Laborda, J. (2006). Analizando críticamente la Selectividad ¿Todos los estudiantes tienen las mismas posibilidades? TESOL-Spain Newsletter, 30 (2), 9-12.
García Laborda, J. (2010). ¿Necesitan las universidades españolas una prueba de acceso informatizada? El caso de la definición del constructo y la previsión del efecto en la enseñanza para idiomas extranjeros. Revista de Orientación y Psicopadagogía, 21(1),71-80.
García Laborda, J. (2012). Presentación. De la Selectividad a la Prueba de Acceso a la Universidad: pasado, presente y un futuro no muy lejano. Revista de Educación, 357, 17-27.
García Laborda, J., Amengual Pizarro, M., Litzler, M. F., García Esteban, S. & Otero de Juan, N. (2017). Student Perceptions of the CEFR Levels and the Impact of Guided Practice on APTIS Oral Test Performance. British Council.
García Laborda, J. & Fernández Álvarez, M. (in press). Multilevel Language tests: Walking into the land of the unexplored. Language learning & Technology.
García Laborda, J. Giménez López, J. L., & Magal Royo, M. T. (2011). Validating Mobile Devices in the Spanish University Entrance Exam English Paper. New Educational Review, 25(3), 160-171.
García Laborda, J. & Gimeno Sanz, A. (2008). Adaptación del examen de inglés de las pruebas de acceso a la universidad a un entorno informático: Estudio sobre la tipología de preguntas. Proceedings of the XXV Congreso Nacional de Lingüística Aplicada, Murcia, Spain (pp. 723-730). Servicio de publicaciones de la Universidad de Murcia.
García Laborda, J., & Magal Royo, T. (2009) Training senior teachers in compulsory computer based language tests. Procedia - Social and Behavioral Sciences, 1(1), 2009, 141-144.
García Laborda, J., & Martín-Monje, E. (2013). Item and test construct definition for the new Spanish Baccalaureate final Evaluation: A proposal. International Journal of English Studies, 13(2), 69-88.
Kramsch (1986). From Language Proficiency to Interactional Competence. The Modern Language Journal, 70(4), 366-372.
McNamara, T. (2000). Language Testing. Oxford University Press.
McLeod, S. A. (2019). Bruner - learning theory in education. Simply Psychology. Retrieved from: https://www.simplypsychology.org/bruner.html
Motallebzadeh, K. & Moghaddam, P. B. (2011). Models of Language Proficiency: A Reflection on the Construct of Language Ability. The International Journal of Language Testing, 1(1), 42-48.
Oller, J. W. (1979). Language Tests at School. Longman.
Oller, J. W. (1983). Issues in language testing research. Newbury House.
Pacheco B. A. and Souza-Concilio, I. A. (2013). Multimodal interfaces: An enactive approach, IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom), Budapest, Hungary, 51-58, doi: 10.1109/CogInfoCom.2013.6719302
Poehner, M. E. (2007). Beyond the Test: L2 Dynamic Assessment and the Transcendence of Mediated Learning. Modern Language Journal, 91(3), 323-340.
Poehner, M. E. (2008). Dynamic Assessment. A Vygotskian Approach to Understanding and Promoting L2 Development. Springer.
Poehner, M. E. (2018). Probing and provoking L2 development: The object of mediation in Dynamic Assessment and mediated development. In Lantolf, J. P., Poehner, M. E. & Swain, M. (Eds.), The Routledge handbook of Sociocultural Theory and second language acquisition (pp. 249-265). Routledge.
Poehner, M. E., Zhang, J. & Lu, X. (2015). Computerized Dynamic Assessment (C-DA): Diagnosing L2 development according to learner responsiveness to mediation. Language Testing, 32, 337–357.
Poehner, M. E. & Lantolf, J. P. (2005). Dynamic assessment in the language classroom. Language Teaching Research, 9(3), 233-265.
Ruiz-Lázaro, J., González Barbera, C. y Gaviria Soto, J.L. (2021). Las pruebas de inglés para acceder a la Universidad. Una comparación entre Comunidades Autónomas. Educación XX1, 24(1), 233-270, http://doi.org/10.5944/educXX1.26746
Sanz Sainz, I. & Fernández Álvarez, M. (2005). La validez del examen de inglés en Selectividad. In Herrera Soler, H. & García Laborda, J. (Eds.). Estudios y criterios para una selectividad de calidad en el examen de inglés (pp- 149-164). Editorial de la Universidad Politécnica de Valencia.
Vergara Cabarcas, L. K., López Caraballo, J. L., Castellón Barrios, D. J., Vásquez Rossi, C. A. & Becker Arroyo, E. A. (2019). Dynamic Assessment Approach in Language Teaching: A Review. Zona Próxima, 30, 82-99.
Winke, P., Lee, S., Ahn, J. I., Choi, I., Cui, Y., & Yoon, H. (2018). The cognitive validity of child English language tests: What young language learners and their native-speaking peers can reveal. TESOL Quarterly: A Journal for Teachers of English to Speakers of Other Languages and of Standard English as a Second Dialect, 52(2), 274-303. http://dx.doi.org/10.1002/tesq.396.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.
Bachman, L. F. & Palmer, A. S. (1996). Language Testing in Practice. Oxford University Press.
Bachman, L. F. & Palmer, A. S. (2010). Language Assessment in Practice. Oxford University Press.
Bruner, J. S. (1961). The act of discovery. Harvard Educational Review, 31, 21–32.
Boals, T., Kenyon, D. M., Blair, A., Cranley, M. E., Wilmes, C., & Wright, L. J. (2015). Transformation in K-12 English language proficiency assessment: Changing contexts, changing constructs. Review of Research in Education, 39(1), 122-164. http://dx.doi.org/10.3102/0091732X14556072.
Campione, J. C., Brown, A. L. (1987) Linking dynamic assessment with school achievement. In Lidz, C. S. (Ed.) Dynamic Assessment: An interactional approach to evaluating learning potential (pp. 82-115). Guilford Press.
Canale, M. (1983). From communicative competence to communicative language pedagogy. In J. C. Richards & R. W. Schmidt (Eds.), Language and communication (pp. 2-27). Longman.
Canale, M. & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1): 1-47.
Celce-Murcia, M., Dornyei, Z. & Thurrell, S. (1995). Communicative Competence: A Pedagogically Model with Content Specifications. Issues in Applied Linguistics, 6(2), 5-35.
Chapelle, C. A., Enright, M. K., Jamieson, J. M. (Eds.) (2008). Building a validity argument for the Test of English as a Foreign Language. Routledge.
Chapelle, C., Grabe, W. & Berns, M. (1997). Communicative Language Proficiency: Definition and Implications for TOEFL 2000. Educational Testing Service.
Choi, J., & Yi, Y. (2016). Teachers’ Integration of Multimodality into classroom practices for English Language Learners. TESOL Journal, 7(2), 304-327.
Cinganotto, L. (2019). Online Interaction in Teaching and Learning a Foreign Language: an Italian Pilot Project on the Companion Volume to the CEFR. Journal of e-Learning and Knowledge Society, 15(1), 135-151.
Chomsky, N. (1965). Aspects of the theory of syntax. MIT Press.
Council of Europe (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment – Companion volume. Council of Europe Publishing.
Díez Bedmar, M. B. (2011). The English exam in the University Entrance Examination: An Overview of Studies. Revista Canaria de Estudios Ingleses, 63, 101-112.
Fatemipour, H., & Jafari, F. (2015). The Effect of Dynamic-Assessment on the Development of Passive Vocabulary of Intermediate EFL Learners. Journal of Educational and Management Studies, 5(1), pp. 41-51.
Fernández Álvarez, M. & García-Hernández, S. (en Prensa). Teachers’ perceptions of linguistic mediation in the Curriculum for Advanced English in Madrid Secondary Schools. Language Teaching Research.
García Laborda, J. (2006). Analizando críticamente la Selectividad ¿Todos los estudiantes tienen las mismas posibilidades? TESOL-Spain Newsletter, 30 (2), 9-12.
García Laborda, J. (2010). ¿Necesitan las universidades españolas una prueba de acceso informatizada? El caso de la definición del constructo y la previsión del efecto en la enseñanza para idiomas extranjeros. Revista de Orientación y Psicopadagogía, 21(1),71-80.
García Laborda, J. (2012). Presentación. De la Selectividad a la Prueba de Acceso a la Universidad: pasado, presente y un futuro no muy lejano. Revista de Educación, 357, 17-27.
García Laborda, J., Amengual Pizarro, M., Litzler, M. F., García Esteban, S. & Otero de Juan, N. (2017). Student Perceptions of the CEFR Levels and the Impact of Guided Practice on APTIS Oral Test Performance. British Council.
García Laborda, J. & Fernández Álvarez, M. (in press). Multilevel Language tests: Walking into the land of the unexplored. Language learning & Technology.
García Laborda, J. Giménez López, J. L., & Magal Royo, M. T. (2011). Validating Mobile Devices in the Spanish University Entrance Exam English Paper. New Educational Review, 25(3), 160-171.
García Laborda, J. & Gimeno Sanz, A. (2008). Adaptación del examen de inglés de las pruebas de acceso a la universidad a un entorno informático: Estudio sobre la tipología de preguntas. Proceedings of the XXV Congreso Nacional de Lingüística Aplicada, Murcia, Spain (pp. 723-730). Servicio de publicaciones de la Universidad de Murcia.
García Laborda, J., & Magal Royo, T. (2009) Training senior teachers in compulsory computer based language tests. Procedia - Social and Behavioral Sciences, 1(1), 2009, 141-144.
García Laborda, J., & Martín-Monje, E. (2013). Item and test construct definition for the new Spanish Baccalaureate final Evaluation: A proposal. International Journal of English Studies, 13(2), 69-88.
Kramsch (1986). From Language Proficiency to Interactional Competence. The Modern Language Journal, 70(4), 366-372.
McNamara, T. (2000). Language Testing. Oxford University Press.
McLeod, S. A. (2019). Bruner - learning theory in education. Simply Psychology. Retrieved from: https://www.simplypsychology.org/bruner.html
Motallebzadeh, K. & Moghaddam, P. B. (2011). Models of Language Proficiency: A Reflection on the Construct of Language Ability. The International Journal of Language Testing, 1(1), 42-48.
Oller, J. W. (1979). Language Tests at School. Longman.
Oller, J. W. (1983). Issues in language testing research. Newbury House.
Pacheco B. A. and Souza-Concilio, I. A. (2013). Multimodal interfaces: An enactive approach, IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom), Budapest, Hungary, 51-58, doi: 10.1109/CogInfoCom.2013.6719302
Poehner, M. E. (2007). Beyond the Test: L2 Dynamic Assessment and the Transcendence of Mediated Learning. Modern Language Journal, 91(3), 323-340.
Poehner, M. E. (2008). Dynamic Assessment. A Vygotskian Approach to Understanding and Promoting L2 Development. Springer.
Poehner, M. E. (2018). Probing and provoking L2 development: The object of mediation in Dynamic Assessment and mediated development. In Lantolf, J. P., Poehner, M. E. & Swain, M. (Eds.), The Routledge handbook of Sociocultural Theory and second language acquisition (pp. 249-265). Routledge.
Poehner, M. E., Zhang, J. & Lu, X. (2015). Computerized Dynamic Assessment (C-DA): Diagnosing L2 development according to learner responsiveness to mediation. Language Testing, 32, 337–357.
Poehner, M. E. & Lantolf, J. P. (2005). Dynamic assessment in the language classroom. Language Teaching Research, 9(3), 233-265.
Ruiz-Lázaro, J., González Barbera, C. y Gaviria Soto, J.L. (2021). Las pruebas de inglés para acceder a la Universidad. Una comparación entre Comunidades Autónomas. Educación XX1, 24(1), 233-270, http://doi.org/10.5944/educXX1.26746
Sanz Sainz, I. & Fernández Álvarez, M. (2005). La validez del examen de inglés en Selectividad. In Herrera Soler, H. & García Laborda, J. (Eds.). Estudios y criterios para una selectividad de calidad en el examen de inglés (pp- 149-164). Editorial de la Universidad Politécnica de Valencia.
Vergara Cabarcas, L. K., López Caraballo, J. L., Castellón Barrios, D. J., Vásquez Rossi, C. A. & Becker Arroyo, E. A. (2019). Dynamic Assessment Approach in Language Teaching: A Review. Zona Próxima, 30, 82-99.
Winke, P., Lee, S., Ahn, J. I., Choi, I., Cui, Y., & Yoon, H. (2018). The cognitive validity of child English language tests: What young language learners and their native-speaking peers can reveal. TESOL Quarterly: A Journal for Teachers of English to Speakers of Other Languages and of Standard English as a Second Dialect, 52(2), 274-303. http://dx.doi.org/10.1002/tesq.396.
Downloads
Published
2022-01-29
How to Cite
Fernandez Alvarez, M., García Laborda, J., & Magal-Royo, T. (2022). Underrepresentation of the construct in Spanish standardized foreign language exams: a computer assisted exam proposal. Porta Linguarum An International Journal of Foreign Language Teaching and Learning, 27–45. https://doi.org/10.30827/portalin.vi.21393
Issue
Section
Articles
License
The authors who publish in this journal agree to the following terms:
- The authors retain copyright and guarantee to the journal the right to be the first to publish the work as well as licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the authorship of the work and the initial publication in this journal.
- Authors may separately enter into additional agreements for non-exclusive distribution of the version of the work published in the journal (e.g., placing it in an institutional repository or publishing it in a book), with acknowledgement of its initial publication in this journal.
- Authors are allowed and encouraged to disseminate their work electronically (e.g., in institutional repositories or on their own website) before and during the submission process, as this can lead to productive exchanges as well as earlier and greater citation of published work (see The Effect of Open Access).
