A Rasch-based Validation of the Evaluation Rubric for Consecutive Interpreting Performance

Foroogh Khorami Nia; Ghasem Modarresi

doi:10.30827/sendebar.v30i0.8512

Authors

Foroogh Khorami Nia Department of English, Quchan Branch, Islamic Azad University, Quchan, Iran
Ghasem Modarresi Department of English, Quchan Branch, Islamic Azad University, Quchan, Iran

DOI:

https://doi.org/10.30827/sendebar.v30i0.8512

Keywords:

Evaluation rubric, testing and assessment, interpretation performance, Rasch measurement, SEM

Abstract

The present study is aimed at developing and validating a rubric for consecutive interpreting performance. Firstly, the researchers interviewed with interpreting experts and teachers to identify the factors involved in assessing student performance on consecutive interpreting. Then they designed an interpreting evaluation checklist including 38 items based on the commonalities that emerged from the interviews. Rasch Measurement determined that 25 items of the checklist fitted the Rasch model. Following this, the researchers constructed 25 items in a Likert-type scale with four levels. Having employed the rubric to measure 105 homogeneous interpreting student performance on consecutive interpreting, Factor Analysis revealed the presence of seven factors with eigenvalues greater than 1.0, which accounted for 77% of the variance. At the final stage, SEM results indicated a good fit to the data. The final rubric consisted of four factors with 22 items. The study offers practical implications for interpreting students and teachers.

Downloads

Download data is not yet available.

Author Biography

Ghasem Modarresi, Department of English, Quchan Branch, Islamic Azad University, Quchan, Iran

Ghasem Modarresi is an assistant professor of Applied Linguistics at IAU of Quchan, and he has been teaching English language and translation courses at BA and MA levels at IAU of Quchan. He has published more than 15 papers in scientific leading journals, and he has presented in national and international conferences. He has translated 3 short story books and has written 2 textbooks on General English. He has been selected as the best researcher in IAU of Quchan in 2019. His main interests include Psychology of Education and Testing and Assessment.

References

Angelelli, C. (2009). Using a Rubric to Assess Translation

Ability: Defining the Construct. In C. Angelelli, & H. Jacobson, Testing and assessment in translation and interpreting studies (pp. 13-48). John Benjamins Publication Co.: Amsterdam.

Bachman, L. F. (2014). Ongoing Challenges in Language Assessment. In A. J. Kunnan (Ed.), The companion to language assessment (Vol. 3), (pp. 1586-1603). Oxford: John Wiley and Sons.

Bachman, L. F., & Palmer, A. S. (1996). Language Testing in Practice: Designing and Developing Useful Language Tests (Vol. 1). New York: Oxford University Press.

Beeby, A. (2000). Teaching Translation from Spanish to English. Ottawa: University of Ottawa Press.

Bond, T. G. & Fox, C. M. (2007). Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Mahwah, NJ.: Lawrence Erlbaum.

Brookhart, S. M. (2011). Educational Assessment Knowledge and Skills for Teachers. Educational Measurement: Issues and Practice, 30 (1), 3-12.

Campbell, S., & Hale, S. (2003). Translation and Interpreting Assessment in the Context of Educational Measurement. Translation Today: Trends and Perspectives 14, 205-224.

Care, E. & Griffin, P. (2009). Assessment is for Teaching. Independence 34 (2), 56-59.

Clifford, A. (2001). Discourse Theory and Performance-based Assessment: Two Tools for Professional Interpreting. Meta 46 (2), 365–378.

Clifford, A. (2005). A Preliminary Investigation into Discursive Models of Interpreting as a Means of Enhancing Construct Validity in Interpreter Certification. Doctoral dissertation, University of Ottawa. Ottawa, Canada.

Cohen, A. D. (1994). Assessing Language Ability in the Classroom. Boston: Heinle & Heinle Publishers.

Conde, T. (2011). Translation Evaluation on the Surface of Texts: A Preliminary Analysis. The Journal of Specialized Translation 69 (15), 69-86.

Dornyei, Z. (2007). The Psychology of the Language Learner Individual Differences in Second Language Acquisition. Mahwah, NJ: Lawrence Erlbaum Associates.

Earl, L. M. (2013). Assessment as Learning: Using Classroom Assessment to Maximize Student Learning (2nd ed.). Thousand Oaks, California: Corwin.

Griffin, P., & Nix, P. (1991). Educational Assessment and Reporting. Sidney: Harcout Brace Javanovich Publisher.

Hatim, B., & Mason, I. (1997). The Translator as Communicator. London & New York: Routledge.

House, J. (1997). Translation Quality Assessment: A Model Revisited. Nehren: Gunter NarrVerlag Tubingen.

House, J. (2001). Translation Quality Assessment: Linguistic Description versus Social Evaluation. Translators' Journal 46 (2), 243-257.

Lamprianou, I. & Athanasou, J. A. (2009). A Teacher’s Guide to Educational Assessment (Revised edition). Rotterdam: Sense.

Lantolf, J. P., & Poehner, M. E. (2004). Dynamic Assessment: Bringing the Past into the Future. Journal of Applied Linguistics 1(1), 49-74.

LeCompete, M. & Preissle, J. (1993). Ethnography and Qualitative Design in Education Research. San Diego, CA: Academic Press.

Linacre, J. M. (2009). Winsteps (Version 3.68) [Computer Software]. Beaverton, Oregon: Winsteps.com.

McMillan, J. H. (2014). Classroom Assessment: Principles and Practice for Effective Standards based Instruction (6th ed.). Boston: Pearson Education.

Mertler, C. A. (2001). Designing Scoring Rubrics for your Classroom. Practical Assessment, Research & Evaluation 7 (25), 1-10.

Moss, A., & Holder, C. (1988). Improving Student Writing. Dubuque, IO: Kendall/ Hunt.

Newmark, P. (1988). A Textbook of Translation. New York: Prentice-Hall International.

Popham, W. J. (2014). Classroom Assessment: What Teachers Need to Know. Boston: Pearson Education.

Rea-Dickins, P. (2008). Classroom-based language assessment. In E. Shohamy & N. H. Hornberger (Eds.) Encyclopedia of Language and Education: Language Testing and Assessment (pp. 257–271). New York, NY: Springer.

Sawyer, D. B. (2004). Fundamental Aspects of Interpreter Education: Curriculum and Assessment. Amsterdam & Philadelphia: John Benjamins.

Scriven, M. (2007). Key Evaluation Checklist. Retrieved March 2016 from:

http://www.wmich.edu/evalctr/archive_checklists/kec_feb07.pdf

Stobart, G., & Gipps, C. (1997). Assessment: A Teacher's Guide to the Issues (3rd ed.). London: Hodder & Stoughton.

Tabachnick, B. G., & Fidell, L. S. (2001). Using Multivariate Statistics (4th ed.). New York: HarperCollins.

Tannen, D. (2007). Talking Voices: Repetition, Dialogue, and Imagery in Conversational Discourse (Vol. 26). London: Cambridge University Press.

Waddington, C. (2001). Should Translations be Assessed Holistically or through Error Analysis? Hermes, Journal of Linguistics, 26. Retrieved August 2016 from http://download2.hermes.asb.dk/archive/download/H2603.PDF

Walvood, B. E., & Anderson, V. J. (1998). Effective Grading. San Francisco: Jossey-Bass.

Wiggins, G. (1998). Educative Assessment: Designing Assessments to Inform and Improve Student Performance. London: Jossey-Bass Publishers.

Wiliam, D. (2010). An Integrative Summary of the Research Literature and Implications for a New Theory of Formative Assessment. In H. L. Andrade & G. J. Cizek (Eds.), Handbook of formative assessment (pp. 18-40). New York: Routledge.

Wilss, W. (1998). Decision Making in Translation. In M. Baker (Ed.), Routledge encyclopedia of translation studies (pp. 57-60). London/New York: Rutledge.