Questionnaire validation on the perception and satisfaction of academics with professional evaluation systems

Authors

DOI:

https://doi.org/10.30827/relieve.v31i1.28345

Keywords:

Questionnaire, validation, psychometric study, perceptions, professional satisfaction, academics evaluation, academics
Agencies: Agencia Estatal de Investigación, Ministerio de Ciencia e Innovación de España, R D i project ‘La influencia del neoliberalismo en las identidades académicas y en el nivel de satisfacción profesional’ -NEOACADEMIC- (PID2019-105631GA-I00/AEI/10.13039/ 501100011033), Grants for the support and promotion of research at the University of Granada in the field of equality , Project ‘Gender inequalities in the new managerial university and their impact on female academics’ -DAMA- (INV-IGU199-2022), Ministry of Universities (Spain) , University Teacher Training Grants Programme (FPU19/00942)

Abstract

One of the purposes of the transformation of higher education is the improvement of quality. At this regard, evaluation and accreditation processes have been incorporated, although they still need to be settled. The objective of this work is to validate the questionnaire "Perceptions and satisfaction of academics on the development and evaluation of their professional activity" (PSPU). To do this, a search and review of instruments was carried out; it allowed the design of a questionnaire to measure the perceptions and level of satisfaction that academics have on evaluation systems. Next, the content of the questionnaire was validated, through the judgment of ten experts. The questionnaire was administered to the population, made up of academics from Andalusian public universities, obtaining a sample of 2183 subjects (12.4%). A construct validity study was carried out through a multigroup confirmatory factor analysis that allowed the identification of three factors. Also, a reliability study was carried out whose results led to the creation of a theoretical model configured by three dimensions, with a Cronbach's Alpha greater than 0.80. The results of this study support the validity and reliability of the questionnaire, which measures both the satisfaction and the perceptions of academics about the evaluation systems of their professional performance. It is concluded that its application will generate key information for designing fairer and more effective university policies and evaluation practices, improving academic well-being and the quality of their functions, to the benefit of the Higher Education system.

Downloads

Download data is not yet available.

Author Biographies

Katia Caballero Rodríguez, University of Granada

Associate professor in the Department of Didactics and School Organisation at the University of Granada. Her research focuses on higher education, academic identity and professional development. She has led and participated in various research projects funded by public organisations and has completed different stays at international universities. She is a member of the FYDAD research group (Training, Development and Teaching).

Javier Mula-Falcón, University of Cádiz

PhD in Educational Sciences at the University of Granada. Teacher and researcher at the University of Cadiz and member of the FYDAD research group (Training, Development and Teaching). His main lines of research focus on higher education, teacher professional development and educational policy.

Eduardo García-Jiménez, University of Seville

Full professor of Educational Evaluation at the University of Seville. Member of the research group ‘LITERACIES: Literacy, multilingualism, diversity and social justice’. His research activity focuses on three main areas: educational assessment, research methodology and competence in linguistic communication.

References

Abrizah, A., Shah, N. A. K., & Nicholas, D. (2019). Malaysian early career researchers on the ethics of scholarly publishing. Malaysian Journal of Library & Information Science, 24(1), 75-96. https://doi.org/10.22452/mjlis.vol24 no1.5 DOI: https://doi.org/10.22452/mjlis.vol24no1.5

Addisu, M. (2018). Determinants of Job Satisfaction Among Employees of Gondar College of Teacher Education, Ethiopia. Research on Humanities and Social sciences, 8(21), 22-32.

Albert, C., Davia, M., & Legazpe, N. (2018): Job satisfaction amongst academics: the role of research productivity. Studies in Higher Education, 43(8). 1362-1377. https://doi.org/10.1080/03075079.2016.1255937 DOI: https://doi.org/10.1080/03075079.2016.1255937

Brøgger, K. (2019). How education standards gain hegemonic power and become international: The case of higher education and the Bologna Process. European Educational Research Journal, 18(2), 158-180. http://doi.org/10.1177/1474904118790303 DOI: https://doi.org/10.1177/1474904118790303

Brunner, J., Labraña, J., Ganga, F., & Rodríguez-Ponce, E (2019). Circulación y recepción de la teoría del “capitalismo académico” en América Latina. Archivos Analíticos de Políticas Educativas, 27(79), 1-32. https://doi.org/10.14507/epaa.27.4368 DOI: https://doi.org/10.14507/epaa.27.4368

Caballero, K. (2009). Construcción y desarrollo de la identidad profesional del profesorado Universitario [Tesis Doctoral]. Universidad de Granada. http://hdl.handle.net/10481/2200

Caballero, K. (2013). Nivel de satisfacción del profesorado universitario con los sistemas de evaluación. Revista de Educación, 360, 483-508. https://doi.org/10.4438/1988-592X-RE-2011-360-124

Cardoso, S., Rosa, M. J., & Santos, C. S. (2013). Different academics' characteristics, different perceptions on quality assessment? Quality Assurance in Education, 21(1), 96-117. https://doi.org/10.1108/09684881311293089 DOI: https://doi.org/10.1108/09684881311293089

Castelló, M., McAlpine, L., & Pyhältö, K. (2017). Spanish and UK post-PhD researchers: writing perceptions, well-being and productivity. Higher Education Research & Development, 36(6), 1108-1122. https://doi.org.10.1080/07294360.2017.1296412 DOI: https://doi.org/10.1080/07294360.2017.1296412

Cohen, L., & Manion, L. (2002). Métodos de Investigación educativa. La Muralla.

Fernández-Cano, A. (2021). Letter to the Editor: publish, publish… cursed! Scientometrics 126, 3673-3682. https://doi.org/10.1007/s11192-020-03833-7 DOI: https://doi.org/10.1007/s11192-020-03833-7

García-Garnica, M. (2016). Elaboración y validación de un cuestionario para medir prácticas eficaces de liderazgo pedagógico de la dirección. Profesorado Revista de Currículum y Formación del Profesorado, 20(3), 493–526.

Hazelkorn, E. (2018). The accountability and transparency agenda: Emerging issues in the global era. En A. Curaj, L. Deca & R. Pricopie (Eds.), European higher education area: The impact of past and future policies (pp. 423–440). Springer. https://doi.org/10.1007/978-3-319-77407-7_26 DOI: https://doi.org/10.1007/978-3-319-77407-7_26

Hernández, F., & Cuesta Saéz, J. D. (2009). Métodos Cuantitativos de Investigación. En M. P Colás, L. Buendía & F. Hernández (2009). Competencias Científicas para la realización de una tesis doctoral. Davinci.

Huisman, J. (2018). Accountability In Higher Education. En P. Nuno & J. Shin, Encyclopedia of International Higher Education Systems and Institutions (pp.1-5). Springer. https://doi.org/10.1007/978-94-017-9553-1_156-1 DOI: https://doi.org/10.1007/978-94-017-9553-1_156-1

Joshi, A., Kale, S., Chandel, S., & Pal, D. (2015). Likert Scale: Explored and explained. British Journal of Applied Science & Technology, 7, 396-403. https://doi.org/10.9734/BJAST/2015/14975 DOI: https://doi.org/10.9734/BJAST/2015/14975

Martín-Romera, A., & Molina, E. (2017). Valor del conocimiento pedagógico para la docencia en Educación Secundaria: diseño y validación de un cuestionario. Estudios Pedagógicos, 43(2), 195-220. https://doi.org/10.4067/S0718-07052017000200011. DOI: https://doi.org/10.4067/S0718-07052017000200011

McCune, V. (2019). Academic identities in contemporary higher education: sustaining identities that value teaching. Teaching in Higher Education, 26(1), 20-35. https://doi.org/10.1080/13562517.2019.1632826 DOI: https://doi.org/10.1080/13562517.2019.1632826

Mohan, P., & Fauzi, M. (2021). Job Satisfaction among Academics in Higher Education: A Systematic Review. Turkish Online Journal of Qualitative Inquiry, 12(3), 1457-1483.

Mula-Falcon, J., & Caballero, K. (2022). Neoliberalism and its impact on academics: A qualitative review. Research in Post-Compulsory Education, 27(3), 373-390. https://doi.org/10.1080/13596748.2022.2076053 DOI: https://doi.org/10.1080/13596748.2022.2076053

Mula-Falcón, J., Lucena, C., Domingo Segovia, J., & Cruz-González, C. (2021). Early career researchers' identity: A qualitative review. Higher Education Quarterly, 76, 786-799. https://doi.org/10.1111/hequ.12348 DOI: https://doi.org/10.1111/hequ.12348

Newton, J. (2000). Feeding the beast or improving quality? Academics’ perceptions of quality assurance and quality monitoring. Quality in Higher Education, 6(2), 153–162. DOI: https://doi.org/10.1080/713692740

Pattaro, A., Moura, P., & Kruijf, A. (2022). Transparency and Accountability in Higher Education as a Response to External Stakeholders and Rules: A Comparison Between Three Country-Case Studies. En E. Caperchione & C. Bianchi (Eds.), Governance and Performance Management in Public Universities (pp.15-49). Springer. ttps://doi.org/10.1007/978-3-030-85698-4_2 DOI: https://doi.org/10.1007/978-3-030-85698-4_2

Raaper, R. (2016). Academic perceptions of higher education assessment processes in neoliberal academia. Critical Studies in Education, 57(2), 175-190. http://dx.doi.org/10.1080/17508487.2015.1019901 DOI: https://doi.org/10.1080/17508487.2015.1019901

Reyes, O., & Hernández Moncada, M. C. (2021). Formato: Validación de contenido por juicio de expertos. Instrumentos cuantitativos [Manuscrito no publicado]. https://doi.org/10.13140/RG.2.2.26812.36486

Santos, A., Muñoz-Rodríguez, D., & Poveda, M. (2015). “En cuerpo y alma”: Insatisfacción y precariedad en las condiciones de trabajo del profesorado universitario. Arxius, 32, 13-14.

Shams, F. (2019). Managing academic identity tensions in a Canadian public university: the role of identity work in coping with managerialism. Journal of Higher Education Policy and Management, 41(6), 619-632. https://doi.org/10.1080/1360080X.2019.1643960 DOI: https://doi.org/10.1080/1360080X.2019.1643960

Shin, J. C., & Jung, J. (2014). Academics job satisfaction and job stress across countries in the changing academic environments. Higher Education, 67, 603-620. http://dx.doi.org/10.1007/s10734-013-9668-y DOI: https://doi.org/10.1007/s10734-013-9668-y

Singh, K. (2007). Quantitative Social Research Methods. SAGE. https://doi.org/10.4135/9789351507741 DOI: https://doi.org/10.4135/9789351507741

Stensaker, B. (2018). External Quality Assurance in Higher Education. En J. Shin & P. Teixeira (eds.), Encyclopedia of International Higher Education Systems and Institutions. Springer. https://doi.org/10.1007/978-94-017-9553-1_523-1 DOI: https://doi.org/10.1007/978-94-017-9553-1_523-1

Szromek, A., & Wolniak, R. (2020). Job satisfaction and problems among academic staff in higher education. Sustainability, 12(12). https://doi.org/10.3390/su12124865 DOI: https://doi.org/10.3390/su12124865

Teichler, U., Arimoto, A., & Cummings, W. (2013). The Changing Academic Profession: Major Findings of a Comparative Survey. Springer. https://doi.org/10.1007/978-94-007-6155-1 DOI: https://doi.org/10.1007/978-94-007-6155-1

Tesouro, M., Corominas, E., Teixidó, J., & Puiggalí, J. (2014). La autoeficacia docente e investigadora del profesorado universitario: relación con su estilo docente e influencia en sus concepciones sobre el nexo docencia-investigación. Revista de Investigación Educativa, 32(1), 169-186. http://dx.doi.org/10.6018/rie.32.1.172771 DOI: https://doi.org/10.6018/rie.32.1.172771

Thoms, P., Dose, J., & Scott, K. S. (2002). Relationships Between Accountability, Job Satisfaction, and Trust. Human Resource Development Quarterly, 13(3), 307-323. https://doi.org/10.1002/hrdq.1033 DOI: https://doi.org/10.1002/hrdq.1033

Tomicic, A. (2019). American dream, Humboldtian nightmare: Reflections on the remodelled values of a neoliberalized academia. Policy Futures in Education, 17(8), 1057-1077. https://doi.org/10.1177/1478210319834825 DOI: https://doi.org/10.1177/1478210319834825

Ursin, J., Vähäsantanen, K., McAlpine L., & Hökkä, P. (2020). Emotionally loaded identity and agency in Finnish academic work. Journal of Further and Higher Education, 44(3), 311-325. https://doi.org/10.1080/0309877X.2018.1541971 DOI: https://doi.org/10.1080/0309877X.2018.1541971

Von Der Embse, N. P., Sandilos, L. E., Pendergast, L., & Mankin, A. (2016). Teacher stress, self-efficacy, and job satisfaction. Learning and Individual Differences, 50, 308-317. https://doi.org/10.1016/j.lindif.2016.08.001 DOI: https://doi.org/10.1016/j.lindif.2016.08.001

Ylijoki, O., Lyytinen, A., & Marttila, L. (2011). Different research markets: a disciplinary perspective. Higher Education, 62, 721-740. https://doi.org.10.1007/s10734-011-9414-2 DOI: https://doi.org/10.1007/s10734-011-9414-2

Zenatta, E., Ponce, T., García, L., Sánchez, C., & Gama, J. (2017). Diseño del cuestionario: Estrategias identitarias de académicos universitarios ante las reformas educativas. Revista de Psicología, 35(2). https://doi.org/10.18800/psico.201702.011 DOI: https://doi.org/10.18800/psico.201702.011

Zhang, L., Fu, M., & Li, D. (2020). Hong Kong academics’ perceived work environment and job dissatisfaction: The mediating role of academic self-efficacy. Journal of Educational Psychology, 112(7), 1431-1443. https://doi.org/10.1037/edu0000437 DOI: https://doi.org/10.1037/edu0000437

Downloads

Additional Files

Published

2025-06-30

How to Cite

Caballero Rodríguez, K., Mula-Falcón, J., & García-Jiménez, E. (2025). Questionnaire validation on the perception and satisfaction of academics with professional evaluation systems. RELIEVE – Electronic Journal of Educational Research and Evaluation, 31(1). https://doi.org/10.30827/relieve.v31i1.28345

Issue

Vol. 31 No. 1 (2025): Revista RELIEVE

Section

Research Articles

License

Copyright (c) 2025 Katia Caballero Rodríguez, Javier Mula-Falcón, Eduardo García-Jiménez

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

The authors grant non-exclusive rights of exploitation of works published to RELIEVE and consent to be distributed under the Creative Commons Attribution-Noncommercial Use 4.0 International License (CC-BY-NC 4.0), which allows third parties to use the published material whenever the authorship of the work and the source of publication is mentioned, and it is used for non-commercial purposes.

The authors can reach other additional and independent contractual agreements, for the non-exclusive distribution of the version of the work published in this journal (for example, by including it in an institutional repository or publishing it in a book), as long as it is clearly stated that the Original source of publication is this magazine.

Authors are encouraged to disseminate their work after it has been published, through the internet (for example, in institutional archives online or on its website) which can generate interesting exchanges and increase work appointments.

The fact of sending your paper to RELIEVE implies that you accept these conditions.